Cool new tool – Hexawise

I just checked out a new tool called Hexawise (hexawise.com).  It certainly has peaked my interest!!  In a nut shell, if you are testing an application that has a lot of configuration options, it gives you a simple, clean interface to load all of your config options and then the tool will generate test scenarios for you using the pairwise approach.  If you are not familiar with this technique, check out this link : http://en.wikipedia.org/wiki/All-pairs_testing

At first glance, I thought this would be a really cool test planning tool.  Then, the more I thought about it and played with it, I discovered that I am also able to use this to create my fitnesse test tables (inputs only).  I was able to plug in the input fields and the options, then use hexawise to generate the test cases to ensure all pair combos were covered, and then export that to excel and paste it right into a fitnesse table.  I think the same approach would work for creating data fed into tests in QTP, WinRunner, etc.

I can see that this would save me a lot of time manually thinking through the combinations I want to test with, etc when I am tasked with creating tests for functionality that has thousands of possible test cases (and you can’t test for every scenario).

The tool is not fancy (which I could care less about) and is early in its life but I think there is a lot of great potential here.  Definitely worth checking out and following.  I’ll be interested to see how the tool grows.

Diary Post 2 – First 2 weeks of work

Okay….so posting once a week seems too much for me!

So, I have been at my new job for 2.5 weeks now and things have already started to change and shift around for the better.  However, I am running across some much expected resistance from some folks as well.

I am also faced with the added challenge of needing to stay billable while working on implementing change and establishing testing practices.  This means I don’t have the luxury of taking a step back to get a full picture across all of our projects.  Instead, I am staffed on 3 projects that I can directly work on and bill my time to. So, all change and testing activity is focused on those 3 projects for now.

Week 1:

I spent the first week (only 4 days due to the holiday) in our headquarters meeting with a small number of folks to get a lay of the land on the 3 initial projects I am working on.  Project A is a very typical legacy software application with all of the typical legacy baggage.  Project B is a research project that has about 6 months left.  Project C is a Greenfield software development project that is just getting started and is currently contracted for the next 18 months.

Learning the company’s DNS is going to take some time…I have never been surrounded by so many acronyms before in my life!  My goal for the end of week 1 was to provide a document to the president that outlined my observations regarding areas for improvement and proposed recommendations (in priority order) on where to start.

A tough challenge I always face when I am “interviewing” folks to understand how things work today is to make sure it doesn’t feel like an interview.  I try to ask leading questions so that I am not talking much at all.  Then, as people are answering them, I try to keep a poker face.  So, when someone says, “Environments?  No, we just check in our code from our laptop and then push the build to the production server”  my gut reaction is to drop my jaw, grasp my heart, and stifle a scream.  Instead, I try to look calm and unaffected and then follow that up with a, “How is that working for you?”  What I find most valuable in these sessions is to have the people I am talking to come to their own conclusions on what may not be going so well.  This way, when the time comes to start implementing change, they are already on board with the need for it.  Or, even better, they start changing practices all on their own!

I won’t list all of the issues and recommendations I came up with but here are a few themes:

1. Planning and Tracking projects
2. Configuration Management (builds, environments, branching/merging practices)
3. Testing
4. Information Sharing within the company

Week 2:

One thing I failed to mention earlier is that I live in NC and the company HQ is in VA.  About ½ of the employees work in VA and the other half are scattered around the country working from home.  This obviously adds another dimension to implementing change and makes it harder.

After reviewing the recommendations document with my boss, we agreed to focus initially on planning/tracking and testing with the 3 projects.  I agreed to do a 2 hour webex with a core group of project leads focused on project planning and tracking.  The company already has Jira and has been using it sporadically on some projects.

I spent most of the first few days of my second week going through Jira and cleaning up some old projects, tasks, etc.  I found some fields that were not being used correctly and cleaned those up as well.  Project C (the Greenfield project) had a complete Microsoft Project Plan built out by the project lead so I imported all of that into Jira to use for the webex as well.

On Wednesday of week 2, I gave a 2 hour presentation that focused on the 5 levels of planning (vision, roadmap, release, iteration, daily).  For some political reasons, I am avoiding Scrum specific language and am avoiding the word Agile as well.  I am focusing my points on Lean and it seems to be received well.  We discussed tracking tools and the expected challenges and benefits from planning and tracking this way.  I was then able to show them a company specific project in Jira set up with a full roadmap and release plan and a burndown chart started for the current iteration (yay Project C!)  As expected, some people were excited, some were a little skeptical, and some don’t buy it all.  Also as expected, the non-believers at this point are the ones that historically have provided little to no direction for their project teams and things are pretty chaotic on that team.  This new approach will require them to get organized, plan better, and communicate regularly with the team to make sure they have what they need.

I spent the later half of the week getting FitNesse running for Project C as well and built a few fit tables to test the first feature that was just completed.

Finally, I am eating my own dog food and I translated the recommendations doc I gave to my boss into a project in Jira as well called “Continual Improvement.” This is where I am going to track the efforts that are non-project specific.  This includes things around CI, Planning, Infrastructure, Tools, etc.   I have monthly releases laid out with high level stories in each month.  For July, I have broken out those stories into detailed tasks with estimates and am updating my time daily as I work on those tasks.  I have set up a call every other Friday with my boss to review this backlog of work together and re-prioritize as necessary.

All in all, the first 2 weeks were a lot of fun and I am really excited about the work.

Week 3 is already throwing some expected curve balls at me….will share soon.

Diary of a Live Process and Quality Improvement Initiative

I just made a move out of the consultant world. I just joined a small company (about 35 folks). They have been in business for almost 5 years and do consulting and contracts mostly for NASA and the FAA. The company has grown quite a bit in the last year and recognized the need to establish some better quality practices across their programs and projects. So, enter me!

I was hired with a fairly loose job description which is a good and bad thing. It is good because I am able to really create this job and have pretty free reign on what I am allowed to do. The bad part is that there isn’t any real clear direction on what they want from me….they just know they need process and testing improvement. So, I don’t really have any solid goals, objectives or priorities to work from.

I will be posting an update every 1 or 2 weeks on what I am doing and how I am doing it (and most importantly, how well it is working!).  I hope this will bring some interesting conversations, suggestions, and feedback from others. My first post will be up next Monday on my first week of work.

Communicating vs. Discussing – Lesson Learned

I was visiting a customer last week and had an “ah-ha” moment thanks to a comment by someone there.
We were talking about communication and the need for regular communication with the entire division regarding the work they are doing towards an agile transformation.  Linda then said the following (or something close to it..), “communication and discussing are NOT the same thing.  We need to make sure we give the staff the time to discuss.”

It was such a simple statement but really hit me hard.  I have always felt I was a good communicator…..I try to send updates to my team every week and share as much information as I have to ensure we are all on the same page.  However, I haven’t always thought about the discussing side and how very important that is.
I can communicate all day long but if I don’t provide the situation and environment for everyone to discuss the information that was just communicated, then I am missing the boat.  I know that I always feel more comfortable with change when I have a chance to ask questions, get clarification, and discuss with others.
So, this is my new goal for the rest of the year…continue to communicate but also ensure there is ample opportunity for discussion.
Thanks Linda!

DON’T RUN! (tips for communicating I learned from my nephew)

I just finished teaching an Agile Testing Class to an awesome testing team in Dallas.  We spent a good amount of time discussing how to communicate effectively with other team members. Specifically, we were talking about how to approach a developer when you need them to implement a solution in a different way that lends itself to easier testability.  I often hear these conversations start with words like, “Don’t put the code…..” or “I can’t complete my tasks if you…..” or “Every time you do that I can’t….” etc.  You get the point…they are very negative statements.  Without meaning to, you are putting up a wall and the person you are talking to is likely to go on the defensive immediately.  Not a great way to foster teamwork.
I have a 4-year-old nephew that, like all other boys his age, absolutely can NOT walk anywhere.  His feet only know how to move at a run.  I have spent several hours of my life saying (or shouting), “Don’t run!!!”  “Stop Running!”  I recently learned a great tip from his pre-school teacher.  Rather then constantly telling him what NOT to do, just remind him of what he can do.  So, I have learned to use the phrase , “Use your walking feet!”  Not only do I sound like a much nicer person when I say that in crowded public places, he responds to it so much better.  He is allowed and encouraged to use his walking feet.  He is just being reminded of good behavior.
I encourage team members to apply this same principle when you are communicating with one another.  Instead of telling a developer what NOT to do, offer up a suggestion of what would work great.  If you aren’t sure what will work, show them what you are trying to do and ask for their help in crafting a solution.  By communicating this way, you are fostering better collaboration and team work instead of putting each other on the defensive.
Oh, and use your walking feet when you go talk to them ☺

Tips for Working Effectively with Distributed Team Members

In a few classes I have taught recently, we have had some interesting conversations around working on distributed agile teams.  I always recommend trying to keep your Scrum team co-located whenever possible but in today’s world, distributed teams are unavoidable.  Here are some suggestions to help keep your distributed team members more involved:

 

1.    If the majority of your team is in one location and a few members are remote, get a large stuffed animal, blow up doll (not the adult variety please), etc to represent your distributed team members.  Bring the doll, stuffed animal, etc to all of your meetings and put them in a chair.  You can even write the names of the people it represents on a sticky note and stick it on the doll. The visual reminder of the people on the phone helps keep the folks in the room aware of the team members on the phone.

2.    Draw a table on a whiteboard where the meeting is initiated.  Write the names of all attendees on the phone around the virtual table.

3.    Use an on-line, real time collaboration tool that all team members can access during meetings. The Scrum Master can then put relevant data here for everyone to see.  If you have a distributed team, a physical card wall may not be a good choice.  Look into using tools like the GreenHopper plug-in for Jira or Rally.

4.    Get a web-cam and use it often.  I know a company that has a webcam running all the time in each office and has it is pointing to the task board.  Everyone can log in and see it whenever they need to.  The web cams are also used for daily stand-ups, etc.

5.    In meetings where the purpose is to gather information and form an action plan such as planning meetings, retrospectives, etc, try using the round robin style of gathering information.  Go around the virtual room and ask each person to contribute.  This way, everyone has a chance to be heard.

6.    Publish any decisions made somewhere where the entire team can see it.  Don’t bury information on an intranet or share point site that no one goes to. 

7.    Create some simple, fun games with prizes that encourage team members to visit the place (intranet, wiki, etc)  where you keep important information (like action items from retrospectives, etc).  For example, put trivia questions from Jeopardy on the site and give a fun award at the end of your stand-up to whoever responds first with the correct answer.  Or, you can track the results each day during the sprint and then give an award at your sprint demo.

8.    Devote a few minutes during each retrospective to discuss remote team issues and to brainstorm on how to make things better.  Follow-up with an action plan and re-visit it after it sprint.

9.    Pick a few times a year where everyone travels to the same location at once.  This gives everyone a chance to meet each other face to face.

10.   If you are not remote, then act like a remote employee for a day every now and then (i.e. work from home).  See first hand what the remote people experience so you can better understand the challenges they may be facing.

 

The point here is to get creative and go beyond simple conference calls.  Your Scrum Master should own the responsibility of managing effective communication and collaboration with distributed team members.  Make sure you are getting regular feedback from your team members on what is working well and what isn’t.  Collaboration on your distributed teams should be a top priority and you should be willing to make the necessary investments to ensure that all team members feel like they are heard and are part of the team.

Quick, great read on agile testing

This blog post by Simon Baker is excellent and does a great job describing the role of a tester on an agile team.  http://www.think-box.co.uk/blog/2008/05/testers-in-our-agile-team.html

Manual Testers vs. Automation Engineers – Why the Divide?

I just got back from the stp conference in San Francisco (www.stpcon.com).  After attending several talks and speaking one on one with folks there, I started to notice a recurring undercurrent.  There seems to be a divide in the testing community between manual testing and automated testing.  I am noticing more and more that papers, blogs, etc that I read and any talks I attend have either one focus or the other but never combine the two.  For example, the automated community rarely mentions manual testing or the need for it.  Conversely, the manual testing community (especially in the exploratory testing world) seems to really down-play the need and/or importance of automated testing.  I think this is in turn perpetuating a divide in skill sets of testers as well.  Teams have manual testers and then separate automation engineers instead of just having test staff that do both.

 

In my experience, the real sweet-spot in testing is when you have a combination of both manual and automated tests.  They both serve a real purpose and they both provide a lot of value.  I would never want to be on a project that was either 100% manual testing or 100% automated testing.  The projects I have been on that have been the most successful and produced the highest quality results were projects that used a combination of testing strategies.

 

Automated testing allows you to get more testing done.  When you have a large portion of your test scripts automated, that frees up time for testers to do exploratory testing that can’t be put under automation.  If none of your tests are automated, then you will likely spend a good portion of your testing time validating happy path, “good” user scenarios and will never get to spend time really digging into the product to test the more unusual scenarios that tend to have hidden defects in them.

 

The strongest testers I have worked with are ones that can do both.  They are technical enough to get high functioning, easily maintainable automation scripts in place but also spend time using their “soft” skills to manually test as well.

 

Here is why I think we are seeing this divide in our community: 

 

1. Most teams separate their manual testers from their automated testers.  I think this is a recipe for disaster.  When you have a team that only does automated testing, their scripts are often given to them by the manual testers.  Often times, the automation engineers code exactly what they are told to do without any real in-depth knowledge of the product.  Then, because the manual testers don’t really understand the automation or how it works, (and because testers tend to struggle with severe trust issues), the manual testers often spend a large amount of time manually testing the same functions that are automated….just so they can be sure it really works and that the automation didn’t miss anything.  What a waste of time!!!
2. The manual testing community doesn’t want to learn how to automate.  The idea of learning that technology is scary.
3. The automation community thinks manual testing is boring and doesn’t want to do it.  A lot of the strong automation testers I have worked with and met come from development backgrounds and really just want to write code.  They have no desire to really “play” with the system to see what they can find.

 

I think manual only testers have a chip on their shoulders with automation because they don’t know how to do it and are scared of it.  Automation engineers tend to make more money then manual testers because of the required technical skill set – they are treated almost like developers in terms of rank and pay.  Often times, they act like they are more valuable as well.  Not good.  The skills that are required to be a good manual tester are hard to measure so they are often not considered as valuable as developer skills.  This isn’t fair, either.  I think it is harder to train someone how to think like a good tester then it is to train them how to create an automation script. 

 

When testers are able to play in both spaces….do automation AND manual testing, then they are able to build better automation scripts and spend their manual testing time focusing on those high risk, interesting parts of the application that lend themselves to interesting what-if scenarios.  They won’t waste time manually testing what is under automation because they can trust the automation is doing what it was built to do.  However, finding these people is hard (at least it has been for me!). 

 

I hope that in the future we can see more folks in the testing community talking about how manual and automated testing can work together and highlight the strengths and limitations of both.  One is not better then the other – they each serve a specific purpose and each provides significant value.  I believe that every project needs and should have manual and automated tests going on all the time.

 

STP Conference Day 2

The morning started off with a Keynote from Robert Sabourin called “What is so Different about Testing in Scrum.”  I wouldn’t say the title of the talk really fit the talk itself but it was a good kick-off for the day.  If you have seen him speak before, then you know he is high energy and enthusiastic. 

His talk essentially gave a brief overview of Scrum and then walked through some case studies of teams he has worked with that were adopting Scrum and what some of their challenges and proposed solutions were.  It wasn’t surprising to hear that the challenges were centered on either lack of product ownership and/or testing team challenges.  I was hoping he would actually dive into some of the key differences in testing in Scrum like the title of the talk mentioned but he really didn’t go there.  No one seems to go there…again, another topic for another time. 

 

I went to 4 different talks.  Two of these talks were topics around agile testing.  I was disappointed by both of these talks in that they really didn’t get into any meat on the topics.  It was all high level principles with no actual techniques or take-aways on what to do and how to do it. 

 

One talk was on metrics.  The speaker was good and had good content for the most part.  However, I would argue that several of the metrics that were discussed really don’t provide any value to an organization.  For example, she mentioned tracking requirements stability as a metric.  Let’s say you do this and you discover that 40% of your requirements change.  So what.  What is that metric going to do for your team or organization?  Requirements always change.  Period.  We all know that…we should all be prepared for it.  Spending a significant amount of time (and therefore money) tracking how many requirements change doesn’t really help you to handle those changing requirements.  Why not invest that time and money into something that will help your team work effectively in the changing requirement world?

 

I ended the day with a talk by Matt Heusser called “Software Testing from the Ground Up.”  The published excerpt on the talk didn’t seem to match the talk itself.  I did enjoy some of the comparisons he made between testers and other professions, though. 

STP Conference – Day 1 review

I am currently in San Francisco at the STP Conference (www.stpcon.com).  I am really starting to clearly see what I consider a divide in the testing world between automated and manual testing that I believe is negatively affecting all development methodologies.  I am currently collecting my thoughts on this for an upcoming blog post….stay tuned J

 

Yesterday was the first day of the conference and was a full-day tutorial.  I spent the day at Michael Bolton’s “Rapid Intro to Rapid Software Development” class.  This is usually a 3-day class that was condensed into one day.  Because of that, the class was a lot of lecture and not as much hands-on as I was hoping for.  That said, I have condensed 2 day classes into 4 hour “overviews” so I know the challenges with that.  I think I would really enjoy the 3-day class much more.

 

Overall, I enjoyed the class and picked up some great tips/tricks/ideas particularly around exploratory testing.  I have attended a few talks by James and John Bach in the past and have read many of their articles, etc on exploratory testing.  This workshop really drove home how this technique works and the value it adds.

Some of my key take-aways from this class were:

 

1. As a tester, you should report what you see, not what you infer.  Basically, the idea here is that we should report just the facts and not infer what the problem is.  By doing this, we are able to keep our credibility with the rest of the team.  If you document an issue and infer what you think the problem is and that isn’t the actual problem, then the developers may start to think of you like the boy that cried wolf.  Just reporting the facts avoids this issue.
2. Manual tests should discover things – not prove things.  I love this – especially in the Scrum/Agile context.  Agile teams focus heavily on test automation (as they should).  However, manual testing is still important and necessary.  Manual testing allows you to “discover things.”  Test automation is where you prove things – you prove that the new code that was checked in didn’t break the existing code – you prove that the system is still functioning as it was before.  However, your automation doesn’t discover new things.  You discover behaviors, unfriendly user interfaces, patterns, etc by manual testing.  Both are important.
3. Asking the right question is more important then getting the right answer.  Food for thought.  This ties directly into the 3C’s of User Story development – card, conversation, confirmation.  During your conversation, you should be focused on asking the right questions.  These questions will hopefully illicit those implied assumptions, those “what-if” scenarios no one had thought of.  What you learn in asking those “right questions” can then be documented in the confirmation (the acceptance tests).
4. Any bug that slows down testing is a bad bug.  The example that brought this to light in the class was what most teams would likely call a usability issue.  The application we were testing had an entry field and a button to click to calculate the answer.  As a user, you could not enter the value and click the enter key on your keyboard to calculate the answer.  You had to either mouse or tab to the button on the UI first to click it.  This wasn’t necessarily a “bug.”  I am sure developers, testers, and product folks could argue for hours as to whether or not it was a “bug” or a usability issue.  It would like get ranked a low priority since the system essentially worked.  However, this functionality really slowed down testing.  If we could have just entered data and clicked enter, we could have tested more cases quickly.  Having to tab or mouse for every test slowed down my rate of testing.  Testability of the application you are working on should be considered.  This is why I like the concepts of ATDD and TDD.  If the tests are defined before development starts, then the developers can build the system in such a way that lends itself to easy testing.  As testers, it is absolutely our right to ask for testable applications.

 

More stpcon reviews to come as the conference continues……

What makes a ScrumMaster a “good” ScrumMaster?

I was recently asked to help form a job description for a new ScrumMaster (SM) position in a company.  So, it got me thinking about what qualities and skills make up a good SM.  I thought back on teams I have been on that had good and some not-so-good SMs.  Most of the skills that come to mind when I think of a good SM are more of the “soft” skills.  Here are the ones I came up with: 

Intimately familiar with Scrum.  Since the SM owns the process, they must be intimately familiar with how Scrum works and be able to guide a team to find solutions to help them succeed.  This takes experience.  Most SMs come fresh out of CSM training and jump into the role.  The challenge here is that they often do not have an experienced SM available to mentor and coach them.  If at all possible, hire someone with proven experience or get help from an experienced coach. 

Excellent facilitator.  SMs organize and facilitate several of the Scrum meetings.  This requires organizational and excellent facilitation skills.  If you are a SM today and are looking for help in this area, I highly recommend the Collaboration Explained class (see previous posts on this class).  Facilitation is very different from controlling meetings.   

Highly available to the team.  If your SM has a full time role elsewhere in the company and just runs in for the daily standup, they are not going to be very helpful to the team.  They need to manage impediments and be available as needed to the team to work through any roadblocks.  The SM should always know the status of the team and how things are going. 

Be quiet.  This is a tough one.  Often times, skills that make a person a good SM are the same skills that make it hard for them to keep their mouth shut.  I have seen several SMs that can’t help themselves and start telling team members what tasks to do in the daily standup or start questioning the team’s estimates during estimation sessions.  As a SM, you need to know when to keep your mouth shut and when to step in.  You are not a traditional project manager and you should not operate in a command and control style. You are a “servant leader.” 

All About the Team.  As a SM, you are always focused on the team, not on yourself.  You need to thrive in watching the team succeed and do whatever it takes to help them do so.  If you need a lot of personal recognition and praise, then a SM may not be the best job for you.  

Book Review: Test Driven .Net Development with FitNesse

I just finished Gojko Adzic’s book called Test Driven .Net Development with FitNesse.  I have used FitNesse on .Net projects and I still learned a ton from this book.  I highly recommend it for anyone working with FitNesse on a .Net project, even if you are already comfortable navigating the waters of FitNesse.  If you have heard of FitNesse and are interesting in learning more about it and playing with it, this is the perfect place to start.  The book is geared more towards developers but is still an excellent read for analysts and testers as well.  You can download the source code behind the examples and follow along very easily.

Thank you, Gojko, for writing this much needed book!

Notes from a Recovering Crackberry Addict

I was out to lunch the other day with some friends I hadn’t seen in a long time.  I was really looking forward to catching up and seeing them.  However, by the time the check came, I was half irritated with them both and ready to leave.  Why?  Because the entire lunch was spent with them half listening and checking their Crackberry’s every 2 minutes.   

In the spirit of full disclosure, I must first admit that I WAS one of those people about a year ago.  My Blackberry was strapped to my side 24/7.  It was the last thing I looked at before I went to sleep and the first thing I looked at when I woke up in the morning.  Every time it vibrated to let me know an email came in, I stopped what I was doing to check it out.  I just couldn’t help myself.  I was addicted.  Surely every email that came through (and there was well over 100 a day), needed my immediate attention or something awful was going to happen.  I just knew it. 

When I changed jobs last year, my Blackberry was left behind.  I almost bought a new one but decided on a simple cell phone instead with no text messaging package included.  My last day at work, I handed over my Blackberry and immediately felt 20 pounds lighter.  I was free.   

However, later that night, I swear I began to get nervous and jumpy.  I was out with friends and they all had Blackberry’s they were diligently checking every few minutes and all I had was my little cell phone that only made and received calls.  I started to panic. What if someone had sent me an email?  What if they all knew something I didn’t?  How would I know what was going on?   Maybe I needed to go and trade in my cell phone for a Blackberry or Trio the next day. 

I decided to see if I could stay Crackberry free for 1 month.  If the world fell apart during that time because of an email I didn’t answer immediately, then, I would go back and get fully connected again. As the weeks went by, I slowly started to calm down.  I was shocked to find that the world (and my job) managed to do just fine without me getting emails immediately all day long.  If there truly was an emergency at work, my co-workers knew where to find me or they could actually pick up the phone and call me if needed.  The funny thing is, they never did.  My new cell phone remained quiet. 

So, I am now the worst thing possible…I am a reformed Blackberry user (kinda like a smoker that finally quit).  I get irritated when I see people madly scrolling with their thumb and typing furiously with both thumbs.  (I will say that whoever invented the Blackberry is clearly not a woman…you try typing on that with long nails…it takes some talent!)   I challenge all you Blackberry users out there to actually turn it off for a few hours one day and see what happens.  Or if you are having a nice meal out with friends or family, leave it in the car and really participate in the analog conversation (nothing digital for an entire meal!!).   I can almost guarantee that nothing catastrophic will happen. 

If you are an employer that hands out Blackberry’s to your staff, reconsider it.  What is wrong with the old fashioned pager?  I used to carry one of those years ago for work emergencies.  And, I can honestly say, it only went off in emergencies.  People will think twice before paging you at 9pm at home but there is no reason not to send an email to someone at all hours of the day.  And, if you are like most Blackberry users, you will feel obliged to stop what you are doing and check that email no matter what time of day it is. 

Now, I realize that there are some of you out there that can carry a Blackberry and are not addicted to it.  You can actually use it in moderation.  Well, good for you.  I was not one those people (and neither are most of my friends that carry them).  So, I had to make a complete break and go cold turkey.  And, it was the best thing that ever happened to me.  My husband also agrees J 

Who gets to decide when to ship software?

I recently got in a lengthy debate with some folks over this very topic.  As I quickly found out, most folks have a very strong opinion on this.  And here I thought I was the only oneJ.  It seems as though most folks fall into 3 different camps: 

Camp #1 – The Testers decide 

This “camp” was formed of folks on waterfall and agile teams (which I found interesting).  They felt that the team that tests the software and was most familiar with the quality and test coverage should get the final say on whether or not to ship it. 

Camp #2 – The Project Manager decides 

This “camp” applied to people on more traditional waterfall projects.  They felt that the project manager knew all of the data points with the project and based on that data should be responsible for determining when to release software.  However, this project managers did not have ROI responsibility. 

Camp #3 – The Product Manager (aka: ”the business”) decides 

This “camp” applied to people that were staffed on more agile projects.  On these teams that had a strong product owner presence, they felt that since the product owner was ultimately responsible for the ROI, then they should decide what and when to ship. 

Camp 1 had the most votes followed by camp 2 and then camp 3 had the least amount.  In some ways this surprised me but in other ways, it was what I was expecting. 

I will admit that I used to be a member of Camp #1.  As a matter of fact, I believe at one point I might have actually been the camp founder and directorJ  However, over time, I have moved to Camp #3.  Let me tell you why. 

Testers typically do have the most insight into the quality of the software.  We are intimately familiar with what is working, what isn’t, and what the high risk areas of the application are.  We can tell you the defect trends, defect turn around time, severity break downs, etc.  Because of this, it is easy for us to feel like we should be the decision maker as to whether or not we should put this code into our customer’s hands.  However, often times, there are several things we do NOT know that also play in to the decision of whether to ship code or not.  Perhaps there is a large business deal that will fall through if we don’t ship.  Perhaps the customer (or customers) wanted the new code so badly that they were willing to take the risk of using it with the list of known issues.  Perhaps the business wants it shipped for demo purposes but isn’t planning on having customers use it until it is cleaner.  Perhaps the business is planning a Beta / trial period.  The list can go on and on.  You get my point.   

Here is what I think about the testers’ role in deciding when to ship software.  I think the tester is responsible for delivering all of the metrics around the quality of the code to the business.  Deliver the facts, not your emotion.  Include test coverage, defect count, defect find rate, etc.  Make sure the business has a solid grasp on the current state of the code and any risks associated with shipping this code.  Then, do the hardest thing possible…..trust that the business team will make the right decision based on those facts and any other data points they may have.

Then, I also firmly believe that the business should CLEARLY COMMUNICATE with the delivery team why they made the decision to ship (or not ship) the software.  This is particularly important if they decide to ship software that the testers don’t feel is ready to ship.  The business should let the team know that they understood the risks but made the decision to ship based on the following facts (at this point, they should clearly list out the reasons why they are shipping).

 If you think a bad decision was made and don’t know why, then by all means ask for an explanation!

Should we be called QA Analysts?

I have been thinking a lot about titles lately and just how much I detest them :)  I do love the fact that my current title (QA Architect) is something most people have never heard of so it prompts them to ask me what I really do.  It gives me a chance to explain to folks what I actually do at work, not what they will assume I do based on my title!  Common titles can be very misleading to what people actually do.  The title of QA Analyst or QA Engineer is a perfect example.  I am going to pick on this because I had this title for a long time.  (The same goes for QA Manager).  There are several reasons why I hate these titles.  Here are the top 3 reasons why:

1.   If someone on your team has the title of QA Analyst, they typically bear the responsibility of the end quality of your product.  People start to say things like, “The QA folks will test it…don’t spend too much time testing your code.”  Or, “QA will catch it if there are bugs.”  Since when did the end quality of the product rest solely on the QA staff?  Everyone on your team should be responsible for quality, not just the folks testing the software.

2.   QA Analyst of what exactly?  It sure is vague.  Everyone just assumes it means the quality assurance analyst or engineer of the application you are building.  Why are there people with the word “quality” in their title only on the development side?  What about the quality of the entire development process, the quality of the designs, the quality of the sales team, quality of support?  By putting a “QA” title in development only it seems like we are really missing the boat by not looking at quality across the organization in all roles.

3.   When you tell someone that you are a QA person, you have just stuck a huge label on yourself.  It may be a good label or a bad label depending on who you are talking to.  Some QA folks focus solely on testing.  Some QA folks are really focused on SDLC only.  By telling someone that you do “QA”, you may not be telling them what you really do at all.   

Here is what I would love to see.  I would like to see people that focus on testing called “Testers.”  Nothing more, nothing less.  If the main purpose of your job is to test the application to ensure that it is working as it should, then just call yourself a tester.  People that develop code are just called developers, not “Quality Developers”.  Let’s call a spade a spade here.  If you test, then just say you are tester.  You should not be ashamed of that! 

Then, ideally, you WOULD have someone in your organization that is overseeing quality.  However, this person would be looking at quality across the board.  They would be analyzing processes, etc from the sales team all the way through development and then to the team that supports the customers.   Depending on the size of your company, this could be a whole team of folks.  Make sure these people know what they are doing and have been in the trenches actually “doing it” before.  Also, make sure these folks are trained on the Lean Principles of software development  to get the biggest bang for your buck.

What is your current title?  Do you think it reflects what you really do?

Collaboration Explained Class – Day 2 (Last Day)

I can honestly say that I have never been so exhausted after a 2 day class/workshop.  The last 2 days were filled with so much great information that my head is spinning.  Jean and Ronica did an amazing job with this class.  I don’t want to list out too much detail around my key take-aways.  If I did, this blog post would likely be a novel and I also don’t want to give away all my new found “subtle skills.”  That said, here are 3 things that really hit home for me (in no particular order):

1. I talk too much   As I am writing this, I swear I can hear my friends and family laughing.  Seriously, when I am in a facilitator role, I talk too much and don’t listen enough.  I really learned just how important it is to be quiet and neutral as a facilitator.  I think this is an area that most facilitators can improve on as well.
2. I don’t prepare enough for the meetings I run.  If Jean and Ronica had prepared for this class the way I prepare for my meetings, it would have been a disaster.  I am committing to myself to spend the appropriate amount of time preparing for meetings I facilitate (if I am planning a 2 hour meeting, then I need to spend 4 hours preparing for it).  If people are going to give up time in their day to come to a meeting I set up, then I owe it to them to make it a productive and worthwhile use of their time.
3. Managing conflict isn’t easy but I now feel very well equipped with new tools and techniques to do this.  I am no longer afraid of having conflict in meetings but am actually looking forward to the opportunity to use some of my new found skills to help with this.

One of the real benefits of this class that I was not expecting was how much I learned about myself and what type of person I am.  I am much more aware (painfully so ) of some character traits I have that I am hoping to improve on in the work place.

I would highly recommend this class for any and all CSM’s.  I can guarantee that you will come out of it prepared to really facilitate collaborative decision making on your agile teams.

Collaboration Explained Class – Day 1

I am currently in Boulder taking the Collaboration Explained class at Rally.  We just completed day 1 of 2 and so far, this class has been just awesome!

One of my biggest pet peeves around the Certified Scrum Master class/workshop is that anyone that wants to go and sit in on a 2 day class and pay the money for it is suddenly a “certified” ScrumMaster.  I know from personal experience that by getting “certified” I was no where near being equipped or qualified to serve as a good Scrum Master on my team (or any team for that matter).

In my opinion, THIS class would be a much better CSM class.  This class is all about giving you the tools and techniques needed to be a good servant leader and facilitator.  I am already overloaded with a ton of great tools and techniques that I will be able to take back to my job and start using immediately.  

I am a huge fan of the Collaboration Explained book by Jean Tabaka and this class is the greatest compliment to it.   If you haven’t read the book, do so as soon as you can.  Once you have read it, do everything you can to attend this class – it is well worth it! 

I am going to wait until the class is over to post my favorite moments and take-aways.

“I am an Agile Tester” article

I have been actively recruiting to find agile testers for over a year now at different companies I have worked for.  This has been a very hard position to fill.  As part of the recruiting process, I had to come up with a job description for an agile tester.  Not a trivial task J  So, I was very excited to see an article through StickyMinds.com called “I am an Agile Tester.  And this is Why” by Erik Boelen. 

All in all, I think Mr. Boelen covered a pretty good checklist for what makes an agile tester (especially from the soft-skill perspective).  His checklist was the following: 

1.    I love my customer

2.    I work in a very structured way

3.    I adore unconventional test tools

4.    I communicate from an open position

5.    I am a VIP in my project team, just like the others

6.    I don’t spend time ‘covering my ass’

7.    I know my boundaries

8.    I am a tester 

On his list, my favorite item was #6.  I have never really thought about that before but it is so true.  I have wasted hours of my life on traditional software projects playing the CYA game.  However, since I have moved to agile projects, I don’t waste time doing this any more.  It is so refreshing!  Instead, the entire team works together to get things done.  No one is blaming others, we aren’t ‘signing off’ on documents to prove anything, and we aren’t wasting time documenting things to protect ourselves. 

I do think that Mr. Boelen missed one crucial item on his checklist, though.  I would add the following:         

I love to automate tests. 

Agile testers should embrace, practice, and live test automation.  Without it, your projects will never become truly agile.

Software Testers: Have we become our own worst enemy?

More and more companies are either switching to agile practices or talking about switching to them.  Agile development highlights high-quality software delivery and yet the testing community seems to be the most resistant to adopting agile.  Have the qualities that make us great software testers become our own worst enemy when it comes to adopting agile practices?

I recently attended the Agile2007 conference in Washington DC.  I went to almost all of the testing focused talks and tutorials and as the week wore on, I came to an interesting yet disturbing realization.  After listening to dozens of testers ask questions during presentations, I realized that Software Testers, QA analysts, QA engineers, or whatever your team calls them have become their own worst enemy.  What gives me the right to say this?  I am a software tester and have been one for over 10 years.  I know how testers think and why we are the way we are. 

Agile development is all the rage these days.  If your company isn’t currently trying it out in one way, shape, or form then chances are that someone is talking about it.  After attending several sessions at the agile conference (and talking to dozens of testers there), the testing community seems to be the one that is resisting agile practices the most!  What really shocks me about this is that we (the testers) have the most to gain from good agile practices!  Agile focuses on high quality.  So, why are we resisting this so much?   

Let’s start by taking a look into the mind of a tester.  Testing is not just a skill set, but it really is a way of thinking.  Some people would even go so far as to say it is a way of life.  We generally have a “guilty until proven innocent” mentality.  When we are given code to test, the first thing we want to do is to prove that it doesn’t work.  That is our job.  That is why we get a paycheck.  That is why we come to work every day.  We are hired and paid to break things.  So, it is no surprise that when a new process (Scrum, XP, etc) is introduced, our first inclination as testers is to “break” the process.  We start looking for all of the ways that the process will fail and try to find defects in it.  We immediately assume (whether we realize it or not) that we don’t trust that the process has really been unit tested well and it is therefore our job to find all of the issues and challenges with it.   

So, all of you testers out there, I challenge you to change your way of thinking for a few minutes.  Instead of immediately trying to “break” agile practices, I challenge you to first analyze the process you are using today and list out the defects with it.  Here is a quick list of the typical “defects” that testers complain about on software projects: 

1.     We are never invited to meetings early in the project to discuss requirements, design, etc.  We have no input into what is being built so we have a hard time creating test plans.

2.     By the time we start testing, the requirements have changed 10 times and no one has told us.  So, we have no idea what we are testing anymore and our test plans are not valid.

3.     We have no idea what development is doing when they are coding and code is “thrown over the wall” for us to test.

4.     We never have the amount of time we need to test.  If development delivers late, then we don’t get to push out our testing time.

5.     The code we get from developers is really hard to automate and is of poor quality.

6.     We feel the full burden of quality.  If we miss a bug, everyone will blame QA! 

How does agile solve these issues?  The beauty of agile development (when done correctly) is that it puts quality first every step of the way.  This sounds like a testers dream come true, doesn’t it? 

Let’s take a closer look at the 6 problems listed above and how good agile practices help solve these issues: 

1.     Agile is all about including the entire team in release planning, iteration planning, estimation, etc.  Testers are involved upfront and in every step of the way.  If priorities change, the entire team knows about it, including the testers.

2.     Agile promotes test first activities.  On good agile teams, the acceptance tests are written before development even starts.  The entire team sees the tests and agrees that they need to pass in order for that story to be considered “done.”   

3.     The tests are written first, so the testers always know what the developers are doing – they are writing code to make the tests pass!  The testers and developers are sitting side by side working as a team to ensure the tests are passing.  There are no walls on agile teams….just constant collaboration and communication.

4.     In agile teams, nothing is considered “done” until it has passed the acceptance tests.  Therefore, developers don’t talk about being “done” until the tests pass.  We don’t discuss design complete, coding complete, and testing complete.  Nothing is done until the tests pass and that is what matters.

5.     The developers know what the tests are before they start coding.  This encourages them to build code that can be automated easily.  Their focus is to build code that makes the tests pass.

6.     It the entire team’s responsibility to drive stories to completion with all tests passing.  The entire team carries the responsibility of producing high quality software. 

So, why do testers want agile to fail?  Why in the world would we want to stay in our waterfall world full of problems when there is clearly a better, higher quality way of building and delivering software?  What are we afraid of?  I think there are 4 main reasons that make testers resist a change to agile.  Fear, accountability, skill set, and the need to be right. 

Most testers out there are afraid of the unknown.  We may not like the delivery process we are currently in but it is comfortable and we know what to expect.  Moving into the “unknown” means a loss of control.  That isn’t a happy place for testers.  We tend to fall into the Type-A, control-freak side of the spectrum.  I know this because I am one.   

Our current waterfall process gives us something to blame when we miss bugs that go into production.  We can always say, “I didn’t have enough time” or “the requirements changed and I didn’t know it” or “I couldn’t automate the tests so we didn’t have time to run them more than once.”   

We are also comfortable with our current skill set, especially if we are manual testers.  Agile testing relies heavily on test automation to be successful.  Therefore, when teams start discussing a change to agile, we hesitate to get on board because we are scared that we won’t provide value on those teams with our current skill set.  When you think about this, it is really a double standard for developers and testers.  As technology changes, we expect our developers to learn new languages, architectures, etc.  Why should we as testers be allowed to stay stagnant in our technologies and skill set?  We should constantly work to increase our skill sets to match the changing needs of the software industry, too!  Testing on an agile team is very different then testing on a waterfall team, especially if all of your testing has been manual.  Adopting good agile practices may require new skills, tools, and techniques.  Instead of pushing back, I challenge you to look at this an opportunity to beef up your skill set and learn new things.  Get engaged!     

Testers do not like to be wrong.  It is our job to ensure that what we are testing is “right.”  I believe that many testers worry that if they start championing agile practices, they will be looked at as having been “wrong” with their former processes.  I argue that learning a new process or skill doesn’t mean that the old one was wrong.  That is just as crazy as saying that old development languages were “wrong” and everyone that used them were wrong.  In reality, we were all doing what we knew to be the best at that time.  Processes and technologies improve and evolve over time.  Testers need to embrace this as much as the development community does. 

As testers, we are champions of quality at our organizations.  If you are a tester that is currently resisting a change to agile practices, I ask you to take an honest look as to why.  If you can truly say that your process has no room for quality improvement, then I would love to hear what you are doing.  You must sleep very well at night J  If not, then push yourself out of your comfort zone and take that first step.  Stop trying to “break” agile and start practicing it.

TISQA 2007 Conference Review

I spent the day yesterday at the TISQA conference in Chapel Hill, NC.  It is a one day local conference that the TISQA group puts on every year. 

The opening key note was from Jon Bach.  As usual, it was a great talk.  I love listening to him talk about exploratory testing.  I always learn something new.  The afternoon keynote was a talk by Rex Black called “Testing in an Outsourced World.”  As usual, Rex was an awesome presenter.  While I disagree with some of his views on managing outsourced testing, I got some great ideas from him around how to manage cultural differences in that situation. 

I only got to listen to one other presenter (TR Buskirk) since I was presenting in the afternoon.  It was a fun talk that highlighted some of the challenges that QA groups face in all companies.

This was the first testing only conference I have been to in several years.  It was an eye-opener for me.  The conference was run extremely well as usual and the TISQA group did a great job scheduling, etc.  The main thing that stuck out for me was the atmosphere difference from this conference to others I have attended in the last few years.  My personal opinion is that the atmosphere was different because it was a group of testers, not a mix of developers, testers, product folks, etc.  It definitely didn’t feel like an “agile-friendly” environment.  That said, it was still a good experience and I hope that at the end of the day, at least one person there may think about approaching testing differently.