I just checked out a new tool called Hexawise (hexawise.com).  It certainly has peaked my interest!!  In a nut shell, if you are testing an application that has a lot of configuration options, it gives you a simple, clean interface to load all of your config options and then the tool will generate test scenarios for you using the pairwise approach.  If you are not familiar with this technique, check out this link : http://en.wikipedia.org/wiki/All-pairs_testing

At first glance, I thought this would be a really cool test planning tool.  Then, the more I thought about it and played with it, I discovered that I am also able to use this to create my fitnesse test tables (inputs only).  I was able to plug in the input fields and the options, then use hexawise to generate the test cases to ensure all pair combos were covered, and then export that to excel and paste it right into a fitnesse table.  I think the same approach would work for creating data fed into tests in QTP, WinRunner, etc.

I can see that this would save me a lot of time manually thinking through the combinations I want to test with, etc when I am tasked with creating tests for functionality that has thousands of possible test cases (and you can’t test for every scenario).

The tool is not fancy (which I could care less about) and is early in its life but I think there is a lot of great potential here.  Definitely worth checking out and following.  I’ll be interested to see how the tool grows.

Okay….so posting once a week seems too much for me!

So, I have been at my new job for 2.5 weeks now and things have already started to change and shift around for the better.  However, I am running across some much expected resistance from some folks as well.

I am also faced with the added challenge of needing to stay billable while working on implementing change and establishing testing practices.  This means I don’t have the luxury of taking a step back to get a full picture across all of our projects.  Instead, I am staffed on 3 projects that I can directly work on and bill my time to. So, all change and testing activity is focused on those 3 projects for now.

Week 1:

I spent the first week (only 4 days due to the holiday) in our headquarters meeting with a small number of folks to get a lay of the land on the 3 initial projects I am working on.  Project A is a very typical legacy software application with all of the typical legacy baggage.  Project B is a research project that has about 6 months left.  Project C is a Greenfield software development project that is just getting started and is currently contracted for the next 18 months.

Learning the company’s DNS is going to take some time…I have never been surrounded by so many acronyms before in my life!  My goal for the end of week 1 was to provide a document to the president that outlined my observations regarding areas for improvement and proposed recommendations (in priority order) on where to start.

A tough challenge I always face when I am “interviewing” folks to understand how things work today is to make sure it doesn’t feel like an interview.  I try to ask leading questions so that I am not talking much at all.  Then, as people are answering them, I try to keep a poker face.  So, when someone says, “Environments?  No, we just check in our code from our laptop and then push the build to the production server”  my gut reaction is to drop my jaw, grasp my heart, and stifle a scream.  Instead, I try to look calm and unaffected and then follow that up with a, “How is that working for you?”  What I find most valuable in these sessions is to have the people I am talking to come to their own conclusions on what may not be going so well.  This way, when the time comes to start implementing change, they are already on board with the need for it.  Or, even better, they start changing practices all on their own!

I won’t list all of the issues and recommendations I came up with but here are a few themes:

1. Planning and Tracking projects
2. Configuration Management (builds, environments, branching/merging practices)
3. Testing
4. Information Sharing within the company

Week 2:

One thing I failed to mention earlier is that I live in NC and the company HQ is in VA.  About ½ of the employees work in VA and the other half are scattered around the country working from home.  This obviously adds another dimension to implementing change and makes it harder.

After reviewing the recommendations document with my boss, we agreed to focus initially on planning/tracking and testing with the 3 projects.  I agreed to do a 2 hour webex with a core group of project leads focused on project planning and tracking.  The company already has Jira and has been using it sporadically on some projects.

I spent most of the first few days of my second week going through Jira and cleaning up some old projects, tasks, etc.  I found some fields that were not being used correctly and cleaned those up as well.  Project C (the Greenfield project) had a complete Microsoft Project Plan built out by the project lead so I imported all of that into Jira to use for the webex as well.

On Wednesday of week 2, I gave a 2 hour presentation that focused on the 5 levels of planning (vision, roadmap, release, iteration, daily).  For some political reasons, I am avoiding Scrum specific language and am avoiding the word Agile as well.  I am focusing my points on Lean and it seems to be received well.  We discussed tracking tools and the expected challenges and benefits from planning and tracking this way.  I was then able to show them a company specific project in Jira set up with a full roadmap and release plan and a burndown chart started for the current iteration (yay Project C!)  As expected, some people were excited, some were a little skeptical, and some don’t buy it all.  Also as expected, the non-believers at this point are the ones that historically have provided little to no direction for their project teams and things are pretty chaotic on that team.  This new approach will require them to get organized, plan better, and communicate regularly with the team to make sure they have what they need.

I spent the later half of the week getting FitNesse running for Project C as well and built a few fit tables to test the first feature that was just completed.

Finally, I am eating my own dog food and I translated the recommendations doc I gave to my boss into a project in Jira as well called “Continual Improvement.” This is where I am going to track the efforts that are non-project specific.  This includes things around CI, Planning, Infrastructure, Tools, etc.   I have monthly releases laid out with high level stories in each month.  For July, I have broken out those stories into detailed tasks with estimates and am updating my time daily as I work on those tasks.  I have set up a call every other Friday with my boss to review this backlog of work together and re-prioritize as necessary.

All in all, the first 2 weeks were a lot of fun and I am really excited about the work.

Week 3 is already throwing some expected curve balls at me….will share soon.

I just made a move out of the consultant world. I just joined a small company (about 35 folks). They have been in business for almost 5 years and do consulting and contracts mostly for NASA and the FAA. The company has grown quite a bit in the last year and recognized the need to establish some better quality practices across their programs and projects. So, enter me!

I was hired with a fairly loose job description which is a good and bad thing. It is good because I am able to really create this job and have pretty free reign on what I am allowed to do. The bad part is that there isn’t any real clear direction on what they want from me….they just know they need process and testing improvement. So, I don’t really have any solid goals, objectives or priorities to work from.

I will be posting an update every 1 or 2 weeks on what I am doing and how I am doing it (and most importantly, how well it is working!).  I hope this will bring some interesting conversations, suggestions, and feedback from others. My first post will be up next Monday on my first week of work.

I was visiting a customer last week and had an “ah-ha” moment thanks to a comment by someone there.
We were talking about communication and the need for regular communication with the entire division regarding the work they are doing towards an agile transformation.  Linda then said the following (or something close to it..), “communication and discussing are NOT the same thing.  We need to make sure we give the staff the time to discuss.”

It was such a simple statement but really hit me hard.  I have always felt I was a good communicator…..I try to send updates to my team every week and share as much information as I have to ensure we are all on the same page.  However, I haven’t always thought about the discussing side and how very important that is.
I can communicate all day long but if I don’t provide the situation and environment for everyone to discuss the information that was just communicated, then I am missing the boat.  I know that I always feel more comfortable with change when I have a chance to ask questions, get clarification, and discuss with others.
So, this is my new goal for the rest of the year…continue to communicate but also ensure there is ample opportunity for discussion.
Thanks Linda!

I just finished teaching an Agile Testing Class to an awesome testing team in Dallas.  We spent a good amount of time discussing how to communicate effectively with other team members. Specifically, we were talking about how to approach a developer when you need them to implement a solution in a different way that lends itself to easier testability.  I often hear these conversations start with words like, “Don’t put the code…..” or “I can’t complete my tasks if you…..” or “Every time you do that I can’t….” etc.  You get the point…they are very negative statements.  Without meaning to, you are putting up a wall and the person you are talking to is likely to go on the defensive immediately.  Not a great way to foster teamwork.
I have a 4-year-old nephew that, like all other boys his age, absolutely can NOT walk anywhere.  His feet only know how to move at a run.  I have spent several hours of my life saying (or shouting), “Don’t run!!!”  “Stop Running!”  I recently learned a great tip from his pre-school teacher.  Rather then constantly telling him what NOT to do, just remind him of what he can do.  So, I have learned to use the phrase , “Use your walking feet!”  Not only do I sound like a much nicer person when I say that in crowded public places, he responds to it so much better.  He is allowed and encouraged to use his walking feet.  He is just being reminded of good behavior.
I encourage team members to apply this same principle when you are communicating with one another.  Instead of telling a developer what NOT to do, offer up a suggestion of what would work great.  If you aren’t sure what will work, show them what you are trying to do and ask for their help in crafting a solution.  By communicating this way, you are fostering better collaboration and team work instead of putting each other on the defensive.
Oh, and use your walking feet when you go talk to them ☺

In a few classes I have taught recently, we have had some interesting conversations around working on distributed agile teams.  I always recommend trying to keep your Scrum team co-located whenever possible but in today’s world, distributed teams are unavoidable.  Here are some suggestions to help keep your distributed team members more involved:

1.    If the majority of your team is in one location and a few members are remote, get a large stuffed animal, blow up doll (not the adult variety please), etc to represent your distributed team members.  Bring the doll, stuffed animal, etc to all of your meetings and put them in a chair.  You can even write the names of the people it represents on a sticky note and stick it on the doll. The visual reminder of the people on the phone helps keep the folks in the room aware of the team members on the phone.

2.    Draw a table on a whiteboard where the meeting is initiated.  Write the names of all attendees on the phone around the virtual table.

3.    Use an on-line, real time collaboration tool that all team members can access during meetings. The Scrum Master can then put relevant data here for everyone to see.  If you have a distributed team, a physical card wall may not be a good choice.  Look into using tools like the GreenHopper plug-in for Jira or Rally.

4.    Get a web-cam and use it often.  I know a company that has a webcam running all the time in each office and has it is pointing to the task board.  Everyone can log in and see it whenever they need to.  The web cams are also used for daily stand-ups, etc.

5.    In meetings where the purpose is to gather information and form an action plan such as planning meetings, retrospectives, etc, try using the round robin style of gathering information.  Go around the virtual room and ask each person to contribute.  This way, everyone has a chance to be heard.

6.    Publish any decisions made somewhere where the entire team can see it.  Don’t bury information on an intranet or share point site that no one goes to. 

7.    Create some simple, fun games with prizes that encourage team members to visit the place (intranet, wiki, etc)  where you keep important information (like action items from retrospectives, etc).  For example, put trivia questions from Jeopardy on the site and give a fun award at the end of your stand-up to whoever responds first with the correct answer.  Or, you can track the results each day during the sprint and then give an award at your sprint demo.

8.    Devote a few minutes during each retrospective to discuss remote team issues and to brainstorm on how to make things better.  Follow-up with an action plan and re-visit it after it sprint.

9.    Pick a few times a year where everyone travels to the same location at once.  This gives everyone a chance to meet each other face to face.

10.   If you are not remote, then act like a remote employee for a day every now and then (i.e. work from home).  See first hand what the remote people experience so you can better understand the challenges they may be facing.

The point here is to get creative and go beyond simple conference calls.  Your Scrum Master should own the responsibility of managing effective communication and collaboration with distributed team members.  Make sure you are getting regular feedback from your team members on what is working well and what isn’t.  Collaboration on your distributed teams should be a top priority and you should be willing to make the necessary investments to ensure that all team members feel like they are heard and are part of the team.

This blog post by Simon Baker is excellent and does a great job describing the role of a tester on an agile team.  http://www.think-box.co.uk/blog/2008/05/testers-in-our-agile-team.html

The morning started off with a Keynote from Robert Sabourin called “What is so Different about Testing in Scrum.”  I wouldn’t say the title of the talk really fit the talk itself but it was a good kick-off for the day.  If you have seen him speak before, then you know he is high energy and enthusiastic. 

His talk essentially gave a brief overview of Scrum and then walked through some case studies of teams he has worked with that were adopting Scrum and what some of their challenges and proposed solutions were.  It wasn’t surprising to hear that the challenges were centered on either lack of product ownership and/or testing team challenges.  I was hoping he would actually dive into some of the key differences in testing in Scrum like the title of the talk mentioned but he really didn’t go there.  No one seems to go there…again, another topic for another time. 

I went to 4 different talks.  Two of these talks were topics around agile testing.  I was disappointed by both of these talks in that they really didn’t get into any meat on the topics.  It was all high level principles with no actual techniques or take-aways on what to do and how to do it. 

One talk was on metrics.  The speaker was good and had good content for the most part.  However, I would argue that several of the metrics that were discussed really don’t provide any value to an organization.  For example, she mentioned tracking requirements stability as a metric.  Let’s say you do this and you discover that 40% of your requirements change.  So what.  What is that metric going to do for your team or organization?  Requirements always change.  Period.  We all know that…we should all be prepared for it.  Spending a significant amount of time (and therefore money) tracking how many requirements change doesn’t really help you to handle those changing requirements.  Why not invest that time and money into something that will help your team work effectively in the changing requirement world?

I ended the day with a talk by Matt Heusser called “Software Testing from the Ground Up.”  The published excerpt on the talk didn’t seem to match the talk itself.  I did enjoy some of the comparisons he made between testers and other professions, though. 

I am currently in San Francisco at the STP Conference (www.stpcon.com).  I am really starting to clearly see what I consider a divide in the testing world between automated and manual testing that I believe is negatively affecting all development methodologies.  I am currently collecting my thoughts on this for an upcoming blog post….stay tuned J

Yesterday was the first day of the conference and was a full-day tutorial.  I spent the day at Michael Bolton’s “Rapid Intro to Rapid Software Development” class.  This is usually a 3-day class that was condensed into one day.  Because of that, the class was a lot of lecture and not as much hands-on as I was hoping for.  That said, I have condensed 2 day classes into 4 hour “overviews” so I know the challenges with that.  I think I would really enjoy the 3-day class much more.

Overall, I enjoyed the class and picked up some great tips/tricks/ideas particularly around exploratory testing.  I have attended a few talks by James and John Bach in the past and have read many of their articles, etc on exploratory testing.  This workshop really drove home how this technique works and the value it adds.

Some of my key take-aways from this class were:

1. As a tester, you should report what you see, not what you infer.  Basically, the idea here is that we should report just the facts and not infer what the problem is.  By doing this, we are able to keep our credibility with the rest of the team.  If you document an issue and infer what you think the problem is and that isn’t the actual problem, then the developers may start to think of you like the boy that cried wolf.  Just reporting the facts avoids this issue.
2. Manual tests should discover things – not prove things.  I love this – especially in the Scrum/Agile context.  Agile teams focus heavily on test automation (as they should).  However, manual testing is still important and necessary.  Manual testing allows you to “discover things.”  Test automation is where you prove things – you prove that the new code that was checked in didn’t break the existing code – you prove that the system is still functioning as it was before.  However, your automation doesn’t discover new things.  You discover behaviors, unfriendly user interfaces, patterns, etc by manual testing.  Both are important.
3. Asking the right question is more important then getting the right answer.  Food for thought.  This ties directly into the 3C’s of User Story development – card, conversation, confirmation.  During your conversation, you should be focused on asking the right questions.  These questions will hopefully illicit those implied assumptions, those “what-if” scenarios no one had thought of.  What you learn in asking those “right questions” can then be documented in the confirmation (the acceptance tests).
4. Any bug that slows down testing is a bad bug.  The example that brought this to light in the class was what most teams would likely call a usability issue.  The application we were testing had an entry field and a button to click to calculate the answer.  As a user, you could not enter the value and click the enter key on your keyboard to calculate the answer.  You had to either mouse or tab to the button on the UI first to click it.  This wasn’t necessarily a “bug.”  I am sure developers, testers, and product folks could argue for hours as to whether or not it was a “bug” or a usability issue.  It would like get ranked a low priority since the system essentially worked.  However, this functionality really slowed down testing.  If we could have just entered data and clicked enter, we could have tested more cases quickly.  Having to tab or mouse for every test slowed down my rate of testing.  Testability of the application you are working on should be considered.  This is why I like the concepts of ATDD and TDD.  If the tests are defined before development starts, then the developers can build the system in such a way that lends itself to easy testing.  As testers, it is absolutely our right to ask for testable applications.

More stpcon reviews to come as the conference continues……