TISQA 2007 Conference Review

I spent the day yesterday at the TISQA conference in Chapel Hill, NC.  It is a one day local conference that the TISQA group puts on every year. 

The opening key note was from Jon Bach.  As usual, it was a great talk.  I love listening to him talk about exploratory testing.  I always learn something new.  The afternoon keynote was a talk by Rex Black called “Testing in an Outsourced World.”  As usual, Rex was an awesome presenter.  While I disagree with some of his views on managing outsourced testing, I got some great ideas from him around how to manage cultural differences in that situation. 

I only got to listen to one other presenter (TR Buskirk) since I was presenting in the afternoon.  It was a fun talk that highlighted some of the challenges that QA groups face in all companies.

This was the first testing only conference I have been to in several years.  It was an eye-opener for me.  The conference was run extremely well as usual and the TISQA group did a great job scheduling, etc.  The main thing that stuck out for me was the atmosphere difference from this conference to others I have attended in the last few years.  My personal opinion is that the atmosphere was different because it was a group of testers, not a mix of developers, testers, product folks, etc.  It definitely didn’t feel like an “agile-friendly” environment.  That said, it was still a good experience and I hope that at the end of the day, at least one person there may think about approaching testing differently.


Taking Ownership of Quality with Outsourcing

I have recently had the opportunity to speak to some QA folks at different companies.  During our conversations, the topic of quality and QA practices came up with regards to outsourcing.  I have never personally worked on a project that involved development from outsourced companies so I have never had to deal with this first hand.  However, it is a topic that I found very interesting and it got me thinking about it.  So, I started doing some research and investigating on this issue.

Most of what I am discovering is that QA seems to function one of 3 ways with outsourced development.  For these examples, I will refer to the outsourced company as the Child and the company that is paying them and ultimately owns the code as the Parent.  Here are the re-occurring scenarios I have come across:

1.  Child is responsible for QA.  This includes creating the test scripts (manual mostly, may have a few GUI automated scripts), running them, and ensuring that the code is working.  Child gets to use whatever QA practices they currently operate under.  They deliver code to Parent and Parent trusts that Child did their job well enough and distributes the code.

2.  Parent develops the test cases the Child needs to run (manual scripts).  Child develops code and then executes test cases from Parent (may use manual or GUI automated tests).  Child reports on the statistics of the test cases to Parent.  Parent then distributes code.

3.  Child builds and tests code.  Child reports on QA metrics to Parent.  Parent then has their own QA staff re-run all test cases on delivered code to ensure that the code is functioning correctly.  Parent also does exploratory testing if time permits.

I personally am not comfortable with any of these options and would not sleep well at night if I was on a project in any of these scenarios.  Why?  Because in all of these scenarios, no one is really owning quality. 

Scenario 1 is flat out irresponsible.  Parent owns the outsourced code and owns the quality of it and is doing nothing to ensure that quality is being met.  Parent has no visibility into what Child is doing and Child is not educating Parent. 

In Scenario 2, Parent probably feels like they are owning quality.  By creating the test plans, they feel like they are controlling the quality.  They get a report that lists out the stats of the passing test cases and they feel good.  I argue that Parent is not really owning the quality in this option either.  As we all know, a lot can be lost in interpretation.  I can write out a manual test case that 2 different people can run differently based on their interpretation of it.  One may pass and the other may fail.  This has been proven time and time again with detailed requirement documents where what the analyst wrote is not what the developer coded due to interpretation differences.  This scenario also leaves a huge gap with negative testing and exploratory testing. 

Scenario 3 is the most interesting.  Awhile ago, I likely would have thought this was the best way to go – not so much any more.  I guarantee that Parent really feels like they own the quality here.  They are providing the test cases to Child and then making sure Child didn’t mis-interpret them (or the Child didn’t run them)  by running them again before delivering to production.  If we keep our view of quality to just mean that the code is doing what we asked for it to do, then yes, Parent is taking ownership of the quality. 

But what about quality of the entire SDLC in this scenario??  Parent is spending money for Child to execute tests and then spending even more money to have the Parent QA team re-run those same tests.  That is a very low-quality cost solution.   By re-running manual test cases multiple times by multiple teams, this is also a very low-quality time solution.

QA Professionals should expand their view of quality to include more then just the code in the application they are testing.  Quality needs to include everything that happens to get that application into your customer’s hands.  It should obviously include how the application performs (both functionality and performance) but it should also include the process around that delivery.  We should be looking for ways to cut cost and time such that we help deliver a high performing application in a low cost, speedy timeframe.  This is hard for QA folks – we are control freaks 🙂  We like to dot every I and cross every T.  We want to test, re-test, and re-test again and again to ensure that we didn’t miss anything.  When our co-worker says they tested that scenario and it is working, we want to test it for ourselves to ensure that it really is working J  However, if doing so adds a large cost to the delivery of the software (either time or staff related), it is our job and we should feel obligated to look for ways to reduce that time and cost without hurting the functional quality of the application.

So, with all that said, here is the scenario I would want to be in on an outsourced project.  First and foremost, I don’t see any reason why both Parent and Child shouldn’t work together to ensure joint ownership of quality.  I don’t ever want to be in a situation where Parent throws requirements to Child and then waits 1, 3, 6 months for Child to build them.  Instead, why not have Parent and Child work together?  In an ideal situation, I would want to have Child work in an iterative fashion where they can deliver code AND automated tests to Parent every few weeks.  It may not be easy to find a company that is willing to do this but I think it would be worth the investment to train them how.  That way, Parent not only gets working code every few weeks but also gets automated tests that they can set up and run at will.  This also provides the opportunity for regular feedback between the teams.  You can even take this one step further and have Parent write the requirements as executable requirements and deliver those to Child.  It is then Child’s job to deliver code to make those executable requirements pass.  In this scenario, Parent is certainly driving quality but Child definitely has quality responsibility as well.  Both companies are contributing to the ownership of quality (from and application and process perspective). 

Yes, I know…this is agile developmentJ  This why I love agile practices and principles.  They really focus on quality from every perspective and push it all the way up-stream to the very start of your project.  It is a quality dream come true! 

There are some out-sourcing companies out there today that follow good agile development practices and I hope this number continues to grow.  I also hope that if I am ever on a project that outsources pieces of development, that it will be to a company that is willing to work this way.  As a QA professional, I certainly feel that it is my responsibility to push for this and do everything I can to sell this to the folks that make the ultimate decision.   

GreenHopper – great tool!

I haven’t played with a new tool in a long time that I have liked this much out of the gate.  I have been setting up and customizing Jira for an upcoming project.  I downloaded the trial version of GreenHopper last week after seeing it briefly at Agile 2007.  So far, I have been really impressed!  It is probably the best tool I have seen yet for handling a virtual card wall and virtual release planning.  I still think that nothing beats a real card wall for tracking your sprints. However, if you have distributed team members, then a physical card wall isn’t helpful for the whole team.

My favorite features so far include the chart board that automatically calculates your burndown real time (and has several lines you can choose to view), the click and drag feature for the task board (card wall) and the ability to add custom fields in Jira and have them available for calculations in GreenHopper.

I am also impressed with the customer service from these guys.  I have had several questions and they went as far as to set up a call with the developer that was able to walk me through my list.  He also took the time to point out other items that may be helpful to me as well.

 I am looking forward to finishing the set-up of this and using this “real time” on my project!