Taking Ownership of Quality with Outsourcing

I have recently had the opportunity to speak to some QA folks at different companies.  During our conversations, the topic of quality and QA practices came up with regards to outsourcing.  I have never personally worked on a project that involved development from outsourced companies so I have never had to deal with this first hand.  However, it is a topic that I found very interesting and it got me thinking about it.  So, I started doing some research and investigating on this issue.

Most of what I am discovering is that QA seems to function one of 3 ways with outsourced development.  For these examples, I will refer to the outsourced company as the Child and the company that is paying them and ultimately owns the code as the Parent.  Here are the re-occurring scenarios I have come across:

1.  Child is responsible for QA.  This includes creating the test scripts (manual mostly, may have a few GUI automated scripts), running them, and ensuring that the code is working.  Child gets to use whatever QA practices they currently operate under.  They deliver code to Parent and Parent trusts that Child did their job well enough and distributes the code.

2.  Parent develops the test cases the Child needs to run (manual scripts).  Child develops code and then executes test cases from Parent (may use manual or GUI automated tests).  Child reports on the statistics of the test cases to Parent.  Parent then distributes code.

3.  Child builds and tests code.  Child reports on QA metrics to Parent.  Parent then has their own QA staff re-run all test cases on delivered code to ensure that the code is functioning correctly.  Parent also does exploratory testing if time permits.

I personally am not comfortable with any of these options and would not sleep well at night if I was on a project in any of these scenarios.  Why?  Because in all of these scenarios, no one is really owning quality. 

Scenario 1 is flat out irresponsible.  Parent owns the outsourced code and owns the quality of it and is doing nothing to ensure that quality is being met.  Parent has no visibility into what Child is doing and Child is not educating Parent. 

In Scenario 2, Parent probably feels like they are owning quality.  By creating the test plans, they feel like they are controlling the quality.  They get a report that lists out the stats of the passing test cases and they feel good.  I argue that Parent is not really owning the quality in this option either.  As we all know, a lot can be lost in interpretation.  I can write out a manual test case that 2 different people can run differently based on their interpretation of it.  One may pass and the other may fail.  This has been proven time and time again with detailed requirement documents where what the analyst wrote is not what the developer coded due to interpretation differences.  This scenario also leaves a huge gap with negative testing and exploratory testing. 

Scenario 3 is the most interesting.  Awhile ago, I likely would have thought this was the best way to go – not so much any more.  I guarantee that Parent really feels like they own the quality here.  They are providing the test cases to Child and then making sure Child didn’t mis-interpret them (or the Child didn’t run them)  by running them again before delivering to production.  If we keep our view of quality to just mean that the code is doing what we asked for it to do, then yes, Parent is taking ownership of the quality. 

But what about quality of the entire SDLC in this scenario??  Parent is spending money for Child to execute tests and then spending even more money to have the Parent QA team re-run those same tests.  That is a very low-quality cost solution.   By re-running manual test cases multiple times by multiple teams, this is also a very low-quality time solution.

QA Professionals should expand their view of quality to include more then just the code in the application they are testing.  Quality needs to include everything that happens to get that application into your customer’s hands.  It should obviously include how the application performs (both functionality and performance) but it should also include the process around that delivery.  We should be looking for ways to cut cost and time such that we help deliver a high performing application in a low cost, speedy timeframe.  This is hard for QA folks – we are control freaks 🙂  We like to dot every I and cross every T.  We want to test, re-test, and re-test again and again to ensure that we didn’t miss anything.  When our co-worker says they tested that scenario and it is working, we want to test it for ourselves to ensure that it really is working J  However, if doing so adds a large cost to the delivery of the software (either time or staff related), it is our job and we should feel obligated to look for ways to reduce that time and cost without hurting the functional quality of the application.

So, with all that said, here is the scenario I would want to be in on an outsourced project.  First and foremost, I don’t see any reason why both Parent and Child shouldn’t work together to ensure joint ownership of quality.  I don’t ever want to be in a situation where Parent throws requirements to Child and then waits 1, 3, 6 months for Child to build them.  Instead, why not have Parent and Child work together?  In an ideal situation, I would want to have Child work in an iterative fashion where they can deliver code AND automated tests to Parent every few weeks.  It may not be easy to find a company that is willing to do this but I think it would be worth the investment to train them how.  That way, Parent not only gets working code every few weeks but also gets automated tests that they can set up and run at will.  This also provides the opportunity for regular feedback between the teams.  You can even take this one step further and have Parent write the requirements as executable requirements and deliver those to Child.  It is then Child’s job to deliver code to make those executable requirements pass.  In this scenario, Parent is certainly driving quality but Child definitely has quality responsibility as well.  Both companies are contributing to the ownership of quality (from and application and process perspective). 

Yes, I know…this is agile developmentJ  This why I love agile practices and principles.  They really focus on quality from every perspective and push it all the way up-stream to the very start of your project.  It is a quality dream come true! 

There are some out-sourcing companies out there today that follow good agile development practices and I hope this number continues to grow.  I also hope that if I am ever on a project that outsources pieces of development, that it will be to a company that is willing to work this way.  As a QA professional, I certainly feel that it is my responsibility to push for this and do everything I can to sell this to the folks that make the ultimate decision.   


One Response

  1. very interesting. i’m adding in RSS Reader

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: