Thank you for bring this up. I have been having the question about how to bring manual and automation together for sometime.

In the structure where we have people do both manual and automation testing, automation efforts failed because automation was on when testers have free time. We always have some unscheduled tasks for manual testing. The end results are: 1) Automation scripts are useless when we really need them because lacking of time to maintain. 2) The number of automation scripts increase very slow. 3) Automation scripts have maintenance problems because they are created by multiple people without any architect approach. I understand that all those problems fit in the most common reasons for failed automation efforts.

On the other side of the story, I have been leading an automation team for an automation project. Most people in my team know very little about the product. We were following manual steps. Results that I observed: 1) The Automation scripts increase much faster. 2) Automation scripts are used in the regression testing cycles. 3) Manual test steps are unclear or not align with automation efforts which reduces the speed of automation.The speed of automation can go up if we do not follow the exact manual steps.

I guess we are getting back of old questions. Should we use testers who have less programming experience in general do both manual and automation testing? Should we divide the teams into manual and automation?

I have done both manual and automation testing. I love them both and agree that there should not be any differences between good manual testers and good automation testers.

Observed from both side, I think there should be at least one person in the qa team understand both from technical side. This person should have experiences in both manual and automation testing to try to find a way to bring manual and automation together to increase productivity in a certain environment especially in a relatively small agile company.

Good testers as you mentioned are difficult to find.

I will continue my experiments to try to bring manual and automation more closely. But each environment is different. It is difficult to find one solution that fits for all environments. I believe the most important thing is to find a solution for your environment.

Lynn

If you look at testing in terms of the full spectrum of tasks involved in imagining, designing, implementing, debugging, documenting, executing, interpreting and results-reporting a test, any of these tasks can be automated. GUI regression testing automates the execution and a simple level of results-evaluation. It is also typically subject to a significant maintenance cost when the software changes. For those of us who want to support change, rather than change-control it, a high-maintenance test suite adds to the project’s inertia rather than its velocity.

The other problem with GUI regression testing is that it automates traditional scripted tests. Rerunning the same test over and over and over and over and over gives very little useful information. You CAN design test automation schemes that introduce much more variation, rather than snoozing over the same old scripts, but that kind of work is beyond the technical skills of many people who learned how to use the simple scripting tools.

I think the divide is not between high technical skill and low but between (a) valuation of scripted regression tests at the GUI (not unit) level, (b) willingness to consider technical approaches for tasks like computer-assisted test design, computer-assisted test implementation, computer-assisted result evaluation, computer-assisted pattern analysis of results, rather than keeping our heads locked in the vise of computer-assisted test execution.

You won’t see me talking about how to reconcile GUI test automation with exploratory testing because I believe that most regression tests worth automating at the GUI level are better automated at the unit or glass-box-integration (e.g. FIT) level and most of the interesting questions that we can address with brains or technology are not well targeted by the GUI regression tools. That’s not because of a fear of technology. It’s because of a fear of wasting my time on low-value technology.

– Cem Kaner