Essential Software Testing Tools Blog

Overview of PDCA

1. Plan a small-scale project, e.g. learn and use a test automation tool.

2. Do. Execute the plan, e.g. assign a resource to learn the tool and automate a suite of regression tests.

3. Check. Review the project, analyze the results and determine what you have learned.

4. Act. Take action based on what you have learned in the project:

i.    If the project did not work, go through the steps again with a different plan, e.g. maybe use a different tool if the first one did not work out.

ii.    If the project was successful, incorporate what you learned from the project into wider changes, e.g. automate all of your regression tests, or use the tool on other projects.

iii.    Use what you have learned to plan new improvements, beginning the PDCA cycle again, e.g. make use of advanced automation techniques such as data driven testing.

PDCA Project Example

PLAN

Project: Develop a project plan to incorporate a test automation tool into your current testing process on a trial basis and automate a series of regression tests. Keep in mind that most tool vendors have a 30 to 60 day free trial period for their tools.

Process Definition: Define your current process. Incorporating a tool will fundamentally change your process, so benchmark your testing current process.

Data collection: To justify the investment in the tool you will need to determine the value it has added to your process, i.e. this is a cost-benefit analysis. Existing data to collect for the cost- benefit analysis will be:

i.    Number of manual regression tests.
ii.    Number of times regression tests were executed in previous releases.
iii.    Number of people required to execute the tests.
iv.    Time taken to execute the tests.

DO

Implement the plan. Collect data for the cost-benefit analysis and record observations during the project. The data to collect for the cost-benefit analysis will be:

i.    Number of regression tests automated.
ii.    Number of times regression tests were executed.
iii.    Number of people required to automate and execute tests.
iv.    Time to automate the tests.
v.    Time to learn how to use the tool.

CHECK

Review the project, analyze the results and identify what you’ve learned. You can now determine the value using tool has added to your process, with:

Value = Benefits – Costs

Where:

Benefits, represent the time and resources saved in executing automated tests instead of manual test.

Costs, represent the time and resources invested in automating the tests.

In addition to the Value added, you now have a basis for training others to use the tool.

ACT

Let’s assume the project was a success, i.e. the tool added value to your process, now you can act on what you have learned in this project, therefore;

i.    Update the current test process to incorporate automation.

ii.    Start collecting cost and benefit data during other projects that incorporate automation.

iii.    Develop a training regime for others to learn the tool.

At this point you are in a strong position to plan your next PDCA cycle for test automation. For example, you could automate the remaining regression tests for your next project, and follow that project by incorporating automation into other projects, or incorporating advanced automation techniques, etc.

Continuous Improvement

You now have a basis for continuous process improvement using test automation; however, as you continue to use automation to improve your test processes, and as test automation is adopted throughout your organization, the following overall pattern will emerge;

• There will be an increase in the people and resource investments required for things like training, user support and documentation.
• As such, there will need to be greater understanding, attention and commitment from management towards automation.
• Consequently, you will need to have further controls, measurements, and reporting in your processes to provide the feedback that will tell you if your improvements are adding value.

Why Implement Test Automation Tools?

If you currently perform manual software regression testing for each product release, have a large number of tests that need to be executed for each release, and you have frequent product releases then this is an ideal situation to make use of a test automation tool.

Manual software regression testing is an inefficient use of a test analyst’s time, especially if they are merely repeating documented tests.  You can reduce your test cycle times and improve the accuracy of your testing by letting a software test automation tool execute these tests instead.

How to Start – The “Ad Hoc” Approach

So let’s assume you purchase a software automation tool for the test analysts to use but you find things do not go as you expected:

• Some analysts do not want to use the tool to automate their tests preferring to keep on testing manually.
• Some analysts give up using the tool, but some users are using the tool successfully.
• After the several releases the application has changed and a significant number of automated test scripts need to be updated.
• The tool was expensive to purchase and so much time and resources are being used to maintain the automated test scripts it seems the tool is of marginal benefit.

This is a small sample of the problems that are likely to occur when you attempt to adopt a software test automation tool in to your organization in an ad hoc manner.

A Better Approach – “Plan – Do – Check – Act”

When you adopt any tool into your organization you intend to improve your current process but remember you are also changing the current process and this can be problematic.

Quality assurance and quality control people have developed a system that can make changes like these less problematic. It is an ideal way to adopt test automation into your organization.

This system consists of four steps, plan-do-check-act (PDCA).

Have you ever spent any time defining and documenting the requirements for your software testing process?

No? I Didn’t think so.

I rarely see software test teams that have stepped back and worked out where it is they want to get to. We’re all  so busy working to the next software release deadline that we forget to invest any time looking to the future.

We’re all guilty of looking to software solutions in the hope that they’ll free up time once we’ve implemented  them successfully.

It NEVER works this way!

I can’t give you quick solutions to finding the time you need. I will stress that you’ll waste even more time if you pick the first quick fix for your software testing process. But you knew that anyway.

So why not find the time to implement the right software testing system, the right way, the first time round?

Consider thought that if you don’t know what you want then you won’t find what you need.

You see it’s impossible to select the right software if you don’t know what you are trying to achieve.

Lewis Carroll said – ‘if you don’t know where you are going, any road will take you there’.

Which is why selecting the first good looking piece of software to help your software testing process will  usually take you somewhere you weren’t quite expecting.

In the long run it will save you time and money if you get your requirements right at the start. Then work  towards implementing your selected tools to improve your test process. If you do it the other way round you can  guarantee you’ll end up with at least some rework. Or you’ll start from scratch again.

The clearer your requirements, the less time you’ll spend on re-work finding the right solution.

I think it’s fair to say that we spend a lot of time slinging mud at our development teams and customers over poor requirement specifications. Then, when we come to implement software solutions to help our software testing process, we make exactly the same mistakes.

In any project, effective requirements capture separates the successful projects from the failures.

And that goes for us in the software testing arena too!