Essential Software Testing Tools Blog

Software Test coverage is one of the most important indicators of your software test progress. There are many different ways to track software test coverage; bugs, requirements, components, code, configuration, change, data, etc. Using just one method will never present you with an accurate picture. Using multiple methods to track software test coverage will always give you a clearer picture.

The most common methods for tracking test coverage include:

Bug count: this helps demonstrate that a product is not ready for release. However, an absence of recorded bugs won’t show that the product is ready for release.

Requirements coverage: testing each requirement will help prove that the product meets the customer’s needs. Where requirements are subjective it can be difficult to gauge how well a requirement is covered.

Component coverage: tracking tests against individual components of a product can help show components that haven’t been covered. Again the problem here is that it is difficult to accurately gauge how well a component has tested.

Code coverage: the lines-of-code covered measure is useful for showing how much of the application you have exercised. However, this method won’t tell you if functionality is missing or if the features implemented meet the customer’s requirements.

Test case coverage: the number of test cases covered as a percentage of the test cases you planned to run is one of the simplest test coverage measures. This measure can be very misleading though as it depends largely on the scope of coverage of each test case.

Taken together a range of different software coverage figures can be a very powerful indication of test coverage. For example you may know that you have completed test cases covering all of the products components. You may also know that your code coverage is 10% higher than you achieved for a previous release. In addition, bug counts for this version may list only a few minor bugs. These values taken together would be a far stronger indication of good software test coverage than just one value which says we’ve completed 70% of our software test cases.

How often do you feel overwhelmed with the amount of software testing you need to complete for your next product release? Twice a week? Four times a week? Every day? It is a frustrating fact of testing that there is never enough time to complete all the testing. Rather than finding a way to deal with this, most software test teams simply hope that they’ve done enough.

If that is what you’re doing why not consider prioritising your software testing? When you prioritise your software testing you:

• Find the serious bugs sooner
• Deliver the results needed by the team faster
• Don’t waste time on irrelevant tests

It would be great if we had more testers and realistic deadlines but in the real world we often get little influence over those variables. You have the resources you have, and you need to make the most of them.

We can’t help you decide on the precise priorities for each of your tests, as you need knowledge of the product to do that. However, the following 6 points will help guide you :

1. Review defect trends from previous product releases
2. Identify areas where bugs have been fixed or new functionality added
3. Balance test priorities across early and later builds for the product
4. Make sure you have well defined meanings for each priority you use
5. Ensure you have consistent priorities across all your tests
6. Review test results from previous test runs

The solution is not just to hope that you’ve done enough software testing. The solution is to find a system that enables you to prioritise software tests and focus on what matters most.