Software Testing metrics are essential to keep the track of all the testing activities and measure them. In other clear words, you need to measure the test activities not only to know where you stand but also to calculate the test team’s productivity, progress, and the quality of the product you just tested. You need them to have a better understanding of what and how you tested with so much effort. Several organizations use different factors to measure and that’s mostly is influenced by what they want to enhance, control, and track in the project.
A metric typically communicates a prediction or results basing a fusion of data.
Result Metrics: Metrics that are of an accurate measure of a completed process or an activity.
For instance: The amount of time taken to execute a bunch of Test Cases in a suite.
Predictive Metrics: These metrics can be considered as an early red flag of an unfriendly result.
For instance: When you compare the Defects and The Defects Resolved chart it gives a picture of the rate of your
defect fixing pace. Now this drives your team’s attention towards the things that need to move faster, like at the desired rate.
Why Do You Need Software Testing Metrics?
The aim of compiling these metrics is to utilize them to enhance the testing process, unlike the fancy reports. It also involves finding the real answers to certain test questions. For instance:
- How long will it take to finish the testing?
- The budget required for Testing?
- How critical are the bugs?
- The number of bugs that are found and fixed, closed, deferred, and reopened?
- The number of bugs that the team couldn’t manage to find?
- Percentage of the software testing?
- Can the team complete the Testing on time?
- Will the product go live on time?
- How good are the tests? Is the team operating on low-value test cases?
- What is the total cost of Testing?
- Is the test effort appropriate or up to the mark? Is it possible to push in more testing in the same release?
And so to perfectly answer these big questions you need some good measurements and tactics. Well, this particular blog addresses around 40 metrics of absolute, derivative, and predictive metrics that are crazily in practice now (mostly by testers and QA Managers)
The Fundamental Metrics
This can be a good starting point on the big road to software testing metrics. Fundamental QA Metrics are this fusion of absolute numbers which again can be utilized to derive derivative metrics. Let’s see what they are.
Absolute Numbers
- Total No. of Test Cases
- The total number of Test Cases passed
- The total number of Test Cases that failed
- The total number of Test Cases blocked
- The total number of bugs/defects you found
- The total number of bugs accepted
- The total number of bugs rejected
- The total number of deferred
- The total number of really critical bugs
- The total number of planned test hours
- The total number of test hours taken
- The total number of bugs found post shipping
Derivative Metrics
Now, the absolute numbers are a fine start but they aren’t just enough to conclude.
For instance, just providing the data of the above absolute numbers will only create more questions than answers. You need more to support your testing and to take the credibility of the amount of work done each day. That’s why we have the derivative metrics. They help you go deeper into answers that can perhaps encourage you and your team to solve several problems in the testing process.
Test Tracking and Efficiency
The below ones are the derived metrics that help you track your testing and the efficiency of your testing and team.
13. 
14. 
15.
16. 
17. 
18. 
19. 
20. 
21. 
Test Effort
The typical test effort metrics will provide answers to the dramatic questions like how long or how many or how much or all of them together of testing. Why are they great and why should you go for them? Because they tremendously help in becoming a foundation for further testing and planning. Note: All this being said these metrics are just average. Half of them may fall over average and the other half under average.
Here are some specific measures for you:
22. 
23. 
24. 
25. 
26. 
27. 
Test Effectiveness
The Test effectiveness metrics typically answers the questions like how great are the tests? Is
the team using the high-value test cases? With this, you can find out or measure the bug-finding capacity and quality of your tests. The test effectiveness metrics are to particularly obtain the percentage value of the difference between the number of bugs found (via test team) and the total number of bugs found on the entire software.
28. Metrics Based: Test Effectiveness Using Defect Containment efficiency
The higher the test effectiveness percentage you obtain the better your test is and the lower the whole test case management will be in the future.
For instance: If the test effectiveness of a certain project is 75% it means that the rest 25% of the bugs are still away from the hands of the test team.
- To improve the defect identification rate you need to send these numbers for investigation, retrospection and then accordingly will follow the corrective actions.
- Now, this metric of test effectiveness will never be 100%. So, don’t accept the same in the first rather than feeling disappointed in the end. However, your goal should be to gain a maximum number to feel encouraging.
- An average effectiveness rate will give you an idea of the efforts on test improvement are going positive or not which is paramount.
29. Context-Based
Deriving Test Effectiveness via Team Assessment:
At times when defect containment efficiency metrics don’t work you need to look into another approach to evaluate the test effectiveness. Which is possible by context-based or opinion-based.
Now the defect containment efficacy can’t work in these cases:
- Due to the maturity of the product
- The product is too buggy
- When testing isn’t done properly due to limited time/resources
Simply you can ask your team to give their feedback on the test so that you would know how good it is in general. And some instructions to communicate before your team proceeds to the feedback. Tell them to be neutral and unbiased and ask to opinionate what a good test means. Ask them to be realistic about the standards of testing and probably focus more on the difficult parts or high-risk requirements of your product.
You can also try using the subjective scaling method for sure. On a scale of 1 to 10, you can ask your team members to rate the test set in terms of how complete, up to date, and how effective the test is right now. When you get the average score you get the average test effectiveness.
This way you could also define a more narrowed test focus via the help of your team and a common goal set. Again doing so the team has to be unbiased about what a good test means.
30. Test Coverage
Software Testing metrics are all to support the health of the product that you are testing and to ensure that you need full test coverage and metrics to measure it. The test coverage metrics help you find answers to questions like how much of the application is being tested so far and more.
More metrics are coming your way soon! And further, if you like the type of content you are reading, be sure to subscribe to our QA Touch blog posts for more interesting content. We create and send so much Testing joy to your inbox without making a noise. Also, give us a thumbs-up on social media, where we do all the fun and exciting content on Testing and Tech.
