The Software testing metrics provide quantitative approach to measure the quality and effectiveness of the software development and testing process. It provides the visibility into the readiness of the product and gives clear measurement of its quality and completeness. Moreover, with the assistance of testing metrics the team of testers can keep a track of the software quality at every stage in the software development cycle. One can also get information to control and reduce the number of errors through testing metrics. Some examples of test metrics are:
How many defects are existing within the module?
What is the Test Coverage %?
How many test cases were executed per person?
Importance of Metrics in Software Testing:
There are several ways of measuring the functionality and effectiveness of a software development and testing process. But, the importance of Test Metrics in Software Development Life Cycle (SDLC) is extremely high. Without metrics, there cannot be any improvement in the software system or application, which will further lead to no control of software developer or tester on that error or defect. Hence, following are few points that define the importance of metrics in software testing:
Without measurement, it is impossible to tell whether the process implemented in the software application is improving or not.
With the help of metrics one can easily take decisions about the activities that will be executed in the next phase of software development or testing, such as estimating the cost and schedule of the future projects.
Metrics helps in understanding the type of improvements required in a project and assist with decisions regarding process or technology changes.
One can track the improvements in the software.
Phases of Test Metrics
Known as Test Metrics Life Cycle, there are four phases of test metrics. During these phases different factors related to the software production are measured for the development process as well as reported to the stakeholders. These phases are:
Analysis Phase -In this phase the metrics, which have been generated are identified, which are then defined by the tester.
Communicate Phase- Here, the need of metrics is explained to the stakeholders. Moreover, the testing team is educated about the data points required for generating Metrics.
Evaluation Phase- The data used for generating the Metrics is captured and verified. It is based on these captured data that the metrics are calculated.
Reporting Phase- A metrics report is developed with all the required information, which includes graphs and pie charts and is distributed to stakeholder. After the evaluation of this report, a feedback is provided by stakeholders for any intended improvements.
Types of Manual Test Metrics :
Testing Metrics can help developers and testers in their endeavour to provide a product of exceptional quality. It allows them to understand the type of improvement required in the software and provides assistance in taking decisions regarding the process and technology changes. These changes and improvements are usually implemented through manual testing, which is further categorized into two types. The Manual Test Metrics that are used to measure the quality and effectiveness of the software are as follows:
Base Metrics (Direct Measure) -Base Metrics constitutes the raw data gathered by the Test Analyst during the test case development and execution. These metrics are used to provide project status report to the test lead and the project manager. Moreover, the base metrics provide the input data that is used to feed the formulas, which derive Calculated Metrics.
Examples:
Number of test cases.
Number of test cases executed.
Calculated Metrics (Indirect Measure)-Derived from the data gathered in Base Metrics, Calculated Metrics are generally tracked by the test lead for the test reporting purpose. Calculated Metrics convert the Base Metric data into more useful information, which is used to track the progress of the project at different levels, such as, Module level, Tester Level and more. The information provided here is extremely valuable, which when used and implemented often leads to significant changes and improvements in the overall Software Development Life Cycle (SDLC).
Percentage of test cases executed = (number of test cases executed/total number of test cases written) * 100
Percentage of test cases not executed = (number of test cases not executed/total number of test cases written) * 100
Defect density = (number of defects identified/size) [defect density determines the number of defects identified per requirement]
Defect removal efficiency = (number of defects found during QA testing)/ number of defects found during QA testing + number of defects found by end user * 100
Defect leakage = (number of defects found in User Acceptance Testing)/(number of defects found in QA testing) * 100 [defect leakage aims to find out the defects that may have been missed out during QA testing]
Defects by priority = This metric identifies the number of defects fetched based on its severity.
There are various types of metrics which serve a purpose. Few of the metrics are :
Defect Category - This metric helps to understand the various quality aspects of a product. Quality aspects include usability, functionality, documentation, performance, installation, stability, compatibility etc. Defects are divided according to the category to which it belongs.
Defect cause distribution chart - This metric identifies the cause of a defect.
Defect finding rate - Used to track the pattern of defects captured during a certain period of time.
Defect Severity - Determines the extent of impact of a defect. Higher the impact, lower is the quality of the product.
Test Case Effectiveness - The efficiency of test cases to find out defects.
Test Coverage - Defines the extent to which the program is covered by the test cases. This metric assures the completeness of testing. It thus checks the flow of the code and the functionalities.
Time to fix a defect - Aims to determine the effectiveness of the amount of time and effort spent on resolving defects.
Traceability Matrix - It helps to determine features that has a source in requirements and each requirement has a corresponding feature.
Schedule Adherence - The metric aims to find the difference between planned and actual schedule followed.
Test Metrics Infographics:
Conclusion :
Defining parameters is a good way to analyse and map the progress of the project with the requirements. Different metrics have their own defined pattern of analysis, therefore wise selection of metrics is an important decision. Using metrics in software testing is an efficient way to track defect analysis and its detection.
.png)
.png)
.png)
.png)
