Test Metrics

Introduction :

Metric refers to measurement. A test metric is a measurement criteria by which we can analyse the test process in quantifiable terms. Test metrics are essential in evaluating the quality of a software project in terms of its cost and effectiveness. Metrics are applicable in all fields - electronics, manufacturing etc. There are various techniques for calculating the output that helps us to validate the results as per the expectations. Software testing metrics offer ways to predict the effectiveness of the testing process.

Metric's Life Cycle :

  • Analysis - This process deals with identification of the metrics and define them.
  • Communicate - Convey the need of using such metrics to the stakeholders and testing team. Train the testing team about key points required to capture the processing of metric.
  • Evaluation - Capture the results and evaluate whether they meet the desired criteria.
  • Report - Prepare report which includes the results. This should include an effective conclusion. This report is distributed among the stakeholders and representatives, and necessary feedback is collected.

Types of Manual Test Metrics :

  • Base Metrics -These metrics are the ones derived from the data gathered by test analysts during test case development and execution phases. This data is tracked throughout the entire Test Life Cycle - number of test cases developed, number of test cases to be executed and number of test cases passed or failed.
  • Calculated Metrics -Data gathered from the base metrics forms the basis for calculated metrics. The test lead/ manager tracks the progress of the test metrics. There are few formulae for calculating metrics.
    • Percentage of test cases executed = (number of test cases executed/total number of test cases written) * 100
    • Percentage of test cases not executed = (number of test cases not executed/total number of test cases written) * 100
    • Defect density = (number of defects identified/size) [defect density determines the number of defects identified per requirement]
    • Defect removal efficiency = (number of defects found during QA testing)/ number of defects found during QA testing + number of defects found by end user * 100
    • Defect leakage = (number of defects found in User Acceptance Testing)/(number of defects found in QA testing) * 100 [defect leakage aims to find out the defects that may have been missed out during QA testing]
    • Defects by priority = This metric identifies the number of defects fetched based on its severity.

    There are various types of metrics which serve a purpose. Few of the metrics are :

    • Defect Category - This metric helps to understand the various quality aspects of a product. Quality aspects include usability, functionality, documentation, performance, installation, stability, compatibility etc. Defects are divided according to the category to which it belongs.
    • Defect cause distribution chart - This metric identifies the cause of a defect.
    • Defect finding rate - Used to track the pattern of defects captured during a certain period of time.
    • Defect Severity - Determines the extent of impact of a defect. Higher the impact, lower is the quality of the product.
    • Test Case Effectiveness - The efficiency of test cases to find out defects.
    • Test Coverage - Defines the extent to which the program is covered by the test cases. This metric assures the completeness of testing. It thus checks the flow of the code and the functionalities.
    • Time to fix a defect - Aims to determine the effectiveness of the amount of time and effort spent on resolving defects.
    • Traceability Matrix - It helps to determine features that has a source in requirements and each requirement has a corresponding feature.
    • Schedule Adherence - The metric aims to find the difference between planned and actual schedule followed.

Conclusion :

Defining parameters is a good way to analyse and map the progress of the project with the requirements. Different metrics have their own defined pattern of analysis, therefore wise selection of metrics is an important decision. Using metrics in software testing is an efficient way to track defect analysis and its detection.