Test Prioritization

Test case prioritisation is simply a method of prioritising the test cases that could solve bigger problems and save the trouble of facing issues later in the application’s lifecycle. It is a way of segmenting the different aspects of an application that require a close vigil.

The root cause of occurrence of a certain defect is due to the fact ‘exhaustive testing is not possible’, by which we mean that while testing an application it could be possible that certain areas of an application remained left out. Therefore it is advisable to prepare a strategy in advance so that a list is prepared to categorise the parts of an application. The different segments are planned as per their degree of importance in the context of an application’s performance, test cases are created by testers accordingly.

The team of testers and developers invest a considerable amount of time to decide whether the test case prioritisation activity is in line with the business objectives in terms of resources – manpower, computer systems, tools, hardware and various other configurations or installations must adhere to time and cost constraints.

The core objective of test case prioritisation is thus to schedule execution of test cases on the basis of a test scenario’s priority.

Now, let us throw some light on the types of test cases that are categorised into low, medium and high priority.

Test Cases:

  • Test Case 1: Payment gateway approves the transaction of the customer – high priority
  • Test case 2: User credentials are validated on new registration – high to medium priority
  • Test case 3: User interface notifies the user about any new updates about a product – low priority

The test cases are just a snapshot of the typical manner in which functionalities are grouped in order to prioritise them.

Categories of Test Cases:

  1. Blockers: Such a category includes those feature tests without which the software will become dysfunctional as a whole. Therefore as and when a defect catches tester’s attention, the same shall be dealt with on a priority basis.
  2. Critical: This includes all those test cases that are important from a customer’s perspective. The functionalities are then verified against the desired performance.
  3. Major: The features that add some uniqueness to our product among other competitors are categorised under this section. These feature functionalities must be dealt with a careful consideration to maintain a certain reputation in the market.
  4. Minor: This category comprises of suggestions for improving the product or applying UI modifications. Such features has no impact on the core functionality of the software.

Test Prioritization Techniques:

Prioritising a test case could be dependent on a variety of factors such as cost, feature specifications or simply by observing the pattern of occurrence of defects in past projects. Whichever factor leads to prioritising tests, the ultimate goal is to minimise the chances of failure of the application under test.

Some of the common types of test prioritisation techniques are:

  1. Customer Requirement Based Technique: Requirement specification document forms the central object based on which test cases are designed. According to this technique user’s perspective towards the application matters, hence the SRS becomes the core of decision.
  2. Coverage based technique: Such test cases are based on the ‘coverage’ aspect. Coverage criteria is defined in terms of functionalities that should to be covered by test cases.
  3. Cost Effective Based Technique: Test cases are prioritised on the basis of cost considerations by analysing impact of cost on the project.
  4. Chronographic history based technique: Such test cases are prepared on the basis of test execution history.

Prioritisation Based on ‘Risk’:

Among the different reasons, risk is one factor one factor based on which risks are prioritised. Among a set of test cases priority is assigned to each based on some factors that govern the pattern of risk occurrence.

  • Determine Risk Exposure For Each Test Case: Risk exposure states the degree of any functionality or a feature being exposed to the vulnerability of certain threats. This step is thus to determine the extent to which a test case could possibly be affected by a risk.
  • Calculate Risk Reduction Leverage (RRL): ‘Risk reduction leverage’ is derived by calculating the difference between ‘risk exposure before testing’ and ‘risk exposure after testing’ and dividing it by the total efforts. ‘Effort’ is the amount of time invested in performing test and associated activities.
  • Prioritise test cases based on RRL: After a figure is arrived at by calculating RRL, risk prone test cases could be prioritised based on the scores – highest score means high priority, low score is lower and so on.

Points to be considered while prioritising test cases:

  1. Identify the components that are crucial to the objectives of the project with respect to requirement specification.
  2. Enlist the features that are complex in nature. Information regarding such facts can be sought from developers who would be in a position to provide a detailed insight into the various intricacies.
  3. Test cases for features, that could possibly have the most frequent change requests, should also be in the priority list.
  4. Identify the areas that have consistently seen a series of issues, form an important part of test case prioritisation.
  5. Make a note of probability of adding some new functionality which seems as a preferable consideration on the part of the end user.