Delta Testing

April 28, 2020

The term delta is used in the context of change. Delta testing is a technique of partially releasing a product's modified parts only or until the complete update of a module or unit takes place.Delta testing does not require maintaining the record of expected results, rather the process of delta testing is implemented by elaborating a test plan first which further leads to design of various test suites. After the test suites are ready, they are executed quite often to ensure correctness and stability of the system.

The test case is primarily composed of an input data and its corresponding output value. If the input produce the desired output, then the test is considered successful. But in scenarios where the requirements keep changing, it becomes essential that the business changes do reflect in the test suites as well. Therefore, an organisation needs to find a way to partially automate the testing of the incremental changes. Changes are made as per some business rules.

Delta testing or incremental testing approach believes in following one strategy that is, what changes one shall expect as compared to the previous one, in the expected output. This approach allows quick definition and execution of test cases that belong to very large data sets.

delta testing

Delta Testing Framework :

A typical delta testing framework includes two types of rule sets -champion and challenger. The input data has to pass through each of the rule sets, then the results are compared and evaluated to verify if they deliver the same result, as per expectations.

Challenger Rule set are the re-factored rule set
Champion Rule set original rule set

Delta testing falls under following categories : Quantitative and Qualitative Delta testing

Quantitative Delta Testing :

In this technique, the output is measured with the expected outcome. For instance, there may be a situation where the business rule may require a rate of a product to change frequently. Hence, to test that the change is reflecting correctly, we need an equivalence function that compares the difference between the base rate of one rule set with the other rule set and confirm that the difference in base rates reflects the changes in business rules.

Qualitative Delta Testing :

Qualitative testing is generally used to assess the following aspects :

Correctness of rules :After we re-factor our rule set according to the requirements, by restructuring the rules, redesigning the rule flow with an aim to accommodate the application needs for extensions or performances. Post re-factoring, we must test all possible test input combinations, to verify if we get the same output from both champion and challenger rule sets. If all the test cases produce the same set of results, then the re-factoring process is considered successful.
Stability of system and extent of change : After a certain business requirement change has been implemented in the rule set, one may derive different set of test cases which can shows some impact from the changes and those that do not show any impact from the change. The latter helps us to ascertain the stability of the system the changes and the former lets us verify whether the intended target system solution is met.

Conclusion:

Delta testing offers an effective way of releasing parts of a system as and when they are ready. This approach is helpful in assessing the system from every possible dimension as we resort to evaluation techniques to verify the conformance of outcome with the desired/expected outcome, as per the business rules.

It offers a great deal of flexibility in terms of aligning the testing process with the business rules that are bound to change from time to time.