A Systematic Literature Review on Test Case Prioritization Techniques

1. Introduction

Regression activity ensures the proper functioning of the working software under trial during the development phase (Li et al., 2007). Regression testing is usually employed when any alteration takes place in the software. These alterations are frequent due to customer’s requirement, availability of the new and better algorithms for computing task as well as owning to the revamping in design of the software for reducing the complexity of existing software. Regression testing is a critical and expensive task (Li et al., 2007). Regression testing tends to be an integral part of the software system, and hence plays a pivotal role in enhancing its functionality (Do, 2016; Elbaum et al., 2014; Hao, Zhang, & Mei, 2016; Liang et al., 2018; Ralph, 2016). Perf Ranker is claimed to be the first approach in performance regression testing for collection intensive software (Elbaum et al., 2014). Software testing is as an expensive phase owning to being a repetitive task (Hao, Zhang, & Mei, 2016), therefore, consistent efforts need to be made by software developers to eliminate the testing phase owning to financial and time constraints.

Test case prioritization techniques are applied in continuous integration (CI) environments (Liang et al., 2018). The CI server monitors make alteration in the code repository (CR) and ensure whether changes have occurred and executed the software build and the test cases with respect to the change taken place in CR. Software tends to evolve with time and hence the 33% of total cumulative cost needed for software development is invested over regression testing (Chittimalli & Harrold, 2009; Mostafa et al., 2017). Test case selection, minimization, and prioritization are accountable for regression testing of a software (Jeffrey & Gupta, 2007; Yoo & Harman, 2007).

Test suit minimization is used to eliminate the test cases from the test suit to minimize the number of those test cases, which are required to run the software (Jeffrey & Gupta, 2007). The minimization is sometimes called test suit reduction. The elimination of test cases is said to be permanent in test case reduction process.

Test case selection approach is used to execute those test cases, which are aware of modification and needs to be executed. Test case selection approach mitigates the number of test cases by selecting the test cases that have relation with the modified test cases (Li et al., 2007).

The evidence-based software engineering (EBSE) practices finds applicability for research in test case prioritization (TCP) (Kitchenham & Brereton, 2009). TCP orders the test cases in such a way that it achieves optimization based on some properties (i.e., time and cost) of preference (Kitchenham, 2004). Prioritization techniques (Li et al., 2007) create a sequence of execution of test cases based on matrix using fitness function. For instance, If T is a test suit and PT is the permutation of the T. Let f be a function from PT to a real number (Li et al., 2007). The problem is to find T Î PT such that (∀ T’’ (T’ Î PT) (T’ ¹ T’) [f(T’) ³ (T’’)].

Here, function f is used to find T from all possible permutation (i.e., from PT) based on the value that makes it better member than others.