Towards Multi-Faceted Test Cases Optimization Manoj Kumar 1, Arun Sharma 2, Rajesh Kumar 3

The target of software engineering is to produce high quality software product at low cost. Software testing is labour-intensive, ambiguous and error prone activity of software development. How to provide cost-effective strategies for software test cases optimization problem such as classification, minimization, selection, and prioritization has been one of the research focuses in software testing for a long time. Many researchers and academicians have addressed the effectiveness/fitness and optimization of test cases, and obtained many interesting results. However, one issue of paramount importance in software testing i.e. the intrinsic imprecise and uncertainty of test cases fitness, fitness parameters, multi-objective optimization, is left unaddressed. Test cases fitness depends on several parameters. Vagueness of fitness of test cases and their fitness parameters have created the uncertainty in test cases optimization. Cost and adequacy values are incorporated into multi-faceted optimization of test cases. This paper argues test cases optimization requires multi-faceted optimization in order to adequately cater realistic software testing. In this paper, authors have identified several parameters for test cases fitness and multiple objectives for test cases optimization. In addition above, authors have formulated the test cases optimization problem in three different ways using multi-faceted concept. These formulations can be used in future by authors and researchers.


Introduction
Software testing is the process of exercising the programs with specific intend of finding errors prior to deliver.Although software testing is a very human-intensive, time consuming and itself an expensive activity, yet launching of software without proper testing may lead to cost potentially much higher than that of testing, specially in systems where human safety is involved [1,2].Test cases are the inputs to the program under test.A test case is a set of conditions or variables under which a tester will determine whether an application or software system is working correctly or not.Test cases pool may contain some redundant, irrelevant and unfit test cases.Since, testing is very expensive activity, unnecessary execution of redundant, irrelevant and unfit test cases will increase unnecessary burden of cost.The solution is to choose the fittest test cases and removing the unfit, redundant unnecessary ones, which in turn leads to test cases optimization [3][4][5].Measuring fitness of test cases is always a daunting task.The term "fitness" refers to the appropriateness of test cases to check the quality of soft-ware.A test suite is a collection of test cases that are intended to be used to test a software program to show that it has some specified set of behaviours.Multi-criteria test cases fitness evaluation, multi-objective test cases optimization may be the crucial problem for next generation software testing sorority.It requires to device next generation technologies to solve multi-faceted test cases optimization problem.The effectiveness of this verification and validation process depends upon the number of errors found and rectified before releasing the system.This, in turn, depends upon the fitness and number of test cases exercised.So, test cases optimization is necessary.
An optimization problem is the problem of finding the best solution from all feasible solutions.Multi-objective optimization (MO) also known as multi-criteria or multifaceted or multi-attribute optimization is the process of simultaneously optimizing two or more conflicting objectives subject to certain constraints.The objective of MO optimization is to find the set of acceptable solutions and present them to the decision maker to take decision.Test cases optimization is the problem of finding the best subset (class) of test cases from a pool of test cases to be audited.It will meet all the objectives of testing concurrently.Most of the researchers evaluated the fitness of test cases only on single parameter fault detecting capability.Though, the fitness of test case depends on several parameters but consideration of only one parameter is not appropriate.Though, there are several objectives of test case optimization such as execution time, cost of data, cost of risks, and coveragebility, discussed in details in Section 3.But some objectives of test cases optimization are conflicting in nature, coveragebility of one objective will suffer other objective like cost, fitness of test cases and number of test cases in suite while considering all objectives concurrently.However most of test cases optimization approaches found in the literature are single objective and focused on the problem of maximizing the attainment of adequacy value without taking cost into account.It was not until 2001 that detailed empirical study was presented taking cost into account [6,7].It was single objective optimization of test cases incorporating coveragebility and cost in ratio.Two objective test case optimization can be solved in such manner.If we want to optimize the test cases for three or more objectives, this approach is not fruitful.So it is not appropriate to estimate fitness of test cases just on single parameter and classify, select, prioritize the test cases on single objective.Single objective formulation of test cases fitness and optimization are not meeting the objectives of testing.Test cases should be optimized in such a way that it will achieve maximum of code coverage, maximum requirements coverage, high fault detecting capability, maximum mutant killing score and so far.The objective of test cases optimization is to reduce the number of test cases in suite to be audited and improve the effectiveness/fitness of test cases.So, test cases fitness evaluation, classification and selection of test cases should be treated as multi-faceted concept.It will surely reduce the cost & efforts of software testing and improve the quality of testing and reduce the number of test cases to be audited also.

Single Objective Formulation for Test
Cases Optimization  [13,14].To be worthwhile, the sum of the cost of test cases filtration, execution and audit of selected test cases should be less than the cost of that of all of the test cases of the original pool.The goal of test cases filtration is to chunk/ filter out irrelevant, redundant and less fit test cases from the test suite.Test cases filtration is to chunk out subset of closely related test cases, so that a large portion of the defects would be found as if the whole test suite was to be used.It is often desirable to filter a pool of test cases for a program in order to identify a subset that will actually be executed and audited at a particular time.When it is uncertain that how many test cases can be run and audited, it is advantageous to order or rank the test cases as per priorities, so that the tester will select the test cases as per their rank or order, which permit tester to start quick, early fixing the most of the defects [11].Single objective formulation of test cases minimization, selection and prioritization are as follows: Definition 1 (Test Case Minimization): Let T be a set of test cases and R be a set of test objectives.M is a test case minimization of T with respect to R if and only if M is a subset of T that maximizes R(T).
Traditionally, R is defined with respect to a set of syntactic elements of the program under test to be covered by the test suite, such that R(X) is defined to be the number of elements in  1 2 Typically, we seek a set M that is smaller than T, one that is, therefore, a proper subset of T. Ideally, M will be minimal; no other minimized test suite will be smaller.In some approaches, we may seek to cover all elements of , rather than merely maximizing their coverage.When R is defined in terms of coverage, test case minimization becomes an instance of the set cover optimization problem.
. That is, F is monotonic over T.

Sketch of Multi-Faceted Test Case Optimization
Tester will desire to find the subset of test cases that accomplish multi-objectives concurrently in order to maximize the value obtained from inherently expansive process of executing several test cases, investigate the output produced by them.The test problem specification includes three main parts, the purpose of testing, test coverage criteria and the test strategy that will be employed.
Testing objectives can be categorized into two classes which fall on adequacy value or fitness value (those to be maximum) and those which fall on cost side (those to be minimum).Minimization objective can usually be inverted to convert it into maximize objective.Some objectives have natural fitness function, also known as constraint that delimits the set of valid test cases.In search based engineering, these constraints are treated as objectives.In most realistic testing scenarios there will be at least one cost side and one value-based objective.There are several constraints and may be used as test objectives [15]

Objective and Parameters Identification
Existing literatures are the evidence that there are several objectives for test cases optimization and fitness evaluation, whose details are as following (Table 1).

Cost-Oriented Objectives
The quality/fitness of the test cases is not only concern but cost is also one of the essential criteria, the whole purpose of test case optimization is to achieve more efficient testing in terms of the cost.Selection time and exe- It can also lead to many false negatives.Data access cost includes cost of real data population (Human cost or payment to data provider), data retrieving cost.Some systems interact with third parties software, creating significant testing costs.There may be a price for accessing the systems of a third party.However, without such third party service access, it may not be possible to test the system fully [17].Embedded systems are tightly coupled with their environment.These systems may consume resources that have a non-trivial cost.Cost of technical resources used in testing, is important cost driver.There may be setup costs associated with certain test cases.Setup cost includes cost of required devices, services, files.However, the setup costs for the test case may be significant in time and other costs.Such setup costs may also introduce dependencies, leading to an interaction between objectives and constraints.Testing automotive software is very expensive and requires simulation.The efforts of developing or deploying a simulation of the real system constitute a significant cost.Cost oriented objectives should be minimized in test case optimization problem [15].[2,20].

Project Change Volatility (PCV)
PCV is based on the how many times consumer is modifying the project requirements during the software development cycle.PCV is one of the criteria which help to assess the requirement changes after the start of the implementation.High PCV increases the test efforts significantly and make it difficult to complete the project on time.PCV is also important for test case optimization.Test cases of high PCV value are first executed [12].

Multi-Objective Optimization Constraints
Most optimization problems involve various factors influencing the optimal results.In multi-objective test cases optimization problem, the factors that have an effect on the optimal solution are superiority, concurrency, exclusivity, dependability of test cases.In superiority constraint, some test cases have to be performed before others because they establish a system state in which the subsequent test cases become possible, or for which these later tests perform better in some way.We may treat this as an objective.In conjunction constraint, tests cases may be conjoined such that executing one, entails executing another.Such constraints may be soft (it is advantageous to the tester to test these two together) or hard (these two must be executed together).In exclusivity constraint, two test cases may be mutually exclusive.For instance, if one test completely exhausts a resource that is required by another, then these two tests cannot both be performed.Once again, these constraints may be soft or hard, but where they are present, the Test cases optimization process must take them into account.Otherwise, the test case selection and prioritization results produced by test cases optimization may not be viable.In dependence constraint, Inclusion of one test case may affect the cost of another.
For instance, if we undertake the work required by a complex setup process for a certain test case, the same setup may be re-usable by other test cases.In this way there may be dependence between the cost of one test case and the costs of others.If we include one of the test cases, then the cost of all those that remain will be reduced; they share the same setup procedure and the costs associated with them [21,22].

Problem Formulations
Multi-Objective Optimization is defined as problem of finding the vector of decision variables X which satisfies the constraints and optimizes a vector function whose elements represent the objective functions.Generally it can be described as a vector function that maps a tuple of parameters (decision variables) to a tuple of objectives.The decision vector is also called the parameter space and the objective vector is also called the objective space.Find the vector * which will satisfy the m inequality constraints: where 1, 2, , i m   and the p equality constraints Test cases optimization is multi-objective, NP-Complete, peculiar nature problem.Multi-objective test cases optimization is scalable and multi-modal optimization problem.It can be scaled to any number of objectives and constraints.Similarly, the multi-objective formulation of test optimization can be done in various ways.It can be formulated as multiple single objective functions, weighted sum approach, bottom-up approach and many others.

Multiple Single Objective Functions
First approach is most intuitive one and simple.In this approach, N different single-objective functions are used j to construct a multi-objective test cases optimization problem.Different objective functions are simply used as different translation of single objective function.Details of objectives of test cases optimization are given in Ta- bles 1-2.

Maximum/Minimum
where , and 1, 2, 3, ,  .N is the number of objectives and M is the number of constraints.Z 1 is fault detecting capability and Z 2 is execution cost of test case.It lacks Global optimal solution.Since the test case t i optimal for objective function Z 1 is not optimal for objective function Z 2 and vice versa.It provides local optimal solution.It requires Pareto optimal set of solutions.Pareto optimal set contains individual optimal solution of each objective and trade off solutions of all objectives.It requires global Pareto optimal solutions.

Weighted Sum Approach
Second Weighted Sum Approach is common and simple.A set of n objectives, , and a set of weights 1 2 3 can be combined into a single weighted objective (WO),   where 1 2 3 are the weight values test objectives.Weight Values of test objectives can be estimated by using Fuzzy Logic based approach and will be explored in future.Details of objectives can be found in Tables 1-2., , , , n w w w w 

Bottom-up Approach
In bottom-up approach, test cases optimization is considered as search space problem.Pareto optimal front is considered in this approach.Mathematical function describing Pareto optimal front is assumed in objective space.Pareto optimality is a notion from economics with broad range of applications in game theory and engineering.Pareto optimality provides a set of test cases not a single test case to be exercised on software under test.Pareto-optimal solutions are optimal in some sense.Therefore, like single-objective optimization problems, there exist possibilities of having both local and global Pareto-optimal solutions.Before we define both these types of solutions, we first discuss dominated and nondominated solutions.
For a problem having more than one objective function (say, Z i , and ).Any two solutions x (1) and x (2) can have one of two possibilities-one dominates the other or none dominates other.Solution x (1)  is said to be dominate other solution x (2) , if both the fol-lowing condition are true.
The solution x (1) is no worse than x (2) in all objectives.
2) The solution x (1) is strictly better than x (2) in at least one objective.
If any one of above two is violated, solution x (1) does not dominate the solution x (2) .It is also true that if x (1)  dominates the solution x (2) then we can say x (2) dominated by the solution x (1) or x (1) is non-dominated by x (2) .Between two solutions x (1) is non-dominated solution.
Based on this, the multi-objective optimization problem can be defined as the problem of finding a vector of decision variables x, which optimize a vector of N objec- tive functions   i f x where .The objective functions are the mathematical description of the optimization criteria.Without the loss of generality, it is assumed that the goal is to maximize i 1, 2, , i  N f where .A decision vector x is said to dominate a decision vector y if and only if their objective vectors All decision vectors that are not dominated by any other decision vector are said to form the Pareto optimal set, while the corresponding objective vectors are said to form the Pareto frontier.
Above concept can be extended to find a non-dominated set of test cases form the pool of test cases.Considering the set of M test cases, each having N (N >= 1) objective function values.In Local Pareto Optimal set, if every member t i in test suite Ts, there exist no test case t j which dominates any member in the test suite Ts, then solution belonging to test suite Ts constitute local Pareto optimal test case.In Global Pareto Optimal set, if there exist no test case t i in test suites space which dominates any member in the test suites space TS, then solution belonging to test suites space TS constitute Global Pareto optimal test case.The size and shape of Pareto Optimal fronts usually depends on number of objective functions and interaction among individual objective functions.If the objective functions are conflicting in nature to each other, resulting Pareto front may span larger than if the objective are more cooperating.However in multi-objective test cases optimization problem, some objectives are conflicting in nature to others.Resulting Pareto optimal front of multi-objective test cases optimization may contains many solutions, which can be found by using multi-objective optimization algorithm.When we consider the case of finding a set of non-dominated solutions rather than a single-point solution, multi-objective Evolutionary Algorithms (MOEA) have to perform a multi-modal search for global the Pareto-optimal set.Now the multi-objective test cases optimization problem can be defined as follows: Definition 4 (Multi-Faceted Test Cases Optimization) Given: a vector of decision variables, x, and a set of objective functions,

Conclusion and Future Scope
Testing is complex, time consuming, human-intensive, full of uncertainty and expensive activity.Choosing the right fit test cases is an important and critical task in software testing.Literature is evident that there is no direct measure of fault revelation likelihood.There are different parameters for judging the fitness of test cases.
Cost is also an important factor for test cases optimization involved in different ways and cannot be ignored.This complex interplay between cost and fitness/ adequacy value is further compounded by the many additional validity constraints.These constraints, cost and fitness value are making test case minimization, selection and prioritization problems as multi-faceted optimization problem.The present paper advocacies that such a Multi-faceted approach for test cases fitness evaluation and test cases optimization is long overdue.In this paper, authors formulated the multi-faceted test cases optimization problem.It will be beneficial for researchers.In future, evolutionary and soft computing techniques will be explored for multi-faceted measurement framework for test cases optimization and fitness evaluation.

Definition 2 (Test Case Selection): Let
Test cases selection problem can be formulated in terms of test cases minimization problem.It becomes minimal set cover optimization problem.So, test cases minimization is NP-Complete problem.P 0 be a modified version of P and let T be a test suite.Let CA be a function which takes a pair of programs and reports the set of program elements of the form A that are different in P 0 compared to P. In this context, A plays the role of test adequacy criterion, substituted by values such as <branch adequacy> and <statement adequacy>.Let be the number of elements of covered when P 0 is executed on X.A test set T 0 is an A-adequate test case selection of T with respect to (P, P 0 ) if and only if T 0 is a test case minimization with respect to .Let F be a function from test suite elements to some domain on which the relation ≥ imposes a total order.T' is a test case prioritization of T with respect to F if and only if for all i, 1 1

Table 2 . Summary of multi-objective test cases optimization parameters.
[7,16]re the important factors for test cases optimization.Software testing cost includes data access cost, third party software cost, technical resources cost, setup and simulation cost[1,2].Execution time is a natural candidate for test cases optimization.Execution time is one realistic measure of effort.Physical execution time of test cases is hard to measure accurately.Measurement is confounded by many external factors; different hardware, application software and operating system.Execution time is clearly a pressing concern for a tester, given the short build cycles within which they will typically have to perform the testing activities[7,16].Data access cost concerns to databases software.Access to databases determines the coverage of the application under test.The population of the database will significantly affect the effectiveness of testing.We shall prefer realistic test cases rather than a mocked up version of the database, into which data may be systematically, but nonetheless synthetically added.Such synthetic data can lead to many false positives, because integrity constraints are not handled by the automated synthetic test generation algorithm. cution  , the decision vector for t 1 dominates that of t 2 .The multi-objective test case selection problem is to select a Pareto efficient subset of the test suite, based on multiple test criteria.It can be defined as follows: Pareto optimal set with respect to the objective functions, f i , .The objective functions are the mathematical descriptions of test criteria concerned.A subset t1 is said to dominate t 2 when the decision vector for t 1The multi-objective test case prioritization problem is to rank a Pareto efficient subset of the test suite, based on multiple test criteria.It can be defined as follows: The objective functions are the mathematical descriptions of test criteria concerned.A subset t 1 is said to dominate t 2 when the decision vector for t 1