Differential Evolution Algorithm Based on Ensemble of Constraint Handling Techniques and Multi-Population Framework

Aimed at improving the insufficient search ability of constraint differential evolution with single constraint handling technique when solving complex optimization problem, this paper proposes a constraint differential evolution algorithm based on ensemble of constraint handling techniques and multi-population framework, called ECMPDE. First, handling three improved variants of differential evolution algorithms are dynamically matched with two constraint handling techniques through the constraint allocation mechanism. Each combination includes three variants with corresponding constraint handling technique and these combinations are in the set. Second, the population is divided into three smaller subpopulations and one larger reward subpopulation. Then a combination with three constraint algorithms is randomly selected from the set, and the three constraint algorithms are run in three sub-populations respectively. According to the improvement of fitness value, the optimal constraint algorithm is selected to run on the reward sub-population, which can share information and close cooperation among populations. In order to verify the effectiveness of the proposed algorithm, 12 standard constraint optimization problems and 10 engineering constraint optimization problems are tested. The experimental results show that ECMPDE is an effective algorithm for solving constraint optimization problems.

be summed up as optimization problems, which are widely used in real life or production, such as engineering design [1], intelligent control [2], job scheduling [3], traffic optimization [4], network communication [5], financial investment [6], and so on. These problems can be clarified into two categories: unconstraint optimization problem and constraint optimization problem. In comparison, the constraint of the constraint optimization problem leads to the decrease of the feasible domain in the search space, which leads that constraint optimization problem is more complex than the unconstraint problem. Moreover, the optimal solution often locates at the boundary of the feasible domain, so it is necessary to balance the constraints and optimization objectives, which increases the difficulty of finding the optimal solution. Therefore, finding solution to constraint optimization problems, the handling of constraints is always a difficult problem.
The commonly used constraint handling techniques include penalty function method [7], feasibility rule method [8], random sorting method [9], ε-constraint method [10] and ensemble constraint handling techniques [11]. In 2010, Mallipeddi [11] proposed an (the ensemble of constraint handling techniques using the DE, ECHT-DE) algorithm based on multi-population strategy by using ensemble of constraint handling techniques, such as feasibility rules, penalty function method, constraint handling technique, and random sorting method. ECHT-DE can deal with complex optimization problems through mutual learning among populations. In 2011, Elsayed [12] proposed a constraint differential evolution algorithm by randomly combining four differential evolution operations with different mutation operations, two different recombination operations, and two constraint handling techniques (the feasibility rule and ε-constraint handling method), and then formulated 16 different strategies. In the process of evolution, each individual can choose a strategy randomly. The probability of choosing the strategy is determined by its ability of the individual improvement. In 2017, Wang Yong [13] proposed a method to integrate the objective function information into the feasibility rule (the feasible rule with the incorporation of objective function information, FROFI) to achieve an effective balance between the objective function and the constraints. In this method, all the experimental individuals with good objective function are archived and saved, and then the offspring in the archive are replaced by some of the individuals in the new population into the next generation population according to the objective function.
Differential Evolution (DE) [14] is a stochastic global search algorithm first proposed by R. Storn and K. Price. DE can effective solve nonlinear, high-dimensional and complex optimization problems. In 2006, Brest J. [15] proposed a jDE algorithm by adjusting control parameters F and Cr, in which F and Cr mutate randomly with a small probability ζ in each generation. In 2009, J. Zhang [16] proposed a JADE algorithm, which uses a learning scheme to adjust the values of F and Cr. In 2011, Wang Yong [17] proposed a composite differential evolution algorithm (CoDE), for unconstraint optimization problem, which uses three mutation strategies "DE/rand/1", "DE/rand/1", and "DE/current to rand/1" and three groups of F and Cr values randomly to produce three experimental individuals, from which the best individual and the target individual are selected for one-to-one greedy selection. In 2011, Mallipeddi [18] proposed a combined experimental vector generation strategy and control parameter (EPSDE), EPSDE uses different characteristics of variation strategy pool and related parameter value pool, which can compete to produce offspring on the basis of the success of previous generations. In 2018, Guohua Wu [19] proposed a differential evolution algorithm (ensemble of differential evolution variants, EDEV) based on multi-population framework to solve unconstraint optimization problems. The basic idea of this algorithm is to use a framework to integrate three different DE variants with their own characteristics. Many scholars introduce differential evolution algorithm into constraint handling technique to solve constraint optimization problems. In 2012, Wang and Cai [20] proposed a hybrid framework (DyHF), which is composed of global model and local model dynamically. According to the proportion of feasible solutions in the current population, DyHF spontaneously implements the global search model or the local search model to solve the constraint problem. In 2018, based on CoDE, Wang [21] proposed a constraint differential evolution algorithm (CCoDE), which can deal with constraint optimization problems.
According to the principle of free lunch [22], different constraint optimization problems have different characteristics. Thus, a single differential evolution strategy or a single constraint handlinghandling technique is difficult to solve all constraint optimization problems at the same time. Therefore, in order to improve the performance of constraint optimization algorithm, a constraint differential evolution algorithm based on ensemble of constraint handling techniques and multi-population framework is proposed in this paper. The algorithm adopts three complementary differential evolution algorithms, JADE, jDE and EPSDE, and two effective constraint handlinghandling techniques. Each DE variant has a subpopulation, and two constraint handling mechanisms are assigned to each DE variant through the constraint allocation mechanism. The algorithm is combined in three smaller subpopulations, and then the one with better performance is selected to run in the larger reward subpopulation according to the fitness value.
Finally, the population is updated by the combined constraint method.
The structure of this paper is organized as follows. Section 2 introduces the preliminary knowledge. Section 3 introduces the proposed method in detail, including its constraint allocation mechanism, adaptive value calculation and multi-population framework. Extensive experiments and discussions are carried out in Section 4. Section 5 concludes this paper.

A) Constraint optimization problems
Without loss of generality, a constraint optimization problems (minimization problem) can be defined as: where S is decision space with upper boundary i L and lower boundary i U : are the jth inequality constraint and the n-jth equality constraint, respectively.
For COPs, the total constraint violation degree of the solution x is defined as: where ( ) j G x is the degree of constraint violation on the jth constraint, it can be expressed as follows: in (5), δ is a positive tolerance value in equality constraints and usually set to 0.0001. The solution x is a feasible solution when B) Constraint handling techniques 1) Feasibility rule method Feasibility Rule [8], first proposed by Deb, is the most common used constraint handling technique. For each comparison of paired individuals, it first compares the merits and weaknesses of individuals based on whether the individuals are feasible or not. If all of them are feasible, the individuals with small objective function value are better; if none of them are feasible, the individuals with small constraint violation degree are better. For the two individuals, the feasibility rule is used to compare as follows: In the feasibility rule, in the first stage, the infeasible solution with low violation of the overall constraint is selected. In the second stage, we first select all the feasible solutions, and then select the infeasible solutions with low degree of overall constraint default. In the third stage, only the feasible scheme with the best target value is selected. The above criteria (1) and criterion (3) are for constraints, the two infeasible solutions push the infeasible solutions to the feasible region through the comparison of the total constraint violation degree, while the criterion (2) is aimed at the objective. The comparison of the two feasible solutions on the target value improves the global solution.
Feasible rules were first used in genetic algorithms by Ded. Mezura-Montes [23] used feasible rules in an improved differential evolution algorithm. In the algorithm, multiple experimental vectors can be generated from one objective vector at the same time, which improves the probability of generating excellent solutions.
At the same time, when using feasible rules, certain probability is given to allow defaulting individuals with better objective function values to enter the population, which increases the diversity of the population. Huang [24] regards SaDE as the search algorithm and a feasible rule as the individual update condition, and uses the sequential quadratic programming method to enhance the local search ability of the algorithm.
2) ε-constraint handling method The ε-constraint handling method [10] was first proposed by Takahama and Sakai, which can be regarded as an improvement of the feasible rule method. Because the feasible individual in the feasible rule is always better than the infeasible individual, it is easy to lose some excellent infeasible individuals, which makes the population fall into the local optimal solution. The infeasible individuals are relaxed by the constraint treatment method, so that the infeasible individuals with smaller violation of constraints and better objective function value also have a chance to stay in the population. The ε-constraint handling method is used in this paper to deal with constraints. For the solution i x and j x , i x is preference to j x if both of them meet the following conditions: where ε is decreaing as its iterative generation grows, and its calculation formula is as follows: where ε represents intialized as Takahama [25] used the "DE/rand1/exp" strategy as the basic algorithm and the mutation operator based on gradient information as the local search algorithm. When the newly generated experimental individual is an infeasible indi-International Journal of Intelligence Science vidual, the mutation based on gradient information is used to make it a feasible solution. The algorithm has achieved good results in the test function of CEC2006 [26] and won the first place in the competition of CEC2006 constraint evolutionary algorithm. However, the algorithm increases the computational complexity because of the need to calculate the gradient information. Subsequently, Takahama [27] proposed an improved parameter setting method, which can effectively improve the performance of the algorithm when the default value decreases fast enough in evolution, which can effectively improve the performance of the algorithm when dealing with equality constraints. Takahama [28] proposed a method of decreasing probability to use the mutation operator based on gradient information, and cut off and map individuals outside the search space to deal with the case that the optimal solution is at the constraint boundary. After that, Takahama [29] proposed a mutation strategy based on archiving and gradient information. The algorithm uses archiving mechanism to enhance population diversity and proposes a new mechanism to control parameters. The algorithm was tested on the CEC2010 [30] test function and won the first place in the competition of CEC2010 constraint evolutionary algorithm.

C) Differential evolution
Differential evolution algorithm is a real-coded parallel search algorithm. Its initial iteration mainly includes four operation steps: initialization, mutation, crossover and selection. After initializing the population, the algorithm generates a new generation of population through mutation, crossover and selection to guide the search process to approach the global optimal solution.
Firstly, an initial population containing NP objective vectors (NP individuals) is randomly generated in the decision space. After G generation, , , , , , , 1,2, , is expressed as the i vector, where D represents the dimension of the optimization problem to be solved. Let V denote the mutant vector, and the six commonly used mutation operators are enumerated as follows: DE\rand\1 [14]: DE\rand\2 [29]: DE\best\1 [14]: DE\best\2 [30]: DE\current-to-best\1 [16]: DE\current-to-pbest\1 [31]: where F is a scaling factor with values between 0 and 1. where 1 r , 2 r , 3 r , 4 r and 5 r are mutually different integers randomly chosen from [1,NP], "best" is the individual with the best fitness in the population, "current" represents the best individual in the current population, and "pbest" represents p individuals with the best fitness in the population.
represents a random integer uniformly generated between 1 and D, The cross strength is controlled by a parameter When the trial vector is generated, the selection operator will select the best individual as t + 1 generation in the pairwise comparison between otherwise.
JADE is a simple and efficient variant of DE. In JADE, a new "current-to-pbest/1" mutation strategy is adopted as follows: JADE's mutation strategy "current-to-pbest/1" is incorporated with an archive A which contains some recently declared inferior solutions. Let P denote the current population, and  Experimental research shows that JADE can effectively solve unimodal optimization problems, CoDE shows good performance in solving various optimization problems, especially in dealing with some multimodal optimization problems, while EPSDE is particularly effective in solving some highly complex problems.

The Proposed Method
In the process of global searching of the constraint optimization problem, the population is generally clarified into three cases handling methods are put into the constraint mechanism pool, and a constraint optimization algorithm allocation mechanism is designed. In the constraint handling mechanism, there are 8 combinations according to the combination principle.
Suppose the feasibility rule is A, the constraint handling method is B, and the combined pool P is as follows: When each generation triggers the constraint allocation mechanism, a collocation is randomly selected from the constraint mechanism pool and assigned to the corresponding three DE variants, each with a population, for example, Suppose 2 P is randomly selected, JADE algorithm and feasibility rules are run in Here, the feasibility rate is the ratio of the number of feasible individuals to the population size in the t-generation operation of the algorithm. The calculation method of its fitness value is as follows: In the semi-feasible case, the population contains both unfeasible individuals and feasible individuals. For constraint optimization, how to deal with the unfeasible individuals in the population is a very important problem. Since some information carried by some infeasible individuals may be very important to find the optimal solution, it is unreasonable to exclude all infeasible individuals in the semi-feasible case. C) Multi-population based ensemble framework In this paper, similar to EDEV [19], the entire population is divided into three  In ECHDE, the population is divided into four populations of the same size. The four constraint handling mechanisms each have a population. In EDEV, the population is divided into three equal subpopulations and a larger reward subpopulation, which is given to the best DE variants after accumulating a certain algebra. The best combination is selected directly from the three combinations and then given to the reward subpopulation to run.

D) Algorithm framework
In this paper, a new constraint differential evolution algorithm is proposed through the methods of multi-population, multi-DE variants and ensemble of constraint handling techniques. The ECMPDE framework is as follows (Algorithm 1): The ECMPDE framework is as follows (Algorithm 1): International Journal of Intelligence Science

Experimental Study
In this paper, ECMPDE is compared with CCoDE [20] ECHTEP [12], FROFI [13], DyHF [19], jDE [15], JADE [16] and other constraint differential evolution algorithms on standard constraint optimization problems and engineering constraint optimization problems to verify the effectiveness of the algorithm. Twelve standard constraint optimization problems (g01 -g012) of CEC2006 [26] are selected as the test problems of the algorithm. 10 engineering constraint optimization problems [29] included tension-compression spring design, welding beam design, pressure vessel design, reducer design, three-bar truss design, hydrodynamic thrust bearing design, conical wheel design, rolling bearing design, butterfly spring design, gear train design. In this paper, the population size of the ECMPDE algorithm is set as 100 for the experiment, and the parameters in the variant are set as shown in Table 1 below. For each problem, the algorithm runs 30 times independently. According to the results of 30 runs, the best value (MinBest), average (MinMean), middle value (Min-Median), worst difference (MinWorst) and standard deviation (Std) are used to evaluate the performance indicators. According to the results obtained for statistical analysis, this paper uses three statistical analysis methods, including the Mann-Whitney rank-sum test, Iman-Davenport test, and Wilcoxon signed-rank test. International Journal of Intelligence Science

A) Result Analysis of Standard COPs
The MaxFEs of 12 standard test problems is set as 500,000. The population size of CCoDE, ECHTEP, FROFI, DyHF, jDE and JADE was set as 100. The test results of ECMPDE algorithm and six comparison algorithms are shown in Table  2. Table 2 uses the mann-whitney rank sum test to compare the degree of difference between different algorithms in the same problem. The significance level of the mann-whitney rank sum test in the algorithm is set as 0.05, and NAN means that the corresponding optimal value is not found. The fonts with skewed and bold numerical results, conventional, bold marks, bold and gray shading marks in the cell represent the solution results of ECMPDE algorithm, the solution results of ECMPDE algorithm are good, similar to the results of ECMPDE, and the solution results of ECMPDE algorithm are poor. The comparison results are shown in Table 3, where "+", "≈" and "−" respectively represent the number of problems whose numerical results of ECMPDE are better than those of the comparison algorithm, similar to those of the comparison algorithm, and worse than those of the comparison algorithm.
As can be seen from Table 2, except for JADE,ECMPDE, CCoDE, ECHTDE, FROFI, DyHF and jDE algorithms, all the feasible solutions of all problems can be obtained at 100%. The numerical results of G01, G02, G04, G05, G6, G8, G9, G10, G11 and G12 by ECMPDE are all better or better than those of CCoDE, ECHTDE, FROFI, DyHF, jDE and JADE. It is worth noting that the function G02 mainly examines the search ability of evolutionary algorithms, and the numerical results of ECMPDE are better than CCoDE, ECHTEP, FROFI, jDE and DyHF, indicating that the search ability of the algorithm is stronger in the framework of DE variants and multi-constraint mechanisms. For function g03, the numerical results of this algorithm are better than those of jDE and JADE, and slightly worse than those of CCoDE, ECHTDE, FROFI and DyHF. For the function g07, the numerical results of this algorithm are better than those of ECHTDE, jDE and JADE, but slightly worse than those of CCoDE, FROFI and DyHF. From the comparison of the results of Mann-Whitney rank sum test in Table 3, we can see that compared with CCoDE, ECHTDE, FROFI, DyHF and jDE algorithms, the number of numerical results won by ECMPDE is 3, 3, 6, 3, 6 and 9 respectively, the number of similar numerical results is 7, 7, 5, 7, 6 and 3, and the number of differences in numerical results is 3, 2, 1, 3, 0 and 0, respectively. Then the average value and the best value are tested by Iman-Davenport test to further compare the differences of all algorithms in different test problems. The significance level of the test algorithm is set to 0.05, and the test results are shown in Table 3. International Journal of Intelligence Science  From the Iman-Davenport test in Table 3, it can be seen that for the average value, ECMPDE ranks first, which is similar to that of DyHF, while DyHF ranks second, while for the best value, ECMPDE has the smallest rank, which is 2.81. In this part of the comparison, the performance of ECMPDE algorithm is relatively equal to that of DyHF algorithm, and both are better than the other four algorithms.
B) Comparisons of Engineering COPs In this paper, ECMPDE algorithm and CCoDE, ECHTEP, FROFI, DyHF, jDE and other five algorithms are compared and tested on 10 engineering constraint optimization problems to further verify the effectiveness of ECMPDE algorithm in practical problems. The group size setting of CCoDE, ECHTEP, FROFI, DyHF, jDE algorithm is the same as that of ECMPDE algorithm, which is set to 100. In order to facilitate the comparison of the results, the maximum evaluation times of all algorithms for solving engineering optimization problems MaxFEs is set to 500,000 times, the number of runs is 30 times, and the parameter setting is the same as that of IV. The test results are shown in Table 4, and the marks of the data in the table are the same as those described in IV-A. This time, the statistical test is the same as that of IV-A, and the Wilcoxon symbolic rank test is used to test the difference between ECMPDE and other algorithms. Table 5 shows the test results of engineering constraint optimization design problems.
Similar to the previous section, the test results of the three statistical analysis methods are compared in Table 5 below. It can be seen from Table 4 that ECMPDE, CCoDE, ECHTEP, FROFI, DyHF and jDE can find 100% feasible solutions for all engineering problems. The numerical results of ECMPDE solving welded beam design, pressure vessel design, reducer design, three-bar truss design, hydrodynamic thrust bearing design, conical wheel design, rolling bearing design, butterfly spring design, 10-bar plane truss size, multi-disc clutch, robot grip, disconnected beam domain uniform design are all up to or better than the algorithms CCoDE, ECHTEP, FROFI, etc. The solution results of DyHF and jDE.
From the comparison of the results of Mann-Whitney rank sum test in Table  5, it can be seen that compared with CCoDE, ECHTEP, FROFI, DyHF and jDE, the number of problems won by ECMPDE algorithm is 4, 6, 2, 5, 4 respectively, the number of similar results is 10,9,11,9,10, and the number of differences is 1, 0, 2, 1, 1, respectively. From the Friedman test, we can know that on Min-Mean, the rank of ECMPDE algorithm is the first, and its value is 3; on MinBest,  the rank of ECMPDE algorithm is the smallest which is 3.06. In these comparisons, the performance of ECMPDE algorithm is better than that of other comparison algorithms. In the comparison results of Wilcoxon symbolic rank test, the symbolic rank sum is less than that, indicating that the performance of ECMPDE algorithm is the best.
Based on the analysis of the above results, we can know that the overall solution effect of ECMPDE algorithm is better than that of CCoDE, ECHTEP, FROFI, DyHF and jDE in 15 engineering constraint optimization problems, and the performance of the algorithm is the best.

Conclusion
In this paper, we use two constraint handling mechanisms to combine multiple DE variants and combine multiple group frameworks to overcome the shortcomings of weak constraint solving ability and low search ability. In this paper, ECMPDE combines DE variants such as JADE, jDE and EPSDE. The whole population of the algorithm is divided into two categories, namely, three subpopulations and one reward subpopulation. Each constituent DE variant has a subpopulation, and each variant is assigned a constraint handling mechanism through the constraint allocation mechanism, while the reward subpopulation is dynamically assigned to the combination of the best-performing DE variants and constraint handling mechanisms. The improved population division occurs in each generation, and the redistribution of reward subpopulations is triggered by each generation. Through the comparison of the best collocation of the previous generation and the current best collocation, the excellent collocation is saved in the constraint mechanism pool to improve the probability of selection. According to the analysis of the simulation results, ECMPDE is better than DyHF, ECHTEP, FROFI, jDE and JADE in solving test problems, and its performance is similar to that of CCoDE.
This algorithm can improve the searching ability of the algorithm, improve the ability to solve constraints, and successfully inherit the advantages of its DE variant, showing strong robustness and high accuracy. In the future research, it is hoped that there will be excellent constraint handling mechanisms to further improve ECMPDE.