Performance Comparison of Electromagnetism-Like Algorithms for Global Optimization

Electromagnetism-like (EML) algorithm is a new evolutionary algorithm that bases on the electromagnetic attraction and repulsion among particles. It was originally proposed to solve optimization problems with bounded variables. Since its inception, many variants of the EML algorithm have been proposed in the literature. However, it remains unclear how to simulate the electromagnetic heuristics in an EML algorithm effectively to achieve the best performance. This study surveys and compares the EML algorithms in the literature. Furthermore, local search and perturbed point are two techniques commonly used in an EML algorithm to fine tune the solution and to help escaping from local optimums, respectively. Performance study is conducted to understand their impact on an EML algorithm.


Introduction
This paper studied the performance of a new class of evolutionary algorithms called electromagnetism-like (EML) algorithm, recently proposed by Birbil and Fang [1], for optimization problems with bounded variables in the form of: where f(x) is the objective function to be minimized, 1 2, ( , , ) is the variable vector, and L = (l 1 , l 2 , •••, l n ) and U = (u 1 , u 2 , •••, u n ) are the lower bound and upper bound of x, respectively.That is, l i ≤ x i ≤ u i for i = 1 to n.
EML algorithm simulates the interaction caused by electromagnetic force between electrically charged particles.Due to its effectiveness, EML algorithm has been applied to various optimization problems, such as scheduling [2-4], vehicle routing problems [5], feature selection [6], fuzzy neural system [7], and engineering design problems [8] since its inception.
The general scheme of an ELM algorithm [1] is shown in Figure 1.It consists of four phases: initialize a population of particles (step 1 in Figure 1), local search to exploit local optimums (step 3 in Figure 1), calculate the force exerted on each particle (step 4 in Figure 1), and move each particle along the direction of the force (step 5 in Figure 1).
Because of the simplicity of the EML scheme, many EML or EML-hybrid algorithms have been proposed in the literature.These algorithms mainly differ in the last three phases of the above general EML scheme.That is, different local search method can be used, and the force exerted on each particle and the new position of each particle can be calculated differently in different EML algorithms.Many of these EML algorithms have persuasive experimental results showing their superior performance over the original EML algorithm of Birbil and Fang [1].However, due to the lack of comparison among these algorithms, the best way to design an EML algorithm remains unclear.Further, most of the experimental results are for optimization problems in a lower dimensional space, and it is unclear whether these EML algorithms scale well with high dimensionality.Birbil et al. [9] pointed out the premature convergence problem of the original EML algorithm, however, it also remains unclear whether these new EML algorithms can escape from local optimums effectively and efficiently.
The objectives of this study are threefold.Firstly, it surveys the literature for the alternatives for force calculation in the EML algorithms.Secondly, it uses an artifi-cial problem instance together with non-uniformly distributed particles to provide a sanity check of these EML algorithms on their ability to escape from local optimums.Thirdly, it compares the performance of these EML algorithms using a set of well-known benchmark functions, ranging from low to high dimensionality.The results not only provide better understanding of these EML algorithms, but also guide the development of new EML algorithms.

Survey of EML Algorithms
This section reviews various local search methods (step 3 in Figure 1), force calculation methods (step 4 in Figure 1) and particle moving methods (step 5 in Figure 1) that have been adopted in the EML algorithms in the literature.

Local Search Methods
The purpose of local search is to move a particle to its nearby local optimums.Birbil and Fang [1] indicated that local search can be either omitted or applied to all particles or only the current best particle in the population.Omitting local search, an EML algorithm relies solely on the EML heuristics to find the optimal solution.However, applying local search to all particles is time consuming and offers slight improvement over applying local search only to the current best particle [1].Therefore, in this study, local search is either omitted or applied only to the current best particle.
Various local search methods have been used in EML algorithms.Theoretically, any local search method can be adopted in ELM algorithms.Complex local search methods (e.g.chaos optimization [10] and pattern search [11]) help converge to and escape from local optimums.The original EML algorithm [1] uses a simple local search, called random line search, so that the benefit of the electromagnetism heuristics can be better appreciated.Therefore, random line search is also adopted in this study to allow fair comparison among various implementations of the electromagnetic heuristics in ELM algorithms.
Random line search requires two parameters: δ and LsIter.First, the maximum feasible step length r k at each dimension k is calculated as the product of δ and the range of dimension k (i.e.u k − l k ).Then, for each particle i, this method searches along each dimension k for improvement of particle i for no more than LsIter times, as shown in Figure 2. Notably, this local search method is simple but has weak capability of escaping from local optimums.

Calculate Force and Move Particles
Various EML algorithms employ the electromagnetic 1.
For each dimension k do 2.
End while 16.End for Figure 2. Random line search (for particle i at x i ).
heuristics somewhat differently.For example, to move a particle, some algorithms consider the force exerting from all other particles to this particle, while some algorithms only consider the force exerting from an another particle.Furthermore, in some EML algorithms, the magnitude of the force between two particles is not inversely proportional to the square of their distance.The rest of this section surveys how the electromagnetism heuristics are interpreted in various EML algorithms.

Original Method
The original EML algorithm of Birbil and Fang [1] uses an electromagnetism-like attraction-repulsion mechanism to move particles as follows.
First, calculate the charge q i of each particle i using Equation (2): where x best denotes the particle with the best objective value in current population (i.e.x best = argmin{f(x),  i}), m is the number of particles, and n is the number of dimensions.All particles have charge between 0 and 1, and particles with better objective values have higher charges.
Then, the force i j F exerted on particle i from another particle j is calculated using Equation (3): x x  , which contradicts the electromagnetic heuristics that the force between two particles should be inversely proportional to the square of their distance.The total force i F exerted on particle i from all other particles in the population is calculated using Equation (4): Finally, all particles except x best are moved using Equation (5): where k = 1, •••, n, and λ is a random value uniformly distributed between 0 and 1.One advantage of Equation ( 5) is that it does not move particles outside the feasible space.However, Equation (5) does not move each particle exactly in the direction of the force exerted on them, and thus does not closely follow the electromagnetic heuristics.Also notably, the best particle is not moved.

Original Method with a Perturbed Point
Birbil et al. [9] indicated that the original EML method could converge prematurely, and thus they modified the original method by introducing the idea of a perturbed point.The perturbed point, x p is the farthest particle from the current best particle x best in the current population, i.e.

 
best arg max , 1, 2, , Their new method works exactly the same as the original EML method [1] does except that the calculation of the force exerted on x p is modified as follows.First, the force p j F exerted on x p from particle x j is perturbed by multiplying a random value λ ~ U (0, 1), as shown in Equation (7).
Then, the force p j F reverses its direction if λ < ν where ν is a parameter between 0 and 1.Finally, the total force exerted on particle p is calculated using Equation (4).

Debels's Method
In the original EML method, either with or without using rticle x i , another pa  a perturbed point, all particles in a population exert a force on all other particles.Debels et al. [2] proposed a simplified EML method by considering only the force from a randomly chosen particle.This method is adopted as the mutation operation in the hybrid algorithms by Kaelo and Ali [12] and Chang et al. [3].
To calculate the force exerted on a pa rticle x j is selected randomly from the current population.Then, the force exerted on x i from x j is calculated using Equation ( 8): where x worst and x are the worst particle and the best particle in current population, respectively.Next, particle x i is moved to i i j x F  .Notably, if f(x i ) < f(x j ), then it is possible to mo ut of the feasible space of problem (1).An extra step is taken here to restrict x i inside the feasible space using Equation (9).
Strictly speaking, this method is not electrom lik agnetisme because the magnitude of the force i j F is linear proportional to the distance between x i and j , not inversely proportional to the square of their distance. x

Rocha's Method of Shrinking Population eedup e of
Rocha and Fernandes proposed a method to sp EML algorithms by shrinking the size of a population whenever the spread of the objective values reduces by a predefined percentage [11].Essentially, this idea can be applied to any population-based evolutionary methods.
The spread of the objective values w.r.t. the best valu a population is defined as follows.
Initially, SPR ref is set as the SPR of the initial populatio

ML method
The total force F exerted on particle i in the current it-n.Then, every time after local search in an EML algorithm, if the remaining population size is greater than twice the number of dimensions, then the SPR of the current population is calculated.If the current SPR is less than εSPR ref , then half of the population is discarded and SPR ref is set to current SPR.Here, ε is a used-defined threshold.
Rocha and Fernandes proposed another E which only differs from the original EML method on the calculation of the total force exerted on each particle [13].This method takes the change of the force into account.i eration is calculated as follows.First, F i is calculated using Equations ( 2)-(4), as did in the original EML method.Then, the change Δi of the force is set to 0 for the first iteration, and is set to for the rest iterations of the algorithm, where , i prev

F
denotes the total force exerted on particle i in ous iteration.Finally, F i is adjusted as F i + βΔi, whe is a parameter in the interval [0, 1).Notably, if β = 0, this method is the same as the original EML method.The suggested value for β is 0.1.

Rocha
the previ re β 's Method of Modified Charge Rocha and Fernandes proposed two methods that intend cy of EML to improve the efficiency and solution accura algorithms [14].Both methods differ from the original EML method on how the charge of a particle is calculated.Their first and second methods replace Equation (2) of the original EML with Equations ( 11) and ( 12), respectively.
Both Equations ( 11) and ( 12) still yield q i b and 1.Furthermore, both methods replace Equation (3) of etween 0 the original EML with Equation ( 13) such that the magnitude of the force i j F exerted on particle x i from particle x j is inversely proportional to the square of the distance between x i and x o be consistent with the electromagnetism heuristics.

Yurtkuran's Method of Reducing Movemen
Yurtkuran and Emel [5] propose an EML method that     t differs from Debels's method [2] only on how the electromagnetic force moves a particle.Their method reduces the effect of the force as the number of iterations increases.Let iter denote the current number of iterations.After calculating the force i j F exerted on particle x i from particle x j using equation (8) as did in Debels's method, x i is moved to i i j x F iter  instead of x i + i j F .Consequently, this method has the effect of reducing the range of movement as th f iterations increases.As in Debels's method, this method could move particles out of the feasible space, and thus Equation ( 9) should be applied to confine particles inside the feasible space.

Shang's Method of High Charged Particles e number o
Shang et al. proposed an EML method that ignores the s to force exerted from those particles with small charge improve efficiency [15].First, the charge of each particle is calculated using Equation (2), as did in the original EML method.Those particles with charges lower than half of the average charge of all particles cannot exert force on other particle.This can be done by introducing a modified charge i q of particle x i as follows.
Then, Equation ( 15) is used to calculate the force i j F exerted on particle x i from particle x j .
Equation (15) does not closely follow the electrom netic heuristics.The rest of this method is the same a ori e    ags the ginal EML method.The composite force F i exerted on particle x i is calculated using Equation (4), and the new position of particle x i is calculated using Equation (5).

A Sanity Test for Premature Convergenc
Birbil and et al. [9] used a simple example to show the premature convergence problem of the original EML algorithm of Birbil and Fang [1], described in Section 2.2.1.In this section, we adopt the same example and show that all the EML variants described in Sections 2.2.3 -2.2.8 suffer from the same problem.
Consider the objective function f(x), shown in for a one dimensional minimization problem.Supposed that all of the particles (shown as solid circles) in the current population are located on the right of the position marked with a star.Let x i and x j be two particles in the population.No matter x j is on the left or on the right of x i , the direction of the force exerted on x i from x j is toward the right, according to Equation (3), ( 8), ( 13) or (15 Co s erturbed particle is the leftmost pa on the right of th est pa ).nsequently, the electromagnetic force will alway move all particles, excluding the best particle, in the current population to the right, and miss the global optimum on the left at the origin.
Birbil and et al. [9] proved that their perturbed EML method, described in Section 2.2.2, terminates with an "ε-optimal" solution when the number of iterations is large enough.This is achieved by stochastically reversing the direction of the electromagnetic force exerted on the particle (called the perturbed particle) that is farthest from the current best particle.For the example in Figure 3, it is obvious that the p rticle in the population, and it could be moved to the left of the star sign, and consequently converges to the origin, using the perturbed EML method.
Consider another example in Figure 4, where the perturbed particle is the rightmost particle in the population, and the current best particle is the second leftmost particle.In this case, the reversed electromagnetic force does not help because if the perturbed particle is moved by its reversed electromagnetic force, it will be moved to the right, and consequently further away from the origin.
If the perturbed particle (or any particle e current best particle) in Figure 4 is moved by its electromagnetic force, it will be moved to the left.However, this movement is beneficial only if the particle can be moved pass the star sign.In other words, the electromagnetic force provides the direction of movement, but the distance of the movement should not be restricted to the distance between the particle and the current b rticle.For example, Equation ( 5) provides the greatest freedom by allowing the movement up to the boundary of the feasible space.However, such freedom also reduces the convergence speed to the global optimum, and makes the direction of the movement somewhat different from the direction of the electromagnetic force.On the other hand, Debel's method, described in Section 2.2.3, provides less freedom by restricting the distance of the movement up to the distance between the two particles under consideration.Therefore, it is more likely to trap in a local optimum.

Experimental Setting
Sinc the .The shrinking population method is not included because its idea is applicable to onary algorithm.For ease of ethods are listed in without local search; method 2L refers to D e our interest is on the various interpretations of electromagnetic heuristics, all methods, described in Section 2, except the shrinking population method are implemented for this study any population-based evoluti exposition, the seven implemented m Table 1.
Furthermore, the idea of the perturbed point method [9], described in Section 2.2.2, is extended to and implemented as an option for all seven methods in Table 1.
Local search for the current best particle is also implemented as an option.Therefore, method 2 refers to Debel's method without perturbed point and local search; method 2P refers to Debel's method with perturbed point but ebel's method with local search but without perturbed point.
A set of six well-known benchmark functions, listed in Table 2, was used in this experiment.The number n of dimensions ranges from 10 to 50.The first benchmark function is unimodal, while the remaining five are multimodal.The number of particles and the maximum number of iteration are set to 2n and 25n, respectively.
For each setting, 30 runs were conducted, and their average performance was obtained.Each run stops until the maximum number of iterations is reached.For the same run, the same set of initial particles was generated for all methods.This performance study is divided into three parts, which are described in the subsequent three sections.oor local optimum in methods 2 and 6 consider the force exerted on a particle only from another randomly chosen particle, while the rest five methods consider the force from all other particles.This explains why method 2 converges at a slower pace th sults can be observed on the other benchmark functions and different number of dimensions.

This section focuses on the effectiveness of various
Tables 3-7 show the average of the best objective values over 30 runs.Method 6 is the worst performer,  study, EML algorithms with both perturbed point and local search deactivated perform poorly for most benchmark functions, and thus are not recommended.Previous work has shown that local search and perturbed point help improving the performance of EML methods.

Table 5. Average of best objective values (n = 30).
Method Subsequent two sections study their impact on the performance.

Local Search Test
Birbil and Fang [1] suggested that applying local search on the current best particle improves the performance of their EML method without incurring much overhead.This section evaluates the performance of the seven EML methods with local search on the best particle.We adopted the same local search method as Birbil     Among them, method 4P appears to perform the best for n ≤ 30, while methods 5P-7P perform the best for n > 30.
However, when comparing to their non-perturbed counterparts in Tables 3-7

Figure 4 .
Figure 4. Example of a useless perturbed particle.

Figure 5 .
Figure 5. Convergence of EML methods for f 6 with n = 10.

Table 3 . Average of best objective values (n = 10).
M od4.A age of b t objec ve valu s (n = 0).
and Fang [1] did, i.e. random line search, described in Figure2.The two parameters for random line search are set as follows: δ = 1E−3 or 1E−4 and LsIter = 150.No attempt is made to fine tune these two parameters to fit different benchmark functions.Notably, for an EML method without local search, the objective function needs to be calculated m − 1 times in each iteration, where m is the

Table 22 . Average of best objective values (n = 50, perturb).
Reyc R. Leus et al., "A Hy rid Scatter Search/Electromagnetism Meta-Heuristic for Project Scheduling," European Journal of Operational Research,