Investigation Effects of Selection Mechanisms for Gravitational Search Algorithm

The gravitational search algorithm (GSA) is a population-based heuristic optimization technique and has been proposed for solving continuous optimization problems. The GSA tries to obtain optimum or near optimum solution for the optimization problems by using interaction in all agents or masses in the population. This paper proposes and analyzes fitness-based proportional (roulette-wheel), tournament, rank-based and random selection mechanisms for choosing agents which they act masses in the GSA. The proposed methods are applied to solve 23 numerical benchmark functions, and obtained results are compared with the basic GSA algorithm. Experimental results show that the proposed methods are better than the basic GSA in terms of solution quality.


Introduction
Optimization process in computer science is to find the best solution from all feasible solutions, in which the best solution maximizes the profit function or minimizes the cost function.Especially if optimization problems have high dimensions or non-linear characteristics, to find optimal solution is so hard because search space of optimization problem increases exponentially with dimension increasing.To overcome this situation, many optimization algorithms especially inspired from nature have been suggested in recent years such as particle swarm optimization developed by inspiring bird flocking or fish schooling [1], ant colony algorithm which simulates behavior of real ants between nest and food source [2], bee colony algorithms which were inspired intelligent behavior of honey bees [3,4].In addition to these algorithms, some algorithms which were inspired by various natural events were developed.Such as harmony search algorithm inspired by natural musical performance process when a musician seek for a better condition of harmony [5], genetic algorithm based on natural evolution [6] and the GSA simulated Newton's law of gravity [7].
The GSA is a heuristic optimization technique which is inspired by Newton's law of gravity [7].In this algorithm, the main rule is that each agent has features attract masses to extend each other.When all agents attract each other, the effect which agents have big masses to solution is decrease, so to find optimum solution is difficult and the convergence of the method to the optimum or near optimum has slowed.Which masses will be attraction effect and which masses will be disregard is a problem in the GSA.To overcome this problem, selection mechanisms in the GSA are analyzed in this study.
The rest of paper is organized as follows: Section 2 presents a literature review for the GSA.The basic GSA and the selection mechanism are explained in Section 3. The experiments are presented in Section 4. Section 5 discusses the experimental results and the study and finally, conclusion and future works are given in Section 6.

Literature Review on GSA
GSA is heuristic optimization technique which is inspired by Newton's law of gravity.Each agent is named as object and success in optimization algorithm are related as masses in GSA [7].Using the mass of each agent and distance of between agents are calculated new position of agent.Mass and distance are inversely.Effect of big and close agent is more the solution.
While new position of each agent is calculated, to avoid fast convergence is used random number in basic GSA.Main problem of fast convergence is to get stuck of local minimum the method.To overcome fast convergence and local minimum, Han and Chang [8] suggested using chaotic variables instead of random numbers in modified GSA.
Unconstrained acceleration has caused more diversification in the population.Khajehzadeh et al. [9] proposed velocity clamping for velocities of the agents in order to prevent diversification in the population.Therefore, velocity of the agent is constrained between maximum and minimum values of the velocities.
GSA starts to search with random solutions on the search space.An opposition based learning method is proposed for the initialization and also continuation working of the GSA by Shaw et al. [10].In this way, the convergence rate of GSA has improved and the robustness of GSA is increased.
In order to improve global search ability of GSA, "Disruption" operator, inspired from astrophysics, has been added to basic GSA [11].The new method is used for optimizing 23 numeric functions and obtained results are compared with basic GSA, PSO and GA.The experimental results show that proposed method is superior to basic GSA, PSO and GA.
Li and Zhou [12] developed a new version of GSA by combining the search strategy of PSO and the improved GSA (IGSA) is applied to parameters identification of hydraulic turbine governing systems.The experimental results show that (IGSA) is capable with respect to PSO, GA and basic GSA in terms of solution quality.
Niknama et al. [13] suggested self-adaptive GSA to improve convergence characteristic in basic GSA.Two methods developed to improve solution in this technique.New solution in GSA moves independently the previous solution because GSA has not memory.Finding best solution is used to find new solution in the first technique.Second technique was developed for which solution has not local minimum.New solution was produced using three different agent are selected in second technique.Produced solution was used new solution with probability.

Gravitational Search Algorithm
One of the newest heuristic optimizers is gravitational search algorithm (GSA) which is based on law of gravity, law of motion and interaction of masses [7].In GSA, potential solutions correspond to the position of masses and masses corresponds the fitness value of the solution produced for the optimization problem.By using law of gravity and law of motion, Rashedi et al. [7] have defined interaction between the masses.The GSA is an iterative algorithm and the algorithm is explained step by step as follows: Step 1. Initialization The masses are randomly produced on the solution space using Equation (1).
( ) where , i j X is the ith mass position value on the jth dimension, max j X and min j X is the upper and lower bound for the jth dimension, respectively, N is the number of masses and D is the dimensionality of the optimi-zation problem.
After the random solution is produced, the sizes of the masses are calculated as follows: ( ) ( ) Step

Law of Gravity
In GSA, interaction between the masses is based on action-reaction.Force acting on a mass is calculated as follows: For each mass i, , F t is the force acting on dth dimension of masses on ith position on iteration time t, () is the gravitational constant number on iteration time t, ( ) i M t and ( ) j M t is the active and passive gravitational masses on iteration time t, respectively, ( ) t is the dth dimension of masses on jth position at iteration time t, ( ) , i j R t is the Euclidian distance between the mass on ith position and the mass on jth position and  is the small constant number.
After the force for each dimension is calculated using Equation ( 4), the total force on the mass is obtained as follows: , 1 ( ) ( ) where    () is the total force acting on dth dimension of the mass on ith position on iteration time t and   is a random number produced in range of [0,1], which is used for providing stochastic characteristic to GSA.

Law of Motion
The motion depends on acceleration of the mass and the acceleration of the mass is calculated as follows: ( ) ( ) ( ) where ( ) a t is the acceleration on the dth dimension of the mass ith position.The velocity and new position of the mass are calculated as follows: ( ) ( ) where ( ) is the velocity for dth dimension of the mass on the ith position on iteration time 1 t + , ( ) is the dth dimension of the mass on ith position at iteration time 1 t + and , i d r is the random num- ber produced for dth dimension of the mass ith position in range of [0,1].
Step 4. Termination After new positions for the masses are obtained by using Equations ( 6)-( 8), the fitness of the solutions are calculated using Equations ( 2) and ( 3).The obtained solution with best fitness value is stored and if a termination condition is met, the algorithm is terminated and the best solution is reported.Otherwise, running of the algorithm is continued from the Step 2. Basic GSA is presented as a flowchart in Figure 1.
In the basic GSA, all agents in the population are used for calculating force acted on a mass.In order to increase convergence and local search capabilities of the method, which agents will be used are decided for determining acting force on a mass by using four selection mechanisms-roulette wheel, tournament, random and rank-based selection mechanisms and these mechanisms are given below.

Random Selection
In order to provide enough diversification in the population, a certain number of particles are selected from the population.Instead of whole population, the selected particles are used for calculating force acting on the mass.In this selection mechanism, each agent has same selection probability and whole solution space is searched by using this mechanism but the local search on the solution space is reduced.

Roulette Wheel Selection
This selection mechanism is based on the fitness of the solutions.For calculating force acting on the mass, a certain number of particles are selected from the population using roulette wheel selection.Being selected probability of a particle is given as follows: where i p is the being selected probability of the ith particle, i fit is the fitness value of the solution of ith particle and N is the number of particles.Being used roulette-wheel selection, it is aimed that the convergence rate of the method is increased because the agent has been affected mostly good solutions obtained previous iteration.

Tournament Selection
Tournament selection covers running several tourneys between the particles randomly selected from the population.The winner of each tourney (the agent with the better fitness) is used for calculating force acting on the mass.The tournament size is important for the tournament selection mechanism because if the tournament is larger, weak particles have a smaller chance to be selected.In the tournament selection, when the number of tournament is increased, the agent has been affected mostly the best solution, and the solutions of the population are quickly improved especially for unimodal functions.When the number of tournament is decreased, the solution with low fitness can be selected and the diversity in the population can increase and global search ability of the method is improved.

Rank-Based Selection
Rank-based selection is an alternative selection mechanism used for obtaining chromosomes which will be subjected to crossover operation in genetic algorithm [20].In the rank-based selection, the agents are sorted in ascending order by using their fitness values.For each rank, a selection probability is calculated as follows: where, pos p is agent's probability of being selected in position pos, N is the number of agents and SP is the selective pressure.In the positions, position of least fit agent is first order 1 and the position of fittest agent is Nth order.

Experimental Results
In order to investigate effects of the selection mechanisms to the performance of the GSA, 23 benchmark functions taken from [7] are used and obtained results are compared the results of basic GSA.

Benchmark Functions
23 test functions given in Tables 1-3 and taken from [7] are divided to different groups.F1-F7 functions have only one local minimum and this local minimum is global minimum.These functions are unimodal functions, and used for investigating local search ability of the method.If a function has more than one local minimum, this function is called as multimodal function and the global search capability of the method is tested on these functions (F8-F13).Another difficulty for a method is dimensionality of the optimization problem [21,22].While F14-F23 test functions are small-sized functions, the dimensionality for F1-F13 functions is taken as 30.

Control Parameters
The population size of the methods is taken as 50 in all experiments.The stopping criterion for the algorithms is ( ) ( ) ( ) [0,1) ( ) ( ) 10 cos 2 10 ( ) ( ,5,100,4) ( ) ( ) ( ) ( ) maximum iteration number (MIN), and MIN is 1000 for F1-F13 functions and 500 for F14-F23 functions.The gravitation constant (G) used in GSA depends on iteration time and is calculated as follows [7]: G t is the gravitation constant at time step t, 0 G is the initial gravitation constant which is taken as 100 in the initialization of the algorithm,  is the maximum iteration number and  is scaling factor and it is set to 20.
Number of masses which will be act force is taken as 10, 15, 20, 25, 30, 35 and 40 in GSAF (GSA with roulette wheel selection), GSAT (GSA with tournament selection), GSAR (GSA with random selection), GSAL (GSA with rank-based selection).Experimental studies show that the more successful results are obtained when the number of masses which will be act is taken as 40.Therefore, the number of masses which will be act is taken as 40 in the comparisons.In GSA with rank-based selection mechanism, the linear ranking is used and the selective pressure is taken as 2.

Comparisons
The mean results obtained by GSAF, GSAT and GSAR, GSAL are compared the mean results of basic GSA and these results are presented in Table 4 for unimodal test functions, Table 5 for multimodal functions and Table 6 for the multimodal test functions with fix dimension.
According to Table 4, the results obtained by GSAR and GSAF methods are relatively better than the results of basic GSA because these functions are unimodal functions and the local search ability of the GSAF and GSAR is better than GSA.
Based on Table 5, due to the fact that the masses in basic GSA have been affected by all the masses in the population, the diversity in the population is kept during the iterations and the results obtained by basic GSA are slightly better than the results of proposed methods.According to Table 6, the dimensionality is the important factor for the methods and the proposed methods are better than the basic GSA, except GSAT.In the GSAT, masses have been affected from the same mass and the diversity in the population has been lost and this has caused the stagnation of the population.This situation is balanced in other selection mechanisms by using fitness values or randomness.

Results and Discussion
In this study, we used four selection mechanisms-roulette wheel, tournament, random and rank-based selections and obtained the better results than GSA.Experimental results show that the selection mechanisms directly affect the performance of GSA because to obtain a new position for the agent in GSA is important for the performance of GSA.For the population-based heuristic approaches, information sharing and interaction between the agents describe behavior of the method and to be selected the agents using their fitness values provides to obtain high quality solutions for the numerical benchmark functions.Experimental results show that fitness-based selection mechanisms such as roulette-wheel and tournament is appropriate for the unimodal and multimodal functions with fix dimension, but for multimodal functions with huge local minimums these mechanisms have caused early saturation of the population, to lost diversification in the population and to get stuck of local minimums.Therefore, the random, rank-based selection mechanisms are more appropriate for solving the multimodal functions instead of fitness-based selection mechanisms.But it should be mentioned that the random selection mechanism can cause the slow convergence to the optimum or optimums and reduce the local search ability of the method.The effect of the selection mechanism used for generating new chromosomes is known on the Genetic algorithm in the literature but in this study, the effect of selection mechanism for the GSA has been investigated and obtained results are used for comparing and discussion.In GSA, the selection techniques do not have the same effect with the GA because velocity updating equation of GSA does not work likewise crossover of GA.The obtained results show that the proposed techniques for GSA have positive effect on solving the numerical function.Consequently, it is shown that the solution quality is improved by using selection mechanism in original GSA and, the suitable selection mechanism for GSA should be used in order to obtain more quality solution depending on structure of the optimization problem.

Conclusion and Future Works
We analyzed four selection mechanisms for the GSA on the 23 benchmark functions based on solution quality.The experimental results show that using the appropriate selection mechanism provides to obtain the quality solutions.Because GSA with the selection mechanism has high performance on the continuous optimization problems, our future works include applications of proposed method in various optimization problems.
the size of inertial mass on the ith position on iteration time t, ( ) i m t is the size of the mass which is on the ith position on iteration time t, ( ) i fit t is the fitness value of the mass on the ith position on iteration time t, best and worst fitness values in the mass population on iteration time t.

Figure 1 .
Figure 1.The flow chart of the GSA.

Table 1 .
The unimodal benchmark functions.

Table 2 .
The multimodal benchmark functions.

Table 3 .
The multimodal benchmark functions with fix dimensions.

Table 4 .
The comparison of the methods on the unimodal test functions.

Table 5 .
The comparison of the methods on multimodal test functions.

Table 6 .
The comparison of the methods on the multimodal test functions with fix dimension.