Comparative Research on Particle Swarm Optimization and Genetic Algorithm

Genetic algorithm (GA) is a kind of method to simulate the natural evolvement process to search the optimal solution, and the algorithm can be evolved by four operations including coding, selecting, crossing and variation. The particle swarm optimization (PSO) is a kind of optimization tool based on iteration, and the particle has not only global searching ability, but also memory ability, and it can be convergent directionally. By analyzing and comparing two kinds of important swarm intelligent algorithm, the selecting operation in GA has the character of directivity, and the comparison experiment of two kinds of algorithm is designed in the article, and the simulation result shows that the GA has strong ability of global searching, and the convergence speed of PSO is very quick without too many parameters, and could achieve good global searching ability.


Introduction
Generally, single natural biology is not intelligent, but the whole biotic swarm can have the ability to treat complex problems.The swarm intelligence (Kennedy, 2001& Peng, 2003, P.1982-1988& Jiang, 2009, P.26-28 & Tarasewich P, 2002, P.62-67) is a method which simulates the natural biotic swarm to realize artificial intelligence, and it emphasizes the simple character of individual behavior, the avalanche character of swarm, and the research strategy from bottom to top.The swarm intelligence has shown better optimization in the existed application domains, so the extensive attentions have been produced in relative domains.At present, the research about swarm intelligence is still in the initial stage, so it has large development potential.Some swarm intelligent algorithms such as GA and PSO all belong to the category of evolvement computation, so they have many similar aspects in principle mechanism.In this article, GA and PSO will be deeply studied, and the principal mechanism, flow implementation and optimization result of these two algorithms will be analyzed and compared in detail.GA (Ge, 2008, P.2911-2916& Zhang, 2003) was proposed by professor Holland of American Michigan University in 1975 and gradually developed based on Darwin' theory of evoluation.This method is a kind of random optimization algorithm to simulate natural selection and genetic mechanism, and it only needs the objective function to be the optimization information, and find the optimal individual by the continual selection, crossing, and aberrance of initial swarms, and it has strong global optimization ability and the character to treat discrete variables.But GA has low computation efficiency and slow optimization speed.PSO was proposed by Dr. Kennedy and Eberhart in 1995 under the elicitation of artificial life research result when they simulated the migration and swarm behaviors in simulation of bird swarm, and it can optimize many complex problems such as nonlinear problems, non-differentiable problems and multiple-peak problems (Kennedy, 1995, P.1942-1948& Parsopoulos, 2002, P.235-306 & Duan, 2005& Grosan C, 2005, P.623-632).The individual of particle swarm presents the possible solution of a problem.Each particle has two description variables including position and speed, and the corresponding objective function value of the particle position coordinate can be regarded as the fitness degree of this particle.PSO measures the quality of particle by the fitness degree, so one significant character is that its parameter quantity is less, but the setup of these key parameters will obviously influence the precision and efficiency of the algorithm.Comparing with GA, the information sharing mechanism of PSO is very different.In GA, chromosomes share information with each other, so the transfer of the whole swarm evenly moves to the optimal region.In PSO, only the optimal particle will transfer the information to other particles, which is single-directional information flow.The whole searching updating process is to follow the optimal solution, and compared with GA, all particles may more quickly be convergent to the optimal solution under most situations.

Basic principle
When solving problems by GA, the solution of the problem can be denoted as chromosome, i.e. the string of binary coding in the algorithm, and many chromosomes can be generated by the random mode to form the initial swarm.Then these chromosomes are put in the environment of problem, and the chromosomes which can better adapt the environment according to the principle of survival of the fittest can be selected and copied, and new-generation chromosome swarm which can more adapt the environment by crossing and aberrance.In each generation swarm, each individual can be evaluated by one numerical value by the fitness function, so the individual with high fitness can join in the inheritance operation and the individual with low fitness will be eliminated.In this way, more generations can evolve, and finally one chromosome which can adapt the environment most will be convergent, and it is the optimal solution of the problem.That is the basic principle of GA.

Algorithm flow
The running parameters of basic GA include N, T, Pc and Pm, which are related with the dimension amount of the variable and the geometric character of the function.Commonly, N is the size of the swarm, i.e. the amount of individual in the swarm, T is the terminal evolvement generation of GA, Pc is the crossing probability, and Pm is the aberrance probability.These four running parameters will influence the solution result and the solving efficient of GA.The concrete flow of GA can be described as follows.
(1) Code and generate initial swarm.Because the disposal objective of GA is the chromosome coding string, and GA can not directly dispose the solution data in the solution space, and the solution space must be denoted as the genetic string structured data of genetic space by the mode of coding.The basic GA uses binary symbol string with fixed length can be used to denote the individuals in the swarm, and its allele (genes in same gene position) is composed by binary symbol set {0, 1}.The gene value or individual of various individuals in initial swarm can be generated by the evenly distributed random values.Aiming at two kinds of usual coding of GA, the advantages and disadvantages of real number coding and binary coding can be combined to code.The implementation of independent variable in real number coding and the operation of binary coding are convenient, so the binary-real number combined coding is used in the article.
(2) Compute the fitness value of each individual in swarm.In the searching process, GA generally needs not other exterior information, and it only uses the fitness function to evaluate the each individual's fitness ability to the environment, and regards it as the reference of heritance operation.The standard GA adopts roulette-wheel-selection mechanism which decides the opportunity degree that each individual in the present swarm is inherited to next generation swarm according to the probability of the positive proportion with the individual fitness.Because the probability that each individual can be copied should be computed, so the fitness of individual must be kept as positive number or zero.Especially, when the objective function is negative, the conversion rule between the objective function to individual fitness must the confirmed beforehand.
(3) Select next generation individual according to certain rule decided by individual fitness value.The essential of selection is the copy of chromosome, and it is the most important cause to keep the properties and achieve the stability of species.In GA, the function of selection is like a boult, and according to the fitness of individual to the objective function, the individual chromosome with high fitness can be selected, so its gene can be inherited and copied to next generation, but the individual chromosome with low fitness can be eliminated.So the essential of selection is filtration, and the function means directional evolvement.By multiple directional accumulations of selection operators, individuals in the swarm will quickly close to the region with high value of objective function, and form the swarm of high-quality solution.The implementation method is to use the roulette-wheel-selection mechanism, i.e. selecting the operator according to the proportion.
The roulette-wheel method is an evaluation method of directional selection based on probability.Suppose that the fitness values of individual in swarm respectively are f(i) (i∈(1, 2, …, pop size)), and the chip in the roulette-wheel is c(i), if i>begin.
Where, "begin" is the subscript of the particle selected last time, and the fitness and the selected probability in the roulette-wheel method is Where, p(i) denotes the probability that the i'th individual is selected, and it obviously seems the probability that the individual with big fitness value can be selected is also higher.When the popsize time roulette-wheel is executed, the father swarm of next generation can be generated.The essential of selection is the copy of chromosome, and it is the most important cause to keep the properties and achieve the stability of species.One important stage in the roulette-wheel method is to compute the fitness of each particle after each selection in the roulette-wheel method.
(4) Implement the crossing operation according to the probability Pc.The essential of crossing operation in GA is that the son-generation individual can be obtained from father-generation individual in small range by large-probability aberrance, and this range is decided by two individuals though "or" or "and" operation.The crossing operator is the core of the GA operation, and it includes two basic contents, and one is to randomly match the individual in the matching base formed by the selection operation, and decide whether each pair needs the crossing operation according to the pre-established crossing probability Pc.And the other is to set up the crossing point of individual to exchange part structures before and after these crossing points.The random method is used in the whole process, and it will produce the crossing individuals, i.e. continually generating random number temp∈[0,1] in the circulation, and if temp is in the interval of [0, Pc], this individual is the matching individual.So the size of Pc is the aberrance probability.The random method produces matching individuals, match=floor(rand*i), match∈[0,i], i.e. it is crossed with former individuals, and this way can simplify the algorithm, and it would not the exchange quality by testing.The random method produces the exchange bit, n=ceil(codel*D*rand), and the single-point crossing method is adopted.
(5) Implement the aberrance operation according to the probability Pc.The introduction of the aberrance operator is to help the swarm to search the whole solution space.In the evolvement process of algorithm, the aberrance plays very important function to recover the diversity of swarm and ensure the global constringency of the algorithm.The aberrance operator means gene value on some gene positions in individual chromosome coding string can be substituted by other alleles on this gene position, and new individual will be generated.The aberrance is the change of son-generation gene generated according to small-probability interference.The intention is to add the diversity of swarm, promote the performance of algorithm, and avoid the immature convergence.The operation uses the basic bit aberrance operator or the even aberrance operator, and it is implemented according to the bit, i.e. the values of certain bit are exchanged.When the aberrance happens to certain gene bit, the potential of new same-order mode will exist.Therefore, the aberrance is also the important part of GA to implement the global searching.Aiming at the strong aberrance of GA, to increase the global searching ability, the dynamic aberrance factor can be adopted because the GA is difficult to be convergent, which makes the aberrance factor to be gradually reduced with the increase of iterative times, and avoid the deficiency of global optimization solution induced by too many mutations.To be similar with crossing operator, the aberrance operator decides the aberrance probability, and the implementation method is similar with the crossing algorithm.The dynamic binary aberrance factor is adopted in the article, because with the maturation of the evolvement of particle swarm, to reduce the aberrance degree, the good individuals should be saved to avoid the loss of global optimal solution.With the running of each evolvement, the experience function of Pm with the change of the evolvement time is Where, P m0 is the initial value, K is the evolvement time, Pc is the crossing factor, and G is the maximum iterative time.
(6) If certain stop condition has not been satisfied, turn to the step (2), or else, enter into the step (7).
(7) Output the chromosome with optimal fitness in swarm as the satisfactory solution or the optimal solution of the problem.

Basic principle
PSO originated from the simulation of bird swarm, and it is one of main algorithm in the domain of swarm intelligence research (Kennedy, 1995, P.1942-1948& Parsopoulos, 2002, P.235-306).similar with other evolvement algorithms, PSO moves the individuals in the swarm to better region according to the fitness to the environment, and the difference is that it doesn't use the evolvement operator to the individual like other evolvement algorithms, but regard the potential solution of each optimization problem as a "particle" without volume and quality in the d dimensional searching space, and all particles have one their own fitness value decided by the objective function, and one speed to decide the flying direction and distance, so particles can fly in the solution space following the current optimal particle, and dynamically adjust the flying speed according to the experiences of individual and collectivity, and finally find the approximately optimal solution by the searching iterative.Concrete mathematical description can be described as follows.
Suppose the i'th particle in the swarm is ) ,..., , ( , and its position is ) ,..., , ( , and the best position is pBest.The best position that all particles go through in the swarm is gBest.The speed of the particle i can be denoted as ) ,..., , ( . For the iteration each time, the updating formulas about the speed and position of the particle i in d (1≤d≤D) dimensional space are Where, ω is the inertia weight, and its range is [0.1, 0.9], and it can keep the inertia of the particle to make it to explore new region, and the updating of ω accords with the formula (7), and the maximum inertia weight max ω descends to min ω with the iteration linearly, and max iter is the maxim iterative time established for the algorithm, and iter is the present iterative time.c and 2 c are acceleration constants, and they make each particle to move to the positions of pBest and Gbest more quickly.Rand is the random number in [0, 1].In addition, the speed of particle i V is limited by the maximum speed max V which decides the searching precision of the particle in the solution space.The formula (8) and the formula (9) show the searching processes of the positions of pBest and gBest.Present is the present position of the particle, and the position with the best fitness is appointed as the best position pBest of this particle, so it is appointed as the global optimal position gBest.
The terminal condition of PSO generally is to achieve the maximum iterative time or search the optimal fitness value to satisfy the threshold condition.

Algorithm flow
The concrete flow of PSO is shown in Figure 4, and there are many parameters to be adjusted.
(1) The particle swarm scale, i.e. the particle amount N (swarm size): The amount is generally in 20-40, and 10 particles can acquire good result, and for the complex problem or special problems, and amount of particle can be 100 or 200, and the amount of particle in the searching space each time in the article is 80 to keep consistent with GA and control the comparison condition possibly.
(2) The length of particle D (space dimension): It is decided by the testing problem, and it is the length of the problem solution.The testing function in the article is Schaffer function, and the length of this function D is 2.
(3) The coordinate range of particle: It is decided by the optimization problem, and each dimension can establish different ranges such as [-2.0, 2.0].
(4) The maximum speed V max : It decides the maximum movement distance of the particle in one circulation, and it is set as the range width of the particle.
(6) Inertia factor ω: To add inertia into initial PSO is to seek the optimal balance between global optimization and local optimization, and big inertia can help to search the global optimization, and small inertia can help to search the local optimization.The researches show that with the increase of iterative time, the inertia descending gradually in [1.4, 0] can obtain good effect.In this article, ω'=ω*u -k (ω∈ [0,1]; u∈[1.0001,1.005]), ω is the inertia weight, and the representative initial value is in [0, 1], and big ω is propitious to global searching, and small ω is propitious to local searching.
(7) Terminal conditions: The conditions include the maximum circulation times and the minimum warp requirement, and the maximum circulation time is the terminal condition in the article.

Experiment result and analysis
According to above two algorithms, corresponding program is developed in the article, and the performances of GA and PSO are studied and researched by the representative testing function Schaffer.Following factors such as easy implementation, good testing effect and variable exchangeability are mainly considered, and closely related with the characters of GA and PSO.Two functions all code variables in one matrix, so variables have same running tendency and numeric area.The maximum of 3D Schaffer function is 1, and the global maximum point is at [0, 0].Schaffer is the testing function because there are infinite local extreme values around the global optimal values.The intensive shocking character and the above character of the Schaffer function make general algorithm to be difficult to find the global optimization of this function.Therefore, this function can be used to test the influence of parameters on the optimization performance of algorithm in PSO.The expression of the Schaffer testing function is the formula (10), and its curved surface of 3D function is shown in Figure 1.
To ensure that the testing result can objectively reflect the characters of two algorithms as much as possibly, both irrelative parameters should be controlled strictly, such as same iterative time (MaxDT), same searching dimension (D), and same searching range [-2, 2].Both common maximum iterative time MaxDT=500, and the variable dimension D=2, and the particle or individual amount of one time iteration size=80.For the PSO algorithm, c 1 =c 2 =2, ω= 0.7298.For the GA, pr=0.6, pc=0.8,pm0=0.01.From Figure 2, the algorithm of PSO first enters into the convergence state.From the particle distribution of Figure 3 and Figure 4, the distribution range of PSO particle is centralized like the nebula state, but the distribution of GA is discrete without obvious convergence.As viewed from the principle of algorithm, the cause is that the PSO is the algorithm of directional convergence, and new particles all will compare with global values and historical values to more effectively confirm the selection direction. .When obtain the maximum, the average iterative time of PSO is less, and the searching effect is better.

Conclusions
In this article, two important swarm intelligence algorithms including GA and PSO are deeply analyzed and compared.GA is mainly evolved by the form of swarm through four operations including coding, selection, crossing and aberrance to seek the optimal solution.Only the selection operation in four operations has the character of direction, and other operation operators are mainly to ensure the diversity of swarm.So the selection operation is selected as the core to implement and promote GA.The simulation experiment result of two algorithms shows that GA has high global searching ability, but the computation efficiency is low and the optimization speed is slow, and it is hard to be convergent, and the complexity of algorithm is higher.PSO can be implemented simply, and the convergence speed is quick without too many parameters, and it has good global searching ability, because the information of particle is single-directional, each particle would remember the past position, and the convergence is very quick, but at the same time, it is easily to fall into local optima.At present, the selection operation in GA is mainly to apply the probability mechanism without the judgment of convergence condition and the selection process of memory, which is the combination point of GA and PSO.

Figure 2 .
Figure 2. Comparison of Iterative Processes of GA and PSO

Figure 4 .
Figure 4. Chromosome Convergence Result of PSO

Table 1
includes the data of comparison experiments ten times of GA and PSO.