curvature flight path for particle swarm optimisation
TRANSCRIPT
Curvature Flight Path for Particle Swarm Optimisation
Cheng Wai KHENG∗
Department of ComputerScience
Universiti Tunku AbdulRahman
Jalan Universiti Bandar Barat31900, Kampar, Malaysia
Day Chyi KUFaculty of Computing and
InformaticsMultimedia UniversityPersiaran Multimedia
63100 Cyberjaya, [email protected]
Hui Fuang NGDepartment of Computer
ScienceUniversiti Tunku Abdul
RahmanJalan Universiti Bandar Barat
31900, Kampar, [email protected]
Mahmoud KhattabSchool of Computer Science
University of Nottingham,Malaysia Campus
43500 Semenyih, [email protected]
Siang Yew CHONGSchool of Computer Science
University of Nottingham,Malaysia Campus
43500 Semenyih, Malaysiasiang-
ABSTRACT
An optimisation is a process of finding maxima or minima ofthe objective function. Particle Swarm Optimisation (PSO)is a nature-inspired, meta-heuristic, black box optimisationalgorithm used to search for global minimum or maximumin the solution space. The sampling strategy in this algo-rithm mimics the flying pattern of a swarm, where each sam-ple is generated randomly according to uniform distributionamong three different locations, which marks the currentparticle location, the individual best found location, and thebest found location for the entire swam over all generation.
The PSO has known disadvantage of premature conver-gence in problems with high correlated design variables (highepistatis). However, there is limited research conducted infinding the main reason why the algorithm fails to locatebetter solutions in these problems. In this paper, we pro-pose to change the traditional triangular flight trajectory ofPSO to an elliptical flight path. The new flying method istested and compared with the traditional triangular flighttrajectory of PSO on five high epistatis benchmark prob-lems. Our results show that the samples generated from theelliptical flight path are generally better than the traditionaltriangular flight trajectory of PSO in term of average fitnessand the fitness of best found solution.
∗Corresponding Author
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full cita-tion on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from [email protected].
GECCO ’16, July 20-24, 2016, Denver, CO, USA
c© 2016 ACM. ISBN 978-1-4503-4206-3/16/07. . . $15.00
DOI: http://dx.doi.org/10.1145/2908812.2908840
CCS Concepts
•Computing methodologies→Continuous space search;Randomized search;
Keywords
Particle Swarm Optimisation; Curvature Flight Path, Ge-ometry, Multi-dimensional Ellipsoid
1. INTRODUCTIONA non-linear optimisation problem is defined with a set
of design (solution) variables x ∈ Rd, where d indicates the
number of design variables (problem dimensions), and theobjective function f : x → R evaluates the fitness of a so-lution. In the context of minimisation, the optimal solu-tion of the problem is the one with the minimum fitnessargmin
x
f(x).
Prior to 1960s, optimisation problems were solved throughnumerical analysis methodology [15] [18]. This involvedanalysis of problem characteristics (e.g. landscape proper-ties such as continuity, multimodal or unimodal) and sub-sequent application of dedicated algorithms based on theunderlying problem characteristics. However, the analysismay be time consuming and dedicated algorithms are de-veloped around problem characteristics – they may not beuseful to new problems having different underlying prob-lem characteristics. Over the last few decades and morerecently with easy access to high-performance computing,researchers could develop general purpose stochastic searchalgorithms. These algorithms are known to be problem in-dependent (as long as the objective function is provided, see[14] for more information) but may require higher computa-tional resource as compared to dedicated algorithms (accord-ing to “No Free Lunch” theorem [19]). Some of these solversare inspired by natural processes. They include population-based stochastic search algorithms such as Evolutionary Al-gorithms (EA) [11] and Particle Swarm Optimisation (PSO)algorithms [13]. Despite early success of PSO algorithms [1]
29
[2], the algorithm has several known weaknesses. First, thetraditional search trajectory in PSO is biased towards thecentre of the solution space [1]. Second, the algorithm haspremature convergence on high epistasis problems [8] (highepistasis is a metaphor from genetic testing, where the effectof one genotypic trait changes multiple phenotypic traits. Inoptimisation, it represents the variation of fitness with highcorrelation between dimensions).
We aim to address the issue of high epistasis in optimisa-tion problems by altering the traditional triangular searchtrajectory into a curvature search trajectory, which we calledcurvature flight path. The remaining of this paper is organ-ised as follows. Section 1.1 presents the basic elements ofPSO and different implementation of velocity updates forthe particles, as well as some recent development of incor-porating cultural evolution in PSO. Section 2 presents theidea of creating curvature flight path, which includes theformulation of d-dimensional ellipsoid and its parametersestimation. Section 3 discusses the experimental settingsand benchmark problems used to test our proposed method.Followed by section 4, which presents our finding and com-parisons between triangular search trajectory and curvatureflight path. This paper ends with conclusion and acknowl-edgment.
1.1 IntroductionThe PSO algorithm first defines a group of particles (or
swarm). Let X be a set of solutions (or the swarm), xi ⊂X, where xi indicates the ith solution (ith individual in theswarm). There is a second set of vectors with identical sizeas X, called V , which stores the velocity (flight direction)for each solution in X. Lastly, there is a set of vectors P ,also having identical size as X, that stores the best knownsolution for each individual xi. The g is a vector that holdsthe best solution found by the entire swarm. The swarm isinitialised as follows:
∀xi ∈ X,xi ∼ U(bl, bu),
where bl and bu are the lower bound and upper bound of theoptimisation problem. At the initial stage of the search, thebest known location for each individual is its initial location.
∀pi ∈ P,pi = xi,
The global best solution is defined as:
∀pi ∈ P, g = xi if f(xi) ≤ f(g),
The PSO has three parameters to generate new samples,the inertia ω and the weights associated in moving the solu-tion towards the individual best φp and the global best φg.To generate new samples, we first generate the new velocityfor each individual solution in X.
∀vi ∈ V,vi = ωvi + φprp(pi − xi) + φgrg(g − xi), (1)
where rp and rg are two uniform random variables between0...1. The algorithm parameters ω, φp and φg are highlydependent on the optimization problem. Past studies hadattempted to find the appropriate algorithm parameters ω,φp and φg. In [7], the parameters were estimated from thepopulation performance through a Bayesian inference on theparameters formulated as probabilities. In [16], the fitness
and Euclidean distance ratio between individuals were usedto estimate the actual parameters.
The new samples x′
i is generated by adding the velocityto the old xi
∀x′
i ∈ X,x′
i = xi + vi.
If a new sample is generated out of bound, the sample is ei-ther discarded or bounced back into the space (increased bybl or decreased by bu). The algorithm then updates the indi-vidual best in P and the global best g by evaluating all newxi, the search process continues until certain terminationcondition is reached (i.e. the maximum number of iteration,which is set by the user). Many variations of velocity andflight path updates are proposed, such as firefly algorithm[20], cuckoo search [22] and bat-inspired algorithm [21]. Thefirefly algorithm updates the flight path based on:
x′
i = xi + β0eγr2ij (xj − xi) + α(u− 0.5),
where β0 is the attractiveness of a particle, γ is the lightabsorption coefficient, u is a uniform random number be-tween 0 and 1. r is the distance between particle xi and xj .The firefly algorithm changes the traditional PSO search be-haviour. Instead of moving towards the best particle, noweach particle moves according to the attractiveness of theneighbouring particles [20]. Empirical results show that thenew firefly algorithm outperforms GA and PSO on bench-mark problems. The cuckoo search simulates the egg-layingbehaviour of cuckoo by having two set of populations [22].The first set denotes the particle, namely the cuckoo andthe second set represents the available nest. The particle incuckoo search no longer moves towards to the best solutionbut move randomly according to Levy distribution. Whena new solution is found through the random search, eachnest is given certain probability of being replaced. The bat-inspired algorithm simulates the echo location (pulse rateand loudness) of bat to update the particle velocity and foodsources [21].
The phase angle-encoded and quantum behaved particleswarm optimization algorithm was proposed in [6]. The au-thors proposed to use the wave form of light particle behav-ior in generating new samples (velocity vector v). However,there is explanation on the use of the quantum behaviorparticle in the search. Second, the particle trajectory anglemove back and forth between the global best g and indi-vidual best p, which does not guarantee continuous angle,changes over iteration.
The cultural based PSO is another branch of PSO thatsimulate the cultural influence between swarms to updatethe velocity vector [23][5][12][5] [4][3]. The research groupfrom Gary Yen et. al has extensive and complete analysisof the characteristics of cultural based PSO algorithm. Thecommon concept of these algorithms is to have multiple pop-ulations / sub swarms (instead of having only one populationin the canonical PSO) to conduct the search. The velocityvector is updated with an additional term,
∀vi ∈ V,
vi = ωvi + φprp(pi − xi)
+φgrg(g − xi) + φsrs(s− xi), (2)
where s is the individual that has the highest score basedon culture fitness (in additional to f). The cultural score is
30
−5−4
−3−2
−10
12
34
5
−5−4
−3−2
−10
12
34
5
−4
−2
0
2
4
6
8
10
12
x1x2
log
sca
led
fitn
ess f
or
Ro
se
nb
rock
Figure 1: Surface Plot for Rosenbrock (BenchmarkProblem).
−5 0 5−5
−4
−3
−2
−1
0
1
2
3
4
5
Figure 2: A Contour Plot for Rosenbrock with 30Samples Generated Based on PSO Velocity Update.
based on six different aspects: representation of knowledge,situational knowledge, temporal knowledge, domain knowl-edge, normative knowledge and spatial knowledge. The samealgorithm has been applied to different optimisation prob-lems, such as dynamic optimisation [5], Multi-objective op-timisation [5] [3], Constrained optimisation [12].
Although all the reviewed past works had attempt to alterthe search trajectory in Equation 1, the shape of the trajec-tory remains unchanged, which is a triangle plane in theEuclidean space. The high epistatis problem landscape usu-ally has a narrow and deep valley, leading towards the globaloptimal solution as depicted in Figure 1. The gradient in-formation helps in leading the search trajectory towards theglobal optimal solution. Other stochastic algorithms suchas Natural Evolution Strategies [17] and Covariance MatrixAdaptation Evolution Strategy [10] estimates the naturalgradient by finding the minimum expected fitness under thesearch distribution. Both of these algorithms work the bestin solving high epistatis problem because the covariance ma-trix allows the search distribution to adapt the correlationbetween dimensions. This motivates us to design a heuris-tics that allows the PSO to move based on these correlationswhile maintaining the derivative-free property. In order toso, we have to change the triangle plane in the traditionalPSO to a curvature flight path.
To demonstrate the triangle plane, we generate thirty and500 samples based on Equation 1, the samples are depictedin Figure 2 and 3 respectively. We generate these samplesby fixing the xi at +, pi at ∗ and g at ×.
−5 0 5−5
−4
−3
−2
−1
0
1
2
3
4
5
Figure 3: A Contour Plot for Rosenbrock with 500Samples Generated Based on PSO Velocity Update.
−5 0 5−5
−4
−3
−2
−1
0
1
2
3
4
5
Figure 4: A Contour Plot for Rosenbrock with 30Samples Generated Based on Curvature Flight Path.
2. METHODOLOGYIn this paper, we propose to alter the search trajectory
into a curvature flight path. Our intention is to plan theflight path in such a way that it aligns with the valley inthe problem landscape. We hope that this increases thechance of finding better solution in lesser iteration. Figure4 shows the intended samples generated from our proposedmethod, where samples are randomly generated based onuniform distribution along the valley. We also hope that thesaturation of coverage on the valley increases as the numberof samples increases, as depicted in Figure 5.
For doing so, we need a basis function to generate sam-ples along the curve that connects xi and g. The basisfunction must support up to d-dimensions, gradient inde-pendent and has parameters that control the curvature ofthe path (so that we can adapt it to any problem landscapethrough learning later). We have select ellipsoid to be thebasis function to conduct the empirical study in this paperbecause it fulfills all the above requirements.
2.1 d-Dimension EllipsoidThere are two parameters required in generating an d-
dimension elliptic curve, the flying angle θ and the majoraxes length m. θ is a vector that contains the angles fromthe centre of an ellipsoid c ranging from −π to π. Figure 6shows the values for multiple flight paths from the startingpoint × to the end point +, each path is generated along
31
−5 0 5−5
−4
−3
−2
−1
0
1
2
3
4
5
Figure 5: A Contour Plot for Rosenbrock with 500Samples Generated Based on Curvature Flight Path.
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1
-0.9
-0.8
-0.7
-0.6
-0.5
-0.4
-0.3
-0.2
-0.1
0
Figure 6: A Contour Plot for Multiple HorizontalAligned Ellipsoids.
the surface of a 2D ellipsoid with different m values. Fordemonstration purpose, we generated a set of fixed intervalsfor θ and randomized the values for m2 (there are only 2major axes for 2D problem and m1 is determined by theEuclidean distance between × and +). The proper selectionof parameters will be discussed in the next section. Fromthe figure, we recognised that m plays an important roleof controlling the curvature of the path. The major axesm ∈ R
d defines the shape of an ellipse (i.e. if the ratiobetween two major axes is 1:1, then, the algorithm generatesa spherical flight path instead of an elliptical one).
The two dimensional ellipse is generated with d−1 numberof angles (θ),
x = m1 cos θ1,
y = m2 sin θ1.
The equation can be generalised to three dimensions with
x = m1 cos θ1 cos θ2,
y = m2 cos θ1 sin θ2,
z = m3 sin θ2.
To generate the d-dimensional elliptic flight path, we firstalign the samples on the ellipsoid horizontally along the firstmajor axis. Let t′j be the j-th dimension of the new sample,the new sample can be generated using Equation 3, where
the centre of the ellipsoid is at 0. Next, we have to transposeand rotate the new particle w.r.t. the midpoint and theangles from xi to g. Let the linear line from xi to g forms the1st major axis, m1 is half of the Euclidean distance between
xi and g. This can be calculated with 12
√
∑d
j=1(xij − gj)2,
where xij indicates the jth dimension of ith individual in the
population. The midpoint is then calculated by c = (xi +g)/2. To rotate t′, we need to compute the rotation matrixM , which can be consolidated by rotating one dimension ata time, up to d − 1 dimensions (keeping the 1st dimensionas the base). The pseudo-code for rotation in d-dimensionalspace is depicted in Algorithm. 1.
t′1 = m1 cos θ1 cos θ2 . . . cos θd−1
t′2 = m2 cos θ1 cos θ2 . . . cos θd−2 sin θd−1
t′3 = m3 cos θ1 cos θ2 . . . cos θd−3 sin θd−2 sin θd−1
t′j = mj cos θ1 cos θ2 . . . cos θd−j sin θd−j−1 sin θd−j−2 sin θd−1
...
t′d = md sin θd−1 (3)
Algorithm 1 d-Dimension Rotation
1: function d-dimension rotation(x, g, d)2: j = d3: v = g − x
4: M = 1
5: for j ≥ 2 do6: θ = tan−1(vj , vj−1)7: aj = θ8: R = rotAngle(d, j − 1, j, θ)9: v = v ×R10: M = M ×R11: j = j − 112: end for13: return M,a14: end function
Notice that we record and return the angles in a vector aat line 7 in Algorithm. 1. This will be used in parametersetting in the next section. With the new midpoint c and
Algorithm 2 Constructing Rotation Matrix
1: function rotAngle(d, j, k, θ)2: R = 1
3: R(j, j) = cos(θ)4: R(k, k) = cos(θ)5: R(j, k) = − sin(θ)6: R(k, j) = sin(θ)7: return R8: end function
the rotation matrix M , we then can update our new sample
x′ = c+Mt
′. (4)
Figure 7 shows five different flight paths from × to + con-structed with Equation 4. For demonstration purpose, weset a fixed interval on θ1 and randomized θ2 and m. Themidpoint c is centered at [−0.488,−0.029, 0.35].
32
00.2
0.40.6
0.8
0.81
1.21.4
−0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
Figure 7: Five Rotated and Transposed EllipticFlight Paths.
2.2 Parameter Selection for θ and m
From the previous subsection, we realised that θ is a d−1dimensional vector whereas m is a d dimensional vector.We also knew that θ1 can be set randomly or with a fixedintervals between [−π, π] and m1 is always half of the Eu-clidean distance between xi and g. The remaining param-eters to be set are [θ2, θ3 . . . θd−1] and [m2,m3,m4, . . .md].These parameters can be estimated if we let the curve passesthrough the individual best pi. To do so, we need to find[θ2, θ3 . . . θd−1] from c to pi. Fortunately, we have solvedthis problem in Algorithm. 1. By passing in c and pi asthe arguments (as oppose to xi to g in previous section),we obtain the flying angle in a and subsequently assign[θ2, θ3, . . . θd−1] = [a2, a3, . . . ad−1]. [m2,m3,m4, . . .md] canbe estimated by finding the corresponding ratio of m1. Wehave to first rotate pi in a reversed order so that it alignswith the base axis (1st dimension). The reversed rotationmatrix M−1 can be found by calling Algorithm. 1 again, bypassing in g then xi.
p′
i = M−1pi − c.
Since we know that p′
i is on the curve, it must obey Equation3. In order to estimate d−1 number of parameters, we needat least d−2 linear equation. However, we only have one ref-erence point on the curve, which is p′
i. If [m1, m2, . . .md] isa vector of weight associated with each axis, we assume thatthe d− dimensional ellipse is a prolate ellipsoid of revolutionwith the weight evenly distributed to m2,m3 . . .md (all ofthem are having the equal value). We will investigate thisratio in future studies as it requires further investigation.We first calculate the t′i for the current p′
i using Equation3. Let pij denotes the j-th dimension of the i-th individualbest, our estimated parameters for m2...d is:
m2...d =
√
∑d
j=2p′2ij
1− (p′2i1/m1). (5)
Figure 8 demonstrates ten different flight paths from thesame x to g passing through a randomly generated pi, whichis marked as +. Finally, we add normal distribution whitenoise N(0, σ) to the the major axis vector m to increase the
−6
−4
−2
0
2
4
6
−6−4−20246
−5
−4
−3
−2
−1
0
1
2
3
Figure 8: Ten Rotated and Transposed EllipticFlight Paths Passing Through Reference Points
-6 -4 -2 0 2 4 6-6
-4
-2
0
2
4
6
Figure 9: Five Rotated and Transposed EllipticFlight Paths Passing Through Reference Points withWhite Noise.
sampling diversity. The final version of our curvature flightpath is shown in Figure 9 with σ = 0.05.
3. EXPERIMENTAL SETTINGWe have selected five different benchmark problems that
are known to have high epistatis to test our approach. Theseproblems are taken from CMA-ES MATLAB source codefrom https://www.lri.fr/ hansen/purecmaes.m [9]. The bench-mark problems are listed in Table 1 and they are all min-imisation problems. We set the number of dimensions tod = 10, 30 and 100 for all the benchmarking test. 100 setsof xi and g are randomly generated in a range of [-5, 5].To simulate the individual best behaviour for pi, we firstgenerate a solution randomly in [-5, 5] then apply Quasi-Newton algorithm to this solution to find the local optimumamongst the neighbouring solutions. Using these settings,we generate thirty samples with Equation 1 and 4 respec-tively. The log scaled average fitness difference (log scaledaverage fitness of curvature flight path − log scaled aver-age fitness of PSO) of these thirty samples for each test setare calculated and recorded in Figure 12, 13, 14, 15, and16 respectively, where negative values indicate that averagefitness found from curvature flight path is better than PSO.
33
Table 1: Benchmark ProblemsName Formular
FRosenbrock
∑d−1
i=1[100(xi+1 − x2
i )2 + (xi − 1)2]
FCigar x21 + 106
∑n
i=2x2i
FDiffPow
∑d
i=1|xi|
2+10i
d−1
FElli
∑d
i=1x2i × (100000
id−1 )
FSchwefel
∑d
i=1(∑i
j=1xj)
2
Number of Test
0 20 40 60 80 100 120
Lo
g S
ca
led
Fitn
ess D
iffe
ren
ce
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Figure 10: Average (Log Scaled) Fitness Differ-ence Found for 100 Random Test for 10 DimensionsFRosenbrock.
4. DISCUSSIONIn general, the proposed method (curvature flight path)
outperforms the traditional triangular search trajectory methodin all five benchmark problems. Figure 12, 13, 14, 15, and16 have shown that the average fitness found by thirty sam-ples generated with curvature flight path are usually betterthan traditional triangular search trajectory. The results areconsistent for all d = 10, 30 and 100. This is an on goingresearch, we can only show the performance of both curva-ture flight path and PSO with d = 10, d = 30 and d = 100for FRosenbrock in Figure 10, Figure 11 and Figure 12 re-spectively. We summarised the best solution found for eachtest case and recorded in Table 2. The results are highlyfavourable to our proposed method, where curvature flightpath found better solutions with an average of 82% of all thetest cases with the exception of 10 dimensions FCigar. Theresults also suggest that the proposed method performs bet-ter in higher dimensions. This is expected because the num-ber of local optima increases w.r.t d. The curvature flightpath is continuous and may pass through multiple local op-tima on its way to g. We have conducted Student’s T -test tojustify the differences between curvature flight path and tri-angular search trajectory. A two tailed pair test is conductedwith the 100 best found solutions from both methods and theP-scores are recorded in Table 2. From the results, we canconclude that there are at least 98% significant differencesbetween the curvature flight path and the triangular searchtrajectory statistically in high dimensions (>30). There isno significant difference between these two algorithm at lowdimensional problems.
Table 2: Performance Measure and Student’s T -TestResults
Name d Percentage ofCurvatureFlight PathFindingBetter
Minimum
Student’sT -TestP-Score
FRosenbrock 10 82% 0.273768FRosenbrock 30 84% 2.07E-35FRosenbrock 100 92% 1.31E-39
FCigar 10 67% 0.002117FCigar 30 71% 1.12E-11FCigar 100 73% 1.31E-39FDiffPow 10 83% 0.134903FDiffPow 30 89% 0.001415FDiffPow 100 89% 2.77E-05FElli 10 79% 2.05E-11FElli 30 88% 1.89E-23FElli 100 93% 1.18E-31
FSchwefel 10 79% 9.8E-11FSchwefel 30 89% 4.37E-10FSchwefel 100 82% 1.78E-11
Number of Test
0 20 40 60 80 100 120
Lo
g S
ca
led
Fitn
ess D
iffe
ren
ce
-2.5
-2
-1.5
-1
-0.5
0
Figure 11: Average (Log Scaled) Fitness Differ-ence Found for 100 Random Test for 30 DimensionsFRosenbrock.
5. CONCLUSIONFrom the empirical results, we conclude that the curva-
ture flight path generates better quality solutions in highepistatis problems. This helps to prevent premature con-vergence for the PSO algorithm. In the future, we wouldlike implement this new velocity update on Standard Par-ticle Swarm Optimisation (SPSO2011) and compare withCMA-ES, NES and Differential Evolution (DE). We wouldlike to investigate free-form curvature flight path, such asusing Bezier function as the basis function to generate newsamples.
Acknowledgment
This project is funded by UTAR Research Fund with theamount of RM33,200, Project Number: IPSR / RMC /UTARRF / 2015-C1 / K01 and Royal Society-Newton Mo-
34
Number of Test
0 20 40 60 80 100 120
Lo
g S
ca
led
Fitn
ess D
iffe
ren
ce
-2
-1.8
-1.6
-1.4
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
Figure 12: Average (Log Scaled) Fitness DifferenceFound for 100 Random Test for 100 DimensionsFRosenbrock.
Number of Test
0 20 40 60 80 100 120
Lo
g S
ca
led
Fitn
ess D
iffe
ren
ce
-2
-1.5
-1
-0.5
0
0.5
1
Figure 13: Average (Log Scaled) Fitness DifferenceFound for 100 Random Test for 100 DimensionsFCigar.
Number of Test
0 20 40 60 80 100 120
Lo
g S
ca
led
Fitn
ess D
iffe
ren
ce
-8
-7
-6
-5
-4
-3
-2
-1
0
Figure 14: Average (Log Scaled) Fitness DifferenceFound for 100 Random Test for 100 DimensionsFDiffPow.
bility Grant with the amount of £11181, Reference Num-ber: NI150289. The authors would like to thank MahmoudKhattab, a PhD Student from University of Nottingham
Number of Test
0 20 40 60 80 100 120
Lo
g S
ca
led
Fitn
ess D
iffe
ren
ce
-1.4
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
Figure 15: Average (Log Scaled) Fitness DifferenceFound for 100 Random Test for 100 Dimensions FElli.
Number of Test
0 20 40 60 80 100 120
Lo
g S
ca
led
Fitn
ess D
iffe
ren
ce
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
Figure 16: Average (Log Scaled) Fitness DifferenceFound for 100 Random Test for 100 DimensionsFSchwefel.
(Malaysia Campus) for carrying out the preliminary inves-tigation on curvature flight path.
6. REFERENCES[1] A. Banks, J. Vincent, and C. Anyakoha. A review of
particle swarm optimization. part i: background anddevelopment. Natural Computing, 6(4):467–484, 2007.
[2] X. Chen, Q. Gu, J. Qi, and D. Chen. Applying particleswarm optimization to pairwise testing. In ComputerSoftware and Applications Conference (COMPSAC),2010 IEEE 34th Annual, pages 107–116, July 2010.
[3] M. Daneshyari and G. G. Yen. Cultural-basedmultiobjective particle swarm optimization. IEEETransactions on Systems, Man and Cybernetics, PartB: Cybernetics, 41(2):553–567, 2011.
[4] M. Daneshyari and G. G. Yen. Cultural multiobjectivepso with sensitivity analysis for parameters usingadditive binary epsion indicator. International Journalof Computational Intelligence and Applications,10(2):121–139, 2011.
[5] M. Daneshyari and G. G. Yen. Cultural based particleswarm for dynamic optimization problems.International Journal of Systems Science,43(7):1284–1304, 2012.
35
[6] Y. F. M. Ding. Phase angle-encoded andquantum-behaved particle swarm optimization appliedto three-dimensional route planning for uav. IEEETransactions on Systems. Man and Cybernetics- PartA: Systems and Humans, 42(2):511–526, 2012.
[7] Z. X. Fengrui Zhang, Jiangshu Cao. An improvedparticle swarm optimization filtering algorithm. InInternational Conference on Communications,Circuits and Systems (ICCCAS), volume 2, pages173–177, 2013.
[8] R. Gras. How efficient are genetic algorithms to solvehigh epistasis deceptive problems? In EvolutionaryComputation, 2008. CEC 2008. (IEEE WorldCongress on Computational Intelligence). IEEECongress on, pages 242–249, June 2008.
[9] N. Hansen. Cma-es source code.https://www.lri.fr/˜hansen/purecmaes.m, 2003-2009.Accessed: 2016-01-17.
[10] N. Hansen and A. Ostermeier. Completelyderandomized self-adaptation in evolution strategies.Evol. Comput., 9(2):159–195, June 2001.
[11] K. A. D. Jong. Evolutionary Computation: A UnifiedApproach. Prentice Hall of India Private Limited,2006.
[12] A. A. Kadkol and G. G. Yen. A cultural-based particleswarm optimization framework for dynamic,constrained multi-objective optimization.International Journal of Swarm Intelligence Research,3(1):1–29, 2012.
[13] J. Kennedy and R. Eberhart. Particle swarmoptimization. In Proceeding of IEEE InternationalConference on neural networks, volume 4, pages1942–1948, 1995.
[14] G. R. Maarten Keijzer, Juan J. Merelo Guervos andM. Schoenauer. Evolving objects:a general purposeevolutionary computation library. Artificial Evolution,2310:829–888, 2002.
[15] J. Nocedal and S. Wright. Numerical Optimization.Springer-Verlag, 1999.
[16] K. V. Thanmaya Peram and C. K. Mohan.Fitness-distance-ratio based particle swarmoptimization. In Proceedings of the 2003 IEEESwarmIntelligence Symposium, volume 3, pages 174–181,2003.
[17] D. Wierstra, T. Schaul, T. Glasmachers, Y. Sun,J. Peters, and J. Schmidhuber. Natural evolutionstrategies. The Journal of Machine Learning Research,15(1):949–980, 2014.
[18] W. T. V. William H. Press, Saul A. Teukolsky andB. P. Flannery. Numerical Recipes. CambridgeUniversity Press, 2007.
[19] D. H. Wolpert and W. G. Macready. No free lunchtheorems for optimization. IEEE Transactions onEvolutionary Computation, 1(1):67–82, 1997.
[20] X.-S. Yang. Firefly algorithms for multimodaloptimization. In In Stochastic Algorithms:Foundations and Applications (SAGA 2009), pages169–178, 2009.
[21] X.-S. Yang. A new metaheuristic bat-inspiredalgorithm. In In Nature InspiredCooperative Strategiesfor Optimization (NISCO2010), pages 65–74, 2010.
[22] X.-S. Yang and S. Deb. Cuckoo search via levy flights.In In World Congress on Nature and BiologicalInspired Computing (NaBIC2009), pages 210–214,2009.
[23] G. G. Yen and Z. He. Performance metrics ensemblefor multiobjective evolutionary algorithms. IEEETransactions on Evolutionary Computation, 18(1):131– 144, 2014.
36