6Y

6 0

Computational Science Technical Note CSTN-160

Random Flights for Particle Swarm OptimisersA. V. Husselmann and K. A. Hawick

2013

Parametric Optimisation is an important problem that can be tackled with a range of bio-inspired problem spacesearch algorithms. We show how a simplified Particle Swarm Optimiser (PSO) can exploit advanced space explo-ration with Levy flights, Rayleigh flights and Cauchy flights, and we discuss hybrid variations of these. We presentimplementations of these methods and compare algorithmic convergence on several multi-modal and uni-modal testfunctions. Random flights considerably enhance the PSO and the Levy flight gives good balance between local spaceexploration and local minima avoidance. We discuss computational tradeo↵s involved in generating such flights.

Keywords: optimisation; multi-modal functions; flights; walks; swarms

BiBTeX reference:

@INPROCEEDINGS{CSTN-160,author = {A. V. Husselmann and K. A. Hawick},title = {Random Flights for Particle Swarm Optimisers},booktitle = {Proc. 12th IASTED Int. Conf. on Artificial Intelligence and Applications},year = {2013},pages = {15-22},address = {Innsbruck, Austria},month = {11-13 February},publisher = {IASTED},institution = {Computer Science, Massey University},keywords = {optimisation; multi-modal functions; flights; walks; swarms},owner = {kahawick},timestamp = {2012.08.24}

}

This is a early preprint of a Technical Note that may have been published elsewhere. Please cite using the informationprovided. Comments or quries to:

Prof Ken Hawick, Computer Science, Massey University, Albany, North Shore 102-904, Auckland, New Zealand.Complete List available at: http://www.massey.ac.nz/~kahawick/cstn

Technical Report CSTN-160

Random Flights for Particle Swarm Optimisers

A.V. Husselmann and K.A. HawickComputer Science, Massey University,

Albany, North Shore 102-904, Auckland, New Zealand{a.v.husselmann, k.a.hawick}@massey.ac.nzTel: +64 9 414 0800 Fax: +64 9 441 8181

September 2012

ABSTRACT

Parametric Optimisation is an important problem that canbe tackled with a range of bio-inspired search algorithms.We show how a simplified Particle Swarm Optimiser (PSO)can efficiently exploit advanced space exploration using Levyflights, Rayleigh flights and Cauchy flights, and we discusshybrid variations of these. We present implementations ofthese methods and compare algorithmic convergence on sev-eral multi-modal and uni-modal test functions. Random flightsconsiderably enhance the efficient simplified PSO and theLevy flight gives good balance between local space explo-ration and local minima avoidance. We also discuss the com-putational tradeoffs involved in generating such flights. Insummary, these modifications show varying success betweenthemselves for problem solving, but outperforms the uniformrandom exploration technique in most cases.

KEY WORDSoptimisation; multi-modal functions; flights; walks; swarms.

1 Introduction

The general problem of searching for a global maximumwithin a problem space [1] is made specific by the problemof parametric optimisation (PO) in the domain of multi-modalfunctions. This problem remains an important one in manyapplications including: economics [2–4]; manufacturing pro-cess improvement [5]; scheduling problems [6]; image com-pression [7]; cryptanalysis [8]; object clustering and recogni-tion [9]; and engineering [10–13].Successfully deployed techniques for addressing the problemdomain of PO include Linear Programming (LP) [14], GeneticAlgorithms (GA) [15], Simulated Annealing (SA) [16], andParticle Swarm Optimisation (PSO) [17], among others thatcould be better classified as metaheuristics. There are also

Figure 1: A comparison of the random walks typically gener-ated by a uniform random walk, Brownian motion (Rayleighflights) and Levy flights. Particles released at the origin areshown taking 10,000 random steps.

relatively recent uses of population-based and biologically in-spired algorithms such as ant-related and various foraging ap-proaches [18–21] and other evolutionary algorithms [22,23] totackle these problems.Our focus revolves around the Particle Swarm Optimiser(PSO) [17, 24, 25] approach. This algorithm has undergoneimpressive research effort, which has resulted in a number ofvariations. Some are very specialised, including some that arefully hybrid methods [26, 27]. Arguably, this initiative wasspear-headed by Kennedy and Eberhart in their early work,which described an lBest PSO as well as the gBest variety [24].The algorithm itself introduces a random step in tandem withan inertial movement towards a superior solution vector in

1

parameter-space. Much consideration has been given to themechanics of the movement itself. There has been recenteffort in upgrading the PSO with more effective search be-haviour [28, 29] using Levy flights.Levy flights [30] are random walks with a long-tailed prob-ability distribution of step-lengths that gives rise to a mixof long trajectories and shorter random movements. Levyflights therefore have quite different properties than the Brow-nian random movements that are commonly found in diffusivemodels. The Levy distribution is part of the stable distribu-tions, as are the Gaussian and the Cauchy distributions [31].When the random step-length is taken from a Gaussian distri-bution or a Cauchy distribution, these give rise to the Rayleighflights and Cauchy flights respectively. These distributionshave slight variations on the frequency of longer step sizes.In solving parametric optimisation problems, one seeks an al-gorithm that is both intrinsically fast, and carries with it goodconvergence properties. In the past these algorithms werederived from agent-based modelling techniques [24, 32, 33],which in turn, were derived mostly from biological phenom-ena, such as flocking behaviour of animals inspired by workssuch as that of Reynolds [34]. This trend has continued forsome time [35]. As with many other meta-heuristic optimisa-tion algorithms, there are a great many variations to the PSOalgorithm, so much so that it is now commonly referred to asthe basic, or canonical PSO.A specific complication which has received much attention inthe literature is the problem of local minima and multi-modalfunctions. This is a ubiquitous problem which has been thesubject of much research [36]. We seek to evaluate the re-sponse of one algorithm named the Many Optimizing Liaisons(MOL) PSO [37] against issues like these. The MOL is suit-ably less computationally expensive than other PSO variants,especially in terms of memory use. Our contribution is to ex-amine the effects of Levy flights on this simplified PSO andreport on the use of Cauchy and Rayleigh flights as well.Our article is structured as follows. Section 2 contains somebackground on the PSO, its variants, and the variant we usein this article. In Section 3 we discuss the random flights wetested, and show their implementations. We then present con-vergence results in Section 4, and following this, we discussand present conclusions in Sections 5 and 6.

2 Background

In this section we give some background to particle basedoptimisation algorithms. In the following we refer to posi-tions and velocities in an n-dimensional problem space, whereeach dimension represents a parameter, and collectively, an n-dimensional vector will represent a candidate solution with n

parameters. It should be emphasised that these are not physi-cal spaces and that the pseudo-velocities we mention represent

the movement of candidate solutions through parameter space.The basic original particle swarm optimisation (PSO) algo-rithm by Kennedy and Eberhart [17,24] operates by maintain-ing a population of particles each with a solution vector and aninertia vector. These can be thought of as a position and veloc-ity vector, as this is the most intuitive way of visualising theexecution of this algorithm, at least in two and three dimen-sions. The particle positions (or solution vectors) are updatedby a pair of formulae defining a recurrence relation for eachframe. These formulae are shown in Eqns. 1 and 2.

vi+1 = !vi + �prp(pi � xi) + �grg(g � xi) (1)

xi+1 = xi + vi (2)

The first term of Eq. 1 is the inertia term, where 0 < ! <

1 represents the weight. The next two terms involve relativevectors to the best global solution found so far (g) and the bestsolution found by particle i (pi). These serve to provide acombination of local search and convergence. The values �p

and �g are the weights for pursuing their respective solutions,and they typically also range from 0 to 1, and finally, rp and rg

are traditionally uniform random deviates between the valuesof 0 and 1.Many variations of this algorithm already exist, from smallvariations to hybridisation and meta-optimisation [27, 37]. Inthis paper we opt to use a simplified PSO algorithm namedMany Optimizing Liaisons (MOL) which was proposed byPedersen and Chipperfield in 2009 [37]. By setting the param-eter �p = 0, MOL removes the need for an extra random de-viate and most importantly, eliminates the need for a previous-best solution for every particle. Pedersen and Chipperfieldnote that this simplification results in an algorithm with com-parable results to the original, as well as improvement in somecases. Perhaps the best result from this modification is the factthat having parametric optimisation algorithms with very fewparameters to fine tune is a very desirable attribute [37].In its own right, tuning a meta-heuristic algorithm to performreasonably well to solve a problem is actually optimisation it-self. In fact, this problem has received much attention underthe heading of “meta-optimisation” - the act of optimising anoptimisation algorithm [37]. Some specialised meta-heuristicalgorithms may give better results, but the mere fact of requir-ing the expert tuning of a vast array of parameters effectivelyoff-loads some of the optimisation work to the operator.An effective mechanism for evaluating and comparing meta-heuristics against others is to test the algorithms against avariety of standardised test functions. Some of the wellknown test functions used in the literature are: Rosenbrock,de Jong (uni-modal); Ackley’s Path function, Rastrigin func-tion, the Schwefel function, the Griewangk function, and theMichalewicz function (multi-modal) [38–40], and the resultsfrom these are given in Section 4.

2

These test functions were specifically designed to address theissue of evaluating the performance of the many metaheuristicoptimisation algorithms and facilitating comparison betweenthem to some degree. Unfortunately it is still difficult to ob-jectively compare these algorithms due to the large variety oftest methods, none of which can be assumed to be represen-tative of every single optimisation problem these algorithmsmay be used on. A great amount of research effort is stillbeing expended on this standardisation problem, with somesuccess [22]. The best we can offer is a comparison study ofour algorithms across a relatively small selection of these. Wealso hold our algorithm parameters static across all tests, of-ten to the detriment of performance and quality of the solutionfound. Among other factors, the scale and spatial positionsof local optima alone demand a mandatory parameter tuningeffort.For each of the functions above, we restrict the parameterspace in all dimensions to absolute numerical values between�2 and 2. This allows us to truncate highly multi-modal testfunctions, and allow us to sample performance across an easyto difficult range of problems, some highly multi-modal, andsome almost uni-modal. Local minima become a considerableproblem when this parameter space is large enough [39].Functions such as the Schwefel function have a customarybound of �500 to 500 in every dimension, which makes it ex-tremely difficult to optimise. This is due to the large number oflocal minima that plague the parameter space. In this scenario,a good course of action is often to increase the number of par-ticles, but this causes significant performance problems. Wehave recently mitigated this problem by the use of graphicalprocessing units [35]. In this paper, we focus on the randomvariables and some simplification to improve the basic conver-gence speed properties of the PSO.

3 Random Flights and the MOL PSO

Our specific implementation of the MOL PSO is shown in Al-gorithm 1. The values obtained from functions which are eval-uated at a point in the parameter space are referred to as fit-nesses.We have implemented a few other options for the random vari-able in this algorithm, and these are explained in detail in Sec-tion 3. We extensively tested this algorithm against a varietyof test functions, and the results from these are shown in Sec-tion 4.Apart from implementing the MOL variant of the PSO al-gorithm, we have also implemented Rayleigh flights, Levyflights, and also Cauchy flights. Another small variation wemake use of is to replace the rg scalar value with a vector ofsize d to carry a random deviate for every dimension. Thisdoes however add to the computational cost of the algorithm.One can attain a qualitative understanding of the different ran-

Algorithm 1 The MOL PSO algorithm.initialise xi where i = 1..n with d uniform random valueseachwith upper and lower bounds u and l.initialise vector g with random locations in all d dimensionsevaluate the fitness of vector gfor i 0 to in�1

docalculate the fitness of vector xi

end forfor i 0 to in�1

dorg U [0, 1)

vi !vi + �grg(g � x)

ensure velocity is within boundsxi xi + vi

ensure position vector is within boundsend for

dom flight algorithms by considering the resulting randomwalk patterns they generate. Brownian motion or diffusioncomes about from what is essentially a normal or Gaussian dis-tribution of random steps. The generalisation to a Levy flightcomes from changing the probability density function (PDF)to allow a long tail. Practically this means that the particle willmill around relatively locally for a while just as it does witha Gaussian distribution of step sizes, but that occasionally theLevy long tailed PDF allows a sudden jump to a different po-sition.In his Firefly algorithm, Yang considers using a normaliseduniform-random direction, with a magnitude given by a singleLevy deviate. In our empirical studies we experienced bettercovergence results by simply using a linearly-scaled Levy de-viate in every dimension. Conceptually, rather than having alarge increase in all parameters (all dimensions), a single par-ticle’s trajectory may make a sudden large variance in a singledimension, not necessarily all. Although, this method does callfor more computation in obtaining more Levy deviates. This iswhat causes the apparent large jumps along the principle axes,as explained next.A sample random walk for Brownian motion, Levy flights anda uniform distribution are shown in Fig. 1. Cauchy flightsare not visually distinctive from the Levy random walk. Thediagram shows 10,000 random steps in (x, y) starting at theorigin, taken by each of the techniques. As can be seen, theGaussian distribution gives rise to what is usually known as aRayleigh flight and the particle stays relatively local, diffusingaround its origin. The Levy flight however shows the charac-teristic series of sudden jumps to a new position around eachof which it briefly diffuses locally.The implementation of the Levy flights involve choosing a uni-form random direction, and then obtaining a magnitude froma Levy deviate. We later discuss the implications of simplyusing a Levy deviate for every dimension, rather than a uni-

3

formly random direction and a Levy magnitude.The algorithm we use to generate deviates from a Levy distri-bution is shown in Algorithm 2 [31, 41]. The c parameter isknown as the scale and controls the width of the distribution,and the ↵ parameter is known as the exponent and controls theshape and tail.Given an ↵ value of 1, the distribution reduces to the Cauchydistribution, and with ↵ = 2, a Gaussian distribution with� = c

p2 [42]. Apart from other characteristics, as the dis-

tribution varies from Gaussian to Cauchy, the tail probabilitiesvary from light to heavy. Unless otherwise stated, we experi-ment with ↵ = 1.5 for an intermediate Levy distribution andflight.

Algorithm 2 Algorithm for generating a Levy deviate.procedure levydeviate(c,↵) begindouble u, vu = ⇡ ⇤ (U(0, 1]� 0.5)

{When ↵ = 1, the distribution simplifies to Cauchy}if ↵ == 1 then

return (c tanu)

end ifv = 0

while v == 0 dov = � log(U(0, 1])

end while{When ↵ = 2, the distribution defaults to Gaussian}if ↵ == 2 then

return (2cpv sin(u))

end if{The following is the general Levy case}return c sin(↵u)

cos(u)1/↵(cos(u(1� ↵))/v)

(1�↵)/↵

end

In Algorithm 2, U is a conventional uniform random devi-ate generator [43, 44]. The algorithm shown consumes twouniform deviates to produce a Levy deviate, although optimi-sations can be made for the special Cauchy case [31]. Thetrigonometrical and power functions typically add consider-ably to the cost of generating these deviates even on modernCPUs.

4 Convergence Results

We tested each of our random exploration implementationsagainst a variety of functions, shown in Table 4. Each casewas averaged over 100 runs, and each run would terminate af-ter executing 3,000 frames.The parameters we used for these simulations are as follows:

! = 0.9,� = 0.3, ✓ = 2 (3)

Rayleigh CauchySchwefel 8D �1400± 200 �1500± 200

�1400± 200 �1400± 200

Ackley 32D 3.7± 0.7 3.9± 0.7

3.7± 0.7 4± 0.7

De Jong 64D 0.05± 0.04 0.024± 0.004

0.026± 0.003 0.104± 0.007

Rastrigin 8D 4.2± 1.9 4.7± 1.9

4.1± 2 4± 1.8

Rosenbrock 4D 0.5± 1.3 0.2± 0.9

0.3± 1 0.4± 1.2

Griewangk 3D 4.5± 2.8 5± 3.4

5.3± 3.6 4.9± 3.4

Michalewicz 5D �4.5± 0.2 �4.5± 0.2

�4.6± 0.1 �4.5± 0.2

Levy OriginalSchwefel 8D �1400± 200 �1400± 210

�1400± 200

Ackley 32D 3.7± 0.7 4.3± 0.7

3.9± 0.8

De Jong 64D 0± 0 3± 1.4

0.074± 0.005

Rastrigin 8D 4.3± 1.6 3.9± 1.6

4.2± 2.2

Rosenbrock 4D 0.1± 0.7 0.5± 1.2

0.3± 1

Griewangk 3D 5.4± 3.3 5± 3.7

5.7± 4

Michalewicz 5D �4.6± 0.2 �4.5± 0.3

�4.5± 0.2

Table 1: Various space exploration technique fitness resultswith the simplified MOL PSO (3,000 frames each, averagedover 100 runs each) accompanied by standard deviations. Lowvalues denote faster solution. The first line corresponding toeach test function is the algorithm making use of an alpha-step with a uniform-random direction and a magnitude givenby either a Rayleigh, Cauchy or Levy flight. The second lineindicates the results obtained from using a Rayleigh, Cauchyor Levy deviate in every dimension.

Lower Upper OptimumRosenbrock -2 2 0Rastrigin -5 5 0Schwefel -500 500 -3351.8632Ackley -20 20 0Griewangk -600 600 0Michalewicz 0 ⇡/2 -4.687

Table 2: The boundaries imposed on the particles in the opti-miser for each test function.

4

� is the parameter used for randomising the pursuit of theglobal best solution, ! is the inertia weight of the velocity vec-tor, and ✓ is the parameter used as the c parameter in the Levydistribution. The results shown in the table are the final valueobtained from the optimisers. In most cases, the best possiblefunction value was 0, except in the case of the Schwefel andMichalewicz functions which have optimum function valuesof �3351.8632 and �4.687 respectively. As these results areaveraged over 100 independent runs, these can be taken as agood indication of reliability. The boundaries of each of thetest functions are given in Tab. 4.The choice of our parameters largely result from empiricaltesting, and while we concede that these may not be optimal inall cases, this allows us to more objectively compare the varia-tions of MOL PSO. The population size in all simulations was50.The results shown in Tab. 4 shows that the Levy flight methodprovides equal or better results than the original MOL PSO.This appears to be similar to the results of Richer’s Levy PSO[28]; albeit with some algorithmic differences, there appears tobe improvement over the standard versions of the optimisers.The other random steps fail to clearly distinguish themselvesfrom the Levy flights, however, they generally do not performas well. The values given in the tables are in the form of x± y

where x is the average fitness result from generated solutionvectors, and y is the standard deviation from this. Each testfunction also has a suffix nD where n is the number of dimen-sions for the generalised test functions.In our selected test functions, we have a reasonable spectrumof problem difficulty. The boundaries we imposed on the parti-cles reflect that of the standard bounds used with each of thesefunctions. Large ranges present considerable difficulty, espe-cially for simulations with low numbers of particles.In our testing, we also considered how each alpha-step typebehaves with a direction chosen by uniform-random deviates,which are then normalised, and then a single Rayleigh, Cauchyor Levy deviate is used as a magnitude. It is very expensive tocompute random deviates from these three distributions, there-fore, it is worth investigating how they respond in this respect.Our empirical observations indicate that there is a considerabledifference in computation time, but in this article, we focus onproviding convergence results.While the non-uniform direction choice provides a slight ad-vantage in optimising the Rosenbrock function, this result ismore expensive to obtain. We believe this may be associatedwith the variable dependence of the Rosenbrock function.We have also experimented with a visualisation techniquefor this algorithm. Fig. 2 shows a 3D rendering of an n-dimensional solution vectorWe have also experimented with a visualisation technique forthis algorithm. Fig. 2 shows a 3D rendering of an n-dimensional solution vector, and Fig. 3 shows visualisations in

higher dimensions with a full population of solution vectors.We found this visualisation to be useful as a tool to observecharacteristic behaviour, especially that of the optimum solu-tion, where the lines are too short to be seen. This clearly givesan indication of fitness in a particular solution vector. Whenviewed amongst other solution vectors, it becomes easier todetermine how the algorithm is behaving in higher dimensions.We found this visualisation to be of assistance when tuning analgorithm by hand. An auxiliary advantage to this is that theglobal optimum has a characteristic shape.

5 Discussion

The different flight patterns largely involve different randomsteps through the pseudo space defined by the optimisationfunction parameters. An interesting question to consider iswhy these different step types actually help. This is really onlya kind of exploration vs exploitation balancing technique [28],an area which has received quite a bit of attention, especiallyin the PSO literature, where the two � values are commonlyreferred to as the exploration and exploitation parameters. Theprobability density functions of these deviates are quite similarvisually, although as we have seen the flight paths generatedare quite startlingly different. The Levy path is quite dramati-cally different from the Rayleigh path with the long tail in thePDF contributing to highly non-local jumps.A uniform random step in the flight by its nature is not spatiallybiased and essentially loses any valuable information about theparticle’s present position in solution space. A Rayleigh flightis strongly centred on the present position and the particle canthoroughly explore its locality but is unlikely to be able to leapout of local minima. Our experiments and empirical observa-tions lead us to believe that the Levy distribution encourages abetter balance between properly exploring the local neighbour-hood prior to leaping or jumping to a new region of solutionspace without becoming stuck. By looking at Fig. 1 and thedifference between the Rayleigh and Levy flights, it becomesmore clear why this is the case. Along with this technique,another design question presents itself. One can choose to usea Levy deviate for every dimension, or use a uniform randomdirection and a magnitude from a Levy deviate. In our obser-vation, a variable-dependent function such as the Rosenbrockfunction is more easily optimised by the latter, than the former.The reason for this we believe lies in the tendency of the lat-ter to present large movements which affect most dimensions,rather than just one independent parameter.We have obtained an indication that random flights such asLevy helps in avoiding local minima (as shown by the Rastri-gin function which presents many local minima in the boundsspecified), but the problem is still prominent, however. Hav-ing a variation in step length (given by the approximation ofthe Levy Distribution) seems to help the search for the global

5

Figure 2: A in-simulation real-time visualisation of a single 64-dimension solution vector.

minimum shake off stagnation brought on by getting stuck inlocal minima. Some authors report that the basic original PSOis very susceptible to being caught in local minima. It appearstherefore that even this little advanced exploration techniquehelps significantly. We have experimented with only a fewof the possible variations and hybrids of these flights. Thereis considerable scope for systematically exploring the use ofmultivariate distributions to help balance the exploitation ver-sus exploration tradeoff.There is also a significant tradeoff in choosing which flightpattern to use based on which sort of random deviates, be-cause as shown, the computational cost of a more sophisti-cated path such as a Levy flight involves evaluation of expen-sive trigonometrical and power evaluation functions. Even onmodern processing systems these functions are still expensive.We typically want to add more particles to the optimizer andgenerating expensive random deviates for each particle addsconsiderably to the prefactor in the computational complexity.Given the cost of sin, cos and exponentiation, this can add tohundreds of CPU clock cycles; this prefactor is significant.

6 Conclusion

We have explored how various random walks or flights can bebe used in particle swarm optimization for various uni-modaland multi-modal test functions. We have considered in par-ticular how more sophisticated probability density functions

give rise to flights such as a Levy flights with a mix of bothlocalised space exploration through diffusion as well as largerjumps in the problem space that avoid (to some degree) be-coming stuck in local minima.Although problems in parametric optimisation vary enor-mously in nature and it is not to be expected that a singlemethod will be optimal for all, we have seen that the Levyflight fares quite well across a range of test problem functions.We have visualised the flights in a simple demonstration of thequalitative nature of the flight, as well as multi vector render-ings of a high dimensional particle trajectory. The dynamic na-ture of these renderings when viewed interactively gives somegood visual indications of how well an algorithm is generat-ing good particle trajectories converging towards a solution inproblem space.Parametric optimisation is a problem of ongoing importancein many fields. Particle swarm optimisation is a useful plat-form to tackle problems of this nature and there is continuedscope for hybrids of the algorithms we have discussed. Ran-dom flights appear to help PSO algorithms considerably andthe interplay between exploration and exploitation of the prob-lem space is an important underlying issue deserving furtherresearch.We have noted the computational expense of generating so-phisticated random deviates and the fact that this adds consid-erably to the computational cost of deploying more particlesin a given search. One approach is to seek an approximationto the Levy distribution that involves fewer or cheaper trigono-

6

Figure 3: A visualisation of a population of 50 high-dimensional solution vectors for the de Jong test function in 128 dimensionsand 256 dimensions.

metric and power function evaluations.Parallel computing techniques might also be usefully appliedto accelerate the generation of flight trajectories.

References

[1] Shlesinger, M.F.: Search research. Nature 44 (2006)281–282

[2] Yang, X.S., Hosseini, S.S.S., Gandomi, A.H.: Firefly al-gorithm for solving non-convex economic dispatch prob-lems with valve loading effect. Applied Soft Computing12 (2012) 1180–1186

[3] Giannakouris, G., Dounias, V.V.G.: Experimental studyon a hybrid nature-inspired algorithm for financial port-folio optimization. SETN 2010, LNAI 6040 1 (2010)101–111

[4] Apostolopoulos, Vlachos: Application of the firefly al-gorithm for solving the economic emissions load dis-patch problem. International Journal of Combinatorics1 (2011)

[5] Aungkulanon, P., Chai-ead, N., Luangpaiboon, P.: Sim-ulated manufacturing process improvement via particleswarm optimisation and firefly algorithms. Prof. Int.

Multiconference of Engineers and Computer Scientists2 (2011) 1123–1128

[6] Khadwilard, A., Chansombat, S., Thepphakorn, T., Tha-patsuwan, P., Chainate, W., Pongcharoen, P.: Applicationof firefly algorithm and its parameter setting for job shopscheduling. First Symposium on Hands-On Research andDevelopment 1 (2011)

[7] Horng, M.H.: Vector quantization using the firefly al-gorithm for image compression. Expert Systems withApplications 38 (2011)

[8] Palit, S., Sinha, S.N., Molla, M.A., Khanra, A., Kule,M.: A cryptanalytic attack on the knapsack cryptosystemusing binary firefly algorithm. Int. Conf. on Computerand Communication Technology (ICCCT) 2 (2011) 428–432

[9] Senthilnath, J., Omkar, S.N., Mani, V.: Clustering usingfirefly algorithm: Performance study. Swarm and Evolu-tionary Computation (2011)

[10] Azad, S.K., Azad, S.K.: Optimum design of structuresusing an improved firefly algorithm. International Jour-nal of Optimization in Civil Engineering 1 (2011) 327–340

7

[11] Gandomi, A.H., Yang, X.S.: Mixed variable structuraloptimization using firefly algorithm. Computers andStructures 89 (2011) 2325–2336

[12] Chatterjee, A., Mahanti, G.K., Chatterjee, A.: Design ofa fully digital controlled reconfigurable switched beamcoconcentric ring array antenna using firefly and particleswarm optimization algorithms. Progress in Electromag-netic Research B (2012) 113–131

[13] Basu, B., Mahanti, G.K.: Firefly and artificial beescolony algorithm for synthesis of scanned and broadsidelinear array antenna. Progress in Electromagnetic Re-search 32 (2011) 169–190

[14] Kantorovich, L.V.: A new method of solving someclasses of extremal problems. Doklady Akad. Sci. USSR28 (1940) 211–214

[15] Holland, J.H.: Adaptation in natural and artificial sys-tems. Ann Arbor: University of Michigan Press (1975)

[16] Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimizationby simulated annealing. Science 220 (1983) 671–680

[17] Kennedy, J., Eberhart, R.C.: Particle swarm optimiza-tion. In: Proc. IEEE Int. Conf. on Neural Networks, Pis-cataway, NJ. USA (1995) 1942–1948

[18] Brown, C.T., Liebovitch, L.S., Glendon, R.: Levy flightsin dobe ju/’hoansi foraging patterns. Hum. Ecol. 35(2007) 129–138

[19] Gazi, V., Passino, K.M.: Stability analysis of social for-aging swarms. IEEE Trans. on Systems, Man and Cyber-netics - Part B 34 (2004) 539–557

[20] Passino, K.M.: Biomimicry of bacterial foraging for dis-tributed optimization and control. IEEE Control SystemsMagazine June (2002) 52–67

[21] Chen, L., Chen, B., Chen, Y.: Image feature selec-tion based on ant colony optimization. In: Proc. 24thAustralasian Joint Conference on Artificial Intelligence,Perth, Asutralia (December 2011) 580–589

[22] Shilane, D., Martikainen, J., Dudoit, S., Ovaska, S.J.: Ageneral framework for statistical performance compari-son of evolutionary computation algorithms. In: Proc.IASTED Int. Conf. on Artificial Intelligence and Appli-cations, Innsbruck, Austria, ACTA (2006) 7–12

[23] Mandal, A., Zafar, H., Das, S., Vasilakos, A.: A modi-fied differential evolution algorithm for shaped beam lin-ear array antenna design. Progress in ElectromagneticsResearch 125 (2012) 439–457

[24] Eberhart, R.C., Kennedy, J.: A new optimizer using par-ticle swarm theory. In: Proc. Sixth Int. Symp. on Mi-cromachine and Human Science, Nagoya, Japan (1995)39–43

[25] Shi, Y., Eberhart, R.: A modified particle swarm op-timizer. In: Evolutionary Computation Proceedings.(1998)

[26] Parsopoulos, K.E., Vrahatis, M.N.: Recent approachesto global optimization problems through particle swarmoptimization. Natural Computing 1 (2002) 235–306

[27] Fu, W., Johnston, M., Zhang, M.: Hybrid particle swarmoptimisation algorithms based on differential evolutionand local search. In: Proc. AI 2010. Number 6464 inLNAI (2010) 313–322

[28] Richer, T.J.: The levy particle swarm. In: IEEE Congresson Evolutionary Computation. (2006)

[29] Huang, G., Long, Y., Li, J.: Levy flight search patternsin particle swarm optimization. In: Seventh InternationalConference on Natural Computation. (2011)

[30] Mandelbrot, B.B.: The Fractal Geometry of Nature.W.H. Freeman (1982)

[31] Chambers, J.M., Mallows, C.L., Stuck, B.W.: A methodfor simulating stable random variables. Journal of theAmerican Statistical Association 71 (1976) 340–344

[32] Husselmann, A.V., Hawick, K.A.: Spatial agent-basedmodelling and simulations - a review. Technical Re-port CSTN-153, Computer Science, Massey University,Albany, North Shore,102-904, Auckland, New Zealand(October 2011) In Proc. IIMS Postgraduate Student Con-ference, October 2011.

[33] Husselmann, A., Hawick, K.: Simulating species in-teractions and complex emergence in multiple flocks ofboids with gpus. In T. Gonzalez, ed.: Proc. IASTEDInternational Conference on Parallel and DistributedComputing and Systems (PDCS 2011), Dallas, USA,IASTED (14-16 Dec 2011) 100–107

[34] Reynolds, C.: Flocks, herds and schools: A distributedbehavioral model. In Maureen C. Stone, ed.: SIGRAPH’87: Proc. 14th Annual Conf. on Computer Graphicsand Interactive Techniques, ACM (1987) 25–34 ISBN 0-89791-227-6.

[35] Husselmann, A.V., Hawick, K.A.: Parallel paramet-ric optimisation with firefly algorithms on graphical pro-cessing units. In: Proc. Int. Conf. on Genetic and Evo-lutionary Methods (GEM’12). Number CSTN-141, LasVegas, USA, CSREA (16-19 July 2012) 77–83

8

[36] Liang, J.J., Suganthan, P.N.: Dynamic multi-swarm par-ticle swarm optimizer with local search. In: EvolutionaryComputation. (2005)

[37] Pedersen, M.E.H., Chipperfield, A.J.: Simplifying par-ticle swarm optimization. Applied Soft Computing 10(2009) 618–628

[38] Rosenbrock, H.H.: An automatic method for finding thegreatest or least value of a function. The Computer Jour-nal 3 (1960) 175–184

[39] Molga, M., Smutnicki, C.: Test functions for optimiza-tion needs. Technical report, Univ. Wroclaw, Poland(2005)

[40] Yang, X.S.: Test problems in optimization. In: Engi-neering Optimization: An Introduction with Metaheuris-tic Applications. Wiley (2010)

[41] Bratley, P., Fox, B.L., Schrage, L.E.: A Guide to Simu-lation. Springer Verlag (1983)

[42] Galassi, M., Gough, B., Jungman, G.: GNU ScientificLibrary Reference Manual. Network Theory Limited(2009)

[43] Preez, V.D., B.Johnson, M.G., Leist, A., Hawick, K.A.:Performance and quality of random number generators.In: International Conference on Foundations of Com-puter Science (FCS’11). Number FCS4818, Las Vegas,USA, CSREA (18-21 July 2011) 16–21

[44] Coddington, P., Mathew, J., Hawick, K.: Interfaces andimplementations of random number generators for javagrande applications. In: Proc. High Performance Com-puting and Networks (HPCN), Europe 99, Amsterdam.(April 1999)

9