engineering optimization (an introduction with metaheuristic applications) || firefly algorithm

CHAPTER 17

FIREFLY ALGORITHM

17.1 BEHAVIOUR OF FIREFLIES

The flashing light of fireflies is an amazing sight in the summer sky in the tropical and temperate regions. There are about two thousand firefly species, and most fireflies produce short and rhythmic flashes. The pattern of flashes is often unique for a particular species. The flashing light is produced by a process of bioluminescence, and the true functions of such signaling systems are still being debated. However, two fundamental functions of such flashes are to attract mating partners (communication), and to attract potential prey. In addition, flashing may also serve as a protective warning mechanism to remind potential predators of the bitter taste of fireflies.

The rhythmic flash, the rate of flashing and the amount of time form part of the signal system that brings both sexes together. Females respond to a male's unique pattern of flashing in the same species, while in some species such as Photuris, female fireflies can eavesdrop on the bioluminescent courtship signals and even mimic the mating flashing pattern of other species so as to lure and eat the male fireflies who may mistake the flashes as a potential suitable mate.

Engineering Optimization: An Introduction with Metaheuristic Applications. 2 2 1 By Xin-She Yang Copyright © 2010 John Wiley & Sons, Inc.

222 CHAPTER 17. FIREFLY ALGORITHM

Some tropic fireflies can even synchronize their flashes, thus forming emerging biological self-organized behavior.

We know that the light intensity at a particular distance r from the light source obeys the inverse square law. That is to say, the light intensity I decreases as the distance r increases in terms of / ex 1/r2. Furthermore, the air absorbs light which becomes weaker and weaker as the distance increases. These two combined factors make most fireflies visual to a limit distance, usually several hundred meters at night, which is good enough for fireflies to communicate.

The flashing light can be formulated in such a way that it is associated with the objective function to be optimized, which makes it possible to formulate new optimization algorithms. In the rest of this chapter, we will first outline the basic formulation of the Firefly Algorithm (FA) and then discuss the implementation in detail.

17.2 FIREFLY-INSPIRED ALGORITHM

17.2.1 Firefly Algorithm

Now we can idealize some of the flashing characteristics of fireflies so as to develop firefly-inspired algorithms. For simplicity in describing our new Fire-fire Algorithm (FA) which was developed by Xin-She Yang at Cambridge University in 2007, we now use the following three idealized rules:

• All fireflies are unisex so that one firefly will be attracted to other fireflies regardless of their sex;

• Attractiveness is proportional to the their brightness, thus for any two flashing fireflies, the less brighter one will move towards the brighter one. The attractiveness is proportional to the brightness and they both decrease as their distance increases. If there is no brighter one than a particular firefly, it will move randomly;

• The brightness of a firefly is affected or determined by the landscape of the objective function.

For a maximization problem, the brightness can simply be proportional to the value of the objective function. Other forms of brightness can be defined in a similar way to the fitness function in genetic algorithms.

Based on these three rules, the basic steps of the firefly algorithm (FA) can be summarized as the pseudo code shown in Figure 17.1.

17.2.2 Light Intensity and Attractiveness

In the firefly algorithm, there are two important issues: the variation of light intensity and formulation of the attractiveness. For simplicity, we can always

17.2 FIREFLY-INSPIRED ALGORITHM 223

Firefly Algorithm

Objective function f(x), x = (xi, ...,Xd)T

Generate initial population of fireflies Xi (i = 1,2, ...,n) Light intensity Ii at x» is determined by f(xi) Define light absorption coefficient 7 wh i l e (t <MaxGeneration) for i = 1 : n all n fireflies

for j = 1 : n all n fireflies (inner loop) i f (Ii < Ij), Move firefly i towards j; end i f

Vary attractiveness with distance r via exp[—77·] Evaluate new solutions and update light intensity

end for j end for i Rank the fireflies and find the current global best gt

end while Postprocess results and visualization

Figure 17.1: Pseudo code of the firefly algorithm (FA).

assume that the attractiveness of a firefly is determined by its brightness which in turn is associated with the encoded objective function.

In the simplest case for maximum optimization problems, the brightness / of a firefly at a particular location x can be chosen as I(x) oc f(x). However, the attractiveness ß is relative, it should be seen in the eyes of the beholder or judged by the other fireflies. Thus, it will vary with the distance r^ between firefly i and firefly j . In addition, light intensity decreases with the distance from its source, and light is also absorbed in the media, so we should allow the attractiveness to vary with the degree of absorption.

In the simplest form, the light intensity I(r) varies according to the inverse square law

I(r) = £ , (17.1)

where Is is the intensity at the source. For a given medium with a fixed light absorption coefficient 7, the light intensity I varies with the distance r. That is

/ - Joe"7", (17.2)

where IQ is the original light intensity. In order to avoid the singularity at r = 0 in the expression Is/r2, the combined effect of both the inverse square law and absorption can be approximated as the following Gaussian form

I(r)=IQe-v*. (17.3)


As a firefly's attractiveness is proportional to the light intensity seen by adjacent fireflies, we can now define the attractiveness ß of a firefly by

ß = ßoe-~>r\ (17.4)

where ßo is the attractiveness at r = 0. As it is often faster to calculate 1/(1 + r2) than an exponential function, the above function, if necessary, can conveniently be approximated as

/ 3 = τ ^ 4 · (17.5) 1 + 77"=

Both (17.4) and (17.5) define a characteristic distance Γ = l/^/γ over which the attractiveness changes significantly from ßo to ßoe-1 for equation (17.4) or ßo/2 for equation (17.5).

In the actual implementation, the attractiveness function ß(r) can be any monotonically decreasing functions such as the following generalized form

ß(r)=ß0e^Tm, ( m > l ) . (17.6)

For a fixed 7, the characteristic length becomes

r = 7 _ 1 / m - » l , m - > o o . (17.7)

Conversely, for a given length scale Γ in an optimization problem, the parameter 7 can be used as a typical initial value. That is

7 = τ τ - · (17·8)

The distance between any two fireflies i and j at x, and Xj, respectively, is the Cartesian distance

Xi . Y^{xiM-xjM)2, (17.9) \ fc=i

where Xitk is the fcth component of the spatial coordinate Xi of ith firefly. In 2-D case, we have

Tu = ^{xi - xi)2 + (vi - yj)2- (17.10) The movement of a firefly i is attracted to another more attractive (brighter)

firefly j is determined by

Xi = Xi + ß0e~~<Tii(xj - Xi) + a ej, (17-H)

where the second term is due to the attraction. The third term is randomization with a being the randomization parameter, and e, is a vector of random numbers drawn from a Gaussian distribution or uniform distribution. For

17.2 FIREFLY-INSPIRED ALGORITHM 225

example, the simplest form is e\ and can be replaced by rand — 1/2 where rand is a random number generator uniformly distributed in [0,1], For most of our implementation, we can take ßo = 1 and a € [0,1].

It is worth pointing out that (17.11) is a random walk biased towards the brighter fireflies. If ßo = 0, it becomes a simple random walk. Furthermore, the randomization term can easily be extended to other distributions such as Levy flights.

The parameter 7 now characterizes the variation of the attractiveness, and its value is crucially important in determining the speed of the convergence and how the FA algorithm behaves. In theory, 7 € [0, 00), but in practice, 7 = O(l) is determined by the characteristic length Γ of the system to be optimized. Thus, in most application, it typically varies from 0.1 to 10.

17.2.3 Scaling and Global Optima

It is worth pointing out that the distance r defined above is not limited to the Euclidean distance. We can define other distance r in the n-dimensional hyperspace, depending on the type of problem of our interest. For example, for job scheduling problems, r can be defined as the time lag or time interval. For complicated networks such as the Internet and social networks, the distance r can be defined as the combination of the degree of local clustering and the average proximity of vertices. In fact, any measure that can effectively characterize the quantities of interest in the optimization problem can be used as the 'distance' r.

The typical scale Γ should be associated with the scale concerned in our optimization problem. If Γ is the typical scale for a given optimization problem, for a very large number of fireflies n 2> m where m is the number of local optima, then the initial locations of these n fireflies should distribute relatively uniformly over the entire search space. As the iterations proceed, the fireflies would converge into all the local optima (including the global ones). By comparing the best solutions among all these optima, the global optima can easily be achieved. Our recent research suggests that it is possible to prove that the firefly algorithm will approach global optima when n —► 00 and ί > 1 . In reality, it converges very quickly and this will be demonstrated later in this chapter.

17.2.4 Two Special Cases

There are two important limiting or asymptotic cases when 7 —> 0 and 7 —► 00. For 7 —> 0, the attractiveness is constant ß = ßo and Γ —> 00, this is equivalent to say that the light intensity does not decrease in an idealized sky. Thus, a flashing firefly can be seen anywhere in the domain. Thus, a single (usually global) optima can easily be reached. If we remove the inner loop for j in Figure 17.1 and replace Xj by the current global best gt, then the Firefly Algorithm becomes the special case of accelerated particle swarm optimization


Figure 17.2: Landscape of a function with two equal global maxima.

(PSO) discussed earlier. Subsequently, the efficiency of this special case is the same as that of PSO.

On the other hand, the limiting case y t rn leads to I? + 0 and P(r) + S(r) which is the Dirac delta function, which means that the attractiveness is almost zero in the sight of other fireflies. This is equivalent to the case where the fireflies randomly roam in a very thick fog region. No other fireflies can be seen, and each firefly roams in a completely random way. Therefore, this corresponds to the completely random search method.

As the firefly algorithm is usually in the case between these two extremes, it is possible to adjust the parameters y and a so that it can outperform both the random search and PSO. In fact, FA can find the global optima as well as the local optima simultaneously and effectively. This advantage will be demonstrated in detail later in the implementation.

A further advantage of FA is that different fireflies can work almost in- dependently, it is thus particularly suitable for parallel implementation. It is even better than genetic algorithms and PSO because fireflies aggregate more closely around each optimum. It can be expected that the interactions between different subregions are minimal in parallel implementation.

17.3 IMPLEMENTATION

17.3.1 Multiple Global Optima

In order to demonstrate how the firefly algorithm works, we have implemented it in MatlabIOctave. The program is given in Appendix B where we have used

17.3 IMPLEMENTATION 227

Figure 17.3: The initial locations of 25 fireflies (left) and their final locations after 20 iterations (right).

a simple function

f(x, y) = (\x\ + \y\) exp[-O.O6250r2 + y2)\, (17.12)

which has four equal peaks at ( - 2 , - 2 ) , ( -2 ,2) , (2 , -2) and (2,2). This function can easily extended to any higher dimensions.

In order to show that both the global optima and local optima can be found simultaneously, we now use the following four-peak function

f(x,y) = e-(*-4)2-(y-i)2+e-(*+*f-(y-*)2+2le-x2-y2+e-x2-(y+V\ (17.13)

where (x, y) € [—5,5] x [—5,5]. This function has four peaks. Two local peaks with / = 1 at (-4,4) and (4,4), and two global peaks with / m a x = 2 at (0,0) and (0, —4), as shown in Figure 17.2. We can see that all these four optima can be found using 25 fireflies in about 20 generations (see Figure 17.3). So the total number of function evaluations is about 500. This is much more efficient than most of existing metaheuristic algorithms.

In the implementation, the values of the parameters are a — 0.2, 7 = 1 and ßo = 1. Obviously, these parameters can be adjusted to suit for solving various problems with different scales.

17.3.2 Multimodal Functions

Now let us use the FA to find the optima of some tougher test functions such as Ackley's d-dimension function

f{x) = -20exp \

1 1

- Σ A } - exP [^ Σ cos(2™i)} + 20 + e, (17.14) 2=1 i=l


Figure 17.4: Landscape of Ackley's 2D function with the global minimum 0 at (0,O).

which has the global minimum f, = 0 at (0,0, ..., 0) in the domain of -32.768 5 xi 5 32.768 where i = 1,2, ..., d. For example, the 2D case of Ackley's function is shown in Figure 17.4.

As this is a minimization problem, so we have to slightly modify the Matlab program of the implemented Firefly Algorithm in Appendix B. The simplest change is to use Ii > Ij instead of Ii < Ij. Figure 17.5 shows the initial locations of the 25 fireflies and their final locations at the 20th iteration.

17.3.3 FA Variants

The basic firefly algorithm is very efficient, but we can see that the solutions are still changing as the optima are approaching. It is possible to improve the solution quality by reducing the randomness.

A further improvement on the convergence of the algorithm is to vary the randomization parameter a so that it decreases gradually as the optima are approaching. For example, we can use

a = a, + ( a o - a,)e-t, (17.15)

where t E [0, t,,] is the pseudo time for simulations and t,,, is the maximum number of generations. a g is the initial randomization parameter while a, is the final value. We can also use the similar function to the geometrical annealing schedule. That is

a = a g e t , (17.16)

where 6 E ( O , l ] is the randomness reduction constant. In addition, in the current version of the FA algorithm, we do not explicitly

use the current global best g,, even though we only use it to decode the final

EXERCISES 229

Figure 17.5: The initial locations of the 25 fireflies (left) and their final locations after 20 iterations (right).

best solutions. Our recent studies show that the efficiency may significantly improve if we add an extra term Xei(xi —gt) to the updating formula (17.11). Here λ is a parameter similar to a and β, and e» is a set of random numbers. These could form important topics for further research.

EXERCISES

17.1 Schubert's function

5 5

f(x) = l·^ i cos ( i + (i + l)xj · ^ i cos ( i + (i + l)y) ,

in the 2D domain —10 < x, y < 10 is multimodal, and it often serves as a tough test function. Modify the Matlab code in Appendix B or write your own computer program to find all the optima. Can your program find all the optima at the same time? If not, what modification is needed to achieve this? How many fireflies are needed.

17.2 Use the implemented FA code, try to use different methods such as (17.15) and (17.16) to reduce the randomness gradually. Observe if the rate of convergence and the quality of the solution improve?

17.3 The mobility of fireflies in the standard Firefly Algorithm is represented by the Gaussian and/or uniform distribution, which are relatively efficient. For some problems, especially those with significant differences in parameter scales, others probability distributions such as Levy distribution in terms of Levy flights may be more efficient. This is true for many meta-heuristic algorithms including PSO. Modify the Matlab code to incorporate the Levy flights or other distributions with a long tail.


REFERENCES

1. S. Lukasik and S. Zak, "Firefly algorithm for continuous constrained optimization tasks", ICCCI 2009, Lecture Notes in Artificial Intelligence (Eds. N. T. Ngugen, R. Kowalczyk and S.-M. Chen), 5796, 97-106 (2009).

2. S. M. Lewis and C. K. Cratsley, "Flash signal evolution, mate choice, and predation in fireflies", Annual Review of Entomology, 53, 293-321 (2008).

3. C. O'Toole, Firefly Encyclopedia of Insects and Spiders, Firefly Books Ltd, 2002.

4. X. S. Yang, Nature-Inspired Metaheuristic Algorithms, Luniver Press, 2008.

5. X. S. Yang, "Firefly algorithms for multimodal optimization", in: Stochastic Algorithms: Foundations and Applications, SAGA 2009, Lecture Notes in Computer Science, 5792, 169-178 (2009).

6. X. S. Yang, "Firefly algorithm, Levy flights and global optimization", in: Re-search and Development in Intelligent Systems XXVI, (Eds M. Bramer et al. ), Springer, London, pp. 209-218 (2010).

7. X. S. Yang, "Firefly algorithm, stochastic test functions and design optimization", Int. J. Bio-inspired Computation, 2, 78-84 (2010).

8. X. S. Yang and S. Deb, "Engineering optimization by cuckoo search", Int. J. Math. Modeling Num. Opt, 1, No. 4, (in press) (2010).

engineering optimization (an introduction with metaheuristic applications) || firefly algorithm

Documents