A Comparison of Computational Efforts betweenParticle Swarm Optimization and Genetic Algorithm

for Identification of Fuzzy ModelsArun Khosla Shakti Kumar Kumar Rahul Ghosh

Department of Electronics and Society for Education and Research Aricent Inc.Communication Engineering Jagadhari - 135003. India Gurgaon - 122015. India

National Institute of Technology shaktikkgmail.com gkumarrahulkgmail.comJalandhar - 144011. India

khoslaak(knitj.ac.in

Abstract - Fuzzy systems are rule-based systems that provide a This paper is structured as follows. Section II reviews theframework for representing and processing information in a fuzzy model structures and the various issues associated withway that resembles human communication and reasoning the fuzzy model identification process. Basic knowledgeprocess. Fuzzy modeling or fuzzy model identification is an about the fuzzy logic, fuzzy sets and fuzzy inference systemarduous task, demanding the identification of many parameters is assumed. The framework for fuzzy model identificationthat can be viewed as an optimization process. Evolutionary through evolutionary algorithms (EAs) is presented inalgorithms are well suited to the problem of fuzzy modelingbecause they are able to search complex and high dimensional Section III. Brief information about the rapid Ni-Cd batterysearch space while being able to avoid local minima (or charger is provided in Section IV. The simulation resultsmaxima). The Particle Swarm Optimization (PSO) algorithm, focusing on the computation requirements for PSO and GAlike other evolutionary algorithms, is a stochastic technique for the identification of fuzzy models are presented in Sectionbased on the metaphor of social interaction. PSO is similar to V and finally, the conclusions are drawn in Section VI.the Genetic Algorithm (GA) as these two evolutionary heuristicsare population-based search methods. The main objective of this II. FUZZY MODELS AND FUZZY MODELpaper is to present the tremendous savings in computational IDENTIFICATIONPROCESSefforts that can be achieved through the use of PSO algorithm incomparison to GA, when used for the identification of fuzzymodels from the available input-output data. For realistic The commonly used fuzzy models are [2]:comparison, the training data, models complexity and some * Mamdani fuzzy modelsother common parameters that influence the computational * Takagi-Sugeno fuzzy modelsefforts considerably are not changed. The real data from the * Singleton fuzzy modelsrapid Nickel-Cadmium (Ni-Cd) battery charger developed hasbeen used for the purpose of illustration and simulation

purposes. ~~~~~~~~~~~InMamdani models, each fuzzy rule iS of the form:Ri: If xl is Ail and... and x, is Ai, then y is B

I. INTRODUCTION In Takagi-Sugeno models, each fuzzy rule is of the form:n

Developing models of complex real-systems is an important R1: If x1 is Ail and... and Xn is Ai then y is 1 +topic in many fields of science and engineering. Models are whereas, in Singleton models, each fuzzy rule is of the form:generally used for simulation, identifying the systems Ri: If x1 is Ail and... and xn is Ainthen y is Cbehavior and design of controllers etc.

where,The principles of fuzzy modeling were outlined by Zadeh x,,..., xn are the input variables and y is the output variable,when he gave the concept of grade of membership and AIl,..., Ain, B are the linguistic values of the input and outputpublished his seminal paper on fuzzy sets [1] that lead to the variables in the i-th fuzzy rule and ai and C are constants.birth of fuzzy logic technology. This concept represents one Infact, Singleton fuzzy model can be seen as a special case ofof the most significant scientific paradigms and provides an Takagi-Sugeno model when ai=O. The input and outputapproximate yet effective means of describing the behaviour variables take their values in their respective universes ofof systems that are too complex or ill-defined to admit use of discourse or domains. The universe of discourse or simplymathematical analysis. Fuzzy modeling techniques allow universe is the working range of a variableextracting and interpreting the knowledge contained in themodel and at the same time take into account a priori The fuzzy model identification process involves the questionknowledge and experience, which is often imprecise and of providing a methodology for development i.e. a set ofqualitative in nature. techniques for obtaining the fuzzy model from information

1-4244-1214-5/07/$25.00 ©B2007 IEEE 245

and knowledge about the system. Generally there are two the error between the desired output and actual output can befuzzy modeling approaches viz. expert or knowledge-driven minimized. Many real-world problems can be translated intoand data-driven. Expert-driven or heuristic-based modeling optimization ones and the design of fuzzy models from theapproach follows Zadeh's idea by trying to build a fuzzy available data is not an exception. The design of fuzzy systemmodel that directly takes advantage of expert's domain or fuzzy model identification can be formulated as a searchknowledge. Unfortunately, there is no general methodology and optimization problem in high-dimensional space, wherefor the implementation of this modeling approach, which is each point corresponds to a fuzzy system i.e. representsmore an art of intuition and experience. This modeling membership functions, rule base and hence the correspondingapproach becomes difficult, when the available knowledge is system behaviour. Given some objective/fitness function, theincomplete or when the problem space is very large. Even system performance forms a hypersurface and designing thethough this design methodology has led to a large number of optimal fuzzy system is equivalent to finding the optimalsuccessful applications, it is time-consuming and subjected to location on this hypersurface. The hypersurface is generallycriticism for its lack of principles and systematic found to be infinitely large, nondifferentiable, complex,methodologies. On the other hand, for the data-driven noisy, multimodal and deceptive [4].modeling approach, no prior knowledge of the system underconsideration is available to formulate the rules. This These characteristics of hypersurface make evolutionarymodeling approach makes use of numeric information algorithms good candidates for searching the hypersurfaceobtained from input-output measurements. It is also possible than the traditional gradient-based methods. Evolutionaryto integrate both the modeling approaches. In this paper, the algorithms have the capability to find optimal or near optimalattention is focused on building fuzzy models from the solution in a given complex search space and can be used toavailable data. modify/learn the parameters of fuzzy model. The idea of

fuzzy model identification through evolutionary algorithm isThe problem of fuzzy model identification includes the illustrated in Figure 2.following issues [2][3]:

* Selecting the type of fuzzy model An optimization problem can be represented as a tuple ofthree components and explained below:

* Selecting input and output variables for the model* Choosing the structure of membership functions * Solution Space - The first step in the optimization* Determining the number of fuzzy rules step is to pick up the variables to be optimized and

define the domain/range in which to search for the* Identifying the parameters of antecedent and optimal solution.

consequent membership functions * Constraints - It is required to define a set of* Identifying the consequent parameters of rules constraints which must be followed by the solutions.

Solutions not satisfying constraints are invalid* Defining some performance criteria for evaluating solutions.

fuzzy models * Fitness/Objective Function - The fitness/objectivefunction represents the quality of each solution and

These issues can be grouped into three subproblems: structure also provides a link between the optimizationidentification, parameter estimation, and model validation as alsoridesathe bl ender optio n

shown in Figure 1. If the performance of the model obtained algorithm and the problem under consideration.is not satisfactory, the model structure is modified and theparameters are re-estimated till the performance iS Thobetvofpimzin prbe is tolo ohsatisfaetory. are re-estimated till theperformanceis values of the variables being optimized that satisfy thesatisfactory. defined constraints, which maximizes or minimizes the

111. FRAMEWORK FOR IDENTIFICATION OF fitness function. Hence, it is required to define the solution

FUZZY MODELS THROUGH EVOLUTIONARY space, constraints and the fitness function when usingALGORITHMS evolutionary algorithms for the identification of optimized

fuzzy models.

Optimization plays an important role in every field and acommon problem which falls under optimization is modelfitting, where the goal is to find the model parameters so that

246

Linguisticand/or Structure Parameter YsNumeric Idenitflication Estimaition ModelM ValidaitionSaifidInformation

No

Figure 1. Fuzzy Model Identification Process

(ILarnng roug vl:u Jonary Agortm X

Fiiuzy Mlodel Identification Process

Inputs Futiy Mode utputs

Figure 2. Fuzzy Model Identification Process through EA

In this paper, Mean Square Error (MSE) defined in Eq. (1) is not the drive of survival that breeds quality solutions,has been used as fitness/objective function for rating the rather the individuals strive to improve themselves byfuzzy model. imitating traits from their successful peers. In PSO, each

particle also has a memory and hence it is capable of(]2 remembering the best position in the search-space ever

MSE= z > [y(k) - (1) visited by it. The position corresponding to the best fitness isk=1 known as pbest and the overall best out of all the particles in

where, the population is called gbest.y(k) - desired outputy (k) - actual output of the model Suppose that the search-space is d-dimensional and i-thZ - number of data points taken for model validation particle in the swarm can be represented by Xi= (xi, Xi2,x

xid) and its velocity can be represented by another d-A very important consideration is to completely represent a dimensional vector V, = (vil, vi2y. Vid). Let the bestfuzzy system and for this, all the needed information about previously visited position of this particle be denoted by Pi =

the rule-base and membership functions is required to be (Pil, Pi2y ...p1)i- If g-th particle is the best particle and thespecified through some encoding mechanism as shown in the iteration number is denoted by the superscript, then theFigure 3. It is also suggested to modify the membership swarm is modified according to (2) and (3).functions and rule-base simultaneously, since they arecodependent in a fuzzy system [4]. Vnd1 = +crd (P,d - Xid) + C2r2 (Pgd - Xd (2)

xn+1 n + n+1A framework for the identification of fuzzy models has been Xid iXd + Vid (3)developed based on PSO algorithm [5]. The PSO algorithm, where,like other EAs, is a stochastic technique that uses a w - inertia weightpopulation of potential solutions (called particles) to probe cl - cognitive accelerationthe search space and also does not require gradient c2 - social accelerationinformation of the objective function under consideration. rl, r2 - random numbers uniformly distributed in the rangeEach particle in PSO flies through the search space with an (0,1).adaptable velocity that is dynamically modified according toits own flying experience and also flying experience of other Thus, the velocity update in a PSO consists of three parts, theparticles. PSO favors collaboration among the candidate "momentum" part, the "cognitive" part, and the "social" part.solutions instead of rivalry. Majority of evolutionary The balance among these parts determines the performance ofalgorithms are based on Darwin's theory of natural selection a PSO. These parameters viz, inertia weight (w), cognitivefor which the noted English philosopher, Herbert Spencer acceleration (cl), social acceleration (c2), alongwith Vmax arecoined the phrase Survival ofthe Fittest. In particle swarms, it known as the strategy/operating parameters of PSO

247

algorithm. These parameters are defined by the user before and its temperature gradient (dT/dt). The charging rate and isthe PSO run. Generally, the inertia weight is dynamically commonly expressed as a multiple of the rated capacity of thedecreased as the algorithm progresses to balance between the battery (for a battery with C=500 mAh, 0. IC means chargingglobal and local search. The parameter Vmax is the maximum current of 50 mA). From the experiments performed, input-velocity along any dimension, which implies that, if the output data was tabulated. This data set consisting of 561velocity along any dimension exceeds Vmax, it shall be points is available at http://research.4t.com. The input andclamped to this value. The parameters cl and c2 determine the output variables identified for rapid Ni-Cd battery chargerrelative pull of pbest and gbest. Random numbers r1 and r2 along with their universes of discourse are listed in Table 1.help in stochastically varying these pulls. Figure 4 depicts the Input-output data consisting of 561 points, obtained throughposition update of a particle for a two-dimensional parameter experimentation is also available atspace. Infact, this update is carried out as per (2) and (3) for http.:www,research.4t.com.each particle of swarm for each of theM dimensions in an M-dimensional optimization. TABLE I

INPUT AND OUTPUT VARIABLES FOR RAPID Ni-Cd BATTERY

The procedure for the identification of fuzzy models through CHARGER ALONGWITH THEIR UNIVERSES OF DISCOURSEINPUT VARIABLES MINIMUM MAXIMUM

PSO algorithm can be represented in the form of a pseudo- VALUE VALUEcode as below: Temperature (T)[°C] 0 50Begin Temperature Gradient (dT/dt)[°C/sec] 01

Define strategy parameters for PSO Algorithm; OUTPUT VARIABLEIteration = O; Charging Rate (Ct)[A] 0 8CCreate initial swarm of particles;

while Iteration < Maximum Iteration V. SIMULATION RESULTSConstrain Swarm The first step in designing fuzzy models through EAs is toBuild fuzzy model for each particle; decide that which parts of the model would be subjected toEvaluate each fuzzy model and calculate MSE using optimizat This din pso n two conflictingequation (1); optimization. This decision depends on two conflicting

Vn+1 objectives: dimensionality and efficiency of the search. AGet id as defined in equation (2); search space with limited parameters results in a faster andDetermine new position of each particle by usingequation (3); simple learning process at the expense of subopimalIteration = Iteration+1; solutions. For a complete search space that considers all the

end parameters of fuzzy model for optimization, theEnd learning/search process would be slow, but more likely toThere are possibilities that during the movement of the provide optimal results.swarm, some particles may move out of the bounds definedby the system constraints. It is therefore necessary to Our encoding mechanism was based on followingconstrain the exploration to remain inside the valid assumptions:hyperspace. Whenever a particle moves to a point i) Fixed numbers of triangular membership functionsrepresenting invalid solution, it is reset within the valid were used for both input and output variables withbounds. Thus all the particles in the swarm are scrutinized their centres fixed and placed symmetrically overafter every iteration to ensure that they represent only valid corresponding universes of discourse.solutions. ii) First and last membership functions of each input

and output variable were represented with left- andThe framework presented above for PSO is quite generic and right-skewed triangles respectively.can be extended to use of GAs for fuzzy modeling, where iii) Complete rule-base was considered, where alleach solution is going to be represented by chromosome in possible combinations of input membershipthe population and the evolution from one generation to next functions of all the input variables were consideredwould be through three basic mechanisms: selection, for rule formulation.mutation and crossover. iv) Overlapping between the adjacent membership

functions for all the variables was ensured throughIV. RAPID Ni-Cd BATTERY CHARGER some defined constraints.

In this paper, the data from the rapid Ni-Cd battery charger With the above assumptions, Mamdani fuzzy model for rapiddeveloped were used. The main objective of development of Ni-Cd battery charger with two input and one outputthis charger was to charge the Ni-Cd batteries as quickly as variables can be encoded as shown in the Figure 5.possible but without doing any damage to them [6]. Based onthe rigorous experimentation with the Ni-Cd batteries, it was For both PS0 and GA, real-coding has been used. Real-observed that the two input variables used to control the coding provides the representation that is very close to thecharging rate (Ct) are absolute temperature (T) of the battery

248

Antecedent Parameters p4

Rules Mapping

Consequent Parameters

Figure 3. Representation of fuzzy models encoding

pbestXn 's

gbest

*. ~~~n+1V

Current motioninfluence

Figure 4. Position update in PSO for 2-D parameter space

Input Variable - T Input Variable - dT/dt Output Variable - Ct

Encodingmethod formembership

s r | | |l l l | s l i ; functions

1 2 3 13 145 15 16 17 18 19 20 21

dT/dt Encodingmethod forCt ~~1 2 3 logic rules

1 Rule 1 Rule 2 Rule

T 2 Rule 4 Rule 5 Rul

3 Rule 7 Rule 8 Rule 9

Figure 5. Encoding of Mamdani fuzzy model for rapid Ni-Cd battery charger

natural formulation of many problems i.e. there is no influences the computation efforts like swarm size anddifference between the genotype (coding) and the phenotype iterations for PSO and population size and generations for(search-space). Real-coded EAs are also computationally GAs were also kept same. For the purpose of comparison ofefficient as they avoid the repeated conversion from genotype computational cost, only the identification of Mamdani fuzzyto phenotype for fitness evaluation. For realistic comparison, model was considered and centre of gravity was used as thereal-coding was used for PS0 as well as for GAs and at the defuzzification technique. The set of operating parameterssame time the training data, model complexity was also kept used for PS0 and GA are listed in Tables II and Table IIIsame. In addition to this, the strategy parameters that greatly respectively.

249

TABLE II TABLE VPSO PARAMETERS FOR MAMDANIFUZZY MODEL SIMULATION TIME DETAILS FOR MAMDANI FUZZY MODEL

IDENTIFICATION IDENTIFICATION THROUGH GAParameter Value Operation TimeSwarm Size 50 (in percentage)Iterations 2000 Applying constraints 0.0750Cl 2 Fitness evaluation 97.500C2 2 Selection 0.0107Wstart 1 Crossover 0.0750(Inertia weight at the start ofPSO run) Mutation 0.2143W'end 0.1 Other timings 2.1250(Inertia weight at the end ofPSO run) TOtal tmn 2.1250Vmax 125 Total 100 (84.52 hours)

TABLE III The total simulation times for the two algorithms show thatGA PARAMETERS FOR MAMDANI FUZZY MODEL for the same problem complexity, swarm (population) size

IDENTIFICATION and number of iterations (generations), GA takes 332.5%PoParameter Value/Method (84.52/25.42*100) more simulation time as compared to

Generations 2000 PSO. The savings reported are for a small system with 21-Crossover rate 0.65 dimensions. For a bigger system, through the use of PSO, theMutation rate 0.01 savings would be of the order of days or months dependingSelection method Roulette wheel selection upon the problem under consideration. Thus the results

presented in this section clearly demonstrate the simplicity ofThe identification of flzzy model throughPb 0 was carried PSO algorithm that resulted into huge savings in theout with Matlab toolbox developed by the authors The computational efforts in comparison to GA. Moreover, thetoolbox viz. PSO Fuzzy Modeler for Matlab has been simulation results reported in [5] also show the ability of PSOreleased as an open-source initiative on SourceForge net algorithm to arrive at satisfactory results.(httv: wwwsourceforge. net projects which isthe world's largest development and download repository of VI. CONCLUSIONSopen-source code and applications. The toolbox has thecapabilities to generate Mamdani and Singleton fuzzy models The paper compares the computational efforts of PSO andfrom the data based on PSO algorithm. The framework for GA when used for identifying fuzzy models of samethe identification of Mamdani fuzzy model through GA was complexity that were generated from the same data. Thedeveloped using GEATbx toolbox (http://wwwgeatbx.com), results bring out the tremendous computational efficiency ofa genetic and evolutionary algorithm toolbox for use with PSO algorithm. For the problem under consideration, theMatlab. saving was of the order of 332%. Although in this paper, the

focus has been on the use of PSO algorithm for fuzzyThe total simulation time recorded for PSO was found to be modeling, but the algorithm has much wider use and may be25.42 hours corresponding to operating parameters defined used for the optimization of other systems as well at muchin Table II, whereas for GA it was 84.52 hours for the less computational efforts as comparedmtoGA.parameters listed in Table III. The simulation timings (inpercentage) for the different modules of PSO and GA arelisted in Tables IV and V respectively. Under the heading REFERENCESother timings are the timings that include: initialization, [1] L.A. Zadeh, "Fuzzy Sets," Information and Control, pp. 338-353, 1965.calling functions, reading training data from the excel file etc. [2] H.Hellendoorn and D. Driankov (Eds.), Fuzzy Model Identification -

Selected Approaches, Springer-Verlag, 1997.[3] John Yen and Reza Langari, Fuzzy Logic - Intelligence, Control and

TABLE IV Information, Pearson Education, Delhi, First Indian Reprint, 2003.SIMULATION TIME DETAILS FOR MAMDANI FUZZY MODEL [4] Y.Shi, R. Eberhart and Y.Chen, "Implementation of Evolutionary Fuzzy

IDENTIFICATION THROUGH PSO Systems," IEEE Transactions on Fuzzy Systems, vol. 7, pp. 109-119,Operation Time 1999.

(in percentage) [5] Arun Khosla, Shakti Kumar and K.K. Aggarwal, "A Framework forApplying constraints 0.03125 Identification of Fuzzy Models through Particle Swarm Optimization,"Fitness evaluation 87.5000 IEEE Indicon Conference, Chennai, India, 11-13 Dec, 2005, pp. 388-Fosritnes vevluaityion 87l.01500 391.Constraint velocity in all 0.03125 [6] Arun Khosla, Shakti Kumar and K.K. Aggarwal, "Design anddimensions to Vmax Development of RFC-_0: A Fuzzy Logic Based Rapid Battery ChargerParticles position update 0.031250 for Nickel-Cadmium Batteries," HiPC 2002 Workshop on Soft

TOtaltmig 100 (025.2hus Computing, Dec 18, 2002, pp. 9-14.

250