This article was downloaded by: [University of Kent]On: 03 December 2014, At: 16:23Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

International Journal ofElectronicsPublication details, including instructions for authorsand subscription information:http://www.tandfonline.com/loi/tetn20

A new global optimizingalgorithm for fuzzy neuralnetworksLIANGJIE ZHANG , YANDA LI & HUIMIN CHENPublished online: 10 Nov 2010.

To cite this article: LIANGJIE ZHANG , YANDA LI & HUIMIN CHEN (1996) A new globaloptimizing algorithm for fuzzy neural networks, International Journal of Electronics,80:3, 393-403, DOI: 10.1080/002072196137237

To link to this article: http://dx.doi.org/10.1080/002072196137237

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of theContent.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone is

expressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Uni

vers

ity o

f K

ent]

at 1

6:23

03

Dec

embe

r 20

14

int. j. electronics, 1996, vol. 80, no. 3, 393± 403

A new global optimizing algorithm for fuzzy neural networks

LIANGJIE ZHANG² , YANDA LI² and HUIMIN CHEN²

In this paper, a new global optimizing algorithm that combines the modi® ed quasi-Newton method and the improved genetic algorithm is proposed to ® nd the globalminimum of the total error function of a fuzzy neural network. A global linearsearch algorithm based on fuzzy logic and combinative interpolation techniques isdeveloped in the modi® ed quasi-Newton model. It is shown that this algorithmensures convergence to a global minimum with probability 1 in a compact regionof a weight vector space. The results of computer simulations also reveal that thisalgorithm has a better convergence property and the times of global search areobviously decreased.

1. Introduction

In recent years, fuzzy neural networks have been studied quite extensively bymany researchers and various fruitful results have been obtained (Linkens and Nie1993, Kajitan et al. 1992). In particular, the back-propagation method (BP method)has been widely used to tune the weights of the fuzzy rules, as well as membershipfunctions, of the linguistic variables for optimizing the fuzzy neural network(Horikawa et al. 1992). However, the most important limitations of this method areas follows.

(1) It does not ensure convergence to the global minimum of the total errorfunction of the fuzzy neural network.

(2) Because it consists of a steepest descent method, which uses the ® xed stepsize rule in the linear search of the objective function, it does not necessarilyensure the monotone decreasing property of the total error function, andtherefore, it does not ensure convergence even to a local minimum of theobjective function.

It has been suggested that the genetic algorithm (GA) is a quasi-parallel’ globaloptimizing technique that can be introduced to resolve the ® rst limitation (Holland1975). It is, however, di cult to modify the selection operator that has an intrinsicmechanism to ensure global convergence (Davis and Principe 1991). To resolve thesecond limitation, it is necessary to improve the original BP method by performinga relatively accurate linear search in each step of the calculation.

In this paper, a hybrid algorithm that combines the modi® ed quasi-Newtonmethod and the improved genetic algorithm is proposed to ® nd the global minimumof the total error function of a fuzzy neural network. It exploits the high convergencerate of the quasi-Newton method in the trust region with an accurate global linearsearch to obtain a local minimum, and it uses a variable mutation rate (VMR) basedon fuzzy reasoning in the improved genetic algorithm to search for a better o springfrom the local minimum point. Fuzzy logic is used in the combinative interpolation

Received 4 April 1995; accepted 12 July 1995.² Department of Automation, Tsinghua University, Beijing 10084, People’s Republic of China.

0020± 7217/96 $12.00 Ñ 1996 Taylor & Francis Ltd.

Dow

nloa

ded

by [

Uni

vers

ity o

f K

ent]

at 1

6:23

03

Dec

embe

r 20

14

L iangjie Zhang et al.394

of the linear search as well as in changing the mutation rate in the genetic algorithmfor adaptive searching purposes. Computer simulations show that the algorithm isfeasible for designing fuzzy neural networks and a fuzzy controller.

In this paper, §2 is devoted to an explanation of the BP method for tuning fuzzyneural networks. In § 3, we propose a hybrid algorithm and give some remarks todiscuss in detail the linear search, quasi-Newton method and genetic algorithm. Atheorem is also given that shows our proposed hybrid algorithm ensures convergencewith probability 1 to the global minimum of the total error function. In the ® nalpart, computer simulations are given to compare the performance of the proposedmethod with that of the BP method, the quasi-Newton method and the canonicalgenetic algorithm.

2. Tuning fuzzy neural networks by using the back-propagatio n method

A fuzzy neural network (FNN) is a kind of neural network based on fuzzyreasoning, thus, it can express human experiences by using linguistic variables andfuzzy rules, and approximate a complex nonlinear system. The basic idea of a fuzzyneural network is to express fuzzy reasoning through the connection weights betweenthe antecedent part and the consequent part of the FNN (Zhang 1994 a). Note thatthe input variables are x1 , x2 , . . . , xn , and the estimated result of each x i is de® ned,so that the ith membership values are A i 1 , A i 2 , . . . , A i n (i =1, 2, . . . , m) .

The fuzzy rules can be expressed as

Rule k: if x1 is A i 1 and . . . and xn is A i n then y is wk (1)

To a neural network with n input variables and m membership functions for eachinput, there exist l=m

n rules in the overlap region of the di erent linguistic inputvariables (see the ® gure).

With fuzzy inference and centoid-defuzzi® cation, the rules and the output of theFNN are de® ned as follows

ok =mAi 1

(x1 )mAi 2

(x2 ) ´ ´ ´ mAi n

(xn ) (k =1, 2, . . . , l ) (2)

y* =�l

k=

1ok wk

�l

k=

1ok

= �l

k=

1oà k wk (k =1, 2, . . . , l, l=m

n) (3)

The membership function mi j can be de® ned as triangle-shaped, or as any othershape. In this paper, we de® ne the triangle-shaped function

mi j (x) =2 |x Õ ai j |

b i j(4)

as the i th membership function of jth input variable.The total error function E is de® ned by the following

E = �i

E i = �p

i=

1[

12 ( y

di Õ y

*i )

2] (5)

where p is the number of training samples, ydi represents the desired value of the ith

sample and y*i represents the estimated value derived from FNN.

Dow

nloa

ded

by [

Uni

vers

ity o

f K

ent]

at 1

6:23

03

Dec

embe

r 20

14

Global optimizing algorithm for fuzzy neural networks 395

The con® guration of the fuzzy neural networks with two-inputs and one-output.

The back-propagation method is described below. The objective function E ,consisting of tuning parameters a i j , b i j and wk is minimized by calculating the gradient

A‚ E

‚ a i j,

‚ E

‚ bi j,

‚ E

‚ wk Bshown as the following equations

a i j ( t+1) =ai j ( t) Õ ga‚ E

‚ ai j(6)

b i j ( t+1 ) =bi j ( t) Õ gb‚ E

‚ b i j(7)

wk ( t+1 ) =wk ( t) Õ gw‚ E

‚ wk(8)

‚ E

‚ ai j=( y

d Õ y*) �k

(w i Õ y*)ok

�l

k=

1ok mi j

‚ mi j

‚ a i j(9)

‚ E

‚ bi j=( y

d Õ y*) �k

(w i Õ y*)ok

�l

k=

1ok mi j

‚ mi j

‚ b i j(10)

‚ E

‚ wk=( y

d Õ y*)ok

�l

k=

1ok

=( yd Õ y*)oà k (11)

ga , gb , gw are learning factors, which are also called step sizes. Although the BPmethod has been considered as one of the smartest algorithms for ® nding appropriateweights in a multi-layered network, it also has several problems. As is well known,one of the most important problems is the potential failure of falling into a local

Dow

nloa

ded

by [

Uni

vers

ity o

f K

ent]

at 1

6:23

03

Dec

embe

r 20

14

L iangjie Zhang et al.396

minimum of the objective function E . Thus, we have to construct a new algorithmthat has the global optimizing property.

3. Hybrid algorithm for ® nding the global minimum of the objective function of a

fuzzy neural network

In this section, we propose a new algorithm combining the modi® ed quasi-Newton method and the improved genetic algorithm to ® nd the global minimum ofthe total error function in a small number of steps. In our hybrid algorithm, themodi® ed quasi-Newton method is introduced to ® nd a local minimal point as theinitial generation for each process of genetic evolution, and the improved geneticalgorithm ensures that it can jump o from the local minimal point (i.e. it can get abetter generation whose ® tness value lowers the error function) until the hybridalgorithm stops as the global minimum. Remarks are added in the subsections togive a brief discussion of the special problems of the global linear search within thetrust region of the quasi-Newton model and the variable mutation rate applied togenetic algorithms.

3.1. Hybrid global optimizing algorithm

Note that W is the vector denoting the tuning parameters ai j , b i j and wk , andE (w) is the objective function of a fuzzy neural network that we have to decrease toa value below a small number e. The algorithm is described as follows.

Step 1. Select an initial weight vector W 0 , let M be the total number of steps, let i=0.

Step 2. Calculate the gradients of objective function E (W i ) , where g(W i ) =‚ E(W i )/ ‚ W i

can be derived from (6)± (11).

Step 3. Use the modi® ed quasi-Newton method with a global linear search to get alocal minimum of E (W ) .

Step 3.1. Let H0 =1

Step 3.2. The calculation can be done as

S i =Õ Hi g(W ) (12)

li =minlµ

0E (W i +lSi ) (13)

W i+

1 =W i +li S i (14)

Step 3.3. If E (W i+

1 )<e, stop the total calculation.Step 3.4. Let

DW i =W i+

1 Õ W i (15)

Dg(W i ) =g(W i+

1 ) Õ g(W i ) (16)

If dDg(W i ) d<e1 or |E (W i+

1 ) Õ E (W i ) |<e2 , then let W*l o c a l =W i

+1 , go

to Step 4.Step 3.5. If DW

Ti Dg(W i ) <Dg(W i )

TH i Dg(W i ) then do the calculation

(Davidson± Fletcher ± Powell)

Hi+

1 =Hi +DW i DW

Ti

DWTi Dg(W i )

ÕHi Dg(W i )Dg(W i )

TH i

Dg(W i )T

H i Dg(W i )(17)

Dow

nloa

ded

by [

Uni

vers

ity o

f K

ent]

at 1

6:23

03

Dec

embe

r 20

14

Global optimizing algorithm for fuzzy neural networks 397

or else do the calculation (Broyden ± Fletcher ± Goldfarb± Shanno)

Hi+

1 =Hi +mi DW i DW

Ti Õ Hi Dg(W i )DW

Ti Õ DW Dg(W i )

TH i

DWTi Dg(W i )

(18)

mi =1 +Dg(W i )

THi Dg(W i )

DWTi g(W i )

(19)

Step 3.6. Let i= i+1.

If i M, then go to Step 2, else let W*l o c a l =W i

Õ1 .

Step 4. Translate W*l o c a l into a binary string (i.e. a gene chain) of length l, which is

determined by the convergent accuracy e.

Step 5. Use the improved genetic algorithm to perform crossover, mutation andselection. The binary string of the evolutionary population is said to be ® tonly when its representative vector W

*G A satis® es the condition

E (W*G A ) <E (W

*l o c a l ) (20)

If there is no ® t o spring after many selection times, then E (W*o ld ) is a global

minimum, else let W 0 =W*G A , and stop GA-searching.

Step 6. If E (W0 ) <e, stop the total calculation, else go to Step 2.

Remark 3.R.1: e1 , e2 are small ® xed parameters that ensure the modi® ed quasi-Newton method stops when falling into a local minimum. In this paper, e1 and e2

are chosen as e1 =0´1e, e2 =0´01e. %

Remark 3.R.2: The accuracy of the linear search in the quasi-Newton method isde® ned as

e3 =min{e*1 , e*2 dg(W i ) d} (21)

e*1 , e*2 are freely chosen within a range of ( 0, e) as long as the quasi-Newton methodensures the monotonous decrease; however, the selection of the values of theseparameters signi® cantly a ects the e ciency of the overall algorithm.

3.2. Global linear search based on combinative interpolation and fuzzy logic in the

modi® ed quasi-Newton model (Zhang 1994 b)The quasi-Newton method has a lot of theoretically good properties of a high

convergent rate (Powell 1984), yet it is based on the quadratic model with a convexcondition of the objective function. However, the total error function of FNN mayhave a lot of di erent convex regions in the output range and the modi® edquasi-Newton method is supposed to work only in one convex region.

It is proved that the searching directions are H-conjugated in optimizing aquadratic convex function, if the value l, derived from the linear search, is to theideal accuracy (Wolfe 1978). Thus, we use an H-conjugated checking method toensure that the optimization is within a trust region. The method is described below.

First, choose an initial number L i >0, and let

li =minE (W i +lS i ) l×( 0, L i ) (22)

Dow

nloa

ded

by [

Uni

vers

ity o

f K

ent]

at 1

6:23

03

Dec

embe

r 20

14

L iangjie Zhang et al.398

Then, note that

ji =g(W i

+1 ), S i

dg(W i+

1 ) d dS i d(23)

The fuzzy rules are de® ned as follows

Rule 1. If ji is ZO, then L i+

1 =L i 12

Rule 2. If ji is PB, then L i+

1 =L i /2

Rule 3. If ji is PM, then L i+

1 =L i

All rules are used to determine the searching region of the vector W i +lS i , whereZO represents Zero’, PB represents Positive Big’, PM represents Positive Medium’.

Thus, a relatively accurate global linear search in the trust region is required.The proposed method of linear search uses combinative interpolation techniques toobtain the characteristic points of global minimal point in fewer steps than theconventional global linear search methods. The method of global linear search(Zhang 1994 b) is described as follows.

Step 1. Set the range of linear search as [left, right], let l0 = left=0, Min=E (left)

Step 2. Search for a range that contains a local minimum. Use second-order polyno-mial interpolation to get a minimum point E (l*) , if E (l*) >Min+ limit thendiscard this point. Otherwise, do an accurate search in this range to the givenextent of accuracy.

Step 3. Use third-order polynomial interpolation to approximate the point ofin¯ ection, and determine whether there exists a new convex region thatcontains a local minimum point.

Step 4. Use fuzzy logic to determine the appropriate step d, and let

l1 =l0 +d, l2 =l0 +2d, l3 =l0 +3d

Thus, a new range is derived.

Step 5. Repeat Steps (2)± (4) until the searching region covers the bound of the giventrust region (0, L i ).

Remark 3.R.3: This method is developed by considering the convex properties andusing fuzzy logic and polynomial interpolation, which achieve an accurate globalminimum point with a small number of steps. The fuzzy logic in getting theappropriate step d can be designed using fuzzy inference as described in § 1.

3.3. Improved genetic algorithm using the uniform mixing method (UMM) and avariable mutation rate (VMR) (Zhang et al. 1994)

Canonical genetic algorithms (CGA) are often used to tackle static optimizationproblems and derive a global optimum in ® nite steps. However, it is proved bymeans of homogeneous ® nite Markov chain analysis that a CGA will never convergeto the global optimum regardless of the initialization, crossover, operator and object-ive function (Gunter 1994). The improved genetic algorithm that has a VMR of themutation operator and uses UMM to crossover is a time-varying mutation and

Dow

nloa

ded

by [

Uni

vers

ity o

f K

ent]

at 1

6:23

03

Dec

embe

r 20

14

Global optimizing algorithm for fuzzy neural networks 399

selection probabilities and it has a satis® ed global convergence property, as discussedin the next sub-section.

First, the genetic algorithm can be sketched as follows.

Choose an initial population and determine the ® tness of each individual performselectionrepeat

perform crossoverperform mutationdetermine the ® tness of each individual

untilsome stop criterion is met

Under the conditions of the four assumptions and the two simpli® cations, thecrossover operator will perform most e ciently when the crossover point is selected inthe middle of the parent strings (Zhang et al. 1994). In the improved genetic algorithm,the method called uniform mixing’ is used in the crossover step. After the initialgeneration ® nishes the crossover, the codes of the o spring are thrown into confusion(this process is called mixing’). At this moment, the crossover point is selected inthe middle of the gene string. Considering the method, i.e. from mixing to crossoverin the middle of the gene string, we ® nd that: the two parent-gene strings with lengthN perform crossover in the [N/2 ] bits randomly. The evolution speed is faster byusing UMM to distribute the bad bits’ homogeneously.

The variable mutation rate of optimizing the objective function of FNN is de® nedas

Pm =P (1| f, df ) (24)

where f is the ® tness value of the objective function and df is the change of f betweenthe generation of parents and their o spring.

The fuzzy rules are listed as

Rule 1. If f is PB and df is NB then Pm is PM

Rule 2. If f is PB and df is NS then Pm is PB

Rule 3. If f is ZO then Pm is PS

where PB represents Positive Big, ZO represents Zero, NB represents Negative Big,PM represents Positive Medium, PS represents Positive Small, and NS representsNegative Small. Rule 1 is mainly aimed at the early period of iterations, Rule 2 ismainly aimed at the possible period of local optimum, and Rule 3 is mainly aimedat the late period of iterations.

3.4. T he convergence analysis of the hybrid algorithm

Lemma 1 (Gunter 1994, p. 98): T he genetic algorithm with mutation probab ility Pm ,

crossover probab ility Pc ×(0, 1 ) and proportional selection is an ergodic Markov chain,

i.e. there exists a unique limit distribution for the state of the chain with non-zero

probab ility to be in any state at any time regardless of the initial distribution.

Lemma 2 (Iosifescu 1980, p. 133): In an ergodic Markov chain the expected transition

time between initial state i and any other state j is ® nite, regardless of the state i and j.

Corollary 1: L et i×S be any state, i.e. S 7{E (W i ) | i=1, 2, . . . , n} and Pti be the

Dow

nloa

ded

by [

Uni

vers

ity o

f K

ent]

at 1

6:23

03

Dec

embe

r 20

14

L iangjie Zhang et al.400

probab ility that GA is in such a state i at step t. T he probab ility that the GA is in

state i converges to P2i >0.

Corollary 2: L et S i 7{E (W j ) |E (W j ) <E (W i ), E (W j )×S} and S i w. Let Pti j be the

probab ility f rom any given state i to state j at step t on the condition that E (W j )×S i .

L et ct =min{P

ti j |E (W j )×S i }. T he probab ility that the GA is in the state belonging

to S i converges to cµc2>0.

Proof: Let Z t =min{E (W( t )

i (k)) |k =1, . . . , n} be a sequence of random variablesrepresenting the best ® tness within a population represented by state i at Step t.

Since S i w, there exists E (W j ) such that

P (Z t ×Si ) µP (Z t =E (W j ) , E (W j )×S i )

From Lemma 2, it leads to

limt� 2

P (Z t ×Si ) µlimt� 2

P(Z t =E (W j ) , E (W j )×S i ) µc2>0 %

Theorem: L et W be a compact region in which we have to ® nd an appropriate weight

vector W . L et WÃ be one of the vectors that gives the global minimum of E (W ) in W

E (WÃ ) = minW

×W

E (W ) (25)

L et Weà be a region such that

Weà 7{W dE (W ) Õ E (Wà ) |<eà , W ×W } (26)

Assume the following

(1) For any positive number d, the euclidean measure of Ud(WÃ ) mW is positive,

where

Ud(WÃ ) ={W | dW Õ WÃ d<d} ( 27 )

(2) T he gene strings are long enough to approximate W to the arbitrary accuracy.

( 3 ) T here is no limit of times in calculating W i (i =1, 2, . . . ) .

( 4 ) E (W ) belongs to class C1

.

T hen the hybrid algorithm ensures that for any positive number eà , E (W i ) converges

to E (WÃ ) with probab ility 1, i.e.

limi� 2

P(W |W i×Weà ) =1 ( 28 )

Proof: Because E (W ) is a continuous function and the gene strings are arbitrarylong, there exists dà >0 (for any given eà ) such that following relation holds

if dW Õ Wà d<dà then |E (W ) Õ E (Wà ) |<eà (29)

Consider the following region A

A 7{W | dW Õ Wà d<dà , W ×W } (30)

Assume that W i has already been found and W i+

1 should be found out by theimproved genetic algorithm. Let PA (W i ) be the probability that W i

+1 enters the

region A (i=1, 2, . . . ). Then from Corollary 2, it is derived that

PA (W i ) µc>0 (31)

Dow

nloa

ded

by [

Uni

vers

ity o

f K

ent]

at 1

6:23

03

Dec

embe

r 20

14

Global optimizing algorithm for fuzzy neural networks 401

From the de® nition of Weà , we can get W

eà 6A , thus

PWeÃ

(W i ) µci >0 (i=1, 2, . . . ) (32)

Once W i enters region Weà , {W t | t= i+1, i+2, . . . } does not leave W

eà since E (W t ) is a

monotone decreasing function of t in the proposed hybrid algorithm. This meansthat the region W

eà is an absorbing region. Consequently

P (W |W i×Weà ) µ1 Õ ( 1 Õ c)

i (33)

where i refers to the i th time using a genetic algorithm to get a better outstandingo spring of W i . Hence

limi� 2

P(W |W i×Weà ) µlim

i� 2

{1 Õ ( 1 Õ c)i}=1 (34)

Hence, E (W i ) converges to E (WÃ ) with probability 1. %

Remark 3.R.4: The modi® ed quasi-Newton method ensures that E (W i ) is monoton-ous decreasing by each step i. This is very important for the global convergenceproperty. %

Remark 3.R.5: Using the improved genetic algorithm alone can also optimize theobjective function, but it needs quite a lot of steps in the evolution process. %

4. Computer simulation results

In order to demonstrate the validity of the proposed hybrid method, threenonlinear objective functions (Equations (35)) are taken as examples. The input± out-put data are prepared by changing the input variables (x1 , x2 ) within [Õ 1, 1]. One-hundred data points are employed for training the total error function. The structureof FNN is presented as Fig. 1, and the initial weights of the objective function arethe same when using di erent learning methods. The learning stops when the object-ive function E for the identi® cation data is less than e=0 0001. The BP method,Broyden’s canonical quasi-Newton method and the genetic algorithm alone areemployed to compare the hybrid learning algorithms. In our simulations e1 =10 Õ

5,

e2 =10 Õ6, e

*

1 =e*

2 =e=10 Õ4, L 0 =1, the encoded length in the GA is 15 bits, and

Pc=0 9, Pm=0 2.

y1 =1/( 3e3 x1

+2eÕ4 x

2)1 / 2

y2 =(x1 +x1 x2 +x22 )/3

y3 =( 4 cos (3x1 ) +2 sin (4x2 ))/6

(35)

Remark 3.R.6: The asterisk in front of the stop error means that the objectivefunction is less than e=10Õ

4 within some steps of descent searching. The geneticalgorithm stops when it cannot get a better weight vector of the total error functionafter 105 steps of total evolution. In Broyden’s quasi-Newton method, there is notrust region regulation used in the linear search.

From Table 1, two points are drawn as follows.

(1) The hybrid algorithm has a higher rate of local convergence than the BPmethod in ® nding the local minimum of the objective function and also hasthe ability to jump o from the local minimum point in optimization. Sincethere are quite a lot of local minimal points in the training error function,the descent methods may stop at di erent local minimal points.

Dow

nloa

ded

by [

Uni

vers

ity o

f K

ent]

at 1

6:23

03

Dec

embe

r 20

14

L iangjie Zhang et al.402

Learning Linguistic Learning Total descent GA evolution Stop errorfunction variables method searching steps times (Ö 10Õ

4 )

BP 43 Ð 25400y1 3 QN 29 Ð 17800GHybrid 57 1878 0 98

BP 23 Ð 25900y2 3 QN 27 Ð 0 97GHybrid 19 Ð 0 98

BP 48 Ð 1 116800y3 3 QN 46 Ð 353200GHybrid 218 2987 49000

BP 18 Ð 0 92y1 5 QN 5 Ð 0 72GHybrid 5 Ð 0 72

BP 3 Ð 0 96y2 5 QN 3 Ð 0 88GHybrid 13 Ð 0 88

BP 82 Ð 21400y3 5 QN 18 Ð 0 87GHybrid 9 Ð 0 88

Table 1. Comparison with BP and Broyden’s quasi-Newton method.

Learning Linguistic Learning Total descent GA evolution Stop errorfunction variables method searching steps times (Ö 10Õ

4 )

GA Ð 12 816 0 99y1 3 Hybrid 57 1 878 0 98

GA Ð 28 465 49 000y3 3 Hybrid 218 2 987 49 000

Table 2. Comparison with the improved genetic algorithm.

(2) The quasi-Newton method is capable of optimizing the objective functiononly within the trust region, where the objective function has a good approxi-mation of a convex quadratic model. Thus, Broyden’s quasi-Newton methodis not a global optimizing algorithm for tuning the error function of FNN.

From Table 2, we can see that the hybrid algorithm uses several fewer steps inobtaining global optimization that GA alone because the modi® ed quasi-Newtonmethod o ers a better mechanism of ® tness evaluation in determining the initialgeneration and the standout o spring.

5. Conclusions

In this paper, a new hybrid algorithm for ® nding appropriate weights andparameters in a fuzzy neural network has been proposed. The improved geneticalgorithm is combined with the modi® ed quasi-Newton method using an accurateglobal linear search within the trust region. It has been shown in several examples

Dow

nloa

ded

by [

Uni

vers

ity o

f K

ent]

at 1

6:23

03

Dec

embe

r 20

14

Global optimizing algorithm for fuzzy neural networks 403

that the proposed hybrid algorithm is quite useful for ® nding the global minimumof the total error function of a FNN. It is believed that the hybrid algorithm can beembedded into other unconstrained nonlinear optimization problems and can havemore applications in designing fuzzy neural networks and expert control systems.

Acknowledgments

This work was supported by the Climbing ProgrammeÐ National Key Projectfor Fundamental Research in China, Grant NSC 92097 and by the Ph.D. Foundationof the National Educational Committee.

References

Davis, T. E., Principe, J. C., 1991, A simulated annealing like convergence theory for thesimple genetic algorithm. Proceedings of the Fourth Conference on Genetic Algorithms,

pp. 174± 181.Gunter, R., 1994, Convergence analysis of canonical genetic algorithms. IEEE T ransactions

on Neural Networks, 5(1), 96± 101.Holland, J. H ., 1975, Adaptation in Natural and Arti® cial Systems (Ann Arbor, Mich.: The

University of Michigan Press).Horikawa, S., Furuhashi, T., and Uchikawa, Y., 1992, On fuzzy modeling using fuzzy

neural networks with the back-propagation algorithm. IEEE T ransactions on Neural

Networks, 5, 801± 806.Iosifescu, M ., 1980, Finite Markov Processes and T heir Applications (Chichester, U.K.: Wiley).Kajitan, R., Katayama, K . Y., and N ishida, K . Y., 1992, Self-generating radial basis function

as neuron-fuzzy model and its application to nonlinear predication of chaotic timeseries. IEEE International Conference on Fuzzy Systems, pp. 407± 414.

Linkens, D . A., N ie, J., 1993, Fuzzi® ed RBF network-based learning control: structure andself-construction. IEEE International Conference on Neural Networks, pp. 1016± 1021.

Powell, M . J. D ., 1984, Some properties of the variable metric algorithms for unconstrainedoptimization, Proceedings of the International Congress of Mathematicians.

Wolfe, M . A., 1978, Numerical Methods for Unconstrained Optimization (Prentice-Hall).Zhang, L., and Li, Y., 1994a, Wavelet based fuzzy neural networks. International Symposium

on Arti® cial Neural Networks; 1994b, A new global linear optimization algorithmbased on fuzzy control and combinative interpolation techniques. Journal of T singhua

University (Special Issue of AI and Pattern Recognition), 4 (in Chinese).Zhang, L., Mao, Z., and Li, Y., 1994, An improved genetic algorithm based on combination

theory and fuzzy reasoning techniques and its applications. International Conference

on Neural Information Processing, pp. 180± 185.

Dow

nloa

ded

by [

Uni

vers

ity o

f K

ent]

at 1

6:23

03

Dec

embe

r 20

14