0-7803-8942-5/05/$20.00 @2005 IEEE.

SMCia/05 2005 IEEE Mid-Summer Workshop on Soft Computing in Industrial Applications Helsinki University of Technology, Espoo, Finland, June 28-30, 2005

Scored Pareto-MEC for Multi-Objective Optimization

1,2Chengyi Sun, 1Wanzhen Wang, and 3X. Z. Gao 1Artificial Intelligence Institute, Beijing City University

Mid Road #269 of North Loop-4, Beijing 100083, P. R. China Phone: +86-10-62322704, Email: scy@hdu.edu.cn

2Computing Center, Taiyuan University of Technology, Taiyuan 030024, P. R. China 3Institute of Intelligent Power Electronics, Helsinki University of Technology

Otakaari 5 A, Espoo, Finland Email: gao@cc.hut.fi

Abstract - This paper proposes a new multi-objective optimization algorithm – Scored Pareto Mind Evolutionary Computation (SP-MEC), which introduces the theory of Pareto into Mind Evolutionary Computation (MEC) for the multi-objective optimization. In our SP-MEC, the selection of individuals is based on their scores that include the Pareto dominance and density information among the individuals. The SP-MEC is compared with the VEGA, NSGA, SPEA, and Pareto-MEC on the basis of four different test problems: convexity, non-convexity, discreteness, as well as non-uniformity. Especially, both the Pareto-MEC and SPEA have shown promising performances in solving various optimization problems. On the test problems, SP-MEC outperforms all the four reference algorithms concerning three measures: the distance from trade-off front to Pareto-optimal front, the uniformity of solutions, and the spread of solutions. Impersonal termination criterion is used in SP-MEC and Pareto-MEC instead of the preset number of generations in other algorithms. SP-MEC has a higher computational efficiency than the VEGA, NSGA, and SPEA. Compared with another our algorithm, Pareto-MEC, the computational efficiency of SP-MEC is a little lower. However, the solution quality of SP-MEC is higher than that of the Pareto-MEC. Therefore, it can be concluded the SP-MEC is a powerful algorithm for multi-objective optimization.

I. INTRODUCTION

It is well known that multi-objective optimization problems are difficult to solve, because there usually exist conflicts among the objectives, and appropriate tradeoffs have to be made in finding the optimal solutions. Furthermore, solutions may have different dominations concerning different objectives. Schaffer applied evolutionary algorithms to solve multi-objective optimization problems in the 1980s [3]. The Pareto technology has been employed in the evolutionary algorithms for the multi-objective optimization problems, among which the SPEA [7] is a typical scheme.

This paper proposes a new multi-objective optimization algorithm – Scored Pareto Mind Evolutionary Computation (SP-MEC). The Parto principle is introduced into MEC[4][2] so that we can derive two optimization algorithms: Pareto-MEC [1] and SP-MEC. In the SP-MEC, the selection of individuals depends on their scores, which reflect the relation of dominance and information of density among the individuals. However, in the Pareto-MEC, the selection of individuals is based on the partial order relation. The SP-MEC is compared with four reference algorithms, VEGA, NSGA, SPEA, and Pareto-MEC in case of four different test problems: convexity, non-convexity, discreteness and non-uniformity. The experiment results show that the SP-MEC outperforms the VEGA, NSGA, and SPEA on all test problems with regard to three measures: the distance from trade-off front to Pareto-optimal front, the uniformity of solutions, and the distribution of solutions. Impersonal termination criterion is used in the MEC instead of a preset number of generations as in other algorithms. Compared with the Pareto-MEC, the number of generations needed by the SP-MEC is larger. However, the qualities of solutions obtained by the SP-MEC are higher than that by the Pareto-MEC.

II. BASIC MIND EVOLUTIONARY COMPUTATION

MEC is a new approach of Evolutionary Computation (EC) proposed by Chengyi Sun in 1998[4]. It is based on the consideration of the disadvantages of GA as well as analysis of human mind mechanism. The basic MEC is an evolutionary computational approach for solving numerical optimization problems. Some concepts of the basic MEC are explained as follows:

Population is the set of all individuals in each evolutionary generation. The whole population is divided

into groups.

Similartaxis is the process in which individuals of a group compete against each other in a local area to search for the local optimum. This operation serves as the local exploitation.

Dissimilation is the process in which groups compete with each other in a global area and investigate new solutions in the whole solution space. This operation serves as global exploration. It has two main functions: (1) some better initial solutions are selected through global competition for every group; (2) the local optimal solutions gained in the process of similartaxis are compared, and some best solutions at this point are selected as the global maxima until the final solutions are found.

The similartaxis and dissimilation are employed in MEC rather than the GA-based crossover and mutation. Different from GA, there is no separate operation selection in MEC. Actually, the operation selection is embedded in processes of both similartaxis and dissimilation.

III. PRINCIPLES OF SP-MEC

The SP-MEC introduces the theory of Pareto into MEC algorithm [4] to optimize multi-objective problems. The principles of SP-MEC are: (1) a number of individuals are scattered in the whole solution space, and according to their scores, some better individuals are selected as the initial ‘centers’ for every group; (2) each group only searches in a local area, and gradually shifts from its initial center to the Pareto front; (3) during the process of shifting to this front, the SP-MEC would bound the searching region of the group, and control its shifting direction. Both functions (1) and (3) belong to the dissimilation, while function (2) is called as similartaxis.

A. Pseudo Codes of SP-MEC algorithm

/* Parameters of SP-MEC */ N /* number of groups */

GS /* size of the group */ S /* size of the population, NS >> , ( NSS G ×= , in

this paper) */ t /* number of generations */

tiG /* the i-th group in the t-th generation */

)(Anond /* sub-set of set A , the nondominated

individuals within the set A */

textP /* the external set in the t-th generation */

)( textPpart /* part of external set in the t-th generation

*/

/* Procedure of SP-MEC */

1. Initialization: 0←t , Φ←textP .

2. Scatter S individuals according to some density function.

3. Calculate the score of each individual. (refer to Section 3.B)

4. Select N individuals with larger scores, and put

them into textP to act as the initial centers for every

group.

5. For the i-th scattering center tic

6. According to some density function, scatter GS

individuals around tic , and form the 'G .

7. Combine 'G and )( textPpart together, and

calculate the scores of all the individuals in

'G � )( textPpart (see Section 3.C).

8. According to some given ratio, select individuals

with larger scores from 'G � )( textPpart to form

the current generation group tiG .

9. According to the position information of each

individual in tiG , confirm the geometry center of

tiG , and predict the next scattering width.

10. End for.

11. If 0≠t ,

∪

= −

=

1

1

text

ti

N

i

text PGnondP � ； else

∪

=

=

00

1exti

N

i

text PGnondP � .

12. Adjust scattering width for every generation group (see Section 3.4), and according to the geometry

center of tiG and vector )( t

iGshift (see section

3.5), create the next scattering center. 13. 1+← tt 14. If the termination criterion is satisfied (see section

3.6), generate textP , and stop the algorithm.

15. End For The above steps 5-10 realize the similartaxis operation, and steps 2-4 and 11-12 implement the dissimilation operation.

B. Scores of Individuals

The score of individual j consists of three parts:

( ) ( ) ( ) ( )jdwjnwjnwjS 3||21 ++=�

, where 01 >w ，

02 >w ， 03 >w are three constants with 1321 =++ www .

(1) )( jn�

denotes the number of individuals, which are

dominated by individual j in set A , i.e.,

{ } | )( ljAlljn ��

∧∈= .

(2) )(|| jn denotes the number of individuals, which are

indifferent to individual j in set A, i.e.,

{ } ||| )(|| ljAlljn ∧∈= .

(3) ( )jd denotes the density information of individual j . This algorithm evaluates the individuals

according to not only the Pareto dominated relation, but also their density information. This can guarantee that the individuals scatter uniformly on the front:

( ) ( ) Ajindexjd = , (1)

where A denotes the number of individuals in set A ,

and ( )jindex represents the sequence number formed

based on the crowded degree in the sequence. Smaller sequence number of the individual implies higher crowded degree, for example, the individual with

sequence number one has the highest crowded degree in set A . To make the solutions scatter uniformly on the front, we should choose those individuals that have lower crowded degrees. The SPEA2 [6] use the Archive truncation method to constrain the size of external sets. We also employed this method here to create the sequence numbers for the individuals.

C. Evaluating Individuals in 'G ∪ )( textPpart

)( textPpart is the set of individuals of t

extP , which fall

in the current scattering area of group i. The scores of all

individuals in 'G ∪ )( textPpart are calculated.

D. Confirm new Scattering Width

Adjusting the group scattering width tiw continuously

can make the group of a generation rapidly shift to the front. During the process of similartaxis, we predict the

scattering width ( )tiGw of 1+t

iG according to the

farthest distance between two individuals in group tiG .

However, this forecast does not have any global information. Therefore, in our algorithm, the scattering width is adapted during the process of dissimilation. That is, if we gain some better individuals, the scattering width would be multiplied by a positive constant larger than one; otherwise a positive constant smaller than one.

E. Confirm new Scattering Center

The scattering center 1+tic of group 1+t

iG is relevant to

both )( tiGc and )( t

iGshift , i.e.,

)()(1 ti

ti

ti GG shiftcc +=+ , where )( t

iGc denotes the

center produced by tiG in the process of similartaxis,

and )( tiGshift denotes a vector produced by t

iG in the

process of dissimilation.

In the process of similartaxis, the center )( tiGc can be

determined based on the positional information of

individuals in the present tiG (the geometrical center of

tiG serves as )( t

iGc ). Unfortunately, the center )( tiGc

may cause some groups to repeatedly search the same areas. The shifting direction of the group must be controlled in the process of dissimilation so that each group can converge to an assigned area of Pareto front

( )( tiGshift is determined by a vector, which reflects the

position of every group in the decision space). The actual

center of 1+tiG is 1+t

ic with the information of both

)( tiGc and )( t

iGshift .

F. The Termination Criterion

A group of individuals mature if the change between the succeeding generations is too small in some consecutive

generations. In our experiments, ε<−∑ ′i

ii aa is used

as the change criterion, where ai is the individual of

element i ordered in 1+tiG , a i′ is the individual of

element i ordered in tiG , and ε is a pre-specified

positive value.

IV. SIMULATION RESULTS

A. The Test Functions

In our simulations, the SP-MEC is verified using four different test problems 1p , 2p , 3p , 4p . These problems are quoted from 1t , 2t , 3t and 6t

described in [5], and we follow the same parameter configurations as [5].

B. Parameter Settings

The following parameters are used in this algorithm: Population size S : 100 Number of the groups N : 5 External set size: 80 The reference algorithms are SPEA, NSGA, and VEGA. The raw data sets of these algorithms are acquired after 250 generations.

C. Experimental Results

The nondominated sets achieved by the VEGA, NSGA, SPEA, Pareto-MEC, and SP-MEC are shown in Figures 1 to 4, where the solid lines represent the Pareto fronts.

Figure 1. The solutions to test function 1p

Figure 2. The solutions to test function 2p .

Figure 3. The solutions to test function 3p .

Figure 4. The solutions to test function 4p .

In the test function 1p of Fig. 1, the distances of the

trade-off fronts of the SP-MEC to the Pareto fronts are more closely than the reference algorithms. Moreover, the solutions obtained by the SP-MEC distribute more uniformly. Concerning test functions 2p , 3p , 4p

(refer to Figs. 2, 3, and 4), the quality of the trade-off front achieved by the SP-MEC outperforms those of all reference algorithms on measures of the spread and distribution.

It should be emphasized that the results of the VEGA, NSGA and SPEA reference algorithms are produced after 250 generations. The SP-MEC and Pareto-MEC have the objective termination criterion, which is different from other reference algorithms. There are multi-groups in the SP-MEC instead of a single population in the reference algorithms. As a result, the numbers of generations are variable for every group. An average generation number

of all groups serves as the general criteria so that the SP-MEC is comparable with the reference algorithms with regard to their computational efficiencies. The numbers of generations for five groups are provided in Table. 1. The above results show that the SP-MEC has not only a better quality of solutions, but also a higher computational efficiency than the reference algorithms of VEGA, NSGA, and SPEA. As for the Pareto-MEC, the SP-MEC has a greater generation number, but its quality of solutions is higher than that of the Pareto-MEC.

The MEC is an effective searching algorithm combining the global search with local search. Our SP-MEC can converge to the Pareto front quickly, and achieve the high quality of the trade-off front for the multi-objective optimization. The score of SP-MEC includes both the optimum and the sub-optimum individual information. Therefore, the solutions acquired by the SP-MEC distribute uniformly on Pareto front. It can be concluded form the experimental results that the distances of the trade-off fronts of the SP-MEC to the Pareto fronts are more closely than that of the reference algorithms.

V. CONCLUSIONS

This paper describes a new algorithm for multi-objective optimization problems, SP-MEC, which introduces the theory of Pareto into the MEC. The basic principles of SP-MEC are: (1) a number of individuals are scattered in the whole solution space, and some better individuals of them are selected as the initial centers for every group; (2) each group only searches a local area, and gradually shifts from its initial center to the Pareto front; (3) during the process of shifting to the Pareto front, this algorithm would bound the searching region of the group, and control its shifting direction.

Table 1. THE NUMBERS OF GENERATIONS FOR FIVE GROUPS ON ALL TEST FUNCTIONS

1p 2p 3p 4p

Grp0-Gen 141 212 214 226

Grp1-Gen 132 249 192 219

Grp2-Gen 114 189 221 263

Grp3-Gen 157 234 187 169

Grp4-Gen 132 173 204 214

Min-Gen 114 173 187 169

Max-Gen 157 249 221 263

Average 135.2 211.4 203.6 218.2

The solution quality of the proposed SP-MEC is compared with that of the reference algorithms on four representative test functions: convexity, non-convexity, discreteness, and non-uniformity. Experimental results show that our SP-MEC can outperform the NSGA, VEGA, SPEA, and Pareto-MEC on all the test problems. On other hand, the computational efficiency of the SP-MEC is higher than many algorithms, such as the NSGA, VEGA, and SPEA. Furthermore, the subjective termination criterion is used in SP-MEC and Pareto-MEC instead of the normal pre-given number of generations. Compared with another algorithm from our recent work, Pareto-MEC, the computational efficiency of the SP-MEC is a little lower than that of the Pareto-MEC. Nevertheless, the solution quality of the SP-MEC is moderately higher. Therefore, it is can be concluded that the SP-MEC is a powerful searching algorithm for solving the multi-objective optimization problems.

ACKNOWLEDGMENT

The work of this paper is a part of Project 60174002 supported by Natural Science Foundation of China. The authors express their great acknowledgements for that. X. Z. Gao's research work was funded by the Academy of Finland under Grant 201353.

REFERENCES

[1] Chengyi Sun, Xiaohong Qi, and Ou Li, (2003), “Pareto-MEC for multi-objective optimization,” in Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Washington DC, pp. 321-328.

[2] Chengyi Sun, Yan Sun, and Wanzhen Wang, (2002), “A Survey of MEC: 1998-2001,” in Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Hammamet, Tunisia, pp. 445-453.

[3] Schaffer J. D.(1984), “Multiple Objective Optimization with Vector Evaluated Genetic Algorithms,” Ph. D. thesis, Vanderbilt University, Nashville, TN, 1984.

[4] Sun Chengyi and Sun Yan,(1998), “Mind-Evolution-Based Machine Learning: Framework and the implementation of optimization”, in Proceedings of IEEE International Conference on Intelligent Engineering Systems, pp. 355-359.

[5] Zitzler E.(1999), “Evolutionary Algorithm for Multiobjective Optimization: Methods and Applications,” Ph. D. thesis, Swiss Federal Institute of Technology(ETH), Zurich, Switzerland.

[6] Zitzler E., Laumanns M., and Thiele L.(2001), “SPEA2: Improving the strength Pareto evolutionary algorithm,” in K. Giannakoglou, D. Tsahalis, J. Periaux, P. Papailou, and T. Fogarty ed., Proceedings of the Evolutionary Methods for Design, Optimization and Control with Applications to Industrial Problems, Athens, Greece.

[7] Zitzler E., Thiele L.(1998), “An Evolutionary Algorithm for Multiobjective Optimization: The Strength Pareto Approach,” Technical Report 43, Computer Engineering and Communication Networks Lab(TIK), Swiss Federal Institute of Technology (ETH), Zurich, Switzerland.