A New Multiobjective Genetic Programmingfor Extraction of Design Information

from Non-dominated Solutions

Tomoaki Tatsukawa

Taku Nonomura

Akira Oyama

Kozo Fujii

ISAS/JAXA, Japan

ISAS/JAXA, Japan

ISAS/JAXA, Japan

ISAS/JAXA, Japan

BackgroundPage. 2

Problem Setting

Multi-objective optimization

Data mining methods

Extraction of design knowledge

© Mitsubishi Heavy Industries, LTD.

Mitsubishi Regional Jet (MRJ)

LE-X

Multi-Objective Design Exploration(MODE)Obayashi, et.al. , “Multi-Objective Design Exploration for Aerodynamic Configurations” , 2005

Multi-Objective Design Exploration- Problem setting -

■ Design paramters

Kink position (x, y)

■ Objective functions

1. Drag to be minimized (M=2.0, AOA = 0)

2. Maximum L/D to be maximized

at subsonic condition (M=0.8)

3. Maximum L/D to be maximized

at supersonic condition (M=2.0)

4. Body volume to be maximized

Page. 3

Ascent phase

Return phase

Lift-off

Nose-entry

Vertical landing

3.33

1.0x

yKink position(x,y)

CLCD

CA

0 [deg]

90 [deg]

A.O.A.

Ascent phase Return phase

Page. 4

Multi-Objective Design Exploration- Multi-objective optimization & Data mining -

Problem Setting

Data mining methods

Extraction of design knowledge

・ It is difficult to reveal non-linear relationship (sin(x1), x1x2, x1/x2)between parameters

Genetic programming is one of the evolutionary technique to reveal non-linear relationship.

Volume

Normalized

0.2,)/( MMAXDL

8.0,)/( MMAXDL

0,0.2, AOAMAC

x

y

Scatter plot matrix with correlation coefficient

Multi-objective optimization

Objective

Page. 5

• Present a new data mining method for MODE to extract design information from non-dominated solutions

Approach– Multi-objective GP(MOGP) for MODE is introduced – MOGP is applied to the practical multi-objective

optimization problem of reusable launch vehicle

Page. 6

About Genetic Programming• Extension of Genetic Algorithm(GA) (Koza, 1990)

sin

1x 2x 3)sin( 1x 23x

1.0 4.2 3.0

1x 2x 3x

GA (Array) GP(Tree expression)

x

6.28 x

X

sin

X

x

10

x

+

X

x

x

X

X

x

• Genetic operatorsCrossover

x

6.28 x

X

sin

X

x

6.28 x

X

cos

X

Mutation

• Genome expression of GP is different from GA

Genetic Programming in MODEPage. 7

Problem Setting

Multi-objective optimization

Data mining methods

Extraction of design knowledge 2. The measure of complexity

Minimization of the number of nodes

1. The measure of accuracyMaximization of correlation coefficient

2x 3

Function nodeTerminal node The number of node = 3

■ Objective functions

GP is used as symbolic regression technique

- Extract effective non-linear terms- Give us hint of unknown new parameters

Page. 8

Proposed GP

Maximization of squared correlation coefficient

max

)ˆˆ()(

)ˆ)(()ˆ,(

2

22

,

,12

ffff

ffffffCor

jiji

jiji

N

j

i

We want to extract “x” as design information

Proposed MOGP- Measurement of Accuracy-

Mi ,...,2,1

Proposed MOGP- Objective Function -

Page. 9

MOGPGP

1) Maximization of correlation coefficient between obji2) Minimization of the number of nodes

1) Maximization of squared correlation coefficient between obj12) Maximization of squared correlation coefficient between obj2

N) Maximization of squared correlation coefficient between objNN+1) Minimization of the number of nodes

obji : ith objective function of MOO

1

1

obj1

obj20

New parameter?

1

obj1

Node

Length

0

1

obj2

0Node

Length

Symbolic Regression Example

• Test function

• Data set

– 40 sample points (random)

• GP

– 1000 Individuals, 1000 Generations

– 15 Trial

Page. 10

10510),,(

10510),,(

10510),,(

21

3

33213

13

3

23212

32

3

13211

xxxxxxf

xxxxxxf

xxxxxxf

]1,1[,, 321 xxx

Results of Proposed MOGP

Page. 11

21

3

33213 2),,(ˆ xxxxxxf

13

3

23212 2),,(ˆ xxxxxxf

32

3

13211 5.0),,(ˆ xxxxxxf

MODE of reusable launch vehicle (RLV)- Problem Setting -

■ Objective functions

1. Drag to be minimized (M=2.0, AOA = 0)

2. Maximum L/D to be maximized

at subsonic condition (M=0.8)

3. Maximum L/D to be maximized

at supersonic condition (M=2.0)

4. Body volume to be maximized

■ Design paramters

Kink position (x, y)

Page. 12

Ascent phase

Return phaseLift-off

Nose-entry

Vertical landing

3.33

1.0x

yKink position(x,y)

CLCD

CA

0 [deg]

90 [deg]

A.O.A.

Page. 13

Gene type Tree expression

Generation 1500

Population 1500

Crossover ratio 0.8

Mutation ratio 0.1

Function sets + , - , * , /

Terminal sets Design parameter (x, y),Constants [-1, 1]

Constraint The number of nodes>1

Computational conditionsObjective functions

１．Maximization of squared correlation between CA, M=2.0, AOA=0

２．Maximization of squared correlation between L/DMAX, M=0.8

３．Maximization of squared correlation between L/DMAX, M=2.0

４．Maximization of squared correlation between Volume

５．Minimization of the number of nodes

in syntax tree

Extraction of design information using Proposed MOGP

CLCD

CA

0 [deg]

90 [deg]

A.O.A.

2x 3

Function node

Terminal node

Results of MOGP

Page. 14

))/(( 0.2, MMAXDL

))/(( 8.0, MMAXDL

)( 0,0.2, AOAMAC

0 1)(VolumeCor

2222 33.08.028.1

3.43.433.5

yyxxyx

xxyyfgp

　　　　　　

The maximum sum of all squared correlation

Correlation

CA,M=20,AOA=0 -0.99

(L/D)MAX,M=0.8 0.44

(L/D)MAX,M=2.0 0.98

Volume -0.93

　Cor

　Cor

　Cor

PreferedRegion

Page. 15

Results of MOGP

x

y

kin

)33.3

5.0arctan(

180)arctan(

180180

x

y

x

ykin

kin

gpf

])[90()1(][])[60

(

])[64.083.452.2180

])[74.08.1653.188(

3(...)42

3

32

32

yOxOyOy

xyOyyy

yOyy

Floor

　　　

　　　

2222 33.08.028.13.43.433.5 yyxxyxxxyyfgp

Page. 16

Volume

Normalized

0.2,)/( MMAXDL

8.0,)/( MMAXDL

0,0.2, AOAMAC

kin

Non-dominatedsolution

Dominatedsolution

Results of MOGP

Original design parameters

x

y

Conclusions• A new multi-objective genetic programming for design

exploration is proposed.– The unique feature of MOGP is the simultaneous

symbolic regression to multiple variables using correlation coefficients.

• MOGP is applied to non-dominated solutions of the design optimization problem of a bi-conical shape reusable launch vehicle.– The result of proposed MOGP presents symbolic equations

which have high correlation to zero-lift drag at supersoniccondition, maximum lift-to-drag at supersonic condition andvolume of shape at the same time.

– These results also have high correlation to kink angle of thebody geometry. It implies that proposed GP is capable offinding composite new design parameters.

Page. 17

Page. 18

Page. 19

About Genetic Programming• Extension of Genetic Algorithm(GA) (Koza, 1990)

sin

1x 2x 3)sin( 1x 23x

1.0 4.2 3.0

1x 2x 3x

GA (Array) GP(Tree expression)

x

6.28 x

X

sin

X

x

10

x

+

X

x

x

X

X

x

• Genetic operatorsCrossover

x

6.28 x

X

sin

X

x

6.28 x

X

cos

X

Mutation

• Genome expression of GP is different from GA

MODE for RVT- Problem Definition -

■ Design parameters

– Kink position(x, y)

■ Constraint conditions

– Base diameter = 3[m]

– Length of the body = 10[m]

– Nose radius(rf) = 0.043[m]

– Base corner radius(rs) = 0.057[m]

– Base radius(R) = 3.0[m]

Page. 20

3.33(10[m])

rf: 0.043

rs: 0.057

1.0

(3[m])

R: 3.0

x

y

Kink position(x,y)

CLCD

CA

0 [deg]

90 [deg]

A.O.A.

Computational methods and conditions

Mach

number

Angle of

Attack [deg.]

Ascent phase 2.0 0

Return phase 2.0 10,25,40

0.8 10,25,40,55

(Re = 1.0x107)

0.0

0.2

0.4

0.6

0.8

1.0

1.2

0.0 20.0 40.0 60.0

A.O.A. [deg.]L

/D

L/D MAX■ Flow analysis

20L Computational Grid( 100x53x93 )

L

■ CFD

– 3rd order MUSCL+SLAU (for shock instability)

– Baldwin-Lomax

■ MOEA

– Non-dominated Solution GA-II (NSGA-II)

■ Computational Methods

■ Computational Conditions

■ Optimization‒ Population size: 20

‒ Generation size: 20 400 body shapes(3200 CFD runs) are evaluated

Page. 21

Comparison of computational time

Page. 22

0 0.5 1 1.5 2 2.5 3 3.5

2 objective GP

Proposed MOGP

Computational Time

Vertical Landing Reusable Launch Vehicle (RLV)

Page. 23

- One of the future space transportation systems

Reusable Sounding Rocket Vehicle(RSRV)Reusable Vehicle Testing(RVT)

- Motivation of development- Low cost, reusability and reliability

Vertical Landing Reusable Launch Vehicle (RLV)

Page. 24

- RLV is a vertical landing SSTO rocket.- RLV flies over a wide range of the flight speed and attack angles.

■ Flight sequence & Design requirements

Lift-off

Nose-entry

Vertical landing

Mission RequirementHigh ascent altitude

Safety & Cost RequirementLarge downrange

More knowledge on aerodynamic shape design is necessary

■ Important feature

It is important to understand relationshipbetween shape parameter and flow field

Proposed GP

A new type of Genetic Programming

Page. 25

1. Change the measure of accuracy

GP

Minimization of mean absolute error

N

yy iiiˆ

min

Proposed GP

Maximization of correlation coefficient

22 )ˆˆ()(

)ˆˆ)((max

yyyy

yyyy

iiii

iii

GP

1) Minimization of mean absolute error between obji2) Minimization of the number of nodes

2. Change the number of objective functions

1) Maximization of correlation coefficient between obj12) Maximization of correlation coefficient between obj2

N) Maximization of correlation coefficient between objNN+1) Minimization of the number of nodes

obji : ith objective function of MOO

Page. 26

x y x2 y2 xy xy2 x2y x3 Const.

Maximum Cor(CA,M=2.0,AOA=0 ) 2.75 -4.1 -2.6 -2 3.1 -1 1 -1.65

Maximum Cor(L/DMAX,M=0.8 ) 0.7 0.6 -1 -4 4 2 0.4

Maximum Cor(L/DMAX,M=2.0 ) -3.87 8.75 1.3 5 -0.1 -0.3

Maximum Cor(Volume) 1 -2 0.2 0.1 0.3

Maximum sum of all correlation 4.3 -5.33 -1.28 -0.33 4.3 -0.8

①

②

③

④

①

②

③

④

kin

⑤

⑤

Results of proposed MOGP

Multi-Objective Design ExplorationPage. 27

■Objective functions

1. Maximization of lift coefficient(CL) 2. Minimization of drag coefficient(CD)

-0.08

-0.06

-0.04

-0.02

0.00

0.02

0.04

0.06

0.0 0.2 0.4 0.6 0.8 1.0

y/c

Control point

y/c

x/c

Fixed point(Trading edge）

Fixed point(Leading edge)

(x1,y1)

(x2,y2)(x3,y3)

(x6,y6)

(x5,y5)

(x4,y4)

■Design parameters

e.g.) MODE for 2D wing shape

- Problem Setting -

Problem Setting

Multi-objective optimization

Data mining methods

Extraction of design knowledge

Multi-Objective Design ExplorationPage. 28

0.00

0.05

0.10

0.15

0.20

-0.2 0.0 0.2 0.4 0.6 0.8 1.0

Designs with constraint violation

Dominated solutions

Nondominated solutions

CD

Cl

CL max solution

CD min solution

L/D max solution

Solutions against constraintsDominated solutionsNon-dominated solutions

LC

Non-dominated solutions are obtained by using multi-objective evolutionary algorithms

- Multi-objective optimization -

Problem Setting

Multi-objective optimization

Data mining methods

Extraction of design knowledge

Page. 29

Cl

0.02 0.06 0.10 -0.06 -0.02 0.02 0.06 0.80 0.84 0.88 0.92 0.10 0.15 0.20 0.25

0.2

0.4

0.6

0.8

0.02

0.04

0.06

0.08

0.10

0.95 Cd

0.90 0.92 y4

0.02

0.04

0.06

0.08

0.10

0.12

-0.06-0.04-0.020.000.020.040.06

0.88 0.93 0.95 y2

0.81 0.78 0.93 0.85 y3

-0.070-0.065-0.060-0.055-0.050-0.045

0.80

0.82

0.840.86

0.880.90

0.92

-0.73 -0.80 -0.78 -0.90 -0.66 x6

-0.70 -0.71 -0.50 -0.55 -0.37 0.38 x20.40

0.45

0.50

0.55

0.2 0.4 0.6 0.8

0.10

0.15

0.20

0.25

-0.52 -0.59

0.02 0.06 0.10

-0.71 -0.67

-0.070 -0.060 -0.050

-0.55 0.57

0.40 0.50

0.19 x4

Scatter plot matrix with correlation coefficient

Multi-Objective Design Exploration

LC

DC

4y

2y

3y

6x

2x

4x

- Data mining methods -

Problem Setting

Multi-objective optimization

Data mining methods

Extraction of design knowledge

Multiple-regression analysis

426324 21.033.033.023.067.05.1 xxxyyyCL

426324 18.024.019.057.018.05.1 xxxyyyCD

Computational methods and conditions

Mach

number

Angle of

Attack [deg.]

Ascent phase 2.0 0

Return phase 2.0 10,25,40

0.8 10,25,40,55

(Re = 1.0x107)

0.0

0.2

0.4

0.6

0.8

1.0

1.2

0.0 20.0 40.0 60.0

A.O.A. [deg.]L

/D

L/D MAX■ Flow analysis

20L Computational Grid( 100x53x93 )

L

■ CFD

– 3rd order MUSCL+SLAU (for shock instability)

– Baldwin-Lomax

■ MOEA

– Non-dominated Solution GA-II (NSGA-II)

■ Computational Methods

■ Computational Conditions

■ Optimization‒ Population size: 20

‒ Generation size: 20 400 body shapes(3200 CFD runs) are evaluated

Page. 30

Results of MOGP

Page. 31

))/(( 0.2, MMAXDL

))/(( 8.0, MMAXDL

)( 0,0.2, AOAMAC

0 1)(VolumeCor

65.126.275.21.31.4 3222 xyxyxxxyy

4.03.187.35575.8 22 xxxyyy

2222 33.08.028.13.43.433.5 yyxxyxxxyy

3.01.02.02 2 xxyyy

4.06.07.0244 222 yxxxyxyy

(A)

(B)

(C)

(D)

(E) The maximum sum value of all squared correlation

CA_M20 0.89

LD_M08 -0.078

LD_M20 -0.94

Volume 1.00

CA_M20 0.98

LD_M08- -0.36

LD_M20 -1.0

Volume 0.96

CA_M20 -1.0

LD_M08 0.46

LD_M20 0.98

Volume -0.91

CA_M20 -0.99

LD_M08 0.44

LD_M20 0.98

Volume -0.93

　Cor

　Cor

　Cor

CA_M20 -0.44

LD_M08 0.97

LD_M20 0.35

Volume -0.11

Page. 32

New Param.(A)

New Param.(B)

New Param.(C)

New Param.(D)

kin

New Param.(E)

Results of MOGP

x

y

kin

)33.3

5.0arctan(

180)arctan(

180180

x

y

x

ykin

Example of GPsymbolic regression

Page. 33

321

2

3

2

121321 )cos()sin(),,( xxx

x

xxxxxxxf

f

1x

2x

3x

Test function ：

Data set

Gene type Tree expression

Generation 1000

Population 1000

Crossover ratio 0.8

Mutation ratio 0.2

Function sets +, - , *, /, sin, cos

Terminal sets Design parameters,Constants [-1, 1]

Constraint The number of nodes>1

Computational Condition

Example of GP- Result -

Page. 34

321

2

3

2

121321 )cos()sin(),,( xxx

x

xxxxxxxf 574th generation →

Residual (Accuracy)

No

de

Len

gth

(Co

mp

lexi

ty)

symbolic regression example

• Test function

• Data set

– 40 sample points (random)

• GP

– 3 type of GP

• TOGP-MAE （2 Objective GP, Mean Absolute Error)

• TOGP-SCC （2 Objective GP, Squared Correlation Coefficient)

• MOGP-SCC （Multi Objective GP, Squared Correlation Coefficient)

– 1000 Individuals, 1000 Generations

– 15 Trial

Page. 35

10510),,(

10510),,(

10510),,(

21

3

33213

13

3

23212

32

3

13211

xxxxxxf

xxxxxxf

xxxxxxf

]1,1[,, 321 xxx

Results

Page. 36

TOGP-SCCTOGP-MAE MOGP-SCC

- History of accuracy

- Comparison of computational time