multiobjective optimization (i) - massachusetts … multiobjective optimization ? while...

1 © Massachusetts Institute of Technology - Prof. de Weck and Prof. WillcoxEngineering Systems Division and Dept. of Aeronautics and Astronautics

Multidisciplinary System Multidisciplinary System Design Optimization (MSDO)Design Optimization (MSDO)

Multiobjective Optimization (I)Lecture 16

31 March 2004

byProf. Olivier de Weck


Where in Framework ?Where in Framework ?

Discipline A Discipline B

Discipline C

Inpu

t

Out

put

TradespaceExploration

(DOE)

Optimization Algorithms

Numerical Techniques(direct and penalty methods)

Heuristic Techniques(SA,GA, Tabu Search)

1

2

n

x

x

x

Coupling

1

2

z

J

J

J

ApproximationMethods

Coupling

Sensitivity Analysis

MultiobjectiveOptimization

Isoperformance

Objective Vector


Lecture ContentLecture Content

• Why multiobjective optimization?• Example – twin peaks optimization• History of multiobjective optimization• Weighted Sum Approach (Convex Combination)• Dominance and Pareto-Optimality• Pareto Front Computation - NBI


Multiobjective Optimization Problem Multiobjective Optimization Problem Formal DefinitionFormal Definition

( )

, , 1, ..., )

min ,

s.t. , 0

, =0

(i LB i i UB i nx x x =

≤

≤ ≤

Design problem may be formulatedas a problem of Nonlinear Programming (NLP). WhenMultiple objectives (criteria) are present we have a MONLP

( ) ( )[ ]

1

2

1

1

1

1

where

( ) ( )

( ) ( )

=

=

=

=

T

z

T

i n

T

m

T

m

J J

x x x

g g

h h


Multiple ObjectivesMultiple Objectives

1

2

3

cost [$]

- range [km]

weight [kg]

- data rate [bps]

- ROI [%]

i

z

J

J

J

J

J

= =

The objective can be a vector J of z system responsesor characteristics we are trying to maximize or minimize

Often the objective is ascalar function, but forreal systems often we attempt multi-objectiveoptimization:

Objectives are usuallyconflicting.


Why multiobjective optimization ?Why multiobjective optimization ?

While multidisciplinary design can be associated with the traditional disciplines such as aerodynamics, propulsion, structures, and controls there are also the lifecycle areas ofmanufacturability, supportability, and cost which require consideration.

After all, it is the balanced design with equal or weighted treatment of performance, cost, manufacturability and supportability which has to be the ultimate goal of multidisciplinary system design optimization.

Design attempts to satisfy multiple, possiblyconflicting objectives at once.


Example F/AExample F/A--18 Aircraft18 AircraftDesign

DecisionsObjectives

Aspect RatioDihedral AngleVertical Tail AreaEngine Thrust Skin Thickness

# of EnginesFuselage SplicesSuspension PointsLocation of MissionComputerAccess Door Locations

SpeedRangePayload CapabilityRadar Cross SectionStall SpeedStowed Volume

Acquisition costCost/Flight hourMTBFEngine swap timeAssembly hours

Avionics growthPotential


Multiobjective ExamplesMultiobjective Examples

Production Planningmax {total net revenue}max {min net revenue in any time period}min {backorders}min {overtime}min {finished goods inventory}

Aircraft Designmax {range}max {passenger volume}max {payload mass}min {specific fuel consumption}max {cruise speed}min {lifecycle cost}

1

2

z

J

J

J

=DesignOptimization

OperationsResearch


Multiobjective vs. MultidisciplinaryMultiobjective vs. Multidisciplinary

• Multiobjective Optimization– Optimizing conflicting objectives– e.g., Cost, Mass, Deformation – Issues: Form Objective Function that represents designer

preference! Methods used to date are largely primitive.

• Multidisciplinary Design Optimization– Optimization involves several disciplines– e.g. Structures, Control, Aero, Manufacturing– Issues: Human and computational infrastructure, cultural,

administrative, communication, software, computing time, methods

• All optimization is (or should be) multiobjective– Minimizing mass alone, as is often done, is problematic


Multidisciplinary vs. MultiobjectiveMultidisciplinary vs. Multiobjective

single discipline multiple disciplines

sing

le o

bjec

tive

mul

tiple

obj

.

single discipline multiple disciplines

Minimize displacements.t. mass and loading constraint

F

δl

mcantilever beam support bracket

Minimize stamping costs (mfg) subject

to loading and geometryconstraint

F

D

$

airfoilα (x,y)

Maximize CL/CD and maximizewing fuel volume for specified α, vo

Vfuelvo

Minimize SFC and maximize cruisespeed s.t. fixed range and payload

commercial aircraft

Image taken from NASA's website. http://www.nasa.gov.


Example: Double Peaks OptimizationExample: Double Peaks OptimizationObjective: max J= [ J1 J2]T (demo)

( )

( )

2 21 2

2 21 2

2 21 2

2 ( 1)1 1

3 511 2

( 2)1 2

3 1

105

3 0.5 2

x x

x x

x x

J x e

xx x e

e x x

− − +

− −

− + −

= −

− − −

− + +

( ) 2 22 1

2 2 2 22 1 2 1

2 ( 1)2 2

(2 )3 522 1

3 1

10 35

x x

x x x x

J x e

xx x e e

− +

− − − −

= +

− − + − −


Double peaks optimizationDouble peaks optimization

Optimum for J1 alone: Optimum for J2 alone:

x1* =0.05321.5973

J1* = 8.9280

J2(x1*)= -4.8202

x2* =-1.58080.0095

J1(x2*)= -6.4858

J2* = 8.1118

Each point x1* and x2* optimizes objectives J1 and J2 individually.Unfortunately, at these points the other objective exhibits a low objective function value. There is no single point that simultaneouslyoptimizes both objectives J1 and J2 !


Tradeoff between Tradeoff between JJ11 andand JJ22

• Want to do well with respect to both J1 and J2

• Define new objective function: Jtot=J1 + J2

• Optimize Jtot

Result:

Xtot* =0.87310.5664

J(xtot*) =3.0173 J13.1267 J2

Jtot* = 6.1439

=

max(J1)

max(J2)

tradeoffsolution

max(J1+J2)


History (1) History (1) –– Multicriteria Decision MakingMulticriteria Decision Making

• Life is about making decisions. Most people attempt to make the “best” decision within a specified set of possible decisions.

• Historically, “best” was defined differently in different fields:– Life Sciences: The best referred to the decision that minimized or

maximized a single criterion.– Economics: The best referred to the decision that simultaneously

optimized several criteria.

• In 1881, King’s College (London) and later Oxford Economics Professor F.Y. Edgeworth is the first to define an optimum for multicriteria economic decision making. He does so for the multiutility problem within the context of two consumers, P and π:

– “It is required to find a point (x,y,) such that in whatever direction we take an infinitely small step, P and π do not increase together but that, while one increases, the other decreases.”

– Reference: Edgeworth, F.Y., Mathematical Psychics,P. Keagan, London, England, 1881.


History (2) History (2) –– Vilfredo ParetoVilfredo Pareto• Born in Paris in 1848 to a French Mother and Genovese

Father• Graduates from the University of Turin in 1870 with a

degree in Civil Engineering– Thesis Title: “The Fundamental Principles of Equilibrium in

Solid Bodies”

• While working in Florence as a Civil Engineer from 1870-1893, Pareto takes up the study of philosophy and politics and is one of the first to analyze economic problems with mathematical tools.

• In 1893, Pareto becomes the Chair of Political Economy at the University of Lausanne in Switzerland, where he creates his two most famous theories:

– Circulation of the Elites– The Pareto Optimum

• “The optimum allocation of the resources of a society is not attained so long as it is possible to make at least one individual better off in his own estimation while keeping others as well off as before in their own estimation.”

• Reference: Pareto, V., Manuale di Economia Politica, SocietaEditrice Libraria, Milano, Italy, 1906. Translated into English by A.S. Schwier as Manual of Political Economy, Macmillan, New York, 1971.


History (3) History (3) –– Extension to EngineeringExtension to Engineering

• After the translation of Pareto’s Manual of Political Economyinto English, Prof. Wolfram Stadler of San Francisco State University begins to apply the notion of Pareto Optimality to the fields of engineering and science in the middle 1970’s.

• The applications of multi-objective optimization in engineering design grew over the following decades.

• References:– Stadler, W., “A Survey of Multicriteria Optimization, or the Vector

Maximum Problem,” Journal of Optimization Theory and Applications, Vol. 29, pp. 1-52, 1979.

– Stadler, W. “Applications of Multicriteria Optimization in Engineering and the Sciences (A Survey),” Multiple Criteria Decision Making –Past Decade and Future Trends, ed. M. Zeleny, JAI Press, Greenwich, Connecticut, 1984.

– Ralph E. Steuer, “Multicriteria Optimization - Theory, Computation and Application”, 1985


Notation and ClassificationNotation and Classification

Ref: Ralph E. Steuer, “Multicriteria Optimization - Theory, Computation and Application”, 1985

Traditionally - single objective constrained optimization:

( )max

. .

f

s t S

=∈

( ) objective function

feasible region

f J

S

If f(x) linear & constraints linear & single objective = LPIf f(x) linear & constraints linear & multiple obj. = MOLPIf f(x) and/or constraints nonlinear & single obj.= NLPIf f(x) and/or constraints nonlinear & multiple obj.= MONLP


Mapping SetsMapping Sets

Linear

Nonlinear

Decision Space(Design Space)

Criterion Space(Objective Space)

x1

x2

x1

x2

J1

J2

S Z

c1

c2

x1

x4

x3x2

J1

J3

J4

J2

J1

J2

Z’

J1

J4

J2x1

MOLP

MONLP

x2

x3

x4S’

S’’ J3

Z’’

Z’’’

Set of images ofall points in set S

19© Massachusetts Institute ofTechnology - Prof. de Weck and Prof. WillcoxEngineering Systems Division and Dept. of Aeronautics and Astronautics

Map: Double Peaks Map: Double Peaks –– manymany--toto--oneone

Xtot*

J(xtot*)

A function f which may (but does not necessarily) associate a given member of the range of f with more than one member of the domain of f.

“Range”“Domain”

A

Domain Range

Bfa

Many-To-One


Formal Solution of a MOO ProblemFormal Solution of a MOO ProblemTrivial Case:There is a point that simultaneously optimizesall objectives

*x S∈, where 1iJ i z≤ ≤

Such a point almost never exists - i.e. there is no pointthat will simultaneously optimizes all objectives at once

Two fundamental approaches:

( ){ }( )

1 2max , , ,

. . 1 i z

z

i i

U J J J

s t J f

S

= ≤ ≤∈

Scalarization Approaches

ParetoApproaches

1 2

1 2

and for

at least one

i i

i i

J J i

J J

i

≥ ∀

>

Preferences included upfront

Preferences included a posteriori


NotationNotation

SOLPSOLP { }max T J S= ∈

S feasible region in decision spaceif S is determined by linear constraints

{ }, 0,n mS = ∈ = ≥ ∈

MOLPMOLP { }

{ }

11max

max . .

T

zTz

J

J s t S

=

= ∈Multiobjectivelinear program

Single objectivelinear program


Notation (II)Notation (II)

C z x n Criterion matrix1 2 zc c c=

Gradientvector of z objectives

1 2

n

Ti i inx x x

∈

=

point in decision space

[ ]1 2

z

T

zJ J J

∈

=

(Criterion) Objective vector


SOLP versus MOLPSOLP versus MOLP

x1

x2

S

c1

x1*

x2*c2constraints

Optimal solutionif only J1 considered

Optimal solutionif only J2 considered

What is the optimalsolution of a MOLP?{ }

{ }1

2

max

max

. . S

J

J

s t

=

=

∈


WeightedWeighted--Sum ApproachSum Approach

x1

x2

Sc1

x1

x2

c2

constraints

Each objective i is multiplied by a strictly positive scalar λi

1

0, 1k

zi i

i

λ λ=

Λ = ∈ > =

x3*Solve thecomposite orWSLP:

{ }max S∈c31 2λ λ=

Strictly convexcombinationof objectives

criterioncone


Weighted Sum: Double PeaksWeighted Sum: Double Peaks( )1 21 where [0,1]totJ J Jλ λ λ= + − ∈

Demo: At each setting of λ we solve a new singleobjective optimization problem – the underlyingfunction changes at each increment of λ

∆λ=0.05


Weighted Sum Approach (II)Weighted Sum Approach (II)

λ=1

λ=0

Weighted sumfinds interesting, solutions but missesmany solutions ofinterest.


Weighted Sum (WS) ApproachWeighted Sum (WS) Approach

1

zi

MO ii i

J Jsf

λ=

=

Max(J1 /sf1)

Min(J2 /sf2)

miss thisconcave region

Paretofront

• convert back to SOP• LP in J-space• easy to implement• scaling important !• weighting determines which point along PF is found• misses concave PF

λ2>λ1

λ1>λ2

J-hyperplaneJ*i

J*i+1

PF=Pareto Front(ier)

weight

scalefactor


What are the weights What are the weights λλλλλλλλ ??

: zU →Decision Maker has a utility function U=f(J), J=J(x)

Then:

1

z

x x ii i

UU J

J=

∂∇ = ∇∂

Chain rule

Gradient of U at x S∈

where

i

i

x i

i

n

J

x

J

J

x

∂∂

∇ =∂∂

For LP’s1

z

ii

w=1

z

xi i

UU

J=

∂∇ =∂ Direction of

Gradient at x

1

ii

U Jw

U J

∂ ∂=∂ ∂

1

ii z

jj

w

wλ

=

=Weights determinedirection of gradientvector of U normalize


Group Exercise: Weights (5 min)Group Exercise: Weights (5 min)We are trying to build the “optimal” automobile

There are six consumer groups:-G1: “25 year old single” (Cannes, France)

-G2: “family w/3 kids” (St. Louis, MO)

-G3: “electrician/entrepreneur” (Boston, MA)

-G4: “traveling salesman” (Montana, MT)

-G5: “old lady” (Rome, Italy)

-G6: “taxi driver” (Hong Kong, China)

Objective Vector:J1: Turning Radius [m]J2: Acceleration [0-60mph]J3: Cargo Space [m3]J4: Fuel Efficiency [mpg]J5: Styling [Rating 0-10]J6: Range [km]J7: Crash Rating [poor-excellent]

J8: Passenger Space [m3]J9: Mean Time to Failure [km]

Assignment: Determine λi , i =1…99

1

1000ii

λ=

=WB=wheelbase, ED=engine displacement

x=[WB ED…]T

( )f=


What are the scale factors What are the scale factors sfsfii??

• Scaling is critical in multiobjective optimization• Scale each objective by sfi:• Common practice is to scale by sfi = Ji*• Alternatively, scale to initial guess J(xo)=[1..1]T

• Multiobjective optimization then takes place in a non-dimensional, unit-less space

• Recover original objective function values by reverse scaling

i i iJ J sf=

Example: J1=range [sm]J2=fuel efficiency [mpg]

sf1=573.5 [sm]sf2=36 [mpg]

“best in GM class”

Saab 9-5

Suzuki “Swift”


Properties of optimal solutionProperties of optimal solution

( ) ( )*optimal if

for and for S

≥

∈ ≠This is why multiobjective optimization is alsosometimes referred to as vector optimization

x* must be an efficient solution

S∈ is efficient if and only if (iff) its objective vector(criteria) J(x) is non-dominated

A point is efficient if it is not possible to move feasiblyfrom it to increase an objective without decreasing at leastone of the others

S∈

(maximization)


Dominance (assuming maximization)Dominance (assuming maximization)

Let be two objective (criterion) vectors.

Then J1 dominates J2 (weakly) iff

Moreprecisely:

z∈1

2i

z

J

JJ

J

= and ≥ ≠

1 2 1 2 and for at least one i i i iJ J i J J i≥ ∀ >

Also J1 strongly dominates J2 iff

Moreprecisely:

>1 2 i iJ J i> ∀


Dominance Dominance -- ExerciseExercise

max{range}min{cost}max{passengers}max{speed}

[km][$/km][-][km/h]

7587 6695 3788 8108 5652 6777 5812 7432321 211 308 278 223 355 401 208112 345 450 88 212 90 185 208950 820 750 999 812 901 788 790

#1 #2 #3 #4 #5 #6 #7 #8

MultiobjectiveAircraft Design

Which designs are non-dominated ? (5 min)


Set TheorySet TheoryNon-dominated Designs: #1, #2, #3, #4 and #8Dominated Designs: #5, #6, #7

Set Theory: DND

D ND∩ = ∅

S∈

J

Z

,D Z ND Z⊂ ⊂

D ND Z∪ =

J*

( ) ND∈

A solution must be feasible

A solution is either dominated or non-dominatedbut cannot be both at the same time

All dominated and non-dominatedsolutions must be feasible

All feasible solutions are either non-dominatedor dominated

Pareto-optimal solutions are non-dominated

Z∈


Procedure (I)Procedure (I)Algorithm for extracting non-dominated solutions:

Pairwise comparison

7587 > 6695321 > 211 112 < 345950 > 820

Score#1

Score#2#1 #2

1 00 10 11 0

2 vs 2Neither #1 nor #2 dominate each other

7587 > 6777321 < 355 112 > 90950 > 901

Score#1

Score#6#1 #6

1 0

1 0

1 01 0

4 vs 0Solution #1 dominatessolution #6

In order to be dominated a solution musthave a”score” of 0 in pairwise comparison


Procedure (II)Procedure (II)for ind1=1:(n-1)

for ind2=(ind1+1):n Ja=Jall(:,ind1).*sign(minmax);Jb=Jall(:,ind2).*sign(minmax); scorea=0;scoreb=0; for indz=1:z

if Ja(indz)>Jb(indz)scorea=scorea+1;

elseif Ja(indz)<Jb(indz)scoreb=scoreb+1;

else% both solutions are equal

endendif scoreb==0&scorea~=0

dmatrix(ind1,ind2)=1; %a dominates belseif scorea==0&scoreb~=0

dmatrix(ind2,ind1)=1; %b dominates aelse

% no domination in this pairend

endend

Pairwisecomparison

PopulateDominationMatrix

paretosort.m


Domination MatrixDomination Matrix

1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

8

12000001

Solution 2 dominatesSolution 5

0 0 0 0 1 1 2 0

Shows which solution dominates which othersolution (horizontal rows) and (vertical columns)

Solution 7 is dominatedby Solutions 2 and 8

Row Σ

j-th row indicateshow manysolutionsj-th solutiondominates

Column Σ

k-th column indicatesby how many solutionsthe k-th solution is dominated

Non-dominated solutions have a zero in the column Σ !


Dominance versus EfficiencyDominance versus Efficiency

Whereas the idea of dominance refers to vectors incriterion space J, the idea of efficiency refers to pointsin decision space x.

Mapping:

( ){ },zZ f S= ∈ = ∈

MOLP

MONLP

{ },zZ S= ∈ = ∈

dominance(objective space)

efficiency(decisionspace)

Z= reachable range in objective space


EfficiencyEfficiency

( ) ( )( ) ( )

A point is efficient iff there does

not exist another such that

and . Otherwise is inefficient.

S

S

∈

∈ ≥

≠

A point x is efficient if its criterion (objective) vectoris not dominated by the criterion vector of someother point in S.

Efficient set ND

Can use this criterionas a Pareto Filter if thedesign space has beenexplored (e.g. DoE).


Double Peaks: NonDouble Peaks: Non--dominated Setdominated Set

Filtered theFull FactorialSet: 3721

Non-dominatedset approximatesPareto frontier:79 points (2.1%)


ParetoPareto--Optimal vs NDOptimal vs ND

max (J1)

min(J2)TrueParetoFront

ApproximatedPareto Front

D ND PO

All pareto optimal points are non-dominatedNot all non-dominated points are pareto-optimal

√√√√√√√√√√√√ √√√√

It’s easier to show dominatedness than non-dominatedness !!!

Obtaindifferentpoints fordifferent weights


Direct Pareto Front CalculationDirect Pareto Front Calculation

SOO: find x*MOO: find PF

PFJ1

J2

PFJ1

J2

bad good

- It must have the ability to capture all Pareto points- Scaling mismatch between objective manageable- An even distribution of the input parameters (weights)

should result in an even distribution of solutions

A good method is Normal-Boundary-Intersection (NBI)


Normal Boundary Intersection (I)Normal Boundary Intersection (I)Goal: Generate Pareto points that are well-distributed

1J

2J

10

1

Reachable Range

NU*

1J

*2J

0

uUtopiaPoint

1. Carry out single objective optimization: ( )* 1,2,...,i iJ J i z= ∀ =

2. Find utopia point

( )i i

ii

J JJ

l

−=

( ) ( ) ( )1 2

T

zJ J J=

5. Normalize

3. Find Nadir Point

1 2

TN N NzJ J J=

( ) ( ) ( )minT

Ni i i iJ J J J=

4. Nadir-Utopia Distance

[ ]1 2

T

zl l l= =

1, 2,...,i z=


Normal Boundary Intersection (2)Normal Boundary Intersection (2)

• Carry out a series of optimizations • Restricted design space: U – Utopia Line between

anchor points, NU – normal to Utopia line• Feasible region restricted to one part of Range• Find Pareto point for each NU setting• Move NU from to in even increments• Yields remarkably even distribution of Pareto points• Applies for z>2, U-line becomes a Utopia-hyperplane

*1J *

2J

Reference: Das I. and Dennis J, “Normal-Boundary Intersection:A New Method for Generating Pareto Optimal Points in MulticriteriaOptimization Problems”, SIAM Journal on Optimization, Vol. 8, No.3, 1998, pp. 631-657


Lecture SummaryLecture Summary

• A multiobjective problem has more than one optimal solution• All points on Pareto Front are non-dominated• Methods:

• Weighted Sum Approach (Caution: Scaling !)• Pareto-Filter Approach• Normal Boundary Intersection (NBI) • More methods in the next lecture

The key difference between multiobjective optimizationmethods can be found in how and when designerpreferences are brought into the process.

…. More in next lecture


Remember ….Remember ….

Pareto Optimal means …..

“Take from Peter to pay Paul”

multiobjective optimization (i) - massachusetts … multiobjective optimization ? while...

Documents