department of mathematics and computer science 1212 1 6bv04 doe: optimization response surface...

department of mathematics and computer science

1

6BV04DOE: Optimization

Response Surface Methods


2

Contents• Optimisation steps• Box method• Steepest ascent method• Practical example• Response surface designs• Multiple responses• EVOP• Software• Literature


3

Optimisation stepsOptimisation is achieved by going through the following phases:

• screening (determine which factors really influence the outcome; tool: screening designs like fractional factorial)• improvement (approach optimum by repeated change of factor settings; tools: Box/simplex or steepest ascent approach)• determination of optimum (find optimal settings of factor settings; tool: response surface designs like CCD or Box-Behnken + analysis of response surface using eigenvalues)


4

current settings

improvement

optimum


5

Regression models used in optimisationStatistical techniques for optimisation assume the following (often reasonably satisfied in practice):“Far away” from the optimum a first order model often suffices. for example:

Y = ß0 + ß1x1 + ß2x2 +

“Near” the optimum often a quadratic (second order) model suffices. For example:

Y = ß0 + ß1x1 + ß2x2 + ß12x1x2 + ß11x12 + ß22x2

2 +

Lack-of-fit techniques must be applied in order to check whether these models are appropriate, since we cannot directly see whether we are near the optimum (cf. next slides).


6

Models

Far away from optimum:first order

model


7

Models

Near optimum:fitting a first order model

shows lack-of-fit

(curvature)


8

Models

Near optimum:second order

model


9

ImprovementIn order to efficiently move from current factor settings to factor setting that yield near-optimal values, 2 methods are available:• Box/Simplex method

– idea: form new full factorials in direction of largest increase in current full factorial

– simple; no statistics needed for implementation– not efficient

• Steepest ascent/descent method – idea: use 1st order regression model from fractional factorial

to obtain direction of largest increase (“steepest ascent”)– perform single runs in direction of largest increase until

increase stops– advanced– recommended since it is more efficient


10

Box method

40.6 41.9

41.8

41.2

41.3

39.3 40.9

41.5 40.0

direction of largest increase

direction of largest increase

stop if one has to return to

previous settings


11

Steepest ascent method

direction of steepest ascent

contour lines of first-order model

perpendicularto contour line

region where 1eorder-model

has been determined


12

start

screening

RSM design(CCD, ...)

single observationin direction

steepest ascent

full factorial+ centre points

1st ordermodel OK?

better observation?

stationary point

optimum?

stationary point

nearby?

go to stationary point

yes

no

yes

no

yes

no

no

acceptstationary point

end

fit 2nd order model

yes

Optimization scheme


13

Practical examplegoal: maximise yield of chemical reactor

significant factors obtained after screening experiment: • reaction time • reaction temperature

current factor setting: time = 35 min. temp = 155 °Ccurrent yield: 40 %


14

Steepest ascent22-design with 5 centre points:time: 30 - 40 min; temp: 150 - 160 °C

results: montgomery14-1.sfx

• there is no significant interaction• there is no significant lack-of-fit• the regression model is significant

Hence, we are not near the optimum.


15

Steepest ascent pathoutcome analysis of measurement:yield = 24.94 + 0.155*time + 0.065*temp

with coding: x1= (time-35)/5 x2 = (temp-155)/5

yield = 40.44 + 0.775*x1 + 0.325*x2

0.775

0.325

direction path: normal vector

step size: 5 min reaction time (choice of chemical engineer!)

5 0.3250.42

5 0.775 coded step size temp (= 2.1°C)


16

Steepest ascent path experiments

40

50

60

70

80

90

0 2 4 6 8 10 12

yiel

d

Further experiments with factor settings of experiment nr. 10.


17

Near the optimumSettings experiment 10:• time = 85 min• temperature = 175 °C

A 22 design with 5 centre points is executed.

results: montgomery14-4.sfx

Lack-of-fit indicates curvature. Hence, we now are probably near the optimum.


18

Quadratic modelsIn order to fit a quadratic model (suitable when we are near the optimum), we must vary the factors at 3 levels. A 2p-design with centre points does not suffice, because then all quadratic factors are confounded.

A 3p-design is possible, but not to be recommended:• number of runs grows fast • uses more runs than necessary to fit quadratic model.


19

Response surface designsThe following designs are widely used for fitting a quadratic model:• Central Composite Design (uniform precision of effect estimates)

• Box-Behnken Design (almost uniform precision of effect estimates, but usually fewer runs required than for CCD)

The choice between these models is usually decided by the availability of these designs for a given number of runs and number of factors.Note that there are other suitable designs (usually available in statistical software that supports DOE).


20

Central Composite Design

A CCD consists of 3 parts:

• factorial points

• axial points

• centre points

A CCD is often executed by adding points to an already performed 2p-design (highly efficient, but beware of blocking!).


21

RotatabilityIn a CCD there are 2 possible choices:• number of centre points• location axial points

By choosing the axial points at the locations (,0,…,0) etc.

with = (# factorial points)¼ , the design becomes

rotatable, i.e. the precision (variance) of the model

depends on the distance to the origin only. In other words,

one has the same precision for all factor estimates.


22

Box-Behnken designsThese are designs that

consists of combinations from 2p-designs.

Properties:• efficient (few runs)• (almost) rotatable• no corner points of

hypercube (these are extreme conditions which are often hard to set)


23

Stationary point

Near the optimum usually a quadratic model suffices:

jiji iji

k

iiii

k

ii xxxxY 2

110

How do find the optimum after we correctly estimated the parameters using a response surface design (CCD or Box-Behnken)?The next slides show the tools to derive optimal settings and the pitfalls that have to be avoided.


24

Recap: optimisation in dimension1• necessary condition for extremum: 1st derivative = 0• not sufficient:

“point of inflection”

• extra sufficient condition: 2nd derivative 0


25

Zero first derivatives: saddlepoint vs. maximum

saddle point (unfavourable) maximum (favourable)

2 2x y 2 2x y


26

Determination of type of optimum • Graphically: make contourplot (if 2

factors)• Analytically:

jiji iji

k

iiii

k

ii xxxxY

2

110

matrix notation: xBxbxY TT

0

Note: B must be chosenas symmetric matrix, see example:

11 12 13

12 22 23

13 23 33

/ 2 / 2

/ 2 / 2

/ 2 / 2


27

Stationarity and matrix analysisstationary point (zero first-order derivatives):

bBxBxbx

Y 10 2

102

characterisation through eigenvalues of matrix B:

xBx • all eigenvalues positive: min • all eigenvalues negative: max• eigenvalues different signs: saddle point

(the ’s are sometimes called “parameters of canonical form”)


28

Stationarity and matrix analysis

In StatGraphics: • augment design• add star points

Please note that additional centre points are addedand a block variable.

We can remove the centre points from the data setand ignore the block variable in the analysis.

StatGraphics results: montgomery14-6.sfx


29

Stationarity and matrix analysisUse Matlab to avoid manual computations: • create matrix B and vector b• compute eigenvalues and location of stationary point

>> B = [-2.75247 0.5/2 ; 0.5/2 -2.00253] / 2Analysis Summary----------------File name: D:\MyDocs\2DS01\collegesheets\montgomery14-6.sfxComment: Montgomery table 14-4

Estimated effects for opbrengst----------------------------------------------------------------------average = 79.94 +/- 0.11896A:tijd = 1.98994 +/- 0.188092B:temperatuur = 1.03033 +/- 0.188093AA = -2.75247 +/- 0.201705AB = 0.5 +/- 0.266003BB = -2.00253 +/- 0.20171----------------------------------------------------------------------Standard errors are based on total error with 7 d.f.


30


In Matlab: • create matrix B and vector b• compute eigenvalues and location of stationary point

>> B = [-2.75247 0.5/2 ; 0.5/2 -2.00253] / 2

B =

-1.3762 0.1250 0.1250 -1.0013


31



>> eig(B)


32



>> eig(B)

ans =

-1.4141 -0.9634

both negative → maximum


33



>> b = [1.98994 ; 1.03033] /2Analysis Summary----------------File name: D:\MyDocs\2DS01\collegesheets\montgomery14-6.sfxComment: Montgomery table 14-4

Estimated effects for opbrengst----------------------------------------------------------------------average = 79.94 +/- 0.11896A:tijd = 1.98994 +/- 0.188092B:temperatuur = 1.03033 +/- 0.188093AA = -2.75247 +/- 0.201705AB = 0.5 +/- 0.266003BB = -2.00253 +/- 0.20171----------------------------------------------------------------------Standard errors are based on total error with 7 d.f.


34



>> b = [1.98994 ; 1.03033] /2

b =

0.9950 0.5152


35



>> spcoded = -0.5 * inv(B) * b


36



>> spcoded = -0.5 * inv(B) * b

spcoded =

0.3893 0.3059

< 1.414 (distance star point) → inside experimental region


37


>> sporiginal = spcoded .* [5 ; 5] + [85 ; 175]

1 2

1 2

85 175,

5 55 85, 5 175

time tempx x

time x temp x



38



>> sporiginal = spcoded .* [5 ; 5] + [85 ; 175]

sporiginal =

86.9463 176.5293


39

start

screening

RSM design(CCD, ...)

single observationin direction

steepest ascent

full factorial+ centre points

1st ordermodel OK?

better observation?

stationary point

optimum?

stationary point

nearby?

go to stationary point

yes

no

yes

no

yes

no

no

acceptstationary point

end

fit 2nd order model

yes

Optimization scheme


40

Multiple responsesIf more than 1 response variable needs to be optimised, then a graphical way of optimising may be achieved by overlaying contour plots in case there are only 2 independent variables.

Overlay Plot

molecular_weightopbrengstviscositeit

tijd

tem

pe

ratu

ur

80 82 84 86 88 90170

172

174

176

178

180


41

Evolutionary Operation (EVOP)

Optimisation of a running production process is not always possible or may not be allowed because of costs:• involves interruption• may (temporarily) yield low quality products

An alternative is Evolutionary Operation:• experimentation within running operation • frequent execution of 2k-designs, starting at current settings• high and low setting of factors are close to each other, thus no risk of low quality products


42

Software• StatLab optimisation:

http://www.win.tue.nl/statlabInteractive software for teaching DOE through cases

• Box: http://www.win.tue.nl/~marko/box/box.html Game-like demonstration of Box method

• Matlab virtual reactor: Statistics toolbox -> Demos -> Empirical Modeling -> RSM demo

• Statgraphics: menu choice Special -> Experimental Design– design experiment with pre-defined catalogue– analysis of experiments with ANOVA


43

Literature• J. Trygg and S. Wold. Introduction to Experimental

Design – What is it? Why and Where is it Useful?, Homepage of Chemometrics, editorial August 2002: http://www.acc.umu.se/~tnkjtg/Chemometrics/editorial/aug2002.html

• DOE booklet from Umetrics: http://www.umetrics.com/pdfs/books/DOEbook.pdf

• Introduction to DOE from moresteam.com http://www.moresteam.com/toolbox/t408.cfm

• StatSoft Electronic Statistics Textbook, chapter on experimental design

• NIST Engineering Statistics Handbook: http://www.itl.nist.gov/div898/handbook/

department of mathematics and computer science 1212 1 6bv04 doe: optimization response surface...

Documents

computer science

department of mathematics

order model slide

c slide

efficient slide

direction steepest ascent

previous settings slide

nd order model