derivative-enhanced variable fidelity kriging approach

Derivative-Enhanced Variable FidelityDerivative-Enhanced Variable FidelityKriging ApproachKriging Approach

Dept. of Mechanical Engineering,University of Wyoming, USA

Wataru YAMAZAKI

23rd, September, 2010

Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ.Wataru YAMAZAKI, Univ. of Wyoming-2-

Motivation*Surrogate models for

- Efficient Design Optimization- Efficient Aerodynamic Data Modeling- Inexpensive Uncertainty Quantification

*For more accurate surrogate models- Gradient/Hessian Information

Efficient adjoint approaches- Variable Fidelity Function Information

Combination of absolute values of high-fid model andtrends of low-fid models

High-Fidelity Model Low-Fidelity Model

Experimental data CFD result

RANS Inviscid

Finer mesh CFD result Coarser mesh CFD result

Converged solution Loose converged solution


Variable Fidelity Kriging ModelConsider a random process model estimating a function valueby a linear combination of variable fidelity function values

3211

331

221

11 ,ˆ321

nnnywywywyn

iii

n

iii

n

iii

x

Minimizing Mean-Squared-Error (MSE) between exact/estimated function

with unbiasedness constraints

0,0,1321

13

12

11

n

ii

n

ii

n

ii www

Solving by using the Lagrange multiplier approach

iforJJJ

w

J

w

J

w

J

wwwyMSEJ

iii

n

ii

n

ii

n

ii

0

1ˆ

321321

133

122

111

321

x

2

133

122

111

321

ˆ YywywywEyMSEn

iii

n

iii

n

iiix


Variable Fidelity Kriging ModelIntroducing correlation function for covariance termsCorrelation is estimated by distance between two pts with radial basis function

ji

ji

ji

RZZCov

RZZCov

RZZCov

ji

ji

ji

xx

xx

xx

xx

xx

xx

,,

,,

,,

222222

122121

112111

L

rw

F

FR

~~

0T

Unknown parameters are determined by the following system of equations

,~,,~,~,~~3121111 1

wwww nT w

Final form of the variable fidelity Kriging model is

T

T

T

TTTy

00

~~~ˆ

3

13

2

121

1

333

122

2

111

yyyF

FR

L

r

ywywywx


Variable Fidelity Kriging Model

FYRrL xx 1ˆ TTy

samplesgivenatninformatioexact

dataobservedbetweennscorrelatio

dataobservedandbetweennscorrelatio

termconstantmean

T

TT

3132121

111

,,

yyyY

R

xr

YRFFRF

x

In matrix form expression

Correlation parameters in R and r, factorsare estimated by a likelihood maximization approach

312121 ,,

33

11222

21

2

1

21

22

1

,0,,0,,0 yyYyY

YFRYYRY

T

TT

Rln2

1ln

2

1ln 2

1321 nnnL


Variable Fidelity Kriging Model

321321

333

3

233

2

133

1

322

3

222

2

122

1

311

3

211

2

111

1

33331

331

3311

23231

231

2311

13131

131

1311

23231

231

2311

22221

221

2211

12121

121

1211

13131

131

1311

12121

121

1211

11111

111

1111

nnnnnn

nnn

n

nnn

n

nnn

n

nnn

n

nnn

n

nnn

n

nnn

n

nnn

n

nnn

n

RR

RR

RR

RR

RR

RR

RR

RR

RR

RR

RR

RR

RR

RR

RR

RR

RR

RR

R

else

dforddddscf

dscfRndv

kkkij

0

13183513

1,

,

226

1

0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0

d

SCF

θ=1.0

θ=2.0

Correlations between all sample points combinations by a RBF


Derivative-enhanced Kriging

Extension of direct approach of gradient-enhanced Kriging

nnnnnn

Nn

Nn

Nn

Nn

Nn

Nn

Ndvn

Ndvn

Nn

Nn

Nn

Nn

Nn

Nn

Nn

Nn

Nn

Nn

Nn

Ndvn

Nn

Ndvn

Nn

dvdv

nn

dv

n

dvdv

nn

dv

nnnn

dv

n

dv

nn

dvdv

nn

dv

n

dv

nn

dv

n

dv

nn

dv

n

dv

n

dv

nn

nnn

dv

nnn

nnn

n

dv

n

n

xx

R

xx

R

xx

R

xx

R

x

R

x

R

xx

R

xx

R

xx

R

xx

R

x

R

x

Rxx

R

xx

R

x

R

xx

R

x

R

x

R

xx

R

xx

R

xx

R

x

R

x

R

x

Rx

R

x

R

x

R

x

RRR

x

R

x

R

x

R

x

RRR

22

,4

211

2

,4

2

,3

11

2

,3

2

,2

2

,2

11

22

,4

11

211

2

,4

11

2

,3

11

11

2

,3

11

2

,2

11

2

,2

2

,3

11

2

,3

2

,2

11

,2

,,

11

2

,3

11

11

2

,3

11

,2

11

2

,2

11

,

11

,

2

,2

11

2

,2

,

11

,,,

2

,2

11

2

,2

,

11

,,,

111

111111111

111

111111111

11

1

111111

111

xxxxxxxxxxxx

xxxxxxxxxxxx

xxxxxxxxxxxx

xxxxxxxxxxxx

xxxxxxxxxxxx

xxxxxxxxxxxx

R

Correlations between F-F, F-G, G-G, F-H, G-H and H-H Up to 4th order derivatives of correlation function Automatic Differentiation by TAPENADE No sensitive parameter Better matrix conditioning than indirect approach

n

iii

n

iii

n

iii yyywy

1

''

1

'

1

ˆ x


Derivative-enhanced Variable Fidelity Kriging

High Fidelity

FunctionGradientHessian

Hessian Vector

1st Low Fidelity


Hessian Vector

2nd Low Fidelity


Hessian Vector

A Kriging surrogate model byabsolute function values of high-fidelity leveland function trends of low-fidelity levels

Results & DiscussionResults & Discussion


1D Analytical Function Case

2 high-fidelity samples 5 low-fidelity samples (+0.5) 5 another low-fidelity samples (-0.5)

-0.5

0.0

0.5

1.0

0.0 0.2 0.4 0.6 0.8 1.0

Exact Function

High only

High+Low1

High+Low2

High+Low1+Low2

High Fid

Low Fid-1

Low Fid-2



-0.5

0.0

0.5

1.0

0.0 0.2 0.4 0.6 0.8 1.0

Exact Function

High only

High+Low1

High+Low2

High+Low1+Low2

High Fid

Low Fid-1

Low Fid-2




2D Cosine function

Analytical gradient/Hessian Latin hypercube sampling for high and low-fidelity samples Comparison by RMSE

highlow

high

ff

xxf

xx

x

1.0

cos

1

21

M

i

exactkrig

iiyy

MRMSE

1

2ˆ

1xx



Only 5 high-fidelity samples Derivative information is useful to construct accurate model

Exact function FuncFunc/GradFunc/Grad/Hess



Exact function

Only function information for both high/low-fidelity samples 5 high-fidelity samples with 0-200 low-fidelity samples Low-fidelity information is useful to construct accurate model



1.E-07

1.E-06

1.E-05

1.E-04

1.E-03

1.E-02

0 10 20 30 40 50

Number of High Fidelity Samples

RM

SE

of

2D C

os F

unct

ions

High_F

High_FG

High_FGH

High_F/Low_F

High_FG/Low_F

High_F/Low_FG

High_FGH/Low_F

High_F/Low_FGH

High_FG/Low_FG

High_FGH/Low_FGH


2D Analytical Function Case# of Low Fidelity Samples = 50

1.E-07

1.E-06

1.E-05

1.E-04

1.E-03

1.E-02

0 10 20 30 40 50

Number of High Fidelity Samples

RM

SE

of

2D C

os F

unct

ions

High_F

High_FG

High_FGH

High_F/Low_F

High_FG/Low_F

High_F/Low_FG

High_FGH/Low_F

High_F/Low_FGH

High_FG/Low_FG

High_FGH/Low_FGH

50 low-fidelity sample points Best performance in FGH for both high/low-fid samples



1.E-05

1.E-04

1.E-03

1.E-02

1.E-01

0 10 20 30 40 50

Number of High-Fidelity Samples

RM

SE

SFVF_MultiVF_ShiftVF_XshiftVF_RndmVF_Lin

21

1.0

21

1.0

5.01.0

0.1

1.0

cos

1

xxff

ranff

ff

ff

ff

xxf

highlowLin

highlowRndm

highlowXshift

highlowShift

highlowMulti

high

xx

xx

xx

xx

xx

x

Only function information for both high/low-fid samples Accuracy of VF model depends on trends of low-fidelity model But anyway helpful at smaller numbers of high-fid samples


Mach-AoA Hypersurfaces

2D aerodynamic data modeling of Cl, Cd, CmMach = [0.5; 1.5]AoA = [0.0; 5.0]

Inviscid steady flow computationsaround NACA0012

Only function informationbecause of noisy design space

High fidelity model by a fine mesh20,000 elementsComputational time factor = 1

Low fidelity model by a coarse mesh1,700 elementsComputational time factor = 1/30



1.E-04

1.E-03

1.E-02

1.E-01

0 20 40 60 80 100

# of High Fidelity Samples

Mea

n E

rror HFonly

VFM(LF050)

VFM(LF100)

VFM(LF200)

Mean error comparison in drag coefficient Improvements at smaller numbers of high fidelity samples

N

i

krigd

exactd CC

NErrorMean

1

1



Uncertainty analysis at M=0.8, AoA=2.5 for both Mach/AoA 1000 CFD evaluations for a specified σ value In total 7000 CFD evaluations (= 1000 x 7) for full-MC

Full-MC results for σ=0.1



-0.11

-0.10

-0.09

-0.08

-0.07

-0.06

0.00 0.02 0.04 0.06 0.08 0.10

Standard Deviation for Mach/AoA

Mea

n of

Cm

Full-MC

IMC_H005L000

IMC_H005L020

IMC_H005L050

IMC_H005L100

IMC_H005L400

Mean of Cm

0.000

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.00 0.02 0.04 0.06 0.08 0.10

Standard Deviation for Mach/AoAV

aria

nce

of C

m

Full-MC

IMC_H005L000

IMC_H005L020

IMC_H005L050

IMC_H005L100

IMC_H005L400

Variance of Cm

More accurate uncertainty analysis by Inexpensive MCwith variable fidelity Kriging model


2D Airfoil Shape Optimization

Unstructured mesh CFD Steady inviscid flow, M=0.755 NACA0012, 9 DVs by PARSEC Objective function as

lift-constrained drag minimization

Adjoint gradient available Geometrical constraint for sectional area Fidelity levels by finer/coarser meshes

(1.0 : 0.1)

22

2target2target

000.02

100675.0

2

12

1

dl

dddlll

CC

CCwCCwF


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI

Exact Function Sample Points Kriging RMSE EI

Infill Sampling Criteria for Optimization How to find promising location on surrogate model ? Maximization of Expected Improvement (EI) value Potential of being smaller than current minimum (optimal) Consider both estimated function and uncertainty (RMSE)

s

yys

s

yyyyEI minmin

min

xxxx

00s

EI,

y

EI



HFonly: Start from 16 HF initials, new samples by HF evaluationsLFonly: Start from 128 LF initials, new samples by LF evaluationsVFM: Start from 128 LF initials, new samples by HF evaluationsAdj: Adjoint gradient evaluations only for new optimal designs

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

0 100 200 300 400 500Number of All Sample Points

Obj

ecti

ve F

unct

ion

HFonly

HFonly_Adj

LFonly

HFeval for LFopt

VFM

VFM_Adj

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

0 100 200 300 400 500Number of All Sample Points

Obj

ecti

ve F

unct

ion

HFonly

HFonly_Adj

LFonly

HFeval for LFopt

VFM

VFM_Adj



To include low-fidelity / derivative information is promising

lowF

highF

highFG NNNCCF 1.00.10.2

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

0 100 200 300 400 500Computational Cost Factor

Obj

ecti

ve F

unct

ion

HFonly

HFonly_Adj

LFonly

HFeval for LFopt

VFM

VFM_Adj

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

0 100 200 300 400 500Computational Cost Factor

Obj

ecti

ve F

unct

ion

HFonly

HFonly_Adj

LFonly

HFeval for LFopt

VFM

VFM_Adj



Shock reduction on upper surface Towards supercritical airfoils in HFonly Additional adjustment of problem definition ?

NACA0012,Obj = 0.121

Optimal by HFonly,Obj = 6.66e-4

Optimal by VFM_Adj,Obj = 1.66e-4

Pressure Distributions


Concluding Remarks / Future Works

Development of derivative-enhanced variable fidelity Kriging model Combination of absolute function values of high-fidelity samples

and function trends of low-fidelity samples

More accurate fitting on exact function Efficient inexpensive Monte-Carlo simulation at much lower cost Faster convergence towards global optimum

Application to Euler/NS/WTT cases and so on

Thank you for your attention !!

Appendix


Gradient/Hessian-enhanced KrigingImplementation Details

Correlation function of a RBF

Estimation of hyper parameters by maximizing likelihood function with GA

Correlation matrix inversion by Cholesky decomposition

Search of new sample point location by maximizing Expected Improvement (EI) value with GA

else

hforhhhhscf

0

13183513

1,

226



12

21

1

1.0

cos

1

xx

x

ff

xxf

21 Distribution of estimated


Aerodynamic Data Modeling

0.208

0.209

0.210

0.211

0.212

1.390 1.395 1.400 1.405 1.410Mach Number

CL

CFD Data

Linear by Adj_Grad

Quadratic by Adj_G/H

0.1080

0.1082

0.1084

0.1086

0.1088

0.1090

1.390 1.395 1.400 1.405 1.410

Mach Number

CD

CFD Data

Linear by Adj_Grad

Quadratic by Adj_G/H

Cl Cd

NACA0012 M=1.4 AoA=3.5[deg] Noisy in Mach number direction


Infill Sampling Criteria for Optimization How to find promising location on surrogate model ? Expected Improvement (EI) value Potential of being smaller than current minimum (optimal) Consider both estimated function and uncertainty (RMSE)

s

yys

s

yyyyEI minmin

min

xxxx

00s

EI,

y

EI

0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0Design Variable

Fun

ctio

n / R

MS

E

0.0E+00

2.0E-03

4.0E-03

6.0E-03

8.0E-03

1.0E-02

EI


EI-based criteria have good balancebetween global/local searching

derivative-enhanced variable fidelity kriging approach

Documents