interval arithmatic and automatic di erentiation euroad workshop... · interval analysis summary...

AimsCalibration of Heston Model

Interval AnalysisSummary

Interval Arithmatic and Automatic Differentiationin Optimization and Model Calibration

Grzegorz Kozikowski

University of Manchester, HPCFinance

Sophia-Antipolis, 11th June 2013Euro AD Workshop

1 / 32

Experience with AD and HPC:

Bachelor Thesis:

Implementation of Automatic Differentiation library using IntervalAnalysis and OpenCL

Forward/Reverse Mode for the Gradient, the Hessian (CPU and GPU)

PARA 2012 Proceedings LNCS 7782, 489-503, 2013

Master Thesis:

Parallel approach to Monte Carlo simulation for Option PriceSensitivities using the Adjoint and Interval Analysis (CUDA)

the Adjoint for the first and the second-order Greeks (CPU and GPU)

PPAM 2013 Proceedings

1 / 32

Agenda

1 Aims

2 Calibration of Heston Model

3 Interval Analysis

4 Summary

2 / 32

Global optimization algorithm for non-linear regression.The approach is based on Interval Analysis and AutomaticDifferentiation using HPC technologies.

Application

Calibration of the Heston Model, ill-posed optimization problems.

Optimization problem

Let us f : Rn → R:

minimizex

subject to:

hi (x) ≤ ai , i = 1, . . . , n.

gj(x) ≤ bj , j = 1, . . . ,m.

3 / 32

Application

minimizex

subject to:

hi (x) ≤ ai , i = 1, . . . , n.

gj(x) ≤ bj , j = 1, . . . ,m.

3 / 32

Application

minimizex

subject to:

hi (x) ≤ ai , i = 1, . . . , n.

gj(x) ≤ bj , j = 1, . . . ,m.3 / 32

Optimization methods:

Consider a contour line plot of some function f:

from starting points (0, 0), (0, 4), (4, 0), converges to 1

from starting points (4, 4), (2, 2), converges to point 2

Unsolved problem in real numbers!!!(gradient based optimization algorithms or heuristics)

4 / 32

Optimization methods:

Global optimization based on IntervalArithmetic/Automatic Differentiation- global extremum guaranteed- embarrassingly parallel problem- intervals

5 / 32

Heston ModelCalibration function

Heston Model

Heston Stochastic Volatility Model

The dynamics of the Heston model can be described as follows:

dSt = µStdt +√

VtStdZ 1t

dVt = κ(θ − Vt)dt + σ√

VtdZ 2t

dZ 1t dZ 2

t = ρdtSt asset price at time tµ - constant drift of the asset pricedt - time incrementVt - variance of the asset price at time tdZt increment of a Brownian motion at time tκ - constant mean reversion factor of the asset price varianceθ - long-term level of asset price varianceσ - volatility of volatilityρ - correlation between Brownian motions

6 / 32

Heston Model - a semi-closed form solution

Solution of Heston PDE is:

C (S ,V , t, ξ) = SP1 − e−rTKP2

where ξ = (V0, κ, θ, σ, ρ, µ) and:

Pj(S ,V , t, ξ,K ) = 12 + 1

∫∞0 Re(

e−iφln(K)fj (S ,V ,t,ξ)iφ )dφ

fj(S ,V , t, ξ) = eC(T−t,φ)+D(T−t,φ)Vt+iφln(St))

C (T − t, φ) = rφir + aσ2 [(bj − ρσφi + d)τ − 2ln( 1−gedr

1−g )]

D(T − t, φ) =bj−ρσφi+d

σ2 ( 1−edr1−gedr )

g =bj−ρσφi+dbj−ρσφi−d

d =√

(ρσφi − bj)2 − σ2(2ujφi − φ2)u1 = 1

2 , u2 = −12 , a = κθ, b1 = κ+ λ− ρσ, b2 = κ+ λ,

7 / 32

Calibration

Minimize errors between prices predicted by the Heston model andthe market option data (different maturities, strike prices, options):

ri (ξ) = Ci (ξ)− Cmkti

Global optimization - Calibration of the Heston Model

The aim is to find the set of input-parameters

ξ = (V0, κ, θ, σ, ρ, µ), ξεX

that minimize an error function:

minξεX

G (ξ) = minξεX

√√√√Nquotes∑i=1

(Ci (ξ)− Cmkti )2

8 / 32

Calibration

Minimize errors between prices predicted by the Heston model andthe market option data (different maturities, strike prices, options):

Global optimization (the minmax form)

The aim is to find the set of input-parameters

ξ = (V0, κ, θ, σ, ρ, µ), ξεX

that:minξεX

maxiε1,...,Nquotes

|Ci (ξ)− Cmkti |

9 / 32

Most optimization require the gradient and the Hessian:

the gradient:

∇G (ξ) = 2

Nquotes∑i=1

ri (ξ)∇ri (ξ) =

2∑Nquotes

i=1 ri (ξ) ∗ dri (ξ)dV0

2∑Nquotes

i=1 ri (ξ) ∗ dri (ξ)dκ

2∑Nquotes

i=1 ri (ξ) ∗ dri (ξ)dθ

2∑Nquotes

i=1 ri (ξ) ∗ dri (ξ)dσ

2∑Nquotes

i=1 ri (ξ) ∗ dri (ξ)dρ

2∑Nquotes

i=1 ri (ξ) ∗ dri (ξ)dµ

the Hessian:

∇2G (ξ) = 2

Nquotes∑i=1

(ri (ξ)∇2ri (ξ) +∇ri (ξ)∇ri (ξ)T ) = [...]NxN

10 / 32

IntroductionReduction box techniquesOverviewStage 1Stage 2Stage 3

Interval arithmetic

Approach to putting bounds in mathematical computation andthus, developing numerical methods that yield reliable results

Each single real-number X is represented as a pair of bounds[xl , xu], where xl < xu

Interval operators are inclusion isotonic(Z ⊂ X ) => f (Z ) ⊂ f (X )Optimization - reduction of an input space

11 / 32

Interval arithmetic

Approach to putting bounds in mathematical computation andthus, developing numerical methods that yield reliable results

Each single real-number X is represented as a pair of bounds[xl , xu], where xl < xu

Interval operators are inclusion isotonic(Z ⊂ X ) => f (Z ) ⊂ f (X )Optimization - reduction of an input space

11 / 32

Interval operators

Each single real-number X is represented as a pair of [x1, x2],where x1 < x2

Elementary arithmetic operations:

[x1, x2] + [y1, y2] = [x1 + y1, x2 + y2][x1, x2]− [y1, y2] = [x1 + y2, x2 + y1]

[x1, x2] ∗ [y1, y2] =[min(x1y1, x1y2, x2y1, x2y2),max(x1y1, x1y2, x2y1, x2y2)]

[x1,x2][y1,y2] = [x1, x2] ∗ [ 1

y1, 1y2

Conclusion: Even if interval operations are more computationally expensive, they take infinite spaces into account

(non differentiable functions). Many numerical algorithms using intervals may efficiently solve the mathematical

problems that are unsolved in real number arithmetics.

12 / 32

Reduction box techniques

Box consistency

Consider an equation f (x1, ..., xn) = 0 where x1εX1, ..., xnεXn.

Replace all the variables except the i-th by intervals:q(xi ) = f (X1, ...Xi−1, xi ,Xi+1, ...Xn) = 0

Eliminate subsets of Xi not satisfying q(xi ) = 0

13 / 32

Reduction box techniques

Contractors

Property that allows us to set constraints for input parameterstaking into account bounds of the result.Consider f (x1, x2) where x1εX1 and x2εX2 and f (x) = y

14 / 32

Contractors - Forward/Backward propagation algorithm

If f (x) = exp(x) + sin(x) and xε[x ] .To obtain the exact domains for y = f (x) we use contractors:

forward propagation: backward propagation:[a1] = exp([x ]) [a2] = ([y ]− [a3]) ∩ [a2][a2] = sin([x ]) [a3] = ([y ]− [a2]) ∩ [a3]

[y ] = [a2] + [a3] [x ] = sin−1([a3]) ∩ [x ][x ] = log([a1]) ∩ [x ]

15 / 32

Forward/Backward algorithm + Automatic Differentiation

If f ′(x) = exp(x) + cos(x) and xε[x ] .To obtain the exact domains for 0 = f ′(x) we use contractors:

forward propagation: backward propagation:[a1] = exp([x ]) [a2] = (0− [a3]) ∩ [a2][a2] = cos([x ]) [a3] = (0− [a2]) ∩ [a3]

[y ] = [a2] + [a3] [x ] = cos−1([a3]) ∩ [x ][x ] = log([a1]) ∩ [x ]

16 / 32

Monotonicity/Concavity tests

Monotonicity test

Let ∇(X ) the interval gradient of a box X.If 0 /∈ ∇(X ), then eliminate X

Concavity test

Let H(X ) the interval Hessian of a box X.If some Hii (X ) < 0, then eliminate X

Disjoint boxes

If X1 ∩ X2 = ∅ and fupper (X1) < flower (X2) then eliminate X2

17 / 32

Taylor’s expansions

Taylor series - the gradient

Consider a Taylor expansion of f around x (the middle of the box):

f (y)εf (x) +n∑

(yi − xi )gi (1)

Eliminate all y that:

f (y)εf (x) +n∑

(yi − xi )gi > f (2)

18 / 32

Taylor’s expansions

Taylor series - the Hessian

Consider a Taylor expansion of f around x (the middle of the box):

f (y)εf (x) + (y − x)Tg(x) +1

2H(x)(y − x) (3)

Eliminate all y that:

f (y)εf (x) + (y − x)Tg(x) +1

2H(x)(y − x) > f (4)

19 / 32

Newton Method - equation f(X)=0

From the mean value theorem:f (xu)− f (xl) = (xu − xl)f ′(ξ) and (xl < ξ < xu)

If xl is a zero of f then: xl = xu − f (xu)f ′(ξ)

If f ′(ξ)εf ′([xl , xu]) and m = mid([xl , xu]) then:

N([xl , xu]) = m − f (m)F ′([xl ,xu ]) ,

[xl , xu] = [xl , xu] ∩ N([xl , xu])20 / 32

Newton Method - equation f(X)=0

From the mean value theorem:f (xu)− f (xl) = (xu − xl)f ′(ξ) and (xl < ξ < xu)

If xl is a zero of f then: xl = xu − f (xu)f ′(ξ)

This method can be extended to multi-dimensional case

21 / 32

22 / 32

Heston Calibration - Interval Approach

1 Begin with a box X in which the global minimum is sought

2 Use bid-ask spread to contract ξ to the conditionCi (ξ) = [Cmkt

i − bid ,Cmkti + ask]

3 Evaluate the joint set: ξ = ξ1 ∩ ξ2 ∩ ... ∩ ξNquotes

4 If ξ = ∅ then go to Stage 2

23 / 32

7 Find minimum fsomemin of G (ξ) by using heuristic/gradientmethods

8 If interval contraction was not successful, fsomemin is globalminimum

9 Contract each |Ci (ξ)− Cmkti | = [0, fsomemin]

10 Evaluate the set: ξ = ξ1 ∩ ξ2 ∩ ... ∩ ξNquotes

11 Contract ∇G (ξ) = 0 or∑Nquotes

i=1 ∇ri (ξ) = 0

24 / 32

i=1 ∇ri (ξ) = 0

24 / 32

i=1 ∇ri (ξ) = 0

24 / 32

i=1 ∇ri (ξ) = 0

24 / 32

i=1 ∇ri (ξ) = 0

24 / 32

i=1 ∇ri (ξ) = 0

24 / 32

13 Use a multidimensional interval Newton method for eachsub-box to solve (∇G (ξ) = 0)

14 Contract ξ to the condition Hii (ξ) < 0 or use Taylorexpansions

15 Eliminate sub-boxes whose lower bound > upper bound of thebox for which G(X) is the lowest

16 Choose the best sub-boxes

25 / 32

Interval results of arithmetic operations are not always sharp(f (xεX ) ⊂ f (X ), but not always f (xεX ) ⊆ f (X )) !!!Conclusion:The sub-box whose error function is the lowest known value is notalways the best.

Glower (ξb1) < Glower (ξb2) < .... < Glower (ξbm)

Answer:Try to contract the difference between the best pair of thesub-boxes. If successful, eliminate all the other. 26 / 32

13 Use multidimensional Newton method for each sub-box tosolve (∇G (ξ) = 0)

18 Repeat Stage 2 until the difference between lower and upperbound of ε is less than error tolerance (for each dimension)

27 / 32

Heston Calibration - Interval Approach1 Begin with a box X in which the global minimum is sought

2 Use bid-ask spread to contract ξ to the condition Ci (ξ) = [Cmkti − bid, Cmkt

i + ask]

5 Use bid-ask spread to contract ξ to the condition Ci (ξ) = [Cmkti − bid, Cmkt

i + ask]

6 Find minimum fsomemin of G(ξ) by using heuristic/gradient methods

7 If interval contraction was not successful, fsomemin is global minimum

8 Contract each |Ci (ξ)− Cmkti | = [0, fsomemin ]

10 Contract ∇G(ξ) = 0 or∑Nquotes

i=1 ∇ri (ξ) = 0

12 Use multidimensional Newton method for each sub-box to solve (∇G(ξ) = 0)

13 Contract ξ to the condition Hii (ξ) < 0 or use Taylor expansions

14 Eliminate sub-boxes whose lower bound > upper bound of the box for which G(X) is the lowest

17 Repeat 6-17 until the difference between lower and upper bound of ε is less than error tolerance (for eachdimension)

28 / 32

QuestionsBibliography

Summary

The Gradient and the Hessian are evaluated by AutomaticDifferentiation routines

If some intermediate result has an infinite bound, useextended interval analysis and contractors

During interval contraction or Gauss-Newton method, theinitial box might be divided into many sub-boxes

All the sub-boxes can be processed in parallel (GPU, multicoreCPUs)

Application of the Pareto frontier algorithm for intervalsand additional contraction

29 / 32

Summary

29 / 32

Summary

29 / 32

Summary

29 / 32

Summary

29 / 32

Questions

30 / 32

Bibliography

Hansen Eldon, Global Optimization using Interval Arithmetics New York Basel, 2006

Grzegorz Kozikowski, Evaluation of Option Price Sensitivities based on the Automatic Differentiation

Methods using CUDA, Warsaw University of Technology 2013, Master Thesis

Grzegorz Kozikowski, Library for Automatic Differentiation using OpenCL, Warsaw University of Technology

2011, Bachelor Thesis

Grzegorz Kozikowski and Bartlomiej Jacek Kubica, Interval Arithmetic and Automatic Differentiation using

OpenCL technology, Springer 2013, LNCS 7738

Max E. Jerell, Automatic Differentiation and Interval Arithmetic for Estimation of Disequilibrium Models,

Luenberger G. David, Ye Yinyu: Linear and nonlinear programming, Springer 2008

Bazara S. Mokhatar, Nonlinear programming, Theory and algorithms, Willey Interscience

31 / 32

Thank You

Thank You very muchfor Your attention

32 / 32

Features

Intervals deal with:

Non-continuous and non-differentiable functions

Uncertainty of machine representation of numbers (outwardrounding)

Discretization errors

Analysis of the simulation results based on uncertaininput-parameters

Numerical problems that are unsolved in real-numbers (globaloptimization)

Interval results of operations are not always sharp(sharp bounds - NP-hard problem)

32 / 32

Upper-bound test

1 Consider f (x1, x2) where x1εX1 and x2εX2

2 Suppose f ∗ - global minimum of f(x)

3 Evaluate f at point/interval x0: fu(x0) > f ∗

4 As fu(x0) > f ∗, reduce the box for which f > fu(x0) bycontracting an inequality: f (x) = [−∞, fu(x0)]

5 As a result we obtain a new reduced box

x0 - the middle of box of the point generated by some optimizationmethods for real numbers.

32 / 32

Upper-bound test

32 / 32

Upper-bound test

32 / 32

Upper-bound test

32 / 32

interval arithmatic and automatic di erentiation euroad workshop... · interval analysis summary...

Documents

fundamental theorem of arithmatic

calculating interval forecasts -...

arithmatic mean

4 arithmatic logic unit (alu)

fixed point arithmatic

n x 2 ค าเฉลี่ยเลขคณ ิต s 1 n...

interval notation: ], not interval...

12 april 2013lecture 2: intervals, interval arithmetic and...

arithmatic numeric ability additional

manual-interval, interval 120cm, interval 140cm-all-va

class vii arithmatic ex 2.1

special arithmatic operations

15 obdg08b tcm summary tables - gsi.ext.gm.com ·...

unit 07: arithmatic and geometric sequence, general

cardinal arithmatic

statistics - interval...

matrix algebra -...

dti2143 chapter 3 arithmatic relation-logicalexpression

computer arithmatic

acomputer arithmatic