Managing & Mitigating Model RiskManaging & Mitigating Model Risk

Model Validation GroupModel Validation GroupArea of MethodologyArea of MethodologySantanderSantander

Alberto ElicesAlberto Elices

Financial Risk ManagementFinancial Risk ManagementMasters in Mathematical EngineeringMasters in Mathematical EngineeringUniversidad Complutense de MadridUniversidad Complutense de MadridMadrid, May 23rd – June 3rd, 2011Madrid, May 23rd – June 3rd, 2011

2

Outline

Introduction. Taxonomy of model risk. Model validation. Reconcile FO and Risk interests: FVA (Fair Value Adjustment). Conclusions.

3

Introduction

After the crisis in the 2nd half of 2007, a big concern about pricing models has been raised. Risk management and model validation raise now considerably more attention. Model validation: Validation of model implementation is no longer enough. Periodic and comprehensive review of pricing models. Estimation of model risk.

Risk management: Calculate and apply fair value adjustment (FVA). Limit model risk exposure (reduce volume of operations).

4

Taxonomy of model risk

Bad implementation. Missing a key source of risk. Uncertain model parameters. Difficulty to estimate market data. Wrong use of model. Market evolution risk.

5

Taxonomy of Model Risk: Bad implementation

Incorrect derivation of equations.

Bugs in the code.

Low performance.

Non-intuitive selection of time/space steps or MC paths.

Unstable or non-converging numerical schemes.

Absence of error trapping routines to avoid bad inputs.

6

Taxonomy of Model Risk: Missing a key source of risk

Inappropriate underlying stochastic process.

Robust calibration to market but unrealistic evolution or vice versa.

Missing market variables which affect pricing.

Under-dimensioned model. Some factors are assumed to be deterministic instead of stochastic.

Incorrect assumptions about relations between underlying variables.

7

Taxonomy of Model Risk: Uncertain model parameters

One or more model parameters cannot be properly estimated from liquid market prices.

Wrong set of underlying calibration instruments.

Smile not accounted for.

Inappropriate treatment of extreme cases.

8

Taxonomy of Model Risk: Difficulty to estimate market data

Unavailable market quotes: data highly dependent on sample length.

Incomplete market quotes: inconsistent and non-arbitrage free data.

Complete market quotes: estimation of data from bid - ask.

9

Taxonomy of Model Risk: Wrong use of model

Model used outside valid range.

Product priced with a model for which it was not designed.

Wrong configuration parameters (insufficient time steps or simulation runs).

10

Taxonomy of Model Risk: Market evolution risk

An initially valid model can become inadequate due to:

Changing market conditions.

Changing market pricing consensus.

Market instability.

11

35

3 Model Validation functionModel Validation

Background and motivation Model testing

• Model adequacy analysis• Test complex models in simple cases• Premium analysis• Greek and stability analysis

Integration in corporate systems Conclusions and recommendations

Validation Process

12

35

• Why was this product developed?• Is it a product designed for a campaign or for a single operation?• What are the market conditions which motivate this product?• Is it a change or improvement of an existing product?• If the product is part of a more complicated deal, what is the whole picture?• Similarities to any other previously validated models

3 Model Validation function

Get important information before validation starts as it may considerably change the strategy of tests

Validation Process

Background and motivation

13

35

• Why a particular model was chosen? Which simplifications are involved, are they reasonable?• Study the adequacy of the calibration and skew treatment• Valid range where inaccuracies or approximations are acceptable

3 Model Validation function

Adequacy of the model from a theoretical and practical point of view

Validation Process

Model Testing – Model adequacy analysis

14

35

• Reduce the model under validation to vanilla products, comparing both premium and sensitivities

- Run a few cases manually varying product inputs- Run a more systematic analysis varying market data (e.g. spot

prices, volatility or interest rate levels)

3 Model Validation function

Test models against simpler known solutions

Validation Process

Model Testing – Complex model in simple cases

15

35

• Define a set of scenarios used to test the premium:- Random: define limits to the deal and market input parameters- Manual: justify why they have been chosen

• Calibration tests• Analyze differences between Front Office and Validation models:

- Do not ignore small discrepancies. Track down their origin!• Display plots or tables with the results:

- A histogram might be appropriate for random scenarios- When a single parameter is varied at a time, a series of plots is

a simple and clear option- When two parameters are varied, display a series of tables

3 Model Validation functionValidation Process

Model Testing – Premium analysis

16

35

• Convergence analysis:- Testing the accurateness of premium varying Monte Carlo

simulations or grid parameters for PDEs• Life cycle analysis:

- Test evolution of premium, cash flow account and convergence to the final payoff throughout the life of the option

• Robustness analysis under realistic market parameters

3 Model Validation function

Justifying differences achieves a common sense compromise between extreme and real market scenarios

Validation Process

Model Testing – Premium analysis

17

35

Front OfficeModel

Risk Dept.Model 0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

9.00

10.00

0.0 2.0 4.0 6.0 8.0 10.0

Discrepancies? NOYES

Hypothesis

error

FVA Validation

3 Model Validation functionValidation Process

Model Testing – Premium analysis

18

35

• Plot premium and sensitivities:- Varying spot prices, volatility levels, maturity, interest rate

levels, correlation, etc.- Testing their variation range and whether they evolve smoothly

• Compare actual premium change and its prediction using Taylor expansion• Life cycle analysis: Test smooth evolution of sensitivities

3 Model Validation function

Sensitivities need to be stable enough for hedging and risk calculation (e.g. gamma stability for VaR in Madrid)

Validation Process

Model Testing – Greek and stability analysis

19

35

• Stress the initial market conditions to check its behaviour

3 Model Validation function

-1.000.000

-800.000

-600.000

-400.000

-200.000

0

200.000

400.000

600.000

800.000

-2 -1 0 1 2Vega

V ariación de la curva de tipos

Is the model stable?

NO if market moves to this area!

Validation Process

Model Testing – Greek and stability analysis

20

35

• Different environments are used for validation:- Excel spreadsheet + add-in- IL text files

• It is necessary to ensure that the model is correctly integrated into the final platform

3 Model Validation function

Models are global and unique, but there are different corporate systems

Validation Process

Integration in corporate systems

21

35

3 Model Validation function

Questions to be adressed:• Has the model passed validation?• Is the model fit for its intended purpose?• What are the boundaries within the model is suitable?• What are the weaknesses and limitations of the model?• What are the recommendations for best use?• If differences between models appear, are they acceptable?• Is the model stable and robust enough for daily management without support?• Is it necessary to take into account a Fair Value Adjustments?

Validation Process

Conclusions and recommendations

22

How to reconcile FO and Risk department interests?: FVA

The FVA should cover the expected hedging loss and its uncertainty: When hedging is carried out with a model with aggresive prices, the expected hedging loss is the fair minus the aggresive price: that difference plus a cushion for its uncertainty is the FVA.

FVA as a means to approve campaigns using limited models with controllable risk: A FVA allows accomplishing campaigns which would not possible with a slower more sophisticated model.

FVA to foster improvement of FO models: Models with limitations should be given FVA which should be released the more the model is improved.

23

How to reconcile FO and Risk department interests?: FVA

FVA calculation philosophy: They should be transparent, easy to compute. They should be dynamic, stable, with smooth evolution through time (they should decrease approaching expiry). They should balance risk limitation and trading mitigation. Front Office should be able to reproduce them.

24

How to reconcile FO and Risk department interests?: FVA

How FVA can be calculated: Use FVA tables calculated from studies (either with real or toy models).

Use FO pricing models to estimate model risk:– Changing unobserved or non-calibrated model parameters (mean reversion, correlations, etc).

– Compare prices of deals valued with different FO models (better models might take too long on a daily basis).

Simulate or back test portfolio hedging: sometimes impractical.

25

Example of provisions: liquidity of volatility surface.

Liquidity provisions should allow unwinding a position.Sensitivities are diffused through time (towards lower more liquid maturities) and moneyness (towards closer to ATM).

Diffusion of offsetting sensitivities for the 10 year tenor

with W=0.5

0

200

400

600

800

1,000

1,200

1 2 3 4 5 7 10 15 20 25 30 40 50 60 70

Tenor

Se

nsi

tivi

tie

s (£

k)

Original Sensitivity 1 tailed distribution2 tailed distribution

Diffusion of sensitivities for the 5% strike with W=0.05

ATM = 6%

0

200

400

600

800

1,000

1,200

2.50%

3.00%

3.50%

4.00%

4.50%

5.00%

5.50%

6.00%

6.50

%7.0

0%7.5

0%8.0

0%

Tenor

Sen

siti

viti

es (

£k)

2 tail diffusion 1 tailed diffusion

Original Sensitivities

26

Example of provisions: liquidity of volatility surface.

Liquidity provisions in practice:Movements of implied volatility surface might be mainly explained by parallel and slope (skew) shifts.Parallel shifts: hedged with ATM options.Slope shifts: hedged with risk reversals sensitive to slope.

27

Example of provisions: uncertainty of volatility.

A Heston model calibrated to market is considered:Confidence intervals for total variance & vol are calculated.Up and down increments of vol are calculated vs maturity.Provision: max(|up & down movement * vega|).

28

Example of calculation of FVA: FX self-quanto options.

Self quanto options are the worst case for correlation is 1. Four valuation methods are compared:

Black Scholes with at-the-money (ATM) volatility (left equation).Black Scholes with volatility selection (VolSelection) to the strike level of the non-quanto option (left equation).

Vega-volga-vanna (VVV) method.Estimation of probability distribution of ST using Gatheral’s parametrization of implied volatility surface (right equation)

( )[ ]),( TtPKSprice EURQ

EUR T

+−= E

tttEURt

USDtQ

t

Qt dWdtrrSdS σσ ++−= )( 2

( )[ ]0

1),(S

TtPSKSprice USDTTEUR+−= E

ttEURt

USDt

t

t dWdtrrSdS σ+−= )(

ATM VolSelection Gatheral VVV0.3704 0.3712 0.3990 0.3950

29

Examples of bespoke validation: FX self-quanto options.

Gatheral’s parametrization is fitted to market data:Left plot: market fitting (maximum difference is 5.65bp).Right plot: implied probability distribution.

Delta 0.05 0.1 0.15 0.2 0.25 0.3 0.5 0.7 0.75 0.8 0.85 0.9 0.95

Vol 35.87 33.80 32.79 32.13 31.65 31.25 30.55 30.59 30.77 31.03 31.47 32.22 33.80

30

Examples of bespoke validation: FX self-quanto options.

A set of a hundred random scenarios changing dates, interest rates and strikes preserving shape of volatility surface is generated. Call option prices are compared.

-0.0600

-0.0400

-0.0200

0.0000

0.0200

0.0400

0.0600

0.0800

0 10 20 30 40 50 60 70 80 90 100

Gatheral-VVVGatheral-VolSelectGatheral-ATM

31

Examples of bespoke validation: FX self-quanto options.

Put option prices give a lot less differences for same scenarios. Volatility selection or ATM methods can significantly diverge from fair value. Best method: volga-vanna.

-0.0100

-0.0050

0.0000

0.0050

0.0100

0.0150

0.0200

0 10 20 30 40 50 60 70 80 90 100

Gatheral-VVVGatheral-VolSelectGatheral-ATM

32

Conclusion: Managing & Mitigating Model Risk

Model validation. Quantifying model risk of a desk. Periodic & comprehensive review of models. Limit calculations. Fair Value Adjustments (FVA). Premium sensitivity to non-calibrated parameters. Comparison with other models. Simulation of hedging strategies.