““Planning for an uncertain Planning for an uncertain futurefuture””

Overcoming the challenges we Overcoming the challenges we face in handling uncertaintyface in handling uncertainty

Dickie Whitaker- Director and joint founder ofDickie Whitaker- Director and joint founder ofLighthill Risk Network, Financial Services KTN and Oasis Loss Lighthill Risk Network, Financial Services KTN and Oasis Loss

Modelling FrameworkModelling Framework

““It is hard to overstate the damage done in the recent It is hard to overstate the damage done in the recent past by people who thought they knew more about the past by people who thought they knew more about the

world than they really did.world than they really did.””

John Kay in John Kay in ““ObliquityObliquity”” 2010 2010

Examples of where we Examples of where we have gone wronghave gone wrong

• Financial crash, Efficient market hypothesis, VAR ++Financial crash, Efficient market hypothesis, VAR ++

If you marked your position to market every day for a million If you marked your position to market every day for a million years, there would still be a less than one in a million chance years, there would still be a less than one in a million chance of experiencing a 25-standard deviation event.*of experiencing a 25-standard deviation event.*

• Sustainable cod management in Grand Banks Newfoundland -Sustainable cod management in Grand Banks Newfoundland -1968 810,000 tons of Cod caught by 1992 fishing collapsed1968 810,000 tons of Cod caught by 1992 fishing collapsed

• Japan assessment of seismicity and Tsunami wall heightJapan assessment of seismicity and Tsunami wall height

Cognitive BiasCognitive Bias

AnchoringAnchoringAnchoring describes the process of using a starting point for Anchoring describes the process of using a starting point for evaluating or estimating unknown valuesevaluating or estimating unknown valuesGamblerGambler’’s fallacy effects fallacy effectIt describes the tendency of decision makers to underestimate the It describes the tendency of decision makers to underestimate the probability of a repetition of an event that has just happened probability of a repetition of an event that has just happened (Cohen, Etner, & Jeleva, 2008).(Cohen, Etner, & Jeleva, 2008).Mean-reversion biasMean-reversion bias Whereby decision makers assume that over time, a trend has to Whereby decision makers assume that over time, a trend has to return to the mean. For example, after an return to the mean. For example, after an ‘‘unusuallyunusually’’ high rate of high rate of a natural disaster striking, decision makers may succumb to the a natural disaster striking, decision makers may succumb to the false believe that the rate of occurrence will return to false believe that the rate of occurrence will return to ‘‘normalnormal’’. .

Mario Weick, Mario Weick, School of Psychology,School of Psychology,University of KentUniversity of Kent

Complex problemsComplex problemsHypothesis weaknessHypothesis weakness

‘‘good participants differed from the bad ones … in good participants differed from the bad ones … in how often they tested their hypotheses. The bad how often they tested their hypotheses. The bad

participants failed to do this. For them, to propose a participants failed to do this. For them, to propose a hypothesis was to understand reality; testing that hypothesis was to understand reality; testing that

hypothesis was unnecessary. Instead of generating hypothesis was unnecessary. Instead of generating hypotheses, they generated hypotheses, they generated ““truthstruths”” ’’

The Logic of Failure -The Logic of Failure -DornerDorner

Losers:Losers:Acted without prior analysisActed without prior analysisDidnDidn’’t test against evidencet test against evidenceAssumed absence of negative meant correct decisions madeAssumed absence of negative meant correct decisions madeBlind to emerging circumstancesBlind to emerging circumstancesFocused on the local not the globalFocused on the local not the globalAvoided uncertaintyAvoided uncertainty

Practical challengesPractical challenges

1.1. Its very challenging, we go from a simpleIts very challenging, we go from a simple  to a complex to a complex (all possible theories and outcomes) question(all possible theories and outcomes) question

2. We often do2. We often do  not have the resources, Time, Money not have the resources, Time, Money Compute power or good enough statistical skillsCompute power or good enough statistical skills

3. Multi Disciplinary nature of the problem3. Multi Disciplinary nature of the problem

4. Communication especially visual  4. Communication especially visual  

How does insurance deal How does insurance deal with uncertainty?with uncertainty?

• Regulation didn't help that muchRegulation didn't help that much

• We cant compute (or get given) all of the We cant compute (or get given) all of the uncertaintyuncertainty

• We struggle to visualise and explain uncertaintyWe struggle to visualise and explain uncertainty

• We struggle to combine models and resultsWe struggle to combine models and results

• But the market and Oasis is driving changeBut the market and Oasis is driving change

Oasis Loss Modelling Oasis Loss Modelling FrameworkFramework

Not for profit owned and funded by 23 global Not for profit owned and funded by 23 global Insurers Reinsurers and BrokersInsurers Reinsurers and Brokers

Identification of Identification of uncertaintyuncertainty

Model Uncertainty Model Uncertainty covers the uncertainties that exist within the scope of the model. covers the uncertainties that exist within the scope of the model. Model inadequacy Model inadequacy represents the unsuitability of the underlying physical hazard and vulnerability represents the unsuitability of the underlying physical hazard and vulnerability models, as well as the effect of ignoring secondary perils such as demand surgemodels, as well as the effect of ignoring secondary perils such as demand surgeModel risk Model risk is the risk that the particular model is wrong or provides an incomplete and is the risk that the particular model is wrong or provides an incomplete and misleading picture. misleading picture. Parameter riskParameter risk is the risk that the modelis the risk that the model’’s parameters (often expressing assumptions or calibrations parameters (often expressing assumptions or calibration factors) are wrong, and factors) are wrong, and calculation errorcalculation error the risk that the model has been insufficiently discretisedthe risk that the model has been insufficiently discretised or sampled. or sampled.

Underwriting decisions made nowadays on the Underwriting decisions made nowadays on the basis of an basis of an ““EP curveEP curve”” from which three statistics from which three statistics

are typically used:are typically used:

MeanMean

MeanMeanStandard DeviationStandard Deviation

1 in 200 year VaR1 in 200 year VaR

Idealised use of modelsIdealised use of models

Seek transparency and ease of interrogation of any model, with clear Seek transparency and ease of interrogation of any model, with clear expression of the provenance of assumptions.expression of the provenance of assumptions.

Communicate the estimates with humility, communicate the uncertainty with Communicate the estimates with humility, communicate the uncertainty with confidence.confidence.

Fully acknowledge the role of judgement.Fully acknowledge the role of judgement.””

D. J. Spiegelhalter and H. Riesch in D. J. Spiegelhalter and H. Riesch in ““DonDon’’t know, cant know, can’’t know: embracing deepert know: embracing deeperuncertainties when analysing risksuncertainties when analysing risks”” Phil. Trans. R. Soc. A (2011) 369, 4730–4750 Phil. Trans. R. Soc. A (2011) 369, 4730–4750

What lessons for other What lessons for other sectors and other areassectors and other areas

• Academia needs to have application of best practices in statistics for Academia needs to have application of best practices in statistics for climate science?climate science?

• We need to have more multi disciplinary approaches for scienceWe need to have more multi disciplinary approaches for science

• Peer review process reinforces status quo rather than stimulating Peer review process reinforces status quo rather than stimulating innovation and multi disciplinary approaches innovation and multi disciplinary approaches

• Can be too much focus on Can be too much focus on ““ImpactImpact”” and not enough on utility and not enough on utility

• Communication and visualisation better approachesCommunication and visualisation better approaches

• Understanding of Bias in judgementUnderstanding of Bias in judgement

• Computation is and will play big part as data can be BigComputation is and will play big part as data can be Big

• Use statistics in peer reviewUse statistics in peer review

Thank YouThank You& &

to Dr Peter Taylor & * Prof. Lenny Smithto Dr Peter Taylor & * Prof. Lenny SmithDickie Whitaker, DirectorDickie Whitaker, DirectorFinancial Services Knowledge Transfer NetworkFinancial Services Knowledge Transfer NetworkTel:           +44(0)7920 502302Tel:           +44(0)7920 502302email:       email:       dickie.whitaker@fs-net.orgdickie.whitaker@fs-net.orgWeb:         Web:         http://ktn.innovateuk.org/web/financialservicesktn

Dickie Whitaker- Project DirectorDickie Whitaker- Project DirectorOasis Loss Modelling FrameworkOasis Loss Modelling Frameworkhttp://oasislmf.org/http://oasislmf.org/Tel: +44(0)7920 502 302Tel: +44(0)7920 502 302dickie.whitaker@oasislmf.orgdickie.whitaker@oasislmf.org

Models and uncertainty Lenny Smith LSE; http://www.lse.ac.uk/CATS/Publications/CATS%20Papers%20-%20Chronological.aspx

““Somewhere along the way, appreciationSomewhere along the way, appreciationfor the inherent uncertainty in risk has been diminished or even lost.for the inherent uncertainty in risk has been diminished or even lost.

Instead of models helping users to become more deeply risk-aware, the opposite can Instead of models helping users to become more deeply risk-aware, the opposite can occur.occur.

Indeed, some now feel more vulnerable to a change in model versions than to the very Indeed, some now feel more vulnerable to a change in model versions than to the very catastrophes that these models were intended to mitigate.catastrophes that these models were intended to mitigate.””

Hemant Shah, CEO RMSHemant Shah, CEO RMS