Climate Modelling and Climate Prediction: Uncertainties and how to deal with them

Mat Collins College of Engineering, Mathematics and Physical Sciences Joint Met Office Chair in Climate Change

@mat_collins

Outline

•  Motivation – dealing with uncertainties in models and projections •  Sources of uncertainties •  Techniques

•  Multi-model ensembles •  Emergent constraints •  Perturbed physics ensembles

•  Challenges/interesting questions to work on

Global Mean Surface Air Temperature Change

5-95% ~ ‘likely’

Anomalies w.r.t 1986-2005 average likely = 66-100% probability

Sources of Uncertainty •  Initial conditions, natural variability •  Boundary conditions, emissions/concentrations

of greenhouse gases and other forcing agents •  Model errors and uncertainty, different models

giving different projections •  Probabilistic climate projections (for e.g. 2100)

cannot be easily verified in the way that probabilistic weather forecasts are

An Example of Prediction in the Face of Uncertainties

•  In weather prediction we cannot estimate the current state of the atmosphere with complete accuracy because of limitations in the observational network •  Lorenz showed that small errors in initial conditions can amplify and spoil a deterministic forecast •  Hence run a number of different forecasts with slightly different initial conditions (an initial condition ensemble) •  Design the system so that the spread in the ensemble is a measure of the uncertainty in the forecast

ECMWF Ensembles Forecast, initialised Monday 12th Sept, 2011

ecmwf.int

Climate Change Projections

•  We can still use ensemble and probabilistic techniques in climate change projection

•  Need a different strategy for generating the ensemble as initial conditions are not the leading source of uncertainty •  The “multi-model” ensemble, MME (CMIP3,

CMIP5 etc.) •  The “perturbed-physics” ensemble, PPE

(perturb parameters in a single model)

Multi Model Ensemble

•  A collection of the world’s climate models •  Sometimes called an “ensemble of opportunity” •  Currently coordinated by projects like CMIP-Coupled Model Intercomparison Project (currently CMIP5) •  A relatively large “gene-pool” of possible models, although it is common to share some components •  Models are developed to reproduce observed data – although formal tuning is not performed

Multi Model Ensemble •  Multi-model ensembles (MMEs) are essentially collections of ‘best-shot’ models that are already constrained by observations •  Simplest approach – take the mean and standard deviation of all the best-shot models/simulators you can find

•  Not all models/simulators are independent •  We don’t know how to interpret such a distribution •  Some models/simulators are better than others •  ‘Everyone knows it is wrong but everyone does it’

MME Error Characteristics •  A commonly observed feature of MMEs is that

the ensemble mean model is better than any individual member

Reichler and Kim, 2008

MME Error Characteristics •  Rank histograms indicate that the MME spread

may be too wide •  Says nothing about future projections

Yokohata et al., 2013

Emergent Constraints: Schematic •  Find relationship •  Find observed value •  Read off prediction •  Find observational

uncertainty •  Add onto prediction •  Add “statistical”

uncertainty from scatter

Reduce uncertainty •  Use better observations •  Find a better

relationship (constraint) /metric

Emergent/Process-Based Constraints

Correlate September sea ice extent trends with future reductions in complex climate models (CMIP3 + PPE)

Boe, J., Hall, A. & Qu, X. September sea-ice cover in the Arctic Ocean projected to vanish by 2100. Nature Geoscience 2, 341-343, doi:DOI 10.1038/NGEO467 (2009).  

Cryosphere

Solid lines – subset of models Shading – min/max Dotted – all CMIP5 models

PM Cox et al. Nature, 1-4 (2013) doi:10.1038/nature11882

Emergent constraint on the sensitivity of tropical land carbon to climate change

Perturbed Physics Ensemble •  Take one model structure and perturb uncertain parameters and possible switch in/out different subroutines •  Can control experimental design, systematically explore and isolate uncertainties from different components •  Potential for many more ensemble members •  Unable to fully explore “structural” uncertainties

Notation

€

c = M(p,R)M = model/function c = climate variable p = model parameters/inputs R = radiative forcing Subscript h=historical, f=future o = observations

General Algorithm: •  Run model/evaluate function at many different input parameters for historical radiative forcing •  Compute metric of fit between model output and observations •  Weight future projections according to the value of the metric

€

ch = M(p,Rh )

€

c f = M(p,Rf )

€

m = (ch − o)T (ch − o) = (ch − o)

2∑

€

w = exp − 12

(ch − o)2∑

⎛

⎝ ⎜

⎞

⎠ ⎟

Basic Approach

€

c = M(p,R)

p ch

cf

parameter space

historical/observable climate

future climate

o

M(p1, Rf) M(p2, Rf)

p1

p2

M = climate model c = climate variable p = model parameters R = radiative forcing Sub. h=historical, f=future o = observations

M(p1, Rh)

M(p2, Rh)

€

p(c |o)∝ p(o | c)p(c)

A Very Simple Climate Model

ATMOSPHERE

OCEAN

N = R - ΔF

F ΔT

Climate Sensitivity: Simple Model

€

ΔT2X =R2Xλ

=R2XΔThΔFh + Rh

= M(ΔFh ,ΔTh ,Rh )

Climate Sensitivity, ΔT2X, is a function, M, of the ocean heat uptake, ΔFh (equivalent to the TOA flux imbalance), the observed temperature change, ΔTh and the radiative forcing, Rh.

All these quantities can be estimated from observations or can be calculated but are all uncertain.

Gregory, J. M.; Stouffer, R. J.; Raper, S. C. B.; Stott, P. A.; Rayner, N. A.; An observationally based estimate of the climate sensitivity. Journal Of Climate, 15, 3117-3121, 2002

Extrapolation of Signals: Allen, Stott & Kettleborough (ASK)

Peter Stott & Jamie Kettleborough, Origins and estimates of uncertainty in predictions of twenty-first century temperature rise, Nature, 416, pp.719-723, 2002.

€

ΔTf = βΔThCO2

Perturbed Physics Ensembles

Bayesian Approach with PPE  

•  Requires a large sample of projections and a way of specifying the prior – develop an emulator •  Likelihood based on comparison with present-day and past climate, climate variability and climate change •  This study based on HadCM3 model constrained by obs of mean T, P, RH, Energy fluxes… •  Projection onto truncated multivariate EOF space •  Discrepancy from CMIP MME to represent ‘structural’ uncertainty

Sexton DMH, JM Murphy, M Collins, M Webb, Multivariate prediction using imperfect climate models part I: Outline of methodology. Climate Dynamics, 2013

Combing Models and Observations

Three ways of using observations with models •  To develop models, e.g. parameterisation schemes •  To test emergent properties of models e.g. existence, frequency and amplitude of the El Niño Southern Oscillation •  To constrain ensembles of models to produce estimates of the uncertainty in projections

€

p(m |o)∝ p(m)p(o |m)posterior = prior x likelihood

Sexton et al., 2012

More Perturbed Physics Ensembles  

Lambert, F.H., Harris, G.R., Collins, M., Murphy, J., D.M.H. Sexton, B.B.B. Booth,, Interactions between perturbations to different Earth system components simulated by a fully-coupled climate. Climate Dynamics, 2013

Bayesian Approach with PPEs  

Harris, G.R., D.M.H. Sexton, B.B.B. Booth, Collins, M, J.M. Murphy, Probabilistic prediction of transient regional climate change. Climate Dynamics, 2013

Summary •  Multi-model ensembles

•  Seen as ‘industry standard’ •  Cannot be easily interpreted using simple statistical approaches •  Emergent constraints possible for some variables, but not all

•  Perturbed physics ensembles •  Can use formal Bayesian approach •  Projections dependent on assumptions but can at least test the sensitivity to main assumptions •  Cannot sample ‘structural’ uncertaintes

•  Try and combine PPE and MME approaches

Challenges/Interesting Problems •  Observational challenges

•  Errors in observations •  Alternatives sources of data – obs4mips

•  Model evaluation that matters •  Process-based emergent constraints?

•  Making regional projections •  Combining information from what we understand about climate variability to inform climate change

•  Natural vs forced response

Observational Challenges

Collins et al. 2013 SAPRISE Project

Example of Regional Projection

•  Rainfall change anchored to an equatorial peak in SST warming across the Pacific •  SST change a result of oceanic processes in the west and atmospheric processes in the east (Xie et al., 2010) •  This change in mean rainfall impacts El Nino teleconnections and increases the frequency of ‘extreme’ El Nino events (Cai et al., 2014, Power et al., 2013)

Role of Natural Variability Deser et al., 2012 Nature Climate Change