School of Earth and Environment

Sensitivity analysis and calibration of a global aerosol model

Lindsay Lee l.a.lee@leeds.ac.uk

Ken Carslaw1, Carly Reddington1, Kirsty Pringle1, Jeff Pierce3, Graham Mann1, Jill Johnson1,

Dominick Spracklen1, Philip Stier2

1. University of Leeds 2. University of Oxford 3. Colorado State University

1. Motivation

http://earthobservatory.nasa.gov/Features/CALIPSO/CALIPSO2.php

2. GLOMAP

• We use the global aerosol model GLOMAP (Mann et al. 2010)

• A microphysical modal model simulating the evolution of global aerosol including sulphate, sea-salt, dust and black carbon

http://www.pnl.gov/atmospheric/research/aci/aci_aerosol_indeffects.stm

3. Uncertainty in modelling

• Structural uncertainty

• Compare multiple models of the same type

• Compare model to observations

• AEROCOM • Aerosol Comparisons between Observations and

Models

• >14 models with the same emissions for 2000 and pre-industrial

5

4. GLOMAP in AEROCOM

6

4. GLOMAP in AEROCOM

7

4. GLOMAP in AEROCOM

8

4. GLOMAP in AEROCOM

• More complex models = more uncertain parameters

• How do these uncertain parameters affect model predictions?

• Which processes in the model are dominating the prediction?

• Together with AEROCOM find the most important uncertainties

9

5. Parametric uncertainty

• Ideally want to carry out sensitivity analysis

• Need a good estimation of the output distribution given uncertain inputs

• Consider GLOMAP output Y = η(X)

• X is uncertain so Y is uncertain

• Want to know f(Y) given G(X)

• Monte Carlo sampling impossible

• Have to use

10

6. Parametric uncertainty in a complex computer model

)(ˆ Yf

• found using Bayesian statistics

• Use a limited number of model runs – training data

• Make some assumptions about the model behaviour – prior distribution

• Find the posterior distribution of the model – and use the mean as

11

7. Bayesian statistics

)(ˆ Yf

)(ˆ Yf

8. The procedure

9. The model output

• Focus on Cloud condensation nuclei (CCN) concentration

• Soluble aerosol (>50nm) form cloud droplets at a given supersaturation

• Uncertainty in CCN concentration due to: • uncertainty in the emission,

• nucleation of new aerosol,

• growth to 50nm,

• solubility,

• loss of aerosol.

10. Expert elicitation

Elicitation: Ask the experts

‘We think these are the uncertain parameters and their values are very unlikely to fall outside of these ranges’

11. Statistical design

– Need maximum information in fewest runs

– Space-filling in 28-dimensions

– Maximin Latin Hypercube used

–good marginal coverage

–good space-filling properties

– Number of runs 168 + 84 validation

– Emulator validation to highlight design issues

• Interpolate ‘well-spaced’ model runs to estimate at untried points

• Gaussian Process - conditional probability • Non-parametric

– Key assumption is that parameter settings give information about model behaviour close by in parameter space

12. Filling in the gaps - emulation

GP mean True Function

X1

X2

Linear Model

13. Filling in the gaps - emulation

Emulator mean

X2

X1

Emulated output distribution

Marginal functions X1 X2

Ou

tpu

t

Input

14. Emulator validation

Is the emulator output a good approximate of the model

output?

YES – use the emulator mean instead

15. Emulator validation I

16. Emulator validation II

17. Estimated CCN and its uncertainty concentration in every surface grid box

January

July

18. Variance-based sensitivity analysis

ji

pij

i

i VVVYVar 12

1

)(

)),|(( iXi XYEVarVi

))|(( ijXij XYEVarVij

)(YVar

VS i

i

112

1

ji

pij

i

i SSS

Main effect sensitivity:

Variance decomposition:

Main effects +

Interactions:

Variance due to each parameter:

Marginal functions

X1 X2

Input

Ou

tpu

t

Emulated output distribution

19. Variance-based sensitivity analysis

)),|(( iXi XYEVarVi

)(YVar

24

20. Contributions to uncertainty in every grid box – January CCN

25

20. Contributions to uncertainty in every grid box – January CCN

ACT_DIAM

AIT_WIDTH

ANTH_SOA

SO2O3_CLEAN

DRYDEP_ACC

BB_EMS

PRIM_SO4_DIAM

BB_DIAM

DMS_FLUX

Surface parameter sensitivities 21. Contributions to uncertainty in every grid box

January January July July

σCCN/µCCN σCCN

22. Summarising global maps

23. Seasonal parameter sensitivities

24. Multi-parameter analysis

24. Multi-parameter analysis

BIO_SOA

BL_NUC

25. Towards structural uncertainty and calibration

• Structural uncertainty

• Considering the model uncertainty is any model configuration near to observations

26. AEROCOM versus AEROS

ACT_DIAM BIO_SOA ANTH_SOA

27. Reduce global problem

Jan CCN

Jul CCN

• PCA and cluster analysis Similar patterns seen in both

28. Cluster analysis

Fossil fuel emissions

Model parameters – Structure and cloud processing

ACT_DIAM

AIT_WIDTH

SO2O3_CLEAN

BB_EMS

BB_DIAM

29. Summary

• Sensitivity analysis provides deeper understanding of model behaviour

• Quantifying parametric uncertainty can help identify structural errors/model errors/modeller errors

• Can help direct research areas/observation strategies

• Will help calibration

30. Calibration discussion points

• Calibration of global means not satisfactory • How do we calibrate the global model with known

structural errors? • Do we expect one set of calibrated parameters given we

know model behaviour is variable? • How can we use the sensitivity information?

– to reduce dimensions • regional and temporal information • identify active parameters

• How do we specify discrepancy? – Use AEROCOM – Use grid box variability

References

• Lee, L. A., Carslaw, K. S., Pringle, K. J., Mann, G. W., and Spracklen, D. V.: Emulation of a complex global aerosol model to quantify sensitivity to uncertain parameters, Atmos. Chem. Phys., 11, 12253-12273, doi:10.5194/acp-11-12253-2011, 2011.

• Lee, L. A., Carslaw, K. S., Pringle, K. J., and Mann, G. W.: Mapping the uncertainty in global CCN using emulation, Atmos. Chem. Phys., 12, 9739-9751, doi:10.5194/acp-12-9739-2012, 2012.

• Lee, L. A., Pringle, K. J., Reddington, C. L., Mann, G. W., Stier, P., Spracklen, D. V., Pierce, J. R., and Carslaw, K. S.: The magnitude and causes of uncertainty in global model simulations of cloud condensation nuclei, Atmos. Chem. Phys. Discuss., 13, 6295-6378, doi:10.5194/acpd-13-6295-2013, 2013.

• Mann, GW; Carslaw, KS; Spracklen, DV; Ridley, DA; Manktelow, PT; Chipperfield, MP; Pickering, SJ; Johnson, CE (2010) Description and evaluation of GLOMAP-mode: a modal global aerosol microphysics model for the UKCA composition-climate model, GEOSCI MODEL DEV, 3, pp.519-551. doi:10.5194/gmd-3-519-2010

• Partridge, D. G., Vrugt, J. A., Tunved, P., Ekman, A. M. L., Struthers, H., and Sorooshian, A.: Inverse modelling of cloud-aerosol interactions – Part 2: Sensitivity tests on liquid phase clouds using a Markov chain Monte Carlo based simulation approach, Atmos. Chem. Phys., 12, 2823-2847, doi:10.5194/acp-12-2823-2012, 2012.

• Roustant, O., Ginsbourger, D., and Deville, Y. (2010). DiceKriging: Kriging methods for computer experiments. R package version 1.1. http://CRAN.R-project.org/package=DiceKriging

• Bastos, L. S. and O'Hagan, A. (2009). Diagnostics for Gaussian process emulators. Technometrics 51, 425-438.

• Pujol, G. (2008). sensitivity: Sensitivity Analysis. R package version 1.4-0. • Saltelli, A., Chan, K., Scott, M. Editors, 2000, Sensitivity Analysis, John Wiley & Sons publishers, Probability

and Statistics series.

38