ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Stochastic representations of model Stochastic representations of model uncertaintyuncertainty
Glenn Shutts Glenn Shutts
ECMWF/Met OfficeECMWF/Met Office
Acknowledgements : Judith Berner, Martin Leutbecher
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
OutlineOutline• Ensemble model spread
• The nature of ‘model error’
• The ‘stochastic physics scheme’ (perturbing parametrized tendencies)
• The spectral stochastic backscatter scheme
• Calibrating the schemes by coarse-graining
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Ensemble Forecast for Thurs 15Ensemble Forecast for Thurs 15thth 2007 2007
Representing initial state uncertainty by an Representing initial state uncertainty by an ensemble of statesensemble of states
2t
0t
1t
analysis
spread
RMS error
ensemble mean
Represent initial uncertainty by ensemble of atmospheric flow states Flow-dependence:
Predictable states should have small ensemble spread Unpredictable states should have large ensemble spread
Ensemble spread should grow like RMS error
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Buizza et al., 2004
Systems
Under-dispersion of the ensemble systemUnder-dispersion of the ensemble system
-------------- spread around ensemble meanspread around ensemble mean
RMS error of ensemble meanRMS error of ensemble mean
The RMS error grows faster than the spread
Ensemble is under-dispersive
Ensemble forecast is over- confident
Under-dispersion is a form of model error
Forecast error = initial error + model error
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Manifestations of model errorManifestations of model error
In medium-range:Under-dispersion of ensemble system (Over-confidence)Can extreme weather events be captured?
On seasonal to climatic scales:Not enough internal variabilityTo what degree do detection and attribution studies for
climate change depend on a correct estimate of internal variability?
Underestimation of the frequency of blocking Tropical variability, e.g. MJO, wave propagation Systematic error in T, Precip, …
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Causes of model error : Unrepresented Causes of model error : Unrepresented processes in weather and climate modelsprocesses in weather and climate models
• Systematic versus random error
• physical parametrization delivers ensemble-mean or ‘most likely’ tendencies ?
• random model error can be associated with :
(i) statistical fluctuations in sub-grid (or filter-scale) transport processes (e.g. convective mass flux)
(ii) unrepresented statistical physical process e.g. ‘turbulent backscatter’
• different systematic errors associated with model framework (e.g. gridpoint vs spectral) and parametrization choices can be used to create an ensemble
forecast system (e.g. multi-model ensemble; Hadley Centre QUMP)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Kinetic energy spectra from aircraftKinetic energy spectra from aircraft
Nastrom and Gage, 1985
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Kinetic Energy spectrum in the ECMWF IFSKinetic Energy spectrum in the ECMWF IFS
Wavelength ~ 600 km Missing mesoscale
energy
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Representing UncertaintyRepresenting Uncertainty
within conventional parameterization schemes
Stochastic parameterizations (Buizza et al, 1999, Lin and Neelin, 2000)
Multi-parameterizations approaches (Houtekamer, 1996)
Multi-parameter approaches (e.g. Murphy et al,, 2004; Stainforth et al, 2004)
Multi-models (e.g. DEMETER, ENSEMBLES, TIGGE, Krishnamurti)
outside conventional parameterisation schemes
Nonlocal parameterizations, e.g., cellular automata pattern generator (Palmer, 1997, 2001)
Stochastic kinetic energy backscatter (Shutts and Palmer 2004, Shutts 2005; Bowler et al, 2009)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Stochastic parameterizations have the potential Stochastic parameterizations have the potential to reduce model errorto reduce model error
Weak noise
Multi-modal
Strong noise
Unimodal
Stochastic parameterizations can change the mean and variance of a PDF
Impacts variability of model (e.g. internal variability of the atmosphere)
Impacts systematic error (e.g. blocking, precipitation error)
Potential
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Spectral stochastic perturbed tendency scheme
(‘New Stochastic Physics’)Revised form of the scheme due to Buizza et al (1999)
use a spectral pattern generator based on triangularly-truncated spherical harmonic expansions to represent a global field ‘multiplier’
at any spatial point the multiplier has a mean of 1 and prescribed variance
the field has Gaussian horizontal auto-correlation function with an adjustable correlation scale (e.g. 500 km)
Each spectral component in the pattern evolves in time according to a first-order autoregressive process with prescribed decay time (e.g. 6 model steps)
model parametrization tendencies are multiplied by the pattern field(excluding boundary layer and stratosphere)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
New stochastic physics pattern generatorNew stochastic physics pattern generator
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Decrease in ensemble mean error Decrease in ensemble mean error
x
Ensemble members
x
Ensemble mean error
Analysisx
Ensemble mean
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Continuous Ranked Probability Skill Score
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
r.m.s. error of 850 hPa temperature in the tropics versus spread for the ensemble-mean
(Crosses are for r.m.s. error)
Under-dispersion
Spread increased with newStochastic physics
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Continuous Ranked Probability Skill Score
Spectral Backscatter SchemeSpectral Backscatter Scheme
Rationale: A fraction of the dissipated energy is scattered upscale and acts as streamfunction forcing for the resolved-scale flow (LES, CASBS: Shutts and Palmer 2004, Shutts 2005); New: spectral pattern generator
Total Dissipation rate from Total Dissipation rate from numerical dissipation, convection, numerical dissipation, convection, gravity/mountain wave drag.gravity/mountain wave drag.
Forcing pattern: temporal and Forcing pattern: temporal and spatial correlations prescribedspatial correlations prescribed
D F
* D F
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Spectral Backscatter Scheme (SPBS)Spectral Backscatter Scheme (SPBS)
Spectral pattern generator:
where
and fjm,n are the complex spectral amplitudes at step j and
are associated Legendre functions| |mnP
( * denotes the complex conjugate)
Rationale: A fraction of the dissipated energy is backscattered upscale and acts as streamfunction forcing for the resolved-scale flow ( Shutts and Palmer 2004, Shutts 2005, Berner et al (2009)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
11stst-order autoregressive process-order autoregressive processfor horizontal pattern generationfor horizontal pattern generation
(n) is a scale-dependent parameter that sets the decorrelation time Currently ~ 0.07 for all nand is chosen so that iss.
g(n) sets the amplitude of the random number noise rjm,n
based on coarse-graining calculations using a big-domain cloud-resolving model g(n) is (1+n)
where j is the step number
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Power spectrum of coarse-grained streamfunction forcing at Power spectrum of coarse-grained streamfunction forcing at z=11.5 km computed from a cloud-resolving modelz=11.5 km computed from a cloud-resolving model
k-1.54
Log(E)
Log(k)
g(n) ~ k-1.27
E~ n g(n)2
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Streamfunction forcingStreamfunction forcing
Streamfunction forcing
Backscatter ratio
Total KE dissipation rate Pattern generator
Dtot = numerical dissipation +
gravity/mountain wave drag dissipation + deep convective production of KE
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Smoothed total ‘dissipation rate’Smoothed total ‘dissipation rate’ D*tot
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Numerical dissipation RateNumerical dissipation Rate
where is the relative vorticity and K is the biharmonicdiffusion coefficient.
Dnum is augmented by a factor of 3 to account for thekinetic energy loss that occurs as a result of interpolationof winds to the departure point in the semi-Lagrangianadvection step
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Gravity wave/orographic dragGravity wave/orographic drag
22 /C dD M w
u and v increments from the orographic drag parametrizationmultiplied by u and v to give a KE increment i.e.
Deep convection KE production
Md is the mass detrainment rate; w is a mean convective updraught speed and is the density
gwdgwd gwd
u vD u v
t t
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Smoothed total ‘dissipation’ rateSmoothed total ‘dissipation’ rate
2
2tot num gwd c
fD D D D
Dtot is smoothed to T30 using a tapered spectral filter
Boundary layer dissipation is omitted on the assumptionthat turbulent eddies of scale < 1 km will not project sufficiently on quasi-balanced, meso->synoptic scale motions
The bracketed term multiplying Dc is the absolute vorticitynormalized by twice the Earth’s angular rotation rate. This represents the dependence of balanced flow production on background rotation.
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Impacts on probability skill scoresImpacts on probability skill scores
Continuous Ranked Probability Skill Score for temperature at 850 hPa (20-90 degrees N)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Continuous Ranked Probability Skill Score for Continuous Ranked Probability Skill Score for temperature at 850 hPa (Tropics)temperature at 850 hPa (Tropics)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
ContinuousContinuous Ranked Probability Skill Score for Ranked Probability Skill Score for u at 850 hPa (20 – 90 degrees N)u at 850 hPa (20 – 90 degrees N)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Continuous Ranked Probability Skill Score for Continuous Ranked Probability Skill Score for u at 850 hPa (Tropics)u at 850 hPa (Tropics)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
ContinuousContinuous Ranked Probability Skill Score for Ranked Probability Skill Score for u at 200 hPa (20 – 90 N)u at 200 hPa (20 – 90 N)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Continuous Ranked Probability Skill Score for Continuous Ranked Probability Skill Score for u at 200 hPa (Tropics)u at 200 hPa (Tropics)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Continuous Ranked Probability Skill Score for Continuous Ranked Probability Skill Score for geopotential height at 850 hPa (20 – 90 degs N)geopotential height at 850 hPa (20 – 90 degs N)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Rms error of the ensemble mean versus spread about Rms error of the ensemble mean versus spread about the ensemble mean for T at 850 hPa (20-90 N)the ensemble mean for T at 850 hPa (20-90 N)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Rms error of the ensemble mean versus spread about Rms error of the ensemble mean versus spread about the ensemble mean of T at 850 hPa (tropics)the ensemble mean of T at 850 hPa (tropics)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Rms error of the ensemble mean versus spread about the Rms error of the ensemble mean versus spread about the ensemble mean of u at 200 hPa (20-90 N)ensemble mean of u at 200 hPa (20-90 N)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Rms error of the ensemble mean versus spread about Rms error of the ensemble mean versus spread about the ensemble mean of u at 200 hPa (tropics)the ensemble mean of u at 200 hPa (tropics)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Experimental Setup for Seasonal RunsExperimental Setup for Seasonal Runs
“Seasonal runs: Atmosphere only” Atmosphere only, observed SSTs 40 start dates between 1962 – 2001 (Nov 1) 5-month integrations One set of integrations with stochastic
backscatter, one without Model runs are compared to ERA40 reanalysis
(“truth”)
No StochasticBackscatterNo StochasticBackscatter Stochastic BackscatterStochastic Backscatter
Reduction of systematic error of z500 over Reduction of systematic error of z500 over North Pacific and North AtlanticNorth Pacific and North Atlantic
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Increase in occurrence of Atlantic and Increase in occurrence of Atlantic and Pacific blockingPacific blocking
ERA40 + confidence ERA40 + confidence intervalinterval
No StochasticBackscatterNo StochasticBackscatter
Stochastic BackscatterStochastic Backscatter
Wavenumber-Frequency SpectrumWavenumber-Frequency Spectrum Symmetric part, background removed Symmetric part, background removed
(after Wheeler and Kiladis, 1999)(after Wheeler and Kiladis, 1999)
No Stochastic BackscatterNo Stochastic BackscatterObservations (NOAA)Observations (NOAA)
Improvement in Wavenumber-Frequency Improvement in Wavenumber-Frequency SpectrumSpectrum
Stochastic BackscatterStochastic BackscatterObservations (NOAA)Observations (NOAA)
Backscatter scheme reduces erroneous westward propagating modes
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Coarse-graining as a method of computing Coarse-graining as a method of computing model errormodel error
Cloud-Resolving Model (CRM) approach
1) Spatially-average model fields and tendencies to a coarse grid2) Compute tendencies implied by the coarse-grained model fields3) Subtract the tendencies computed in 2) from the coarse-grained
CRM tendendies
Forecast model method
1. Run a very high resolution forecast model e.g. IFS at T12792. Coarse-grain the tendency fields to a lower resolution e.g. T1593. Run a forecast at the lower resolution and subtract tendency field early in the forecast from the tendency field computed in 2)
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Computing the streamfunction forcingComputing the streamfunction forcing
1) Run a T1279 forecast for 2 hours and compute the total vorticity tendency from increments of u and v.
2) Smooth to T159 and take the inverse Laplacian to obtain streamfunction tendency
3) Run a T159 forecast for 2 hours.
4) Repeat 1) and 2) without smoothing
5) Compute the difference in the two streamfunction forcing functions
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Vertical section of the difference in u between Vertical section of the difference in u between T1279 run and T159 run at t+8 hrsT1279 run and T159 run at t+8 hrs
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
Streamfunction forcing estimated by Streamfunction forcing estimated by coarse-graining approachcoarse-graining approach
ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009
SummarySummary
• Insufficient ensemble model spread indicates the need to account for the statistical aspects of model error
• The true nature of this model error is not fully understood !
• Statistical fluctuations ignored in conventional physical parametrization maybe included
• Energy backscatter from unresolved flow structures may perturb the balanced flow dynamics
• The coarse-graining methodology provides a method for calibrating/validating assumptions inherent in stochastic parametrization