mesoscale ensemble prediction system: what is it? what does it take to be effective?
DESCRIPTION
Mesoscale Ensemble Prediction System: What is it? What does it take to be effective? What are the impacts of shortcomings?. Maj Tony Eckel, Ph.D. Asst. Professor Naval Postgraduate School, Meteorology Department (831) 656-3430, [email protected] Sep 09. - PowerPoint PPT PresentationTRANSCRIPT
1
Mesoscale Ensemble Prediction System:
What is it?
What does it take to be effective?
What are the impacts of shortcomings?
Maj Tony Eckel, Ph.D.Asst. ProfessorNaval Postgraduate School, Meteorology Department(831) 656-3430, [email protected] Sep 09
2
Exploitation
Products, Marketing, Value Verification,
Forecaster & User Education,…
Ensemble
IC Perturbations, Model Perturbations,# of Members, Skill Verification, Post-processing,…
Foundation
Observations, Data Assimilation, the Model(s), Model Resolution,…
Users
Optimal Decision Making
Ens
embl
e Pr
edic
tion
Syst
em
What is it?
Forecast + Uncertaintyand/or
Decision Recommendation
3
External Internal
Upper BoundaryModeling Limitations
Lower Boundary Conditions- Incomplete and Erred Surface Temperature, Soil Moisture, Albedo, Roughness Length, …
Initial Conditions- Erred Observations- Incomplete Observations- Limitations to Data Assimilation
LateralBoundary Conditions - Inaccuracies- Coarse spatial & temporal resolution
Com
puter
Precisio
nModel Core- Mathematical Model - Numerical Truncation- Limited Resolution
Model Physics Limitations - Assumptions- Parameterizations
NWPModel
Sources of Uncertaintyin NWP
4
What does it take to be effective?- - - Robust IC Perturbations - - -
Perturbation Options:
Methods: Bred Modes (BM) Singular Vectors (SV) Ensemble Kalman Filter (EnKF) Ensemble Transform Kalman Filter (ETKF)
Goal: n dynamically consistent and equally likely analyses that span the
analysis error subspace
Questions: Special considerations for mesoscale, short-range ensemble? Importance of Scale? Cold vs. Warm Start?
SVBMEnKFETKF
Descamps and Talagrand, MWR, 2007
Brie
r S
core
EventZ500 > Z500
5
Goal: Diversity in members’ attractors to attempt to span true attractor, or
at least aim for a statistically consistent forecast PDF
What does it take to be effective?- - - Robust Model Perturbations - - -
Methods: Multi-model – different models and/or different physics schemes
Stochastic Physics – perturb state variables’ tendency during model integration
Stochastic Backscatter – like stoch. phys., but scale-dependent
*Stochastic Parameters – randomly perturb parameters (e.g., entrainment rate) during integration
Perturbed Parameters – randomly perturb parameters, but hold constant during integration
Perturbed Field Parameters – SST, albedo, roughness length, etc.
Questions: Combinations? – Which methods may be used together?
Other Methods? – Stochastic Field Parameters
– Couple to Ocean Model Ensemble and/or LSM Ensemble
– …?
Perturbed Lateral Boundary Conditions – Smaller domain, Longer forecast Bigger issue
Teixeira and Reynolds, MWR, 2008
-Stochastic Convection-IC Perturbations
6
Ensemble Size: Need enough members to consistently depict the PDF from which the members are drawn
8-10 for decent ensemble mean 20-30 for skilled forecast probability 50+ to capture low probability events (PDF tails)
Earth Simulator System122.4 teraflops
Primary Drivers…
What does it take to be effective?- - - Powerful Processors - - -
Model Configuration: Need to meet user requirements
Forecast Length Forecast Update Frequency Domain Coverage $ $ $ Resolution $ $ $ – Can only estimate uncertainty of resolved scales
– Benefits of finer resolution 1) Increased spread
2) Lower uncertainty
3) Increased VALUE
– Resolving convection (grid<4km) is key
better statistical consistency
6-h Precip (bias-corrected)
Forecast Hour
Clark et al., WAF, 2009
Talagrand et al., ECMWF, 1999
Brie
r S
core
EventT850 > Tc
7
Calibration: Need to maximize skill & value
Obtain Reliability – account for systematic errors (significant model biases in meso. models)
Boost Resolution via Downscaling – can greatly improve value of information
Reforecast Dataset – needed to calibrate low frequency events
Adaptable to variety of state and derived variables
Practical – easy to maintain
Preserve Meteorological Consistency?
What does it take to be effective?- - - Robust Post-processing - - -
-0.1
0.0
0.1
0.2
0.3
0.4
0.5
0.6
00 03 06 09 12 15 18 21 24 27 30 33 36 39 42 45 48
*ACMEcore
ACMEcore
*ACMEcore+
ACMEcore+
Uncertainty
*UWME
UWME
*UWME+
UWME+
Uncertainty
BS
SEvent: 2m Temp < 0°C
Lead Time (h)
Eckel and Mass, WAF, 2005
Multi-model w/ bias cor.
Multi-model
IC only
8
Parameters – emphasis on sensible weather and user requirements
Ground Truth – emphasis on observations vs. model analysis
Skill Metrics (VRH, BSS, CRPS, etc.) – focus on ensemble performance
Value Metrics (ROCSS, VS, etc.) – focus on benefit to user
Accessible to Users
– Interactive, web-based interface
– Frequent updates
– Link into products and education
What does it take to be effective?- - - Comprehensive Verification - - -
Both feed back into
R&D
Lambert and Roeder, NASA, 2007
http://science.ksc.nasa.gov/amu/briefings/ fy08/Lambert_Probability_ILMC.pps
Lightning % ForecastsCape Canaveral / Patrick AFB
Shuttle Endeavor12 Jul 2009
9
What are the impacts of shortcomings?
Poor Dispersion – Failure to simulate error growth
NCEP SREF20090301-20090531
48-h Sfc Wind Speed
Degraded Valuefor
Decision Making
C/L Ratio
Val
ue
Sco
re
Robust Meso. EPS
Deficient Meso. EPS
NCEP SREF20090301-20090531
Event: 48-h Sfc RH > 90%
UKMet MOGREPS20060101-20060228
Event: 36-h Cloud Ceiling < 500ft
Poor Skill – Weak Reliability & Resolution
Bowler et al., MetOffice, 2007
max
imum
likely
true
pro
babil
ity
mini
mum likely
true p
roba
bility
expe
cted
value
of t
rue
prob
abilit
y
JMA EPS20071215-20080215
Event: 5-d 2m temp 0C
High Ambiguity – Large random error in uncertainty estimate
Eckel and Allen, 2009
11
True Forecast PDF
True forecast PDF recipe for lead time in the current forecast cycle:
1) Look back through an infinite history of forecasts produced by the analysis/forecast system in a stable climate
2) Pick out all instances with the same analysis (and resulting forecast) as the current forecast cycle
3) Pool the verifying true states at to construct a distribution
Combined effect creates a wider true PDF.
Erred model also contributes to analysis error.ErredErred
While each matched analysis corresponds to only one true IC,
the subsequent forecast can match many different true states
due to grid averaging at =0 and/or lack of diffeomorphism. ErredPerfect
Each historical analysis match will correspond to a
different true initial state, and a different true state at time . PerfectErred
Only one possible true state, so true PDF is a delta function.PerfectPerfect
0
0.2
0.4
dnorm( ),,x3.60.9
x
0
0.2
0.4
dnorm( ),,x3.60.9
x
0
0.2
0.4
dnorm( ),,x3.60.9
x
0
0.2
0.4
dnorm( ),,x3.60.9
x
Analysis Model True Forecast PDF (at a specific )
0
0.2
0.4
dnorm( ),,x3.60.9
x
0
0.2
0.4
dnorm( ),,x3.60.9
x
Perfect exactly accurate (with infinitely precision)Erred inaccurate, or accurate but discrete
Not absolute -- depends on
uncertainty in ICs and model
(better analysis/model =
sharper true PDF)
12
ET maintains error variance in more directions than breeding…
Average forecast error covariance matrix eigenvalue spectrum, 24h leadtime
Am
pli
tud
e
Eigenvalue
ET
Breeding
ICs by Ensemble Transform (ET)(from Craig Bishop, NRL)
13
O. Talagrand and G. Candille, Workshop Diagnostics of data assimilation system performanceECMWF, Reading, England, 16 June 2009
14
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
0 12 24 36 48
Lead Time (h)
EF
Sp
read
or
M
SE
of
EF
Mea
n(d) T2(c) WS10
Ens
. Var
. O
R
mse
of
the
Ens
. Mea
n (º
C2 )
Ens
. Var
. O
R
mse
of
the
Ens
. Mea
n (m
2 / s
2 )
c2 11.4 m2 / s2 c
2 13.2 ºC2
mse of *UWME Mean
UWME Variance
*UWME Variance
mse of *UWME+ Mean
UWME+ Variance
*UWME+ Variance
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
0 12 24 36 48
Lead Time (h)
EF
Sp
read
or
M
SE
of
EF
Mea
n
16L
22L
22L04L
04L
10L
10L 16L
0.0
0 12 24 36 48
Eckel and Mass, WAF, 2005