mesoscale ensemble prediction system: what is it? what does it take to be effective?

14
1 Mesoscale Ensemble Prediction System: What is it? What does it take to be effective? What are the impacts of shortcomings? Maj Tony Eckel, Ph.D. Asst. Professor Naval Postgraduate School, Meteorology Department (831) 656-3430, [email protected]

Upload: adriana-synge

Post on 01-Jan-2016

22 views

Category:

Documents


0 download

DESCRIPTION

Mesoscale Ensemble Prediction System: What is it? What does it take to be effective? What are the impacts of shortcomings?. Maj Tony Eckel, Ph.D. Asst. Professor Naval Postgraduate School, Meteorology Department (831) 656-3430, [email protected] Sep 09. - PowerPoint PPT Presentation

TRANSCRIPT

1

Mesoscale Ensemble Prediction System:

What is it?

What does it take to be effective?

What are the impacts of shortcomings?

Maj Tony Eckel, Ph.D.Asst. ProfessorNaval Postgraduate School, Meteorology Department(831) 656-3430, [email protected] Sep 09

2

Exploitation

Products, Marketing, Value Verification,

Forecaster & User Education,…

Ensemble

IC Perturbations, Model Perturbations,# of Members, Skill Verification, Post-processing,…

Foundation

Observations, Data Assimilation, the Model(s), Model Resolution,…

Users

Optimal Decision Making

Ens

embl

e Pr

edic

tion

Syst

em

What is it?

Forecast + Uncertaintyand/or

Decision Recommendation

3

External Internal

Upper BoundaryModeling Limitations

Lower Boundary Conditions- Incomplete and Erred Surface Temperature, Soil Moisture, Albedo, Roughness Length, …

Initial Conditions- Erred Observations- Incomplete Observations- Limitations to Data Assimilation

LateralBoundary Conditions - Inaccuracies- Coarse spatial & temporal resolution

Com

puter

Precisio

nModel Core- Mathematical Model - Numerical Truncation- Limited Resolution

Model Physics Limitations - Assumptions- Parameterizations

NWPModel

Sources of Uncertaintyin NWP

4

What does it take to be effective?- - - Robust IC Perturbations - - -

Perturbation Options:

Methods: Bred Modes (BM) Singular Vectors (SV) Ensemble Kalman Filter (EnKF) Ensemble Transform Kalman Filter (ETKF)

Goal: n dynamically consistent and equally likely analyses that span the

analysis error subspace

Questions: Special considerations for mesoscale, short-range ensemble? Importance of Scale? Cold vs. Warm Start?

SVBMEnKFETKF

Descamps and Talagrand, MWR, 2007

Brie

r S

core

EventZ500 > Z500

5

Goal: Diversity in members’ attractors to attempt to span true attractor, or

at least aim for a statistically consistent forecast PDF

What does it take to be effective?- - - Robust Model Perturbations - - -

Methods: Multi-model – different models and/or different physics schemes

Stochastic Physics – perturb state variables’ tendency during model integration

Stochastic Backscatter – like stoch. phys., but scale-dependent

*Stochastic Parameters – randomly perturb parameters (e.g., entrainment rate) during integration

Perturbed Parameters – randomly perturb parameters, but hold constant during integration

Perturbed Field Parameters – SST, albedo, roughness length, etc.

Questions: Combinations? – Which methods may be used together?

Other Methods? – Stochastic Field Parameters

– Couple to Ocean Model Ensemble and/or LSM Ensemble

– …?

Perturbed Lateral Boundary Conditions – Smaller domain, Longer forecast Bigger issue

Teixeira and Reynolds, MWR, 2008

-Stochastic Convection-IC Perturbations

6

Ensemble Size: Need enough members to consistently depict the PDF from which the members are drawn

8-10 for decent ensemble mean 20-30 for skilled forecast probability 50+ to capture low probability events (PDF tails)

Earth Simulator System122.4 teraflops

Primary Drivers…

What does it take to be effective?- - - Powerful Processors - - -

Model Configuration: Need to meet user requirements

Forecast Length Forecast Update Frequency Domain Coverage $ $ $ Resolution $ $ $ – Can only estimate uncertainty of resolved scales

– Benefits of finer resolution 1) Increased spread

2) Lower uncertainty

3) Increased VALUE

– Resolving convection (grid<4km) is key

better statistical consistency

6-h Precip (bias-corrected)

Forecast Hour

Clark et al., WAF, 2009

Talagrand et al., ECMWF, 1999

Brie

r S

core

EventT850 > Tc

7

Calibration: Need to maximize skill & value

Obtain Reliability – account for systematic errors (significant model biases in meso. models)

Boost Resolution via Downscaling – can greatly improve value of information

Reforecast Dataset – needed to calibrate low frequency events

Adaptable to variety of state and derived variables

Practical – easy to maintain

Preserve Meteorological Consistency?

What does it take to be effective?- - - Robust Post-processing - - -

-0.1

0.0

0.1

0.2

0.3

0.4

0.5

0.6

00 03 06 09 12 15 18 21 24 27 30 33 36 39 42 45 48

*ACMEcore

ACMEcore

*ACMEcore+

ACMEcore+

Uncertainty

*UWME

UWME

*UWME+

UWME+

Uncertainty

BS

SEvent: 2m Temp < 0°C

Lead Time (h)

Eckel and Mass, WAF, 2005

Multi-model w/ bias cor.

Multi-model

IC only

8

Parameters – emphasis on sensible weather and user requirements

Ground Truth – emphasis on observations vs. model analysis

Skill Metrics (VRH, BSS, CRPS, etc.) – focus on ensemble performance

Value Metrics (ROCSS, VS, etc.) – focus on benefit to user

Accessible to Users

– Interactive, web-based interface

– Frequent updates

– Link into products and education

What does it take to be effective?- - - Comprehensive Verification - - -

Both feed back into

R&D

Lambert and Roeder, NASA, 2007

http://science.ksc.nasa.gov/amu/briefings/ fy08/Lambert_Probability_ILMC.pps

Lightning % ForecastsCape Canaveral / Patrick AFB

Shuttle Endeavor12 Jul 2009

9

What are the impacts of shortcomings?

Poor Dispersion – Failure to simulate error growth

NCEP SREF20090301-20090531

48-h Sfc Wind Speed

Degraded Valuefor

Decision Making

C/L Ratio

Val

ue

Sco

re

Robust Meso. EPS

Deficient Meso. EPS

NCEP SREF20090301-20090531

Event: 48-h Sfc RH > 90%

UKMet MOGREPS20060101-20060228

Event: 36-h Cloud Ceiling < 500ft

Poor Skill – Weak Reliability & Resolution

Bowler et al., MetOffice, 2007

max

imum

likely

true

pro

babil

ity

mini

mum likely

true p

roba

bility

expe

cted

value

of t

rue

prob

abilit

y

JMA EPS20071215-20080215

Event: 5-d 2m temp 0C

High Ambiguity – Large random error in uncertainty estimate

Eckel and Allen, 2009

10

Backup Slides

11

True Forecast PDF

True forecast PDF recipe for lead time in the current forecast cycle:

1) Look back through an infinite history of forecasts produced by the analysis/forecast system in a stable climate

2) Pick out all instances with the same analysis (and resulting forecast) as the current forecast cycle

3) Pool the verifying true states at to construct a distribution

Combined effect creates a wider true PDF.

Erred model also contributes to analysis error.ErredErred

While each matched analysis corresponds to only one true IC,

the subsequent forecast can match many different true states

due to grid averaging at =0 and/or lack of diffeomorphism. ErredPerfect

Each historical analysis match will correspond to a

different true initial state, and a different true state at time . PerfectErred

Only one possible true state, so true PDF is a delta function.PerfectPerfect

0

0.2

0.4

dnorm( ),,x3.60.9

x

0

0.2

0.4

dnorm( ),,x3.60.9

x

0

0.2

0.4

dnorm( ),,x3.60.9

x

0

0.2

0.4

dnorm( ),,x3.60.9

x

Analysis Model True Forecast PDF (at a specific )

0

0.2

0.4

dnorm( ),,x3.60.9

x

0

0.2

0.4

dnorm( ),,x3.60.9

x

Perfect exactly accurate (with infinitely precision)Erred inaccurate, or accurate but discrete

Not absolute -- depends on

uncertainty in ICs and model

(better analysis/model =

sharper true PDF)

12

ET maintains error variance in more directions than breeding…

Average forecast error covariance matrix eigenvalue spectrum, 24h leadtime

Am

pli

tud

e

Eigenvalue

ET

Breeding

ICs by Ensemble Transform (ET)(from Craig Bishop, NRL)

13

O. Talagrand and G. Candille, Workshop Diagnostics of data assimilation system performanceECMWF, Reading, England, 16 June 2009

14

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

0 12 24 36 48

Lead Time (h)

EF

Sp

read

or

M

SE

of

EF

Mea

n(d) T2(c) WS10

Ens

. Var

. O

R

mse

of

the

Ens

. Mea

n (º

C2 )

Ens

. Var

. O

R

mse

of

the

Ens

. Mea

n (m

2 / s

2 )

c2 11.4 m2 / s2 c

2 13.2 ºC2

mse of *UWME Mean

UWME Variance

*UWME Variance

mse of *UWME+ Mean

UWME+ Variance

*UWME+ Variance

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

0 12 24 36 48

Lead Time (h)

EF

Sp

read

or

M

SE

of

EF

Mea

n

16L

22L

22L04L

04L

10L

10L 16L

0.0

0 12 24 36 48

Eckel and Mass, WAF, 2005