page 1© crown copyright 2005 use of eps at the met office ken mylne and tim legg
TRANSCRIPT
© Crown copyright 2005 Page 1
Use of EPS at the Met Office
Ken Mylne and Tim Legg
© Crown copyright 2005 Page 2
Outline
Update on verification of First-Guess Early Warnings of severe weather
Example of unusual model and EPS behaviour
Met Office short-range ensemble development
Early Warnings of severe weather –The 4-day skill maximum investigated
© Crown copyright 2005 Page 4
Aims of this investigation
Verification of Early Warnings of Severe Weather presented last year showed a maximum in skill at day 4 for
EPS forecasts Deterministic forecasts from
T511 and EPS control Robust result but
ECMWF could not replicate Very little support in literature
Here we report further investigations of:
Can same result be replicated with Met Office model (UM)?
Definition of weather events.
D+1
D+2
D+4
D+5
© Crown copyright 2005 Page 5
Verification of Early Warnings
Early Warnings are verified against ‘Flash’
Warnings issued at short-range for the same
events (with a high degree of certainty)Warnings are verified on an “event” basis
An event can be a forecast event or an observed event
Each event is counted only once however long it lasts
This will be discussed further
Assessment period: 26 August 2003 to 29 April 2005.
© Crown copyright 2005 Page 6
Verification of UM-based Early Warnings ROC results (Heavy Rainfall)
Heavy Rainfall events
UM
UM results very similar to ECMWF T511 UM also has 4-day skill max
UM ROC Areas:•D+1 0.522•D+2 0.561•D+3 0.586•D+4 0.661•D+5 0.557•D+6 0.531
© Crown copyright 2005 Page 7
ROC results for Gales and Snow
Severe Gale events (Little evidence) Heavy Snowfall events (Similar to Rainfall)
© Crown copyright 2005 Page 8
Cost/Loss results (Heavy Rainfall)
Ensemble Control
T511 Met O U.M.
© Crown copyright 2005 Page 9
Spatial Averaging
Anders Persson suggested the 4-day skill max could be due to predictability on the spatial scale of the UKThis would suggest a shorter period max for smaller
regionsFGEW also gives probabilities for 12 sub-regions of
the UKVerification of these sub-region probs also show
the 4-day max We have not found any evidence to support this
idea but it does merit further investigation
© Crown copyright 2005 Page 10
The effect of how we define an ‘event’
There are two main ways of defining events for verification
(i) On an ‘event-wise’ basis – when an event occurs, did we have an early warning of it? And when an early warning exists, did an event occur? One contingency-table entry per event.
(ii) On a time-wise basis – at fixed time intervals, look to see whether or not an Early Warning and/or a Flash Warning were in force and complete contingency tables
Early Warnings have always been verified on an event basis:
This is different from most standard verification procedures which
use method (ii)
Could this account for the day 4 skill-max?
© Crown copyright 2005 Page 11
Definition of events for verification
Events are defined as:
An event can be a forecast event or an observed event
Each event is counted only once however long it lasts
Event spanning 2 days counted for 1st day only
Changing this to last day did not affect the day-4 skill max
For a warning to be correct (Hit), a warning and a verifying Flash
Warning must coincide for part of their validity period
Flash with no warning is a Miss
Warning with no Flash is a False Alarm
Correct Rejections defined for a complete 24-hour period with
no warning or Flash (except one already verified for previous
day)
© Crown copyright 2005 Page 12
Results for different event definitions
‘Event-wise’ ‘Time-wise’
for Heavy Rainfall warnings (01 Oct 2003 – 03 Nov 2004)
© Crown copyright 2005 Page 13
Conclusions on day-4 Skill Max
First-guess Early Warnings verification designed to assess the skill of warnings issued to end users:
Event-based Events of variable length Each event verified once only Precise timing not required for success – only some overlap
Latest results show that it is this definition which leads to the day-4 maximum in skill apparent in the results We do not claim to fully understand why! Could be related to
spatial averaging as suggested by Anders. We cannot assume that results from “standard” verification of
NWP will apply to user-oriented products based on NWP.email: [email protected] or [email protected]
Skill of EPS Control and EPS ModelExample raised by Met Office Chief
Forecaster
© Crown copyright 2005 Page 15
EPS Model
Forecaster was surprised and concerned that EPS control and T511 were so different at day 6 in this forecast, with only a difference in resolution
Opposite ends of EPS distributionIs this normal/ to be expected?Has the previously reported problem with the EPS model (time-stepping?) been solved?
CTRL T511
Short-Range Ensembles at the Met Office
© Crown copyright 2005 Page 17
Short-range Ensembles
ECMWF EPS has transformed the way we do Medium-Range Forecasting
Uncertainty also in short-range: Rapid Cyclogenesis often poorly forecast deterministically (eg Dec 1999) Uncertainty of sub-synoptic systems (eg frontal waves)
Many customers most interested in short-range Assess ability to estimate uncertainty in local weather
QPF
Cloud Ceiling, Fog
Winds etc THORPEX
Observation targeting Multi-model ensemble contribution
LBCs for future storm-scale ensembles
© Crown copyright 2005 Page 18
Ensemble Prediction Developments
Ensemble under development for short-rangeRegional ensemble over N. Atlantic and Europe (NAE)Nested within global ensemble for LBCsETKF perturbationsStochastic physicsT+72 global, T+36 regional
NAE
© Crown copyright 2005 Page 20
Progress with ETKF
ETKF set up with global UMProcessing all observations used in data
assimilation
12-hour cycle (f/c twice per day)
Running in conjunction with stochastic physics to propagate effect
Encouraging growth rate in case studies
© Crown copyright 2005 Page 21
Stochastic Physics Schemes
Three components to current stochastic physics: Installed in current version:
Stochastic Convective Vorticity (SCV)Random Parameters (RP)
Under test:Stochastic Kinetic Energy Backscatter (SKEB)
Page 29
SKEB
Stochastic Kinetic Energy Backscatter (SKEB) Based on original idea and previous work by Shutts (2004)
Closely related to ECMWF CASBS scheme
Aim: To backscatter (stochastically) into the forecast model some of the energy excessively dissipated by it at scales near the truncation limit
In the case of the UM, a total dissipation of 0.75 Wm-2 has been estimated from the Semi-lagrangian and Horizontal diffusion schemes. (Dissipation from Physics to be added later on)
Each member of the ensemble is perturbed by a different realization of this backscatter forcing
Page 30
SKEB
Streamfunction forcing:1
( , )2
DF K R
K.- Kinetic En.; R.- Random field;
D.- Dissipated en. in a time-step
R is designed to reproduce some statistical properties found with CRMs
Largest at the jets/storm track
Example: u increments at H500
Page 31
SKEB
Preliminary results: Positive increase in spread (comparable to that seen at ECMWF)
SKEB
RP+SCV
Increase in spread respect to an IC-only ensemble
500 hPa geopotential height
© Crown copyright 2005 Page 33
First Full Case Study Run (7-8 July 2004)
ETKF spun-up over 7 daysStochastic physics and ETKF interacting
Forecasts run to 5 daysSpread looks reasonable
© Crown copyright 2005 Page 34
T+48 Postage Stamps from January 8 2005 storm
AnalysisControl
Several members have better lowthan control. Member 4 is deeper. NB. This is global EPS.
© Crown copyright 2005 Page 35
Project Progress
Milestone: Implementation of demonstration ensemble based on NAE model for assessment by forecasters (August 2005)Global ensemble has now been running in our parallel
test suite for almost 2 weeks
NAE suite is nearly complete
Product generation and Verification systems are under development
We are on target to meet the milestone
© Crown copyright 2005 Page 36
Medium-Range (TIGGE)
Global forecasts will be extended to 15 days to contribute to THORPEX multi-model EPS research This will run at ECMWF using UK member state time
ETKF scheme believed suitable for Medium Range as well as Short Range Perturbations scaled to 12h forecast errors – could be
amplified if necessaryPlanned configuration
90km resolution 20 members twice per day
© Crown copyright 2005 Page 37
Conclusions
Closer to understanding 4-day skill max in severe weather warnings from EPSStandard NWP verification results may not always
translate to user-specific products and verification
Good progress with development of ensemble capability at the Met OfficeShort-range regional ensemble for EuropeContribution to global medium-range ensembles for
THORPEX