pertti nurmi juha kilpinen sigbritt näsman annakaisa sarkanen ( finnish meteorological institute )
DESCRIPTION
Pertti Nurmi Juha Kilpinen Sigbritt Näsman Annakaisa Sarkanen ( Finnish Meteorological Institute ) Probabilistic Forecasts and Their Verification as Decision-Making Tools for Warnings against Near-Gale Force Winds WSN05: WWRP Symposium on Nowcasting and Very Short Range Forecasting - PowerPoint PPT PresentationTRANSCRIPT
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
Pertti Nurmi
Juha KilpinenSigbritt Näsman
Annakaisa Sarkanen( Finnish Meteorological Institute )
Probabilistic Forecasts andTheir Verification as
Decision-Making Tools forWarnings against Near-Gale Force Winds
WSN05: WWRP Symposium onNowcasting and Very Short Range Forecasting
Toulouse, 5-9 September 2005
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
• Develop warning criteria / Guidance methods to forecast probability of near-gale force winds in the Baltic Joint Scandinavian research undertaking
– e.g. Finland and Sweden issue near-gale & storm force wind warnings for same areas using different criteria Homogenize !
• 6 Finnish coastal stations c. 15-20 stations from Sweden, Denmark, Norway
• Probabilistic vs. deterministic approach
• HIRLAM ECMWF model input
• Different calibration methods, e.g. Kalman filtering
Goal: Common Scandinavian operational warning practice
Introduction:
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
• HIRLAM (limited area model)
RCR ~ 22 km version MBE ~ 9 km version Data coverage: 9.11.2004 – 31.3.2005 ~ 140 cases
• ECMWF Applied as reference, only
Data interpolated to 0.5o *0.5o Nearest grid point Data coverage: 1.10.2004 – 30.4.2005 ~ 210 cases
• Forecast lead time: +6 hrs (and beyond ECAM paper)
• Forecasts: wind speed at 10m
• Observations: 10 minute mean wind speed
Data:
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
Potential problems:
• with height of instrumentation ?
• with observing site surroundings and obstacles ?
– with the coast ?
– with nearby islands ?
– with barriers ?
– with installations ?
• with low-level stability ?
NE
“Statistical correction”
scheme available at FMI
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
(m)
55
50
45
40
35
30
25
20
15
10
5
Height of the instrumentation - Large filled dots: 6 Finnish stations being used- Yellow dot: Station_981; Results presented here
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
32 m
15,5 m/s
10 m
15
Wind speed dependence:Logarithmic wind profile
14 m/s
979 - Bogskär Unstable
Neutral
Stable
threshold
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
-20 -10 0 10 20ERROR
0
30
60
90
Coun
t0.0
0.1
0.2
0.3
0.4
Proportion per Bar
FITTED DISTRIBUTION
Methods for producing probabilistic forecasts 1:
Deterministic forecasts:
• Error distribution of original sample (~140 cases)
• Approximation of the error distribution with a Gaussian fit (, ):
”Dressing” method
1. ECMWF EPS (51 members) P (wind speed) > 14 m/s
2. Kalman filtering Various approaches No details given here
3. Deterministic forecast, “dressed” with “a posteriori” description of the observed error distribution of the past, dependent sample P (wind speed) > 14 m/s
“Simplistic reference” !
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
Methods for producing probabilistic forecasts 2:4. Deterministic forecast, adjusted with a Gaussian fit
to model forecasted stability( Temperature forecasts from 2 adjacent model levels )
P (wind speed) > 14 m/s “Stability” method~ Scheme used at SMHI (H. Hultberg)
5. “Uncertainty area” method (aka ”Neighborhood method”) (aka ”Probabilistic upscaling”)
Spatial (Fig.) and/or temporal neighboring grid points
Size of uncertainty area ? Size of time window ? c. 50-500 “members”
RCR: ± 3 points ~ 150*150 km2
MBE: ± 6 points ~ 120*120 km2
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
Relative Operating Characteristic
• To determine the ability of a forecasting system to discriminate between situations when a signal is present (here, occurrence of near-gale) from no-signal cases (“noise”)
• To test model performance ( H vs. F ) relative to a given probability threshold
• Applicable for probability forecasts and also for categorical deterministic forecasts Allows for their comparison
“R” statistical package used for ROC computation/presentation
Probabilistic FCs: ROC
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
mbe_nh1_UtoMBEProb6Px_12_6h_H
False Alarm Rate
Hit
Rat
e
0.10.20.3
0.40.5
0.6
0.70.8
0.9
1
ROC AREA 0.82 ( 0.908 )
ROCA = 0.82
ROCA fit = 0.91
0
20
40
60
80
100
120
.05 .15 .25 .35 .45 .55 .65 .75 .85 .96
ROC curve/area; Station_981; +6 hrs; No. of events ~25/130
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
mbe_dress1_UtoMBE06Hh_12DD_06h
False Alarm Rate
Hit
Rat
e
0.1
0.20.3
0.4
0.5
0.60.7
0.8
0.9
1
ROC AREA 0.928 ( 0.911 )
ROCA = 0.93
ROCA fit = 0.91
”Simple reference” (dep. sample):
HIR_MBE_”Dressing”
0
20
40
60
80
100
120
.05 .15 .25 .35 .45 .55 .65 .75 .85 .95
HIR_MBE_”Uncertainty area”
~ 120 * 120 km
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
mbe_dress1_UtoMBE06Hh_12DD_06h
False Alarm Rate
Hit
Rat
e
0.1
0.20.3
0.4
0.5
0.60.7
0.8
0.9
1
ROC AREA 0.928 ( 0.911 )
ROCA = 0.93
ROCA fit = 0.91
0
20
40
60
80
100
120
.05 .15 .25 .35 .45 .55 .65 .75 .85 .95
ROC curve/area; Station_981; +6 hrs; No. of events ~25/130
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
mbe_hh_UtoMBE06Hh_12_06h
False Alarm Rate
Hit
Rat
e
0.1
0.2
0.30.4
0.5
0.6
0.7
0.80.91
ROC AREA 0.844 ( 0.82 )
ROCA = 0.84
ROCA fit = 0.82
0
20
40
60
80
100
120
.05 .15 .25 .35 .45 .55 .65 .75 .85 .95
HIR_MBE_”Stability”
”Simple reference” (dep. sample):
HIR_MBE_”Dressing”
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
Comparison of methods; Station_981; +6 hrs
HIR_MBE HIR_RCR
"Uncertainty area" method
Dr Stb S S + t L L + t S S + t L L + t
ROC A .93 .84 .82 .85 .87 .88 .82 .85 .87 .88
BSS .59 .36 .38 .50 .34 .45 .35 .47 .24 .37
No. events: ~ 25 /130
Dr - "Dressing" of dependent sample
Stb - "Stability" method
"Uncertainty area" method:
S - Smaller area
S + t - Smaller area with ± 3 hour forecast time window
L - Larger (double) area
L + t - Larger (double) area with ± 3 hour forecast time window
9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]
• We’ve only scratched the (sea) surface Need (much) more experimentation with various methods Different methods for different time/space scales ?Apply to data of other Scandinavian counterparts (here, only single station)
• Scores depend on station properties(e.g. observation height; Not dealt with here) (Statistical) adjustment of original observations required !
Finland has an operational scheme for this !
• “Dressing” of dependent sample: quality level hard to reach
• “Uncertainty area” size: a tricky issue
• Higher resolution HIRLAM version produces higher scores Not necessarily a trivial result !
Reach the goal, i.e. common operational practice !!!
Conclusions Future: