bulk statistics on ensemble model forecasts for mdss demo 2003
DESCRIPTION
Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003. Paul Schultz NOAA Forecast Systems Laboratory June 17, 2003. 2. The MDSS ensemble modeling component. What is it? Several computer model forecasts to supplement the NWS model forecast services Why are we doing this? - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/1.jpg)
Bulk Statistics on Ensemble Model Forecasts for MDSS
Demo 2003
Paul Schultz
NOAA Forecast Systems Laboratory
June 17, 2003
![Page 2: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/2.jpg)
The MDSS ensemble modeling component
• What is it?– Several computer model forecasts to supplement the NWS model
forecast services
• Why are we doing this?– Better forecasts. Just seeing if you’re paying attention.
• How does it work?– By combining multiple (imperfect) forecasts of the (imperfectly
observed) atmosphere, we can make a single ensemble forecast that is better than any of the forecasts that went into it.
2
![Page 3: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/3.jpg)
Ensemble modeling
• Did it work during the 2003 Demo?– Not as well as it can. It shows promise. It can be improved.
3
![Page 4: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/4.jpg)
The ensemble for Demo 2003
• Three models, two LBC source models, total of six ensemble members– models: MM5, RAMS, WRF– LBC sources (from NCEP): AVN, Eta– 6-hour cycle– 27-hour forecasts– 12-km grid
5
![Page 5: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/5.jpg)
Bulk statisticsState variables, 12-hr forecasts
Feb 1 – Apr 8, 2003
6
Temperature (K) Wind speed (m/s) Dewpoint (K)
MM5-AVN 3.1 -0.7 2.5 +0.8 5.6 +1.5
MM5-Eta 3.0 -0.5 2.5 +0.8 5.5 +1.6
RAMS-AVN 5.8 -1.1 2.6 +1.6 6.5 -0.9
RAMS-Eta 5.9 -1.1 2.6 +1.7 6.9 -1.0
WRF-AVN 3.1 -0.4 2.4 +1.1 5.7 +1.4
WRF-Eta 3.1 -0.4 2.4 +1.0 5.7 +1.3
![Page 6: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/6.jpg)
Precipitation verification
MDSS 0-3 h QPF Equitable Skill Score 133 runs from 1 Feb - 26 Mar 2003
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.01 0.1 0.25 0.5
Precip Threshold (in)
Eq
uit
able
Ski
ll S
core
MM5
RAMS
WRF
7
MDSS 0-3 h QPF Bias Score 133 runs from 1 Feb - 26 Mar 2003
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0.01 0.1 0.25 0.5
Precip Threshold (in)
Fre
qu
ency
Bia
s
MM5
RAMS
WRF
![Page 7: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/7.jpg)
A closer look
9 pm model runs, verifying only Iowa stations, entire expt
![Page 8: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/8.jpg)
Improving the ensemble
• Remove unhelpful members– If we can’t fix RAMS problems, it’s gone
– Different LBC models don’t seem to help (?????)
8
![Page 9: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/9.jpg)
Unhelpful members
MM5+Avn
WRF+Avn
MM5+Eta
WRF+Eta
The LBC models don’t add enough
diversity
![Page 10: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/10.jpg)
Improving the ensemble
• Add good models– FSL/RUC a very good candidate for Demo 2004
• Change model configurations– WRF cloud/precip physics– Model cycle frequency, lead times, etc.
• Optimize use of available computing resources• Take advantage of what regional models do best
• Improve post-processing– Better PoP (probability of precip) estimates -- FSL– Better tuning procedures -- NCAR– Hope for “better” weather during tuning period
![Page 11: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/11.jpg)
Reliability9
Percentage of expected FSL model runs
0
10
20
30
40
50
60
70
80
90
100
2/3 2/10 2/17 2/24 3/3 3/10 3/17 3/24 3/31 4/7
NCEP data problems Giant
snowstorm in Boulder
Planned power
outage at FSL
![Page 12: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/12.jpg)
Reliability
• MM5 shows good reliability
• Others will improve with better scripting
10
Percentage of expected FSL model runs
0
10
20
30
40
50
60
70
80
90
100
MM5-AVN MM5-ETA RAMS-AVN RAMS-ETA WRF-AVN WRF-ETA
![Page 13: Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003](https://reader036.vdocuments.site/reader036/viewer/2022062804/56814c08550346895db90977/html5/thumbnails/13.jpg)
Photos from MDSS field trip
Bob Stradley and Ron Simmons
Downward-pointed radiometer mounted on rear-view mirror of
Jim Van Sickle’s truck
RWIS tower, I-35 south of Ames