1 nuopc national unified operational prediction capability update to copc 27 – 28 may 2015 dave...
TRANSCRIPT
1 NUOPCNational Unified Operational Prediction Capability
Update to COPC27 – 28 May 2015
Dave McCarren, NUOPC DPM
2 NUOPCNational Unified Operational Prediction Capability
2
Agenda
National ESPC Update
NUOPC UEO Committee Update
NUOPC CMA Committee Update
Questions and Discussion
3 NUOPCNational Unified Operational Prediction Capability
.
The National Earth System Prediction Capability (National ESPC)
3
The National Earth System Prediction Capability ESPC 4
National ESPC Update • National ESPC Strategy 2-pager created for appropriate distribution to
get support for the project; sent to Liaisons for review and taken to AMS meeting in Phoenix
• National ESPC staff drafting Strategy paper for submission to BAMS
• ESG meeting 26 Jan 2015
• ESG Principals +1 meeting hosted by Navy 5 May 2015
The National Earth System Prediction Capability ESPC 5
National ESPC Update
• ESG meeting 26 Jan 2015 Action Items:– Modify the proposed National ESPC Management Structure to show
reporting relationships and communications. Project office will provide proposal with fully coordinated recommendation, options, with pros and cons.
– Coordinate a topical ESG meeting within 90 days for various topics, N2/6E to host. Semi-closed session: just staff and Principals + 1.
– Verify that the ESMF paper contains proper mention of National ESPC. Provide the ESG principals background on ESMF management, funding, and requirements process.
– Coordinate specific sessions for National ESPC at upcoming conferences.
– Coordinate a response to the Presidential Executive Order on Coordination of National Efforts in the Arctic on the role of ESPC in Arctic prediction, including a potential National ESPC brief by the principals to OSTP. This response may include the possible development of an integrated NAVY/NOAA operational Arctic Modeling Strategy.
The National Earth System Prediction Capability ESPC 6
National ESPC Update
• ESG meeting 5 May 2015 draft Action Items:– Revise the ESPC and NUOPC charters and create a new
National ESPC Charter. Include recommendations from Decision Brief on Management Structure/Plan, and Personnel Issues. Work with climate community to help define what is operational for products to support strategic decision making, Define components that will be part of National ESPC - spiral 1
– Coordinate AF and Navy collaboration to determine the DOD way ahead and converge on a common core
National Unified Operational Prediction Capability NUOPC 7
UEO Committee Update
National Unified Operational Prediction Capability NUOPC 8
Unified Ensemble Operations
• NUOPC metrics from NCEP, FNMOC, and AFWA briefed at 26 Jan ESG meeting
• ½ Degree Data Exchange Upgrade on schedule for Summer 2015 implementation at NCEP, CMC, and FNMOC
• NUOPC/NAEFS mini workshop on 24-25 Feb– Face to face meeting of the Co-Chairs of the UEO Committee– Agenda focused on the development of NUOPC that includes data
exchange (of global and wave ensemble data, post process, ensemble week 3&4 forecast and future plans.
– Research following a concern by FNMOC over the timing of the CMC raw ensemble data resulted in NCEP starting raw data processing 90 minutes earlier, much closer to real time and bias corrected data.
• AF moved up operational GEPS processing to take advantage of this; GEPS suite had 5,000 unique users and about 4M product hits in the last 12 months
½ Degree Data Exchange Implementation Schedule
NCEP CMC FNMOC
Test hi-res data Available
Pre-production hi-res data Jan 2015 Sep-Oct 2014
Production hi-res data Apr 2015 Nov-Dec 2014 Oct 2014
Ready to receive hi-res production data Jan 2015
Test hi-res NAEFS products Mar 2015 Apr 2015 (SFC-3HR)
Pre-production hi-res NAEFS products Apr 2015
Production hi-res NAEFS products May 2015 Jul 2015(SFC-3HR)
Earliest practical date for each stage at each center
5-day forecast for surface wind (U)
10-day forecast for surface wind (U)
NH T2m
NA T2m
13
Northern HemisphereWind Speed
The spread ratio for wind speed was greater than one for the entire period (under-dispersive) with NAVGEM and GFS members both being over-dispersive and GEM members also under-dispersive.
There was a spike in May when NOGAPS was replaced by NAVGEM.
GEM members are under-dispersive while NAVGEM and GFS members are over-dispersive
14
Ensemble MeanNorthern Hemi -2 Meter Temps
Ensemble mean in solid red. GFS members in dotted green, NAVGEM members in dotted blue, and GEM members in dotted purple. The solid green is the GFS control member and the solid purple is the CMC control member. We do not currently receive a NAVGEM control member.
Ensemble mean has nearly no bias. GFS members have a cold bias, and NAVGEM members have a warm bias
National Unified Operational Prediction Capability NUOPC 15
CMA Committee Update
National Unified Operational Prediction Capability NUOPC 1616
NUOPC
Common Model Architecture
• Draft manuscript for ESPS paper submitted to BAMS; CMA Committee revising first draft for 1 June deadline
• CMA Committee drafting new whitepaper on Using ESPS
• PI Group building a working Physics Driver prototype with a common physics interface to GFS physics, by June 2015, to address NGGPS requirement to begin testing how it works and use in the selection of a new Dynamc Core, using a common set of physics.
• PI Group presented modernized Kalnay Rules paper at AMS
National Unified Operational Prediction Capability NUOPC 17
Questions & Discussion
NUOPC Verification Metrics
EMC/NCEP
May 18 2015
Acknowledgement: Dr. Yan Luo
For all three individual bias corrected ensemble forecast (NCEP/GEFS, CMC/GEFS and FNMOC/GEFS) and combined (NUOPC) ensemble (equal weights) against UKMet analysis
Period: April 1st 2011 – May 15 2015
Ratio of RMS error over spread
Northern Hemisphere 500hPa height:
30-day running mean scores of day-5 CRP scoreRMS error and ratio of RMS error / spreadAnomaly correlation
All other regions could be seen from: http://www.emc.ncep.noaa.gov/gmb/yluo/naefs/VRFY_STATS/NUOPC_bc_COMB_spr2015_ts.html
NH 500hPa anomaly correlation
5-day forecast
NH 500hPa CRP scores
Under-dispersion
Over-dispersion
NH 500hPa RMS errors
Northern Hemisphere 500hPa height:
30-day running mean scores of day-10 CRP scoreRMS error and ratio of RMS error / spreadAnomaly correlation
All other regions could be seen from: http://www.emc.ncep.noaa.gov/gmb/yluo/naefs/VRFY_STATS/NUOPC_bc_COMB_spr2015_ts.html
10-day forecast
NH 500hPa CRP scores
NH 500hPa anomaly correlation
NH 500hPa RMS errors
NH CRP scoresNH RMS errors
NA CRP scoresNA RMS errors
5-day forecast for surface temperature
10-day forecast for surface temperature
5-day forecast for surface wind (U)
10-day forecast for surface wind (U)
5-day forecast for surface wind (V)
10-day forecast for surface wind (V)
Last winter statistic scores
Dec. 1st 2014 – Feb. 28th 2015
Northern Hemisphere 500hPa height:
Latest 3-month winter scores: CRPS skill scoreRMS error and ratio of RMS error / spreadAnomaly correlation
All other regions/scores could be seen from: http://www.emc.ncep.noaa.gov/gmb/yluo/NUOPC/NUOPC_bc_win1415.html
NH CRPS skill scores
NH anomaly correlation
500hPa Height
NH RMS errors
NH CRPS skill scores NH RMS errors
NA CRPS skill scores NA RMS errors
Surface temperature
NH CRPS skill scores NH RMS/Spread scores
For surface wind (U)
NH CRPS skill scoresNH RMS/Spread scores
For surface wind (V)
Comparing of NCEP/GEFS, NAEFS and NUOPC for all bias corrected ensemble forecasts, against UKMet analysis
Period: April 1st 2011 – May 15th 015
Day-5 NH 500hPa height
Day-10 NH 500hPa height
NH T2m
NA T2m
Aim High…Fly, Fight, Win
NUOPC Ensemble Performance
1 Jan 2015 – 1 Apr 2015
Mr. Bob Craig16 WS/WXN
Aim High…Fly, Fight, Win 34
Overview
Method
Brier Skill Scores
Spread
Ensemble Mean
Aim High…Fly, Fight, Win 35
Method
All models verified against 25km UKMO analysis except for total cloud cover that uses WWMCA
Data shown is mainly for the northern hemisphere domain containing Navy - NAVGEM CMC - GEM NCEP - GFS
On ensemble mean slides, ALL refers to the GEPS ensemble
For skill scores, climatology was used as reference forecast
Aim High…Fly, Fight, Win 36
Brier Skill Score
Brier Score (BS) for ensembles is equivalent to Root Mean Square error for deterministic models. Model probabilities for an event are compared to the
relative frequency of the event To get the skill score, the BS is compared to climatology
Brier skill scores greater than 0 indicate skill
Aim High…Fly, Fight, Win 37
Brier Skill ScoreTotal Cloud > 80%
Total Cloud > 80% - looking at the CONUS domain, significant skill is indicated out to about 108hrs. For global and hemisphere domains, little significant skill is indicated. Some reasons for this can be seen on the next slide.
Aim High…Fly, Fight, Win 38
Typical Total Cloud ForecastTotal Cloud > 80%
Black squares are non-zero probabilities and green “1” are observation hits. Over high latitudes there tends to be less hits compared to lower latitudes. Cloud coverage tends to be over- forecasted
Aim High…Fly, Fight, Win 39
Brier Skill ScoreWind Speed > 25kts
For wind speeds >= 25kts, global coverage and northern hemisphere have significant skill out to 240 hours. Higher winds speeds (>35, and >50) also have skill, but an insignificant number of events resulted in much larger error bars.
25Kts
25Kts
Aim High…Fly, Fight, Win 40
Ensemble Spread
Spread measures the difference between the ensemble mean and ensemble forecasts Typically the RMSE of the ensemble mean is compared
to the standard deviation of the members from the mean (RMSE/Stdev)
An ideal ensemble will have the same size of ensemble member deviation from the mean as Root Mean Square error of the ensemble mean
An ensemble spread greater than one means the ensemble does not contain enough spread in the members to account for the errors in the ensemble system
Aim High…Fly, Fight, Win 41
Northern Hemisphere500mb Heights
This chart depicts the ensemble spread ratio of the ensemble members compared to the ensemble mean over the last year for the 120 hour forecast. Green represents the spread ratio of the combined ensemble (ALL) compared to the spread for GFS (purple), Navy (red), and CMC (blue). Ideally, the spread ratio should be around one. A spread ratio greater than one indicates the ensemble is under-dispersive and the opposite is true for spread ratio less than one.
This chart depicts the ensemble spread by forecast hour. The ensemble starts off under-dispersive in early forecast hour getting better in later forecasts, green line approaching one
Aim High…Fly, Fight, Win 42
Northern HemisphereWind Speed
The spread ratio for wind speed was greater than one for the entire period (under-dispersive) with NAVGEM and GFS members both being over-dispersive and GEM members also under-dispersive.
There was a spike in May when NOGAPS was replaced by NAVGEM.
GEM members are under-dispersive while NAVGEM and GFS members are over-dispersive
Aim High…Fly, Fight, Win 43
Northern HemisphereTemperature
The spread ratio was above 1 (under_dispersive) for most of the winter but approaching 1 as Spring arrives
Over the last several months, the spread starts under-dispersive with early forecast hours but becomes neutral by the end of the period
Aim High…Fly, Fight, Win 44
Ensemble Mean
Ensemble mean is the average value of the ensemble members
The ensemble mean error should be less than the error of all the ensemble members for a good ensemble
One of the ensemble members from each center is the control and is configured like the deterministic model from the center though its resolution is degraded We do not currently get a control member from
NAVGEM, but that will be added when available
Aim High…Fly, Fight, Win 45
Ensemble MeanNorthern Hemi -2 Meter Temps
Ensemble mean in solid red. GFS members in dotted green, NAVGEM members in dotted blue, and GEM members in dotted purple. The solid green is the GFS control member and the solid purple is the CMC control member. We do not currently receive a NAVGEM control member.
Ensemble mean has nearly no bias. GFS members have a cold bias, and NAVGEM members have a warm bias
Aim High…Fly, Fight, Win 46
Ensemble MeanNorthern Hemi -10 Meter Winds
GFS members have the lowest error and NAVGEM members have the highest
The ensemble mean has a positive bias and the NAVGEM members seem to be pulling it that way
Aim High…Fly, Fight, Win 47
Ensemble MeanNorthern Hemi – 500mb Hgts
NAVGEM members have the highest error out to 168 hrs, then all members are similar
Ensemble mean bias is near 0 out to about 168 hours then starts to rise as GEM and NVGEM members pull it positive
Aim High…Fly, Fight, Win 48
Ensemble MeanNorthern Hemi – 250mb Winds
A three model members are close for this field
Ensemble mean has a negative bias with all model members contributing. GFS members show a discontinuity after 192 hrs in their bias
Aim High…Fly, Fight, Win 49
Ensemble MeanNorthern Hemi – 850mb Temps
GFS members slightly lower error in early forecast hours
Ensemble mean has a cold bias with GEM and GFS members pulling it into down
Aim High…Fly, Fight, Win 50
Summary
The GEPS ensemble system is performing as expected Brier skill scores for wind speed indicated the GEPS
ensemble has skill out to 240hrs Precipitation forecasts weren’t included here due to recent
problem with UM GRIB files The GEPS ensemble spread score tends to be greater
than 1 with the mean RMSE > member standard deviation, indicating the ensemble members are under-dispersed which is a common ensemble issue
Aim High…Fly, Fight, Win 51
Summary
Ensemble mean forecasts have lower errors than the members which is expected for correctly designed ensembles.