Ensembles and The Future of Mesoscale Prediction

Cliff Mass, University of Washington

Tony Eckel, USAF and Univ. of WA

This Talk Will:

• Review the history of mesoscale forecasting

• Describe a vision of the future in which ensembles are the key short-range prediction tool, where ultra-high resolution prediction is an important element, where statistical post-processing is essential, and where humans forecasters shift to areas in which they can make a real contribution

1985-2000: Resolution Intoxication• During this period mesoscale meteorology

experienced a major transition– Prior to 1985, mesoscale forecasts were made

subjectively by experience and rules of thumb. (synoptic model resolution of 80-150 km)

– Between 1985-1995 research simulations (case studies) with MM4/5, RAMS, ARPS, COAMPS and other models demonstrated the potential to skillfully forecast mesoscale structures when grid spacing declined below ~15-20 km AND the synoptic scale prediction was realistic.

– Particularly impressive results near terrain and land/water interfaces (diurnal circulations).

5-km grid spacing of 1992 Convergence Zone (Fang-Ching Chien)

Resolution Intoxication• These were heady times…with case studies

promising realistic mesoscale forecasts. • Plus, increasing computer power on

inexpensive workstations and readily available models made it possible to run such forecasts locally.

• Dozens of groups began running mesoscale models around the country—down to 4-km resolution.

• U. Oklahoma received millions to develop a new mesoscale model and to demonstrate deterministic prediction of convection.

Resolution Intoxication• For the military this was a dream come true…offering a

fix for a critical problem: under-trained and time-pressured forecasters that moved around too much to develop local knowledge and skill.

• The Navy developed its own mesoscale model (COAMPS), with local deterministic forecast applications (TAMS-RT, DAMP). Even talk of battlefield NWP with workstations on humvees!

• High-resolution models proved to be excellent educational tools.

• An essentially deterministic mode was in vogue— it was thought that with sufficient observations and state of the art data assimilation plus advanced models, a solution to mesoscale prediction was at hand.

The Sobering Reality• With more experience, it became clear that

there were essential problems with deterministic high resolution forecasting.– Mesoscale models were only as good as the large

scale forecasts…which all too often were seriously in error—particularly near and over oceans.

– High resolution amplified synoptic errors. – Forecasters lost confidence in forecasts during

mesoscale model busts, often turning back to older synoptic scale guidance.

Thanksgiving Day 2001 Wind Forecast Bust

eta-MM5 Initialized 00z, 21 Nov 01 (Tue. evening)

42h Forecast, valid 10AM Thursday

Eta-MM5 model 12-km runs on Tue & Wed forecast severe wind storm for the Puget Sound on Thu AM. Expected widespread damage and power outage was all over the news.

Verification, 10AM Thursday

The storm came ashore weaker and further south giving light and variable winds in the Puget Sound.

The Sobering Reality

– Verification scores seemed to show diminishing returns at high resolution (~12 km and better)

– Problems with physical parameterizations (e.g., moist physics) at high resolution became apparent.

– Univ. of Ok never was able to demonstrate useful forecast skill for convection beyond a few hours.

24-h MM5 Precip. Bias Scores over W. WA

The Sobering Reality• For undertrained or time-pressured

meteorologists high-resolution forecasts were particularly dangerous– The realism and impressive structures led some to

follow the predictions without question…even when they were wrong.

– They lacked the skill or time to evaluate the large scale predictions and thus had little feeling when they should trust the high resolution predictions.

– Lacked objective tools for determining forecast confidence or skill.

Deterministic vs. Probabilistic Mesoscale Prediction

• But even more serious was the central issue: • • Deterministic mesoscale prediction is essentially

inconsistent with the real-atmosphere where there is considerable uncertainty is initial conditions and model physics, and the system is chaotic.

• Mesoscale prediction needs to transition to a probabilistic viewpoint, and ensembles offer a powerful tool in that direction.

Ensembles and the Transition to Probabilistic Mesoscale Forecasting

To make mesoscale (or synoptic ensembles) one can run a mesoscale model many times with varying:

• Initializations, all consistent with observational errors

• Model physics and boundary conditions (e.g., SST, soil moisture)--all within physics and boundary condition uncertainty

• Dynamical cores and numerics

T

48hr forecast state (core)

48hr true state

Forecast pdf

48hr forecast state (perturbation)

ngp

uk

eta

cmc

gsp

avn

Analysis pdfcwb

Cngp

eta

cmc

avn

gsp

cwb

uk

The result is a collection of forecasts--all valid potential predictions--the can be

used in a number of ways:

1. The ensemble mean tends to be more skillful on average than any particular member.

2. The spread or variability among the ensembles can be related to forecast skill. The spread also gives a forecaster an idea of the range of possibilities.

3. The ensemble members can be used to produce probabilistic forecasts.

Ensemble Forecasting• The community has great deal of experience with

large-scale ensembles (e.g., GFS, NOGAPS, ECMWF)

• Much less experience with mesoscale or short-range ensemble forecasting (SREF).

• There have been some short-term studies of short-term ensemble use in the Midwest, mainly for convection (SAMEX). Suggestive results--ensemble mean is more skillful than members.

• In the U.S., NCEP is running a half-hearted SREF effort using the Eta (48 km) and Regional Spectral models. Uses breeding method for generation of initializations. 10 members.

UW Ensemble System

For roughly three years, the University of Washington has run a regional ensemble system at 36-km and 12-km grid spacing for 48-h for the 00 UTC cycle. MM5 model.

UW Ensemble System• In contrast to most ensemble systems, the UW is based on

using the initializations and boundary conditions from major operational centers: – NCEP: GFS and Eta– Canadian: GEM– Navy: NOGAPS– UKMET Office Model– Australian: GASP– Taiwanese Global Model– Japanese Global Model

• Our assumption: the variability among the operational initializations is a measure of initialization uncertainty. Better approach than breeding or singular vectors?

• Upstream of a large data sparse ocean area initialization uncertainty is probably the largest source of forecast error

UW Ensemble SystemIn addition, the UW ensemble system

• makes use of “mirrors”--an attempt to explore the phase space around the initializations we receive

• includes several physics and boundary condition ensembles (e.g., several cumulus parameterizations, SST distributions, etc).

00h cmcg 21 Nov 00z 00h cmcg* 21 Nov 00z

00h cent 21 Nov 00z

Concept of Mirroring

cmcg

C

cmcg

*

Cmcg--canadian model

Cmcg*--the mirror

UW Ensemble System has proven to be an extraordinarily valuable addition to the UW prediction effort.

13 member SREF42h forecast (valid Thu 10AM)

13: avn*

11: ngps*

12: cmcg*

10: tcwb*

9: ukmo*

8: eta*

Verification

1: cent

7: avn

5: ngps

6: cmcg

4: tcwb

3: ukmo

2: eta

- Reveals high uncertainty in storm track and intensity- Indicates low probability of Puget Sound wind event

Ensemble-Based Probability Products

Prob. of (sustained) Winds > 21 kt

42h forecast, valid Thursday 10AM

10%30%50%10%

90%

Probability plots are the most valuable information from an ensemble

- Boils down excessive information from multiple model runs to one product

- % chance of occurrence is based upon our uncertainty of the event

- Can be made for ANY parameter at ANY threshold

Relating Forecast Skill and Model Spread

Mean Absolute Error of Wind Direction is Far Less WhenSpread is Low

GFS Model too slow bringing Precipitation. Could ensemblesgiven us some warning?

Very cheap, but powerful linux clusters using commodity chips are perfect for running ensembles. Computer power is no longer much of an issue.

Future Mesoscale Ensemble Development

The UW Ensemble system is just the beginning. Some major possible enhancements include:•Bias-removal for ensemble members.•Weighting ensemble members by previous skill in computing ensemble mean •Calibrating probabilities produced by ensemble system•Bayesian combination of ensemble members•Inclusion of dprog/dt information

•Systematic forecast biases arise in NWP from model deficiencies - Inaccurate parameterizations - Incomplete representation of small scale phenomenon •Biases depend upon weather regime (time of day, surface characteristics, type of flow, etc.)

Need for Bias Removal

Data Info

GFS-MM5 model grid point in eastern WA

Verification: centroid analysis

38 forecasts (Nov 02 – Dec 02)

Lead time = 36h

Forecast vs Analysis

Gridded Bias Removal Using Centroid Analysis

∑=

⎟⎟

⎠

⎞

⎜⎜

⎝

⎛=

N

n nji

tjitji o

f

Nb

1 ,

,,,,

1 N number of forecast cases fi,j,t forecast at a certain grid point and lead timeoi,j verifying observation

Calculate bias at every grid point and lead time using last 2 weeks of forecasts:

Post-process current forecast to correct for bias:

tji

tjitji b

ff

,,

,,*,, =

Spatial and Temporal Dependence of Bias

GFS-MM5 MSLP Bias at f36

Common Bias Forecast Error

> 1 too low < 1 too high

Data Info

Average of 65 forecasts(25 Nov 02 – 01 Feb 03)

36km domain from Rockies to central Pacific

2-week bias training for each forecast

Verification: centroid analysis

Results

Observation-Based Bias Removal also Needed…But More Difficult

• Might use land use, elevation, and proximity

• Start with systematic bias at observing stations and then at each grid point average nearby observation site biases for sites with similar land use and elevation.

• Could do hourly using the past several weeks of verifications.

Synoptic Drift of Limited Area Domain Models

• When high-resolution models are run nested in a larger domain parent, the limited area model’s synoptic solution can drift from the parent.

• The UW experience is that such drift is generally negative….I.e. the limited area model does not do the synoptics as well. MM5-GFS inferior to GFS for synoptics.

• This can have a serious impact on regional subnests for longer runs.

72h 36-km domain MM5-GFS, GFS Forecasts, Smaller 36-km domain, Strong Nudging, Weak Nudging

Synoptic Drift Solutions

• Nudging outer model domain

• Smaller outer domain

A mixture of ensembles and high resolution deterministic runs?

• Some phenomena demand ultra-high resolution.• The ensemble system can help decide which models to

use to drive the high resolution runs and provide guidance on expected forecast skill.

• The ensemble system can provide probabilistic information at intermediate resolution

• The high resolution runs can document small scale structure…which has an educational value even if problematic.

• For the immediate future the optimal system will probably be a hybrid between lower resolution ensemble runs and a very limited number of high resolution “deterministic” runs.

Modeling Winds in the Columbia Gorge

• Strongest winds are at the exit

Portland

Troutdale

Cascade Locks

4-km grid spacing 1.3 km grid spacing

The Role of Human Forecasters• Biased-corrected ensemble-based systems will soon be

superior to even highly experienced forecasters for short-range forecasts from several hours to several days.

• One indicator of this fact is that veteran NWS forecasters often cannot beat primitive model output statistics.

• Increasing number of examples of humans degrading forecast skill.

• Human forecasters can play a major role in “nowcasting”…using our superior graphics processing capabilities and can be the ultimate quality control on model output.

• Even more important, forecasters are needed to interpret forecasts for users.

Role of Human Forecasters

• Attempts to stay with a deterministic view of forecasting and to keep humans in the direct forecast generation loop are doomed to failure.

• The National Weather Service with their IFPS (Interactive Forecast Preparation System) are going down the wrong road.

• Military weather services don’t have to repeat such NWS mistakes.

Editorial Comment

• Too many mesoscale models in the U.S.• Too little cooperation between modeling

centers and research community. USWRP has been ineffective.

• Need to pool resources to improve parameterizations, improve data assimilation, learn how to verify mesoscale forecasts, and other problems.

So What About Iraq? What Can Be Done Fast?

• Give military forecasts easy to interpret measures of forecast spread based on available synoptic scale and mesoscale models– Spread charts of important parameters– Thumbnail imagery of key fields from various

modeling systems.

• Make sure that synoptic drift in nested domains is reduced.

• Attempt to remove synoptic-scale systematic bias in key surface fields (e.g.., wind speed)

THE END