Steve KeightonNational Weather Service

Blacksburg, VA

OutlineChaos theory and predictability in the atmosphereNumerical Weather Prediction (NWP) and use of

“ensemble” forecast methods

Use of statistical guidance in the forecast process (Model Output Statistics) – if time

Acknowledgments• Josh Korotky – NWS Pittsburgh• Mark Antolik – NWS Meteorological Development Lab

Prelude – What is Chaos and why is it important?Chaos leads us from the laws of

nature to their consequences …shows us that simple systems

can exhibit complex behavior…and vice versa

…demonstrates that unpredictable behavior can develop in a system governed by deterministic laws

As forecasters…chaos shows us the limits of predictability …highlights the importance of

probabilistic thinking …shows us the value of

expressing uncertainty in forecasts

…helps us understand why the future of forecasting will lean heavily on ensemble rather than deterministic approaches

“… one flap of a sea-gull’s wing may forever change the future course of the weather” (Lorenz, 1963)

Small errors in the initial state estimate of a nonlinear system can limit the prediction of later states of the system

Chaos occurs when error propagation, seen as a signal in time, grows to the same size or scale as the original signal...

dxx y

dtdy

xz rx ydtdz

xy bzdt

Elements of ChaosDynamical system – future states caused by past states

(determinism)Nonlinearity – system output (response) isn't proportional to

input (forcing)…a small forcing can lead to a disproportionately large response

and vise versaa system's values at one time aren‘t proportional to the values

at an earlier timeNon-periodic behavior – future states never repeat past

statesExtreme sensitivity to initial conditions – small initial state

uncertainties amplify…a "prediction horizon” is inevitableEven though the governing laws of a system are known, long-

term predictions can be meaninglessChaos occurs only in deterministic, nonlinear, dynamical

systems

Attractors – General StatementsAn attractor is a dynamical system's set of

conditionsIn a phase space diagram, an attractor shows a

system's long-term behavior. It's a compact, global picture of all of a system's possible steady states.

All attractors are either nonchaotic or chaoticNonchaotic attractors generally are points, cycles, or

smooth surfaces (tori), and have regular, predictable trajectories…small initial errors or minor perturbations generally don't have significant long-term effects

Chaotic or strange attractors occur only after the onset of chaos. Long term prediction on a chaotic attractor is limited…small initial errors or minor perturbations can have profound long-term effects

A Strange Attractor is dynamically unstable and non periodic • - A chaotic system is unstable…its behavior

changes with time rather than settling to a fixed point

• - Chaotic systems are non periodic…trajectories do not settle into repeatable patterns …and never cross

• - A chaotic attractor shows extreme sensitivity to initial conditions… trajectories initially close, diverge, and eventually follow very different paths

Strange Attractors

Non-periodic Dynamical System A dynamical system that never settles into a steady state attractor

Non periodic systems never settle into a repeatable (predictable) sequence of behavior.

Prediction of a future state of a non periodic system is eventually impossible, due to nonlinear dynamics (feedback)

The atmosphere illustrates non periodic behavior Broad patterns in the development, evolution, and movement of

weather systems may be noticeable, but no patterns ever repeat in an exact and predictable sequence

The atmosphere is: …damped by friction of moving air and water …driven by the Sun’s energy …the ultimate feedback system

Weather patterns never settle into a steady state attractor

Small uncertainties (minute errors of measurement which enter into calculations) are amplified

Result: system behavior is predictable in the short term…unpredictable in the long term

Sensitivity to Initial Conditions

The Lorenz DiscoveryFrom nearly the same starting point (tiny rounding error), the

new forecast diverged from the original forecast…eventually reaching a completely different solution!

Why? …Slight differences in the initial conditions had profound effects on the outcome of the whole system

Lorenz found the mechanism of deterministic chaos: simply-formulated systems with only a few variables can display highly complex and unpredictable behavior

(.506) vs. (.506127)

Initial condition

Chaos and Numerical Weather Prediction (NWP)

Weather forecasts lose skill because of: Chaos …small errors in the initial state of a forecast grow exponentially

Model uncertainty Numerical models only approximate the laws of physics (important small

scale processes are parameterized) Very small errors in the initial state of a forecast model grow rapidly at

small scales, then spread upscale

Forecast skill varies both spatially and temporally as a result of both initial state and model errors, which change as the atmospheric flow evolves

If a process is chaotic… knowing when reliable predictability dies out is useful, because predictions for all later times are useless.

Models must simulate numerous irresolvable processes

NWP Skill as a Function of Scale and Time

Predictability falls off as a function of scale

Large scale features (planetary waves) may be predictable up to a week in advance

Small systems (fronts) are well forecasted to day 2.. cyclonic systems to day 4

Some skill in 5-10 day QPFGoodVery GoodQPF/ mean clouds

Skill with max/minVery GoodExcellentTemp / wind

--------FairGoodMesoscale banded structuresConvective clusters

----FairGoodExcellentFronts

Low skill-FairFair-GoodVery GoodExcellentCyclone life cycle

GoodVery GoodExcellentExcellentHemispheric flow transitions

Days 6-7Days 3-5Days 1-2< Day1Feature/Variable

FairGoodVery GoodPrecip/ mean clouds

Skill with max/min TempVery GoodExcellentTemp / wind

--------FairGoodMesoscale banded

structuresConvective clusters

----FairGoodExcellentFronts

Low skill-FairFair-GoodVery GoodExcellentCyclone life cycle

GoodVery GoodExcellentExcellentHemispheric flow

transitions

Days 6-7Days 3-5Days 1-2< Day1Feature/Variable

Why can’t we count exclusively on single model NWP?

Overlooks forecast uncertainty Initial condition and model uncertainty Chaotic flows vs. stable flow regimes

Potentially misleading Oversells forecast capability

GFS 84 hr forecastValid 00Z 22 Nov

NAM 84 hr forecastValid 00Z 22Nov

Single Model NWP

Which model do you believe?

Ensembles and PDF

Recognizing the eventuality of chaos…weather forecasts can provide more useful information by describing the time evolution of an ensemble probability density function (PDF)

Initial PDF represents initial uncertainty Single forecast doesn’t account for initial and model error…often fails to

predict the real future state past a certain point Ensemble of perturbed forecasts accounts for initial and model error… PDF

of solutions more likely to contain real future state Ensemble PDF contains additional information, including forecast

uncertainties

High Res Control

RealityPDF PDF

PDF

Time

Single Forecast

Reality

Ensemble Prediction System (EPS) Goals

Represent initial condition and/or model uncertainty

Determine a range of possible forecast outcomes Estimate the probability for any individual

forecast outcomeGeneral: provide a framework for decision

assistance

23

General EPS forecasting toolsSpaghetti Plots (shows all solutions)Mean/Spread (“middleness” and variability)ProbabilitiesMost Likely Event

“Spaghetti” Plots

25

Mean and SpreadCharacteristics of mean

The ensemble mean performs better on average than operational model on which it is based. Why?

Because predictable features remain intact, less predictable features are smoothed out

Characteristics of spreadAllows assessment of

uncertainty, since more spread means more uncertainty

12

3

4

• Helps determine the probability of a specified event.• Gives probability of exceeding meaningful threshold

• Calculation represents count of what % of ensemble members exceed the threshold of interest

• Example here is for 12-hour precipitation exceeding 0.25 inches.

Probability of Exceedance

• Used to show what is most often predicted by the ensemble forecast

• A common example– Precipitation type

(snow, sleet, freezing rain, rain)

Most Likely or Dominant Event Diagram

SummaryChaos and model uncertainties impose a very real

physical limit on predictabilityPredictability falls off (sometimes rapidly) as a

function of scale and timeForecast accuracy varies both spatially and

temporally as a result of initial state and model errors, which change as the atmospheric flow evolves

Ensemble NWP optimizes predictability for all scales, and extends the utility of forecasts…especially at extended ranges (days 4-7)

Allows for quantification of uncertainty, and foundation for decision assistance

Statistical Guidance in the Forecast Process

WHY STATISTICAL GUIDANCE?

●Add value to direct NWP model output

Objectively interpret model - remove systematic biases - quantify uncertainty Predict what the model does not Produce site-specific forecasts (i.e. a “downscaling” technique)

●Assist forecasters “First Guess” for expected local conditions “Built-in” model/climatology

MODEL OUTPUT STATISTICS (MOS)

1. Numerical Weather Prediction (NWP) Model Forecasts 2. Prior Surface Weather Observations3. Geoclimatic Information

Current Statistical Method: MULTIPLE LINEAR REGRESSION (Forward Selection)

Relates observed weather elements (PREDICTANDS) to appropriate variables (PREDICTORS) via astatistical approach.

Predictors are obtained from:

MODEL OUTPUT STATISTICS (MOS)

Properties●Mathematically simple, yet powerful

●Need historical record of observations at forecast points (Hopefully a long, stable one!)

●Equations are applied to future run of similar forecast model

●Probability forecasts possible from a single run of NWP model

●Other statistical methods can be used e.g. Polynomial or logistic regression; Neural networks

MODEL OUTPUT STATISTICS (MOS)

●ADVANTAGES - Recognition of model predictability - Removal of some systematic model bias - Optimal predictor selection - Reliable probabilities - Specific element and site forecasts ●DISADVANTAGES - Short samples - Changing NWP models - Availability & quality of observations

Now approx. 1820 sites

Gridded MOS

●“MOS at any point (GMOS) - Support NWS digital forecast database 2.5 km - 5 km resolution - Equations valid away from observing sites - Emphasis on high-density surface networks - Use high-resolution geophysical data - Some problems over steep terrain or data-sparse regions

Gridded MOS

Use of MOS at a Forecast Office

• Can ingest GMOS directly into local digital forecast database• Can apply bias correction (based on performed in past 30 days)• Can ingest point-based MOS and spread it to entire grid• MOS from single models or from ensemble mean/max/min• We verify our forecast against MOS, so we may use as a starting point but we try to improve on it based on local experience or recent trends

Questions?