1

How Does NCEP/CPC Make Operational Monthly and Seasonal

Forecasts?

Huug van den Dool (CPC)

ESSIC, February, 23, 2011

2

Assorted Underlying Issues

• Jin Huang - R2O - CTB• Which tools are used…• How are tools combined???• CFSv1 CFSv2• Dynamical vs Empirical Tools• Skill of tools and OFFICIAL• How easily can a new tool be included?• US, yes, but occasional global perspective

3

Menu of CPC predictions:

• 6-10 day (daily)• Week 2 (daily)• Monthly (monthly + update)• Seasonal (monthly)• Other (hazards, drought monitor, drought outlook,

MJO, UV-index, degree days, POE, SST) (some are ‘briefings’)

• Informal forecast tools (too many to list)

• http://www.cpc.ncep.noaa.gov/products/predictions/90day/tools/briefing/index.pri.html

4

5

EXAMPLEPUBLICLY

ISSUED

6From an internal CPC Briefing package

7

EMP EMP EMP EMP

EMPDYN

DYN

CONCON

N/A

88

SMLR CCA OCN

LAN

LFQ

(15 CASES: 1950, 54, 55, 56, 64, 68, 71, 74, 75, 76, 85, 89, 99, 00, 08)

OLD-OTLK

CFSV1

ECPIRI

ECA CON

9

Element US-T US-P SST US-soil moisture

Method:CCA X X XOCN X X CFS X X X XSMLR X XECCA X XConsolidation X X X

Constr Analog X X X XMarkov XENSO Composite X XOther (GCM) models (IRI, ECHAM, NCAR, CDC etc):

X X

CCA = Canonical Correlation AnalysisOCN = Optimal Climate NormalsCFS = Climate Forecast System (Coupled Ocean-Atmosphere Model)SMLR = Stepwise Multiple Linear RegressionCON = Consolidation

10

Long Lead Predictions of US SurfaceTemperature using Canonical CorrelationAnalysis. Barnston(J.Climate, 1994, 1513)

Predictor - Predictand Configuration

Predictors Predictand

* Near-global SSTA

* N.H. 700mb Z * US sfc T

* US sfc T

four predictor “stacked” fields one predictandperiod

4X652=2608 predictors 102 locations

Data Period 1955 - last month

11

12

13

About OCN. Two contrasting views:- Climate = average weather in the past- Climate is the ‘expectation’ of the future

30 year WMO normals: 1961-1990; 1971-2000 etc

OCN = Optimal Climate Normals: Last K year average. All seasons/locations pooled: K=10 is optimal (for US T).

Forecast for Jan 2012 = (Jan02+Jan03+... Jan11)/10. – WMO-normal

plus a skill evaluation for some 50+ years.

Why does OCN work?1) climate is not constant (K would be infinity for constant climate)2) recent averages are better3) somewhat shorter averages are better (for T)see Huang et al 1996. J.Climate. 9, 809-817.

14

OCN has become the bearer of most of the skill, see also EOCN

method (Peng et al)

15

16

GHCN-CAMS

FAN

2008

17

Major Verification Issues

• ‘a-priori’ verification (used to be rare)

• After the fact (fairly normal)

18

Source Peitao Peng

After the fact…..

19

(Seasonal) Forecasts are useless unless accompanied by a reliable a-

priori skill estimate.

Solution: develop a 50+ year track record for each tool. 1950-present.

(Admittedly we need 5000 years)

20

Consolidation

21

--------- OUT TO 1.5 YEARS -------

22

OFFicial Forecast(element, lead, location, initial month) =

a * A + b * B + c * C +…

Honest hindcast required 1950-present. Covariance (A,B), (A,C), (B,C), and

(A, obs), (B, obs), (C, obs) allows solution for a,

b, c (element, lead, location, initial month)

23

CFS skill 1982-2003

24

Fig.7.6: The skill (ACX100) of forecasting NINO34 SST by the CA method for the period 1956-2005. The plot has the target season in the horizontal and the lead in the vertical. Example: NINO34 in rolling seasons

2 and 3 (JFM and FMA) are predicted slightly better than 0.7 at lead 8 months. An 8 month lead JFM forecast is made at the end of April of the previous year. A 1-2-1 smoothing was applied in the vertical to

reduce noise.CA skill 1956-2005

25

M. Peña Mendez and H. van den Dool, 2008: Consolidation of Multi-Method Forecasts at CPC. J. Climate, 21, 6521–6538.

Unger, D., H. van den Dool, E. O’Lenic and D. Collins, 2009: Ensemble Regression.Monthly Weather Review, 137, 2365-2379.

(1) CTB, (2) why do we need ‘consolidation’?

26

27

(Delsole 2007)

28

UR MMA COR

RI RIM RIW

Climo

29

3CVRE

SEC

SEC and CV

30

31

Bayesian Multimodel Strategies

Linear regression leads to unstable weights for small sample sizes.

Methods for producing more stable estimates have been proposed by van den Dool and Rukhovets (1994), Kharin and Zwiers (2002), Yun et al. (2003), and Robertson et al. (2004).

These methods are special cases of a Bayesian method, each distinguished by a different set of prior assumptions (DelSole 2007).

Some reasonable prior assumptions: R:0 Weights centered about 0 and bounded in magnitude

(ridge regression)R:MM Weights centered about 1/K (K = # models) and bounded in magnitudeR:MM+R Weights centered about an optimal value and bounded in magnitudeR:S2N Models with small S2N (signal-to-noise) ratio tend to have small weightsLS Weights are unconstrained (ordinary least squares)

From Jim Kinter (Feb 2009)

32

If the multimodel strategy is carefully cross validated, then the simple mean beats all other investigated multimodel strategies.

Since Bayesian methods involve additional empirical parameters, proper assessment requires a two-deep cross validation procedure. This can change the conclusion about the efficacy of various Bayesian priors.

Traditional cross validation procedures are biased and incorrectly indicate that Bayesian schemes beat a simple mean.

From Jim Kinter (Feb 2009)

33

34

35No SEC, (no CV required), ‘raw’)

36

Good to know:

• If a model/methods has no systematic error (by design), leave it alone, please do not apply systematic error correction where none is needed.

• There is more (need for) statistics in dynamical models and their post-processing than in any statistical method

37

38

39

40

See also:

O’Lenic, E.A., D.A. Unger, M.S. Halpert, and K.S. Pelman, 2008: Developments in Operational Long-Range Prediction at CPC. Wea. Forecasting, 23, 496–515.

41

Empirical tools can be comprehensive! (Thanks to

reanalysis, among other things).

And very economic.

42

SST Z500

PrecipT2m

CA

43

SST Z500

PrecipT2m

CFS

Source: Wanqiu Wang

44

Is CPC in good shape to admit more tools???

Even if these tools come from the outside???

Does the outside have the right expectations???

45

extra

46

Dynamics> Empiricism Symbiosis

• Successive generations of Reanalyses are produced because NWP exists, the desire for initial conditions etc, but empirical prediction methods are one of the main beneficiaries of having all this data.

• Early empirical methods have served as an example (for anybody to follow) to produce (honest) hindcasts, a-priori skill assessment, cross validation etc

• Consolidation (of tools) is “color-blind” relative to questions of dynamical or empirical origin

47

Progress in Empirical Methods

• More data to work with (1 year per year)

• More (and global) data to work with (Re-analyses). Oceans, land, atmosphere(note the symbiotic relationship with modeling here)

• New empirical methods

• New applications Inch-by-inch methods

Revolutionary changes

Always be ready for surprises

48

Why are empirical Methods competitive with dynamical

methods (in seasonal prediction)??

• Linearity (define)• A system with skill in <= 3EDOFs is

functionally linear• See Chapter 10 of book

49

Mix of Dynamical & Empirical

• Day 1 – 14: NWP ‘reigns’• Wk 2 – 6 Dynamics & Empirical• SI: Dynamics & Empirical• Decadal ? • Climate Change ?