training course presentation, 19 th april 2010 – thomas jung: systematic model errors. systematic...
TRANSCRIPT
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Systematic Errors
Thomas Jung
European Centre for Medium-Range Weather Forecasts
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Scope of the Lecture
Question: “Do we still have significant systematic errors in the ECMWF forecasting system?”
If so, what are the main problems? How do systematic errors grow? How did systematic errors evolve over the years? How well do we simulate specific phenomena (e.g., blocking, extratropical cyclones)? Does increasing resolution help? Which techniques can be used to understand systematic error?
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Introduction
Two principal sources of forecast error:
Uncertainties in the initial conditions (“observational error”)
Model error
Two kinds of forecast error
Random error (model+initial error)
Systematic error (model error*)
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Concept of Systematic Error
)(ˆ)(ˆ),(ˆ ttttse odfdofd
Relatively straightforward to compute
But there are pitfalls:• finite length (significance tests may help)• Systematic error may be misleading for short time series (loss of predictability) • observations might be biased
Systematic errors can be difficult to interpret.
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Loss of Predictability and “Systematic Error”
-6 -4 -2 0 2 4 6Temperature
0.00
0.01
0.02
0.03
0.04
0.05
0.06P
DF
Observed Anomaly“Systematic Error” when predictability
is lost
Fingerprint of loss of predictability: “Systematic error” resembles the opposite of the observed anomaly
“Observed”
“Forecast”
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Example: Z500 DJF 2005/06
5120
5280
5280
5280
5440
5440
5440
5440
5600
56
00
5600
5600
5600
5760
5760
5760
5760
5760
5760
Mean Z500 Anomaly DJF 2006
-200
-140
-100
-80
-60
-40
-20
20
40
60
80
100
140
200
Observed Anomaly
5120
5280
5280
5280
5440
5440
5440
5440
5600
5600
5600
5600
5600
5760
5760
5760
5760
5760
5760
Mean Z500 Anomaly DJF 2006
-200
-140
-100
-80
-60
-40
-20
20
40
60
80
100
140
200
Mean Error @ D+10
Spatial correlation=-0.78
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Do we know the true state?
Precipitation GPCP (12-2 1990-2005)
-0.5
1
3
5
7
9
11
13
Precipitation ex51-GPCP (12-2 1990-2005)
-10
-4
-2
-0.5
0.5
2
4
10
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Data
• Medium-range forecasts Primarily hindcasts from ERA-40 and ERA-Interim (robust
statistics).
• Seasonal integrations: 13 month long integrations started on 1st November of each of the
years 1962-2005 (or a subset of this period). Diagnosis of standard seasons DJF, MAM, JJA, SON (errors had at
least 1 month to grow, asymptotic errors). Most experiments at TL159 with 91 levels in the vertical.
Observed lower boundary conditions (uncoupled integrations).
• Observational data: ERA-40 Other “observational” data sets.
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Systematic Error Growth
How do systematic errors grow throughout the forecast?
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Systematic Z500 Error Growth from D+1 to D+10
D+1 D+3
D+5 D+10
520
540
560
560
580
580
580
(a) Z500 Difference D+1 FC-AN (DJF 1990-2010 ERA Interim)
-1-0.8-0.6-0.4-0.3-0.2-0.1
0.10.20.30.40.60.81
520
540
560
560
580
580
580
(b) Z500 Difference D+3 FC-AN (DJF 1990-2010 ERA Interim)
-3-2.4-1.8-1.2-0.9-0.6-0.3
0.30.60.91.21.82.43
520
540
560
560
580
580
580
(c) Z500 Difference D+5 FC-AN (DJF 1990-2010 ERA Interim)
-5-4-3-2-1.5-1-0.5
0.511.52345
520
540
560
560580
580
580
(d) Z500 Difference D+10 FC-AN (DJF 1990-2010 ERA Interim)
-10-8-6-4-3-2-1
12346810
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
520
540
560
560
580
580
580
(a) Z500 Difference D+1 FC-AN (DJF 1990-2010 ERA Interim)
-1-0.8-0.6-0.4-0.3-0.2-0.1
0.10.20.30.40.60.81
520
540
560
560
580
580
580
(b) Z500 Difference D+3 FC-AN (DJF 1990-2010 ERA Interim)
-3-2.4-1.8-1.2-0.9-0.6-0.3
0.30.60.91.21.82.43
520
540
560
560
580
580
580
(c) Z500 Difference D+5 FC-AN (DJF 1990-2010 ERA Interim)
-5-4-3-2-1.5-1-0.5
0.511.52345
520
540
560
560
580
580
580
(d) Z500 Difference D+10 FC-AN (DJF 1990-2010 ERA Interim)
-10-8-6-4-3-2-1
12346810
Systematic Z500 Errors: Medium-Range and Beyond
-6-2 -2
-2
-2
-2
-2
-2-2
2
2
2
2
6
Z500 Difference 32R1-er40 (12-2 1963-2006)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
D+10 ERA-Interim Asymptotic: 31R2
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Evolution of Systematic Error
How did systematic errors evolve throughout the years?
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Evolution of D+3 Systematic Z500 Errors
1981-1985 1993-1997 2006-2010
5200
5200
5400
5400
5400
5400
5600
5600
5600
5600
560
0
5600
5800
5800
5800
5800
580
0
5800
Mean Z500 FC Error: D+3 (DJF 1981-1985)
-50
-35
-25
-20
-15
-10
-55
10
15
20
25
35
50
5200
5200
5400
5400
5400
54
00
5400
5600
5600
5600
5600
560
0
5600
5800
5800
5800
5800
5800
5800
Mean Z500 FC Error: D+3 (DJF 1993-1997)
-50
-35
-25
-20
-15
-10
-55
10
15
20
25
35
50
5200
5200
5400
5400
5400
54
00
5400
5600
5600
5600
5600
56
00
5600
5800
5800
5800
5800
5800
5800
Mean Z500 FC Error: D+3 (DJF 2006-2010)
-50
-35
-25
-20
-15
-10
-55
10
15
20
25
35
50
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Evolution of Systematic Z500 Errors: Model Climate
-2
-2
-2
-2
-2
2
2
2
2
Z500 Difference 35R1-er40 (12-2 1963-2006)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
-2
-2
2
2
2
Z500 Difference 33R1-er40 (12-2 1963-2006)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
-2
-2
2
2
2
2
2
2
Z500 Difference 32R3-er40 (12-2 1963-2006)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
-6
-6
-6
-6
-2
-2
-2
-2
-2
-2
-2
-2
-2
2
2
2
2
2
2
6
6
Z500 Difference 32R2-er40 (12-2 1963-2006)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
-6
-2 -2
-2
-2
-2
-2
-2-2
2
2
2
2
6
Z500 Difference 32R1-er40 (12-2 1963-2006)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
-10
-6
-6
-6
-6
-2
-2
-2
-2
-2
-2
-2
-2
2
2
22
2
2
Z500 Difference 31R1-er40 (12-2 1963-2006)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
-6
-6
-6
-6
-6
-2
-2
-2
-2
-2
-2
-2
-2
2
2
2
2
6
Z500 Difference 30R1-er40 (12-2 1963-2006)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
-6
-6
-6
-6
-6
-2
-2
-2-2
-2
-2
-2
-2
-2
2
2
2
2
2
6
Z500 Difference 29R2-er40 (12-2 1963-2006)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
35R1 33R1 32R3 32R2
32R1 31R1 30R1 29R2
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Systematic Z500 Errors: Impact of Recent Changes
-2
-2
-2
-2
-2
2
22
2
2
2
Z500 Difference f127-er40 (12-2 1990-2005)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
-6
-2
-2-2
-2
-2
-2
2
2
2
2
2
2
6
Z500 Difference f3wt-er40 (12-2 1990-2005)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14-2
-2
-2
2
2
2
2
2
2
2
2
2
Z500 Difference f251-er40 (12-2 1990-2005)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
-2
-2
-2
2
2
2
2
2
2
2
2
Z500 Difference f3y0-er40 (12-2 1990-2005)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
-2
-2
-2
-2
-2
-2
-2
2
2
Z500 Difference f3y1-er40 (12-2 1990-2005)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
-2
-2
-2
-2
-2
-2-2
2
2
2
Z500 Difference f3y2-er40 (12-2 1990-2005)
-14
-12
-10
-8
-6
-4
-22
4
6
8
10
12
14
CNT-Reanalysis (33R1) Old Convection-CNT Old TOFD-CNT
Old Vertical Diff-CNT Old Radiation-CNT Old Soil Hydrology-CNT
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Phenomena
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
510
510
510
510
520
520
520
520
520
530
530
530
530
540
540
540
540
550
550
550
550
560
560
560
560
560
570
570
570
570
570
570
570
580
580
ECMWF Analysis VT:Saturday 23 December 2006 12UTC 500hPa Geopotential
Blocking Methodology
L
H
L
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
-180 -120 -60 0 60 120 180Latitude
0
10
20
30
40
Blo
ckin
g F
req
uenc
y (%
)
Blocking Frequency Errors (23r4)
DJF 1990-2001
ERA-40D+1D+4D+7D+10
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Blocking Frequency Errors (31R2)
DJF 1990-2001
ERA-40D+1D+4D+7D+10
-180 -120 -60 0 60 120 180Latitude
0
10
20
30
40
Blo
ckin
g F
req
ue
ncy
(%
)
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Blocking Frequencies: DJF 1990-2005
-180 -120 -60 0 60 120 180Longitude
0
10
20
30
40B
lock
ing
Fre
qu
en
cy (
%)
ERA4033R132R329R2-32R2
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Blocking Frequencies: DJFM
-180 -120 -60 0 60 120 180Longitude
-180 -120 -60 0 60 120 180Longitude
0
10
20
30
40
Bloc
king
Freq
uenc
y(%
)Reanalysis
a
0
10
20
30
40
Bloc
king
Freq
uenc
y(%
)
c
T159T511T1279
-180 -120 -60 0 60 120 180Longitude
0
10
20
30
40
Bloc
king
Freq
uenc
y(%
)
d
ReanalysisT159T511T15915minT5110159
-180 -120 -60 0 60 120 180Longitude
0
10
20
30
40
Bloc
king
Freq
uenc
y(%
)
b
ReanalysisT159T511T1279T2047
ReanalysisT159T1279
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Extratropical Synoptic Systems
Two approaches:
• Compute standard deviation of highpass or bandpass filtered time series (“Eulerian” approach).
• Track individual systems (“Lagrangian” approach).
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
“Observed” Tracks of Long-lived Cyclones
ETC ECMWF Analysis (DJFM 05/06)
940 950 960 970 980 990 1000 1010 1020
n= 515
>2 days
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Sensitivity to Horizontal Resolution: 29R1
Wintertime Cyclones over the Northern Hemisphere
1980 1985 1990 1995 2000 2005Time
100
150
200
250
300
350
400N
um
be
r o
f Cyc
lon
es
ERA-40EC-T95EC-T159EC-T255
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Sensitivity to Horizontal Resolution: 31R1
Wintertime Cyclones over the Northern Hemisphere
1990 1995 2000 2005Time
200
300
400
500
600
700
800
Nu
mb
er
of C
yclo
ne
s
ERA-40T95T159T255T511
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Number of Extratropical Cyclones (DJFM)
Lifetime > 1day
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Synoptic Activity Error: Evolution
2222
22 2 2
44
4
4
4444
44 4 4
6
6 66
6666
66
6
6
6
8
8 88
88
8
8
8
8
8
8
8
8
10
10
10 1010
10
10
10
10
10
1212
12
12
Synoptic Z500 Activity: Difference 33R1-ERA40 (12-2 1962-2005)
-4-3-2.5-2-1.5-1-0.50.511.522.534
2222
22 2 2
44
4
4
4444
44 4 4
6
6 66
6666
66
6
6
6
8
8 88
88
8
8
8
8
8
8
8
8
10
10
10 1010
10
10
10
10
10
1212
12
12
Synoptic Z500 Activity: Difference 32R3-ERA40 (12-2 1962-2005)
-4-3-2.5-2-1.5-1-0.50.511.522.534
2222
22 2 2
44
4
4
4444
44 4 4
6
6 66
6666
66
6
6
6
8
8 88
88
8
8
8
8
8
8
8
8
10
10
10 1010
10
10
10
10
10
1212
12
12
Synoptic Z500 Activity: Difference 32R2-ERA40 (12-2 1962-2005)
-4-3-2.5-2-1.5-1-0.50.511.522.534
2222
22 2 2
44
4
4
4444
44 4 4
6
6 66
6666
66
6
6
6
8
8 88
88
8
8
8
8
8
8
8
8
10
10
10 1010
10
10
10
10
10
1212
12
12
Synoptic Z500 Activity: Difference 31R1-ERA40 (12-2 1962-2005)
-4-3-2.5-2-1.5-1-0.50.511.522.534
2222
22 2 2
44
4
4
4444
44 4 4
6
6 66
6666
66
6
6
6
8
8 88
88
8
8
8
8
8
8
8
8
10
10
10 1010
10
10
10
10
10
1212
12
12
Synoptic Z500 Activity: Difference 30R1-ERA40 (12-2 1962-2005)
-4-3-2.5-2-1.5-1-0.50.511.522.534
2222
22 2 2
44
4
4
4444
44 4 4
6
66
6
6666
66
6
6
6
8
8 88
88
8
8
8
8
8
8
8
8
10
10
10 1010
10
10
10
10
10
1212
12
12
Synoptic Z500 Activity: Difference 35R1-er40 (12-2 1963-2006)
-4-3-2.5-2-1.5-1-0.50.511.522.534
35R1
32R3
31R1
32R2
30R1
33R1
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Resolution and Computational Effort
• 33 times for TL511
• 520 times for TL1279
• 2136 times for TL2047
Increased cost compared to TL159:
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
The Madden-and Julian Oscillation
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Schematic of the MJO
From Madden and Julian (1994)
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Near Global Impact of the MJO (Precipitation)
Conv.: Indian Ocean Conv.: Maritime Continent
Conv.: Central Pacific Conv.: WH/Africa
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
The Madden and Julian Oscillation
Reanalysis T159 T511
T1279 T2047
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Conclusions (1)
Main systematic errors: Concept of systematic error is very straightforward.
But there are pitfalls:
• Short time series (sampling issues + loss predictability).
• True state of the atmosphere not always well known.
Systematic error are (usually) a clear sign of model error.
Understanding systematic error is challenging.
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Conclusions (2)
Main systematic errors:Do we still have systematic errors in the ECWMF model?
• Yes, we do.
• However, most systematic errors have been substantially reduced in recent years.
• Some key-challenges remain!
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Conclusions (3)
Main systematic errors:
Tropical circulation + hydrological cycle
Madden-Julian Oscillation
Indian Summer Monsoon
Quasi-Biennial Oscillation
Euro-Atlantic blocking
Severe weather (e.g. wind storms)
Others (surface related fields)
Key-challenges:
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Further Reading
Bechtold, P., M Koehler, T. Jung, M. Leutbecher, M. Rodwell and F. Vitart, 2008: Advances in simulating atmospheric variability with the ECMWF model: From synoptic to decadal time scales. Quart. J. Roy. Meteor. Soc., 134, 1337-1351.
Jung, T. and A.M. Tompkins, 2003: Systematic Errors in the ECMWF Forecasting System. ECMWF Technicial Memorandum 422. http://www.ecmwf.int/publications/library/do/references/list/14
Jung, T., A.M. Tompkins, and R.J. Rodwell, 2004: Systematic errors in the ECMWF forecasting system. ECMWF Newsletter, 100, 14-24.
Jung, A.M. Tompkins, and R.J. Rodwell, 2005: Some Aspects of Systematic Errors in the ECMWF Model. Atmos. Sci. Lett., 6, 133-139.
Jung, T, et al., 2010: The ECMWF model climate: Recent progress through improved physical parametrizations. Quart. J. Roy. Meteor. Soc, 136, 1145-1160.
Jung, T., 2005: Systematic errors of the atmospheric circulation in the ECMWF model. Quart. J. Roy. Meteor. Soc., 131, 1045-1073.
Jung, T., S.K. Gulev, I. Rudeva and V. Soloviov, 2006: Sensitivity of extratropical cyclone characteristics to horizontal resolution in the ECWMF model. Quart J. Roy. Meteor. Soc., 132, 1839-1857.
Hoskins, B.J. and K.I. Hodges, 2002: New perspectives on the Northern winter storm tracks. J. Atmos. Sci., 59, 1041-1061.
Palmer, T.N., G. Shutts, R. Hagedorn, F. Doblas-Reyes, T. Jung, and M. Leutbecher, 2005: Representing model uncertainty in weather and climate prediction. Ann. Rev. Earth. Planet Sci., 33, 163-193.
Rodwell, MJ and T Jung, 2008: Understanding the local and global impacts of model physics changes: An aerosol example. Quart. J. Roy. Meteor. Soc., 134, 1479-1497.
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Recent Model Changes
29R2 28 Jun 2005 Modifications to convection
30R1 1 Feb 2006 Increased resolution (L60 to L91)
31R1 12 Sep 2006 Revised cloud scheme (ice supersaturation + numerics); implicit computation of convective transports; introduction of orographic form drag scheme; revised GWD scheme
32R1 not operational
New short-wave radiation scheme; introduction of McICA cloud-radiation interaction; MODIS aerosol; revised GWD scheme; retuned ice particle size
32R2 5 Jun 2007 Minor changes to forecast model
32R3 6 Nov 2007 New formulation of convective entrainment and relaxation time scale; reduced vertical diffusion in the free atmosphere; modification to GWD scheme (top of the model); new soil hydrology scheme
33R1 3 Jun 2008 Slightly increased vertical diffusion; increased orographic form drag; retuned entrainment in convection scheme; bugfix scaling of freezing term in convection scheme; changes to the surface model
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Sensitivity to Model Formulation
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Systematic Errors: AMIP Models
Training Course Presentation, 19th April 2010 – Thomas Jung: Systematic Model Errors.
Demeter Models (DJF Precipitation)
CERFACS LODYC
MPIMetOfficeMeteo-France
ECMWF