ETA and GFS ValidationETA and GFS Validation(Southeastern Mexico)(Southeastern Mexico)

John M. DickensJohn M. Dickens

RSIS/CPCRSIS/CPC

OutlineOutline

Models Models DataDataMethodsMethodsResultsResultsConclusionsConclusions

PurposePurpose

To determine whether it is appropriate to To determine whether it is appropriate to replace the 20 km ETA model with the 50 replace the 20 km ETA model with the 50 km GFS model for forecasting precipitation km GFS model for forecasting precipitation amounts in Central America and the amounts in Central America and the Caribbean. Caribbean.

ModelsModels

ETA 20ETA 20 20 km20 km CPC DomainCPC Domain

GFS 50GFS 50 GlobalGlobal 50 km50 km

Show model domain

DataData Central America GTS Stations (75):

The data from these stations are quality controlled downstairs and in the CPC, making them more reliable than their NOAA/USGS counterparts. Due to the sparse data these stations will not be used.

NOAA/USGS Stations (101):These stations were set up as part of the

Hurricane Mitch Program. No money was

set aside for maintenance of these stations

so their reliability is in question. Also, the

exact units of the measurements have never

been determined. Therefore, these stations

will not be used. Mexico Co-Op Stations (~500 out

of 1000):The Mexico stations will be used because

they are fairly dense in the southern half of

the country and therefore will be a decent

representation of the Central American

region.

DataData

Only used stations reporting precipitation Only used stations reporting precipitation amounts greater than zeroamounts greater than zero

Time period: May 1 to August 31, 2004Time period: May 1 to August 31, 2004

August 2 Mexico Data: The Mexico data runs from 12Z to 12Z.

August 1 Forecast

August 2 Forecast

* For the August 1st forecasts of 24 hrs and 48 hrs the Mexico data for August 1st and 2nd are required.

August 3 Forecast

August 1Forecast

Times

F00

F06

F12

F18

F24

F30

F36

F42

F48

F54

F60

F66

F72

F78

The forecasts represent the precipitation for the past 6 hours. • F06 represents rainfall from F00 to F06• F12 represents rainfall from F06 to F12 …….

August 1 Mexico Data: The Mexico data runs from 12Z to 12Z.

DataData

MethodsMethods

Match Co-op stations to model grid points Match Co-op stations to model grid points in a point-to-grid comparisonin a point-to-grid comparison

= model grid points

= Co-op stations

MethodsMethods

RegularRegular 24 hr / 48 hr24 hr / 48 hr forecastsforecasts

No CoastNo Coast 24 hr / 48 hr24 hr / 48 hr

forecastsforecasts

No MountainsNo Mountains 24 hr / 48 hr24 hr / 48 hr

forecastsforecasts

Data PointsData Points

15148

10543

7183

0

2000

4000

6000

8000

10000

12000

14000

16000

Po

ints

Regular No Coast No Mountains

Category

Number of Points for Each Experiment

MethodsMethods

Regular vs No Coast vs No MountainsRegular vs No Coast vs No Mountains 24 hr and 48 forecasts24 hr and 48 forecasts

24 hr vs 48 hr forecasts24 hr vs 48 hr forecasts ETA and GFSETA and GFS

OverallOverall

24 hr Mean Difference24 hr Mean Difference

24 hr Total Mean DifferenceStation - Model

-2

-1

0

1

2

3

4

5

6

Regular No Coast No Mountains

Category

Diffe

ren

ce (m

m)

ETA

GFS

Regular: Mean Difference (24 hr)Station - Model

-20

-100

10

20

3040

50

6070

80

Overall 0-10 mm 10-20 mm 20-50 mm 50+ mm

Station Rainfall Category

Diffe

ren

ce

(m

m)

No Coast: Mean Difference (24 hr)Station - Model

-20-10

010

2030

4050

6070

80

Overall 0-10 mm 10-20 mm 20-50 mm 50+ mm

Station Rainfall Category

Diffe

ren

ce

(m

m)

No Mountains: Mean Difference (24 hr)Station - Model

-20

-10

010

20

30

40

5060

70

80

Overall 0-10 mm 10-20 mm 20-50 mm 50+ mm

Station Rainfall Category

Diffe

ren

ce

(m

m)

24 hr Standard Deviation24 hr Standard Deviationof the Mean Differenceof the Mean Difference

24 hr Total Standard Deviation of Mean Difference

0

5

10

15

20

25

Regular No Coast No Mountains

Category

Devia

tio

n (m

m)

ETA

GFS

No Coast: Standard Deviation of Mean Difference (24 hr)

0

5

10

15

20

25

30

35

40

Overall 0-10 mm 10-20 mm 20-50 mm 50+ mm

Station Rainfall Category

Dev

iatio

n (m

m)

Regular: Standard Deviation of Mean Difference (24 hr)

0

5

10

15

20

25

30

35

40

Overall 0-10 mm 10-20 mm 20-50 mm 50+ mm

Station Rainfall Category

Dev

iatio

n (m

m)

No Mountains: Standard Deviation of Mean Difference (24 hr)

0

5

10

15

20

25

30

35

40

Overall 0-10 mm 10-20 mm 20-50 mm 50+ mm

Station Rainfall Category

Dev

iatio

n (m

m)

48 hr Mean Difference48 hr Mean Difference

48 hr Total Mean Difference

-2

-1

0

1

2

3

4

5

6

Regular No Coast No Mountains

Category

Diffe

ren

ce (m

m)

ETA

GFS

Regular: Mean Difference (48 hr)Station - Model

-20

-10

010

20

30

40

5060

70

80

Overall 0-10 mm 10-20 mm 20-50 mm 50+ mm

Station Rainfall Category

Diffe

ren

ce

(m

m)

No Coast: Mean Difference (48 hr)Station - Model

-20

-100

10

20

3040

50

6070

80

Overall 0-10 mm 10-20 mm 20-50 mm 50+ mm

Station Rainfall Category

Diffe

ren

ce

(m

m)

No Mountains: Mean Difference (48 hr)Station - Model

-20

-10

0

10

20

30

40

50

60

70

80

Overall 0-10 mm 10-20 mm 20-50 mm 50+ mm

Station Rainfall Category

Diffe

ren

ce (m

m)

48 hr Standard Deviation 48 hr Standard Deviation of the Mean Differenceof the Mean Difference

48 hr Total Standard Deviation of Mean Difference

0

5

10

15

20

25

Regular No Coast No Mountains

Category

Devia

tio

n (m

m)

ETA

GFS

Regular: Standard Deviation of Mean Difference (48 hr)

0

5

10

15

20

25

30

35

40

Overall 0-10 mm 10-20 mm 20-50 mm 50+ mm

Station Rainfall Category

Deiv

atio

n (m

m)

No Mountains: Standard Deviation of Mean Difference (48 hr)

0

5

10

15

20

25

30

35

40

Overall 0-10 mm 10-20 mm 20-50 mm 50+ mm

Station Rainfall Category

Dev

iatio

n (m

m)

No Coast: Standard Deviation of Mean Difference (48 hr)

0

5

10

15

20

25

30

35

40

Overall 0-10 mm 10-20 mm 20-50 mm 50+ mm

Station Rainfall Category

Dev

iatio

n (m

m)

Categorical SummaryCategorical Summary

OverallOverall RegularRegular No CoastNo Coast No MountainsNo Mountains

Mean DifferenceMean Difference

(lowest)(lowest)

24 hr24 hrGFSGFS

(3.27 mm)(3.27 mm)

GFSGFS

(4.37 mm)(4.37 mm)

ETAETA

(1.67 mm)(1.67 mm)

48 hr48 hrGFSGFS

(2.19 mm)(2.19 mm)

GFSGFS

(3.19 mm)(3.19 mm)

ETAETA

(3.04 mm)(3.04 mm)

Standard Standard Deviation of Deviation of

Mean DifferenceMean Difference

(lowest)(lowest)

24 hr24 hrETAETA

(4.43 mm)(4.43 mm)

ETAETA

(5.13 mm)(5.13 mm)

TIETIE

(.90 mm)(.90 mm)

48 hr48 hrETAETA

(3.39 mm)(3.39 mm)

ETAETA

(4.28 mm)(4.28 mm)

TIETIE

(.10 mm)(.10 mm)

24 hr vs 48 hr Forecasts24 hr vs 48 hr ForecastsTotal Mean Difference

-2

-1

0

1

2

3

4

5

6

Regular No Coast No Mountains

Category

Dif

fere

nc

e (m

m)

Total Standard Deviation of Mean Difference

0

5

10

15

20

25

Regular No Coast No Mountains

Category

Devia

tio

n (m

m)

Total Correlation

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Regular No Coast No Mountains

Category

Co

rrela

tio

n

24 hr ETA24 hr ETA 48 hr ETA48 hr ETA

24 hr GFS24 hr GFS 48 hr GFS48 hr GFS

ConclusionsConclusions

Mean differences increase with Mean differences increase with increasingly large eventsincreasingly large events

Standard deviations of mean difference Standard deviations of mean difference increase with increasingly large eventsincrease with increasingly large events

ConclusionsConclusions

RegularRegular Overall Overall GFSGFS smaller mean difference smaller mean difference Overall Overall ETAETA smaller standard deviation of the mean difference smaller standard deviation of the mean difference

No CoastNo Coast Overall Overall GFSGFS smaller mean difference smaller mean difference Overall Overall ETAETA smaller standard deviation of mean difference smaller standard deviation of mean difference

No MountainsNo Mountains Overall Overall ETAETA smaller mean difference smaller mean difference Overall Overall ETAETA and and GFSGFS have identical standard deviations of have identical standard deviations of

mean differencemean difference

ConclusionsConclusionsETA 48 hr forecasts more accurate than ETA 48 hr forecasts more accurate than

the ETA 24 hr forecaststhe ETA 24 hr forecastsKnown initialization problems….additional 24 hrs Known initialization problems….additional 24 hrs

for spinupfor spinup

ETA forecasts precipitation in the region ETA forecasts precipitation in the region better than the GFSbetter than the GFS

Mean differences: both < Mean differences: both < ±±6 mm6 mmStandard Deviations: both > 15 mm, ETA ~ 5 mm Standard Deviations: both > 15 mm, ETA ~ 5 mm

lowerlowerCorrelations: ETA .25-.30, GFS near zeroCorrelations: ETA .25-.30, GFS near zero

Thank YouThank You