hydrological-driven validation of mpe precipitation estimatesexh5102/assets/... · 2008-10-11 ·...

HydrologicalHydrological--Driven Validation of Driven Validation of MPE Precipitation EstimatesMPE Precipitation Estimates

Emad Habib & Boone F. LarsonEmad Habib & Boone F. LarsonUniversity of Louisiana at LafayetteUniversity of Louisiana at Lafayette

Jeffrey Jeffrey GraschelGraschel Brian R. NelsonBrian R. NelsonLMRFCLMRFC NCDCNCDC

Joint Assembly, Fort Lauderdale, FloridaMay 2008

ObjectivesObjectives

•• To provide an independent evaluation of MPE To provide an independent evaluation of MPE (NWS(NWS--RFC) at hydrological relevant scalesRFC) at hydrological relevant scales

•• To assess implications of subTo assess implications of sub--pixel variability for pixel variability for MPE evaluationMPE evaluation

•• To gain insight on practical value of MPE To gain insight on practical value of MPE products for hydrological applicationsproducts for hydrological applications

Data SourcesData Sources

•• MPE products:MPE products:– NWS LMRFC products

• Stage III (MPE 2002) ----> Stage IV (NCEP)

• Rain gauge network–– IndependentIndependent–– High quality of dataHigh quality of data

–– High density within scale of MPE productHigh density within scale of MPE product

Rain Gauge Network in Lafayette, LARain Gauge Network in Lafayette, LA

4 Km4 Km

Annual precipitation = 55-60 inches

724 -200

724 -201

723 -201

Rain Gauge StationDischarge Gauge Station

723 -200Vincent Rd.

Gulf South

LaNeuville Rd.

Carriage Light Loop

Millcreek Rd.

STM

FenstermakerCommission Blvd

Lafayette Vineyard

Covenant Church

722 -201

ÊÚ

ÊÚ

0 100 200 Kilometers

KPOE

KLCH New Orleans

KPOE

KLCH New Orleans

~ 116 km from KLCH

• Period of study – 2004-2006

• Scales of interest:– 4x4 km2

– Hourly/ Daily/ Monthly• Daily / Monthly Analysis

– all six pixels• Hourly Analysis

– Two 4-gauge pixels

0.00

0.05

0.10

0.15

0.20

0.25

0 1 2 3 4 5Number of Gauges

VRF

Variance Reduction Factor

Monthly ComparisonsMonthly Comparisons

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec0

100

300

500Rainfall 2004

Dep

th (m

m)


100

200

300Rainfall 2005

Dep

th (m

m)


50

150

250Rainfall 2006

Dep

th (m

m)

GaugeMPE

2004

2005

2006

MP

E(m

m)

0 10 20 30 40 50 60 700

10

20

30

40

50

60

70HOURLY

0 40 80 120 160 2000

40

80

120

160

200DAILY

0 100 200 300 400 5000

100

200

300

400

500MONTHLY

2004

Gauge (mm) Gauge (mm) Gauge (mm)2006

MP

E(m

m)

0 10 20 30 40 50 60 70 800

10

20

30

40

50

60

70

80HOURLY

0 20 40 60 80 100 120 1400

20

40

60

80

100

120

140DAILY

0 40 80 120 160 200 240 2800

40

80

120

160

200

240

280MONTHLY

Gauge (mm) Gauge (mm) Gauge (mm)

MPE

(m

m)

MPE

(m

m)

Hourly Daily Monthly Hourly Daily Monthly

Relative Bias -0.07 0.01 0.02 -0.04 0.00 0.00

Relative RMSE 1.42 0.81 0.28 1.16 0.71 0.19

Correlation Coefficient 0.82 0.90 0.91 0.93 0.94 0.95

2004 2006

-0.4-0.2

00.20.40.60.8

11.21.4

0.13-0.30

0.30-0.50

0.50-0.80

0.80-1.25

1.25-2.0

2-4 4-7 7-10 > 10

Gauge Intensity (mm/h)

Rel

ativ

e B

ias

2004 2005 2006

0

1

2

34

5

6

7

8

0.13-0.30

0.30-0.50

0.50-0.80

0.80-1.25

1.25-2.0

2-4 4-7 7-10 > 10

Gauge Intensity (mm/h)

Rel

ativ

e R

MS

E

2004 2005 2006

Bias

RMSE

X (mm)

Pro

babi

lity

Rat

ioof

Mis

sed

Rai

nfal

l

0 1 2 3 4 5 6 7 80

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.02

0.04

0.06

0.08

200420052006

Categorical metricsCategorical metrics

2004 2005 2006

0.01 0.01 0.02

2004 2005 2006

6% (7.86 cm) 3% (7.93 cm) 4% (11.98 cm)

Total Rainfall Falsely Detected

False Alarm Ratio (FAR)

Probability of Detection

Volume of Missed Rain

SpatioSpatio--Temporal StructureTemporal Structure

Month

Cor

rela

tion

1 2 3 4 5 6 7 8 9 10 11 120

0.2

0.4

0.6

0.8

1

GaugeMPE

2004-2006

Month

Cor

rela

tion

1 2 3 4 5 6 7 8 9 10 11 12

0

0.2

0.4

0.6

GaugeMPE

2004-2006

Spatial Correlation: 4-km lag

Temporal Correlation:1-hour lag

Effect of subEffect of sub--pixel variabilitypixel variability*

**

* **

* **

**

** *

***

**

*

**

*

*

**

**

*

**

*

*

***

***

** *

**

* ** *

*

*

*

**

**

*

*

***

*

*

**

*

*

** *

*

**

** **

**

Distance (km)

Cor

rela

tion

1 2 3 4 5 6 7 8 90

0.2

0.4

0.6

0.8

1

CV <= 0.20.2 < CV <= 0.05CV > 0.5

*

Var(MPE-Gi)

Var(MPE-True)

Var(Gi-True)

2Cov[(MPE-True), (Gi-True)]

=

+

-

Var(MPE-G)/Var(MPE-T)

0

50

100

150

200

250

300

350

1 2 3 4 5 6 7 8Gauge

Varia

nce

Rat

io (%

)

CV<0.2 CV<0.2<0.5

CV>0.5 ALL

Var(MPE-G) / Var(MPE-True)

Hydrologic Application:Hydrologic Application:

724-200

724-201

723-201

Rain Gauge StationDischarge Gauge Station

723-200Vincent Rd.

Gulf South

LaNeuville Rd.

Carriage Light Loop

Millcreek Rd.

STM

FenstermakerCommission Blvd

Lafayette Vineyard

Covenant Church

722-201

•• GSSHA: PhysicallyGSSHA: Physically--based fully distributed based fully distributed hydrologic modeling hydrologic modeling system (Ogden and system (Ogden and Downer, 2002)Downer, 2002)

•• Model Setup: Model Setup: –– 22--d diffusive wave for overland flowd diffusive wave for overland flow–– 11--d explicit diffusive wave for channel flowd explicit diffusive wave for channel flow–– PenmanPenman--MonteithMonteith equation for ET equation for ET –– Green and Green and AmptAmpt infiltration with redistributioninfiltration with redistribution

Calibration/ValidationCalibration/Validation

Time in days since 10/1/2004 00:00

Dis

char

ge(c

ms)

25 30 35 40 45 5005

1015202530354045

MeasuredSimulated

20.7 Km

5.1 Km

0

5

10

15

20

25

30

35

40

45

2004.316 2004.317 2004.318 2004.319 2004.32 2004.321 2004.322 2004.323 2004.324 2004.325

Year

Q (C

MS)

Full Gauge NetworkObserved

0

5

10

15

20

25

30

35

40

45

2004.316 2004.317 2004.318 2004.319 2004.32 2004.321 2004.322 2004.323 2004.324 2004.325

Year

Q (C

MS)

Full Gauge NetworkObservedMPE

0

5

10

15

20

25

30

35

40

45

2004.316 2004.317 2004.318 2004.319 2004.32 2004.321 2004.322 2004.323 2004.324 2004.325

Year

Q (C

MS)

Full Gauge NetworkObservedMPEClose Gauge

0

5

10

15

20

25

30

35

40

45

2004.316 2004.317 2004.318 2004.319 2004.32 2004.321 2004.322 2004.323 2004.324 2004.325

Year

Q (C

MS)

Full Gauge NetworkFar GaugeObservedMPEClose Gauge

Period 1

Period 2

0

5

10

15

20

25

30

35

2004.332 2004.333 2004.334 2004.335 2004.336 2004.337 2004.338 2004.339 2004.34 2004.341

Year

Q (C

MS)


0

5

10

15

20

25

30

35

2004.332 2004.333 2004.334 2004.335 2004.336 2004.337 2004.338 2004.339 2004.34 2004.341

Year

Q (C

MS)


0

5

10

15

20

25

30

35

2004.332 2004.333 2004.334 2004.335 2004.336 2004.337 2004.338 2004.339 2004.34 2004.341

Year

Q (C

MS)


0

5

10

15

20

25

30

35

2004.332 2004.333 2004.334 2004.335 2004.336 2004.337 2004.338 2004.339 2004.34 2004.341

Year

Q (C

MS)


Conclusions & Future WorkConclusions & Future Work

• Overall bias is minimal (2-7% at hourly scale; almost zero at daily/monthly scales)

• MPE has conditional bias: overestimation at low rainfall and underestimation at high rainfall

• MPE has conditional variance: variance decreases with increase of intensity

• high probability of detection (> 90%) except at very small intensities

• Low probability of false detection (1-2%) – results in 3-5% falsely detected rain

Conclusions & Future WorkConclusions & Future Work

• Relying on a single gauge for validation can cause overestimation of MPE errors by ~150%-200%

• For runoff purposes, MPE have noticeable value over typical rain gauge availability situations.

• Future work: use the same network and other similar research networks to look at other QPE products

AcknowledgementsAcknowledgements

•• Funding provided by:Funding provided by:

–– UCAR under sponsorship of NOAA/DOC as part of the UCAR under sponsorship of NOAA/DOC as part of the COMET Outreach Program.COMET Outreach Program.

–– Louisiana Board of Regents, Louisiana Board of Regents, BoRSFBoRSF, agreement , agreement NASA/LEQSF (2005NASA/LEQSF (2005--2010)2010)--LaSPACE and LaSPACE and NASA/NASA/LaSPACELaSPACE, grant NNG05GH22H, grant NNG05GH22H

–– University of Louisiana at LafayetteUniversity of Louisiana at Lafayette

Thank You!Thank You!

Continuous StatisticsContinuous Statistics

( )

RG

n

iRGiMPEi

Rn

RR

B

⎟⎠

⎞⎜⎝

⎛−

=

∑=1 ( ) ( )

RG

n

iMPERGRGiMPEi

R

RRRRnRMSE∑=

⎟⎠⎞⎜

⎝⎛ −−−

= 1

221

( )( )))((

),(2222

MPEMPERGRG

MPERGMPERGMPERG

RRRR

RRRRRR

−−

−=ρ

( )

( )2

1

2

1

2

11

⎟⎟⎠

⎞⎜⎜⎝

⎛−

⎟⎠

⎞⎜⎝

⎛−

−=

∑

∑

=

=

n

iRGRGi

n

iRGiMPEi

RRn

n

RR

E

Pearson Correlation

Normalized Root Mean Square Error

Efficiency

Normalized Bias

Conditional Validation based on Rainfall Conditional Validation based on Rainfall Magnitude and SeasonMagnitude and Season

•• Rainfall Magnitude conditioningRainfall Magnitude conditioning–– Define a relationship between gauge and MPE as Define a relationship between gauge and MPE as

a function of rainfall intensity. a function of rainfall intensity. –– 9 intensity intervals used (0.13 mm/hr 9 intensity intervals used (0.13 mm/hr →→ >10 >10

mm/hr)mm/hr)

•• Seasonal conditioningSeasonal conditioning–– Define any seasonal relationship between gauge Define any seasonal relationship between gauge

and MPE.and MPE.–– Hourly statistics plotted on a monthly basis.Hourly statistics plotted on a monthly basis.

Season-based Statistics

Average Gauge

Single Gauge

Average Gauge Single Gauge Average

GaugeSingle Gauge

Gauge Mean (mm)Normalized Bias 0.06 0.06 -0.04 -0.04 -0.20 -0.20Normalized RMSE 1.01 1.13 1.21 1.34 1.07 1.33Correlation Coefficient 0.92 0.90 0.85 0.81 0.87 0.81Efficiency 0.84 0.80 0.72 0.65 0.74 0.63

Non-uniform (Warm Months)

2.21 2.931.90

Highly Uniform (Cold Months)

Less Uniform (Transitional Months)

Analyzing SubAnalyzing Sub--pixel Rainfall Variabilitypixel Rainfall Variability

Single Gauge or Average GaugeSingle Gauge or Average Gauge

–– Stratified based on CVStratified based on CV•• CV CV ≤≤ 0.20.2•• 0.2 < CV 0.2 < CV ≤≤ 0.50.5•• CV > 0.5CV > 0.5

–– Stratified based on seasonStratified based on season•• Cold months (Dec Cold months (Dec –– Feb)Feb)•• Transitional months (Mar Transitional months (Mar –– May, Sep May, Sep -- Nov)Nov)•• Warm months (Jun Warm months (Jun –– Aug)Aug)

SelfSelf--correlation Statisticscorrelation Statistics

All selfAll self--correlation performed at hourly time scale.correlation performed at hourly time scale.

•• Spatial Spatial (G(G11 vs. Gvs. G22 / MPE/ MPE11 vs. MPEvs. MPE22))

–– Two 4Two 4--gauge pixelsgauge pixels–– Representative of 4 km.Representative of 4 km.

•• Temporal Temporal (G/MPE)(G/MPE)

–– One 4One 4--gauge pixelgauge pixel–– 1 hr time shift1 hr time shift

MPE Development at River Forecast MPE Development at River Forecast Center (RFC)Center (RFC)

--Precipitation Processing System (PPS)Precipitation Processing System (PPS)–– Reflectivity preprocessing (QC)Reflectivity preprocessing (QC)–– Conversion of (Z) to (R). Z=250RConversion of (Z) to (R). Z=250R1.21.2 , Z=300R, Z=300R1.41.4

--STAGE ISTAGE I ~ Digital Precipitation Array (DPA)~ Digital Precipitation Array (DPA)-- Hourly radarHourly radar--only product over (HRAP) gridonly product over (HRAP) grid

--Mean Field Bias CorrectionMean Field Bias Correction-- Rain gauges to correct multiplicative biasRain gauges to correct multiplicative bias

--STAGE IISTAGE II ~ mean field bias correcting of DPA~ mean field bias correcting of DPA

--STAGE IIISTAGE III ~ mosaicking Stage II over each RFC domain. ~ mosaicking Stage II over each RFC domain. (Replaced in 2002 (Replaced in 2002 with with MPEMPE))

--STAGE IVSTAGE IV ~ ~ creation of quilted national product of radar-based estimates by The National Center for Environmental Prediction (NCEP)

Cor

rela

tion

1 2 3 4 5 6 7 8 9 10 11 120.6

0.7

0.8

0.9

1

(d)

Nor

mal

ized

Bia

s1 2 3 4 5 6 7 8 9 10 11 12

-1

-0.5

0

0.5

1

1.5

2

(b)

Nor

mal

ized

RM

SE

1 2 3 4 5 6 7 8 9 10 11 12

1

2

3

4

5

6

(c)

Mea

nR

ainf

all(

mm

)

1 2 3 4 5 6 7 8 9 10 11 120

1

2

3

4

5

6200420052006

(a)

Month

Effi

cien

cy

1 2 3 4 5 6 7 8 9 10 11 12-1

-0.5

0

0.5

1

(e)

In next slides I break up plots to increase visibility

Nor

mal

ized

Bia

s

1 2 3 4 5 6 7 8 9 10 11 12-1

-0.5

0

0.5

1

1.5

2

Mea

nR

ainf

all(

mm

)

1 2 3 4 5 6 7 8 9 10 11 120

1

2

3

4

5

6200420052006

Month

Nor

mal

ized

RM

SE

1 2 3 4 5 6 7 8 9 10 11 12

1

2

3

4

5

6

Month

Effic

ienc

y

1 2 3 4 5 6 7 8 9 10 11 12-1

-0.5

0

0.5

1

Cor

rela

tion

1 2 3 4 5 6 7 8 9 10 11 120.6

0.7

0.8

0.9

1

200420052006

Var(MPE-T)'/Var(MPE-T)

0

20

40

60

80

100

120

140

160

180

200

1 2 3 4 5 6 7 8

Gauge

Varia

nce

Rat

io (%

)

CV<0.2 CV<0.2<0.5

CV>0.5 ALL

Var(MPE-True)’ / Var(MPE-True)

Period 4

0

5

10

15

20

25

30

2005.087 2005.088 2005.089 2005.09 2005.091 2005.092 2005.093 2005.094 2005.095 2005.096

Year

Q (

CM

S)


0

5

10

15

20

25

30

2005.087 2005.088 2005.089 2005.09 2005.091 2005.092 2005.093 2005.094 2005.095 2005.096

Year

Q (

CM

S)


0

5

10

15

20

25

30

2005.087 2005.088 2005.089 2005.09 2005.091 2005.092 2005.093 2005.094 2005.095 2005.096

Year

Q (

CM

S)


0

5

10

15

20

25

30

2005.087 2005.088 2005.089 2005.09 2005.091 2005.092 2005.093 2005.094 2005.095 2005.096

Year

Q (

CM

S)


Validation Metrics Validation Metrics

•• PointPoint-- or gridor grid--based methodsbased methods

• Continuous metrics

• Categorical metrics

• Distribution-oriented metrics

• Spatial/Temporal Structure- metrics

• Intensity stratification

Rainfall (mm)

Cum

ulat

ive

Pro

babi

lity

10-1 100 101 1020

0.2

0.4

0.6

0.8

1

DA

ILY

Rainfall (mm)10-1 100 101 1020

0.2

0.4

0.6

0.8

1

Rainfall (mm)10-1 100 101 1020

0.2

0.4

0.6

0.8

1Rainfall (mm)

Cum

ulat

ive

Pro

babi

lity

10-1 100 1010

0.2

0.4

0.6

0.8

1

GaugeMPE

2004

HO

UR

LY

Rainfall (mm)10-1 100 1010

0.2

0.4

0.6

0.8

12005

Rainfall (mm)10-1 100 1010

0.2

0.4

0.6

0.8

12006

Cumulative Distribution Function

Var(G-T)/Var(MPE-G)

0

25

50

75

100

1 2 3 4 5 6 7 8Gauge

Varia

nce

Rat

io (%

)

CV<0.2 CV<0.2<0.5

CV>0.5 ALL

Var(G-True) / Var(MPE-G)

hydrological-driven validation of mpe precipitation estimatesexh5102/assets/... · 2008-10-11 ·...

Documents