Developing of Evaluation Metrics and Indices for Applications

Galia Guentchev and the NCPP Core and Tech team

Community and Collaboration

NCPP mission - to accelerate the provision of climate information on regional and local scale for use in adaptation planning and

decision making through collaborative participation of a community of scientists and practitioners.

Applications related groups

Applications Related Metrics and Indices

Major focus - the development of a capability for objective and quantitative evaluation of downscaled climate information in

support of applications.

Example of a Health Impacts Case Study– Focus: Historical changes in the temperature related

triggers used in Heat Health Warning Systems (HHWS) – May – September 1971-2000

– Variables – tmax, tmin• The temperature/mortality relationships, when used as trigger

setting approaches for HHWS, have been found to most closely identify the most dangerous days in terms of excess mortality - Hajat et al 2010

– Indices:• 92.5th, 95th and 97.5 percentile exceedances of Tmax and Tmin

with a minimum duration of 2 days (based on relative threshold exceedances for each summer day with a 5 day window);

– Parameters – frequency, timing of occurrence, duration– Areas of interest – 3 metropolitan areas – Boston,

Indianapolis, Washington DC; N Carolina counties

Information that can be obtained from the Advanced Search capability

• Comparisons of downscaled data vs Maurer02v2– Average summer maximum and minimum

temperature comparisons – 95th percentile of maximum or minimum

temperature– Annual number of tropical nights (tmin>20degC)• Average, median, maximum, minimum

Average summer minimum temperature

Minimal differences in the order of -0.5 to 0.5 degrees C regardless of downscaling method or model

(some variation in spatial pattern for the ARRM MIROC MEDRES data).

95th percentile of summer minimum temperature

Average annual number of tropical nightsLarge differences between the methods and the

models used.

Maximum annual number of tropical nightsEven larger divergence between the method and

model used.

Information that can be obtained from the Open Climate GIS capability

• Observed and downscaled minimum and maximum temperature– To be used to calculate exceedances above a

threshold using dynamical percentiles or a simple absolute threshold

– For area of interest – the 3 metropolitan areas and the North Carolina counties

• Pre-calculated indices results from the NCPP evaluation– To be used to extract evaluation results for area of

interest

Metrics calculated on Tmax or Tmin for summer months:

• Mean• Median• Maximum • Minimum • Standard Deviation • Absolute threshold exceedances • Number of values between thresholds• Consecutive occurrences above/below an absolute threshold• Frequency of spells• Exceedances using dynamical percentiles for each day – 90,

92.5, 95, 97.5 percentiles

Comparison between observed and downscaled data for exceedances above the 97.5 dynamical

percentile

6 7 80

5

10

15

20

25

30

35

40

Exceedances of minimum daily temperature above 97.5 p for Boston 1971-2000

MaurerBCCA-GFDLARRM-GFDL

6 7 80

5

10

15

20

25

30

35

40

Exceedances of minimum daily temper-ature above 97.5p for Indianapolis

1971-2000

MaurerBCCA-GFDLARRM-GFDL

Need for Evaluation of Indices

• General evaluation based only on distributional characteristics of variables of interest does not contain sufficient usable information for applications

• There is a need to evaluate applications related indices and metrics

• Following good practices to incorporate uncertainty is important – use of various downscaling methods and an array of GCMs and emissions scenarios is recommended (STARDEX, 2005; TGICA 2004)

During our working times with the applications groups we will be working with

the tools, and will be discussing what is usable interpretation of these evaluation

results and what guidance is needed on the use of climate information in their work.

END

Summer Ave Tmax

95th percentile of summer tmax

Minimum annual number of tropical nights