u.s. ioos model validation capability sura super-regional test bed

US Army Engineer Research and Development Center

U.S. IOOS MODEL VALIDATION CAPABILITY

SURA SUPER-REGIONAL TEST BED

Working across agencies to bring observations and models together in a natural coastal laboratory…

USACE Field Research Facility

Jeff HansonUS Army Corps of Engineers, Field Research Facility, Duck, NC


SURA SUPER-REGIONAL TEST BED

Don Wright (SURA), Yesterday…

Test Bed Goals

1. Develop skill metrics and evaluate model performance…

Year 1 Products

1. Skill metrics and identification of key performance factors for

test bed modeling regimes…


An

alysis

Performance Evaluation Model Diagnostics

Interactive Model Evaluation and Diagnostics System

Inundation Modeling Test BedSkill Assessment Concept

InundationModeling Systems

Waves Circulation

Atmosphere

Storm Surge

Research

SURAServer

Regional Observations


Graphical User Interface


Features

• Stand-alone desktop model validation toolkit

• Based on published NOAA standards (Hess et. al., 2003; Zhang et.al., 2006).

• Robust error metrics: Erms, bias, Scatter Index, Skill Score

• Explore model errors as a function of time, space, event, etc.

Performance Scores

• Performance scores range from 0 to 1 (1 is a perfect match)

• Synthesize error metrics across space and time

• Normalized to mean observed quantities

• A measure of error % relative to mean observed quantities


IMEDS Error Analyses

Statistical Analyses• Temporal correlation

• Quantile-Quantile (distributions)

• Peak event (peak over threshold)

Parameters Added To-Date

Winds

Waves(Windsea and swell)

Storm Surge

Speed, Direction

Height, Period, Direction

Water level, High water marks

Error Metrics

• RMS Error

• Bias, Angular bias

• Scatter Index

• Circular correlation

• Performance (Skill) Scores

Coming Soon

• Target Plots


IMEDS Error Metrics

Bias

RMS Error

Scatter Index

Reference: Hanson, Tracy, Tolman and Scott, 2001. J. Oce. Atms. Tech., 26, 1614-1633

For n pairs of measurement (m) and hindcast (h) data…


IMEDS Error Metrics

Angular Bias

Circular Correlation

Reference: Hanson, Tracy, Tolman and Scott, 2001. J. Oce. Atms. Tech., 26, 1614-1633

And for directional data…


IMEDS Skill Scores

Bias Skill

RMS Error Skill

Scatter Index Skill

Station Skill

Total SkillCombine station scores using sample size weights

Scores range from 0 to 1 1 = perfect agreement 0 = totally uncorrelated

Scores are computed relative to mean observed quantities

Average the 3 scores


Peak Over Threshold Extraction

Peak Event Analysis

Peak Event Statistics


FEMA Region III Storm Surge Project

Hurricane IsabelMax Water Levels


FEMA Region III Storm Surge Study:

Hurricane Isabel September 2003


FEMA Region III Storm Surge Study:

Nor’Ida November 2009


Supported Data Formats

Observed Data

• NDBC buoys

• NODC wind/wave stations

• NOS – includes detide option

• IMEDS Generic Format

Model Output

• WW3 (*spc)

• SWAN (spec2d.out & TAB(opt))

•ADCIRC Water-Level (fort.61)

• ADCIRC Wind (fort.72)

• Extract from SURA OpenDAP Server (netCDF file)

• IMEDS Generic Format


IMEDS Generic Format

Sample File

% IMEDS generic format version 1.0 – water-elevation % year month day hour min watlev (m) NOS UTC NAVD 8635750 37.9950 76.4650 2009 11 9 0 0 0.2440 2009 11 9 1 0 0.1580 2009 11 9 2 0 0.0530 2009 11 9 3 0 -0.0310 2009 11 9 4 0 -0.1000 5461421 36.8432 73.5591 2009 11 9 6 0 -0.1520 2009 11 9 7 0 -0.0900 2009 11 9 8 0 0.0140

Station Data

Station Data


IMEDS Set Up GUI

Sample File


Accomplishments to DateAccomplishments to Date

• IMEDS posted on SURA Server with illustrated users guide

• Develop data interface to SURA OpenDAP server

• Implement generic text file interface

• User-defined graphics export formats (jpeg, png, fig, etc…)

• Prototype Target Plot


Want to use IMEDS?Want to use IMEDS?

• Download IMEDS from SURA Server

• Review fully illustrated users guide

• Ensure observation data are in native format (NOS, NODC, etc) or IMEDS Text format

• Export model results in IMEDS generic text format

• Seek help as needed from CI group and/or USACE

• Put IMEDS to work for you!

u.s. ioos model validation capability sura super-regional test bed

Documents