1

Establishing Confidence in Marine Forecast Systems: The design and skill assessment of the

New York Harbor Observation and Prediction System, version 3 (NYHOPS v3)

Nickitas Georgas1 and Alan F. Blumberg2

Abstract We briefly describe the new NYHOPS v3 OFS (Operational hydrodynamic Forecast System) and quantify its performance against National Ocean Service (NOS) standard OFS evaluation metrics. Given the relatively large area of the NYHOPS v3 OFS (including the NY/NJ Harbor estuary, Long Island Sound, and their coastal ocean), and the proliferation of sensor networks, the presented skill assessment is one of the most extensive performed to date: model results are compared to in situ observations of water level, currents, temperature, salinity, and waves from over 100 locations, collected in a 2 year period. The model’s ability to describe the hydrodynamic conditions in the extensive area it is employed is remarkable. The average index of agreement for water level is 0.98, for currents is 0.87, for water temperature is 0.98, for salinity is 0.77, and for significant wave heights is 0.88. Respective, average root-mean-square errors are: 10cm for water level, 13cm/s and 9° for currents, 1.4°C for water temperatures, 2.8psu for salinities, and 32cm for significant wave heights. 1. Introduction The New York Harbor Observation and Prediction System (NYHOPS, Bruno et al 2006) was established at Stevens Institute of Technology (Stevens) in 2004, through coordinated efforts from academia, industry, local and federal US government to: • Permit an assessment of ocean, weather, and environmental conditions throughout

the New York Harbor and New Jersey Coast regions, • Provide marine forecasts (general circulation and waves) for the said area up to 48

hours in advance, • Establish a continuous history of the marine conditions in and around the New

York / New Jersey Harbor, and, • Provide a test bed for environmental systems integration into situation awareness

scenarios, ranging from flooding alerts, to search and rescue, to chemical spills. The first version of the NYHOPS system included 48hr hydrodynamic circulation predictions based on the Princeton Ocean Model (Blumberg and Mellor 1987) and, specifically, its Estuarine and Coastal Ocean Model derivative (ECOM) as 1Senior Research Engineer, Center for Maritime Systems, Stevens Institute of Technology, 711 Hudson Street, Hoboken, NJ 07030; PH (201) 216-8218; nickitas.georgas@stevens.edu 2George Meade Bond Professor and Director, Center for Maritime Systems, Stevens Institute of Technology, 711 Hudson Street, Hoboken, NJ 07030; PH (201) 216-5289; alan.blumberg@stevens.edu

2

implemented for the New York Harbor region (Blumberg et al 1999). A 2nd version of the NYHOPS forecast model went into effect in January 2007, with a higher resolution numerical hydrodynamic forecast grid, and included uncoupled surface wave forecasts (Georgas et al 2007). After a two-year period of continuous development, updates, testing, and complete automation, the new model has been operational and publicly available (www.stevens.edu/maritimeforecast) since June 2009; under the name NYHOPS v3, it is an integral part of the regional component of the global Integrated Ocean Observing System (IOOS). The new OFS builds upon the older NYHOPS versions, providing marine conditions in a high resolution grid, for a larger area, based on improved representations of physics and physical constraints (such as boundary conditions), and is more accessible (including Google Earth kml files, OpenDAP/THREDDS servers, etc.). The complete NYHOPS v3 environmental system of systems infrastructure is described in Georgas et al 2009.

In section 2 we describe the NYHOPS v3 hydrodynamic forecast model and its implementation to the New York Bight and its estuaries, sounds, and tidal fresh waters (Figure 1). Sections 3 and 4 are the main focus of the paper: the assessment of the new model’s skill in predicting water level, η, currents, U, water temperature, T, salinity, S, and significant wave height, Ho, in the model region. The paper is designed to emulate the model evaluation process followed in the Delaware River and Bay Model Evaluation Environment (DRB-MEE) that resulted in six publications in the previous Estuarine and Coastal Modeling proceedings (ECM10, Patchen 2008 and references therein). We are going to concentrate on a 2 year hindcast period, between 02/01/2007 and 02/01/2009, for which in situ observations were available from a multitude of sensors dispersed throughout the NYHOPS area. Comparisons of the NYHOPS predicted sea surface temperatures (SST) to remote observations (satellite-derived SST) are a focus of another paper in this issue (Bhushan et al 2010).

Figure 1. Map of geographic locations referenced in this paper.

3

2. NYHOPS v3 Model Implementation

The three-dimensional hydrodynamic model ECOM (Blumberg et al 1999), a derivative of the Princeton Ocean Model (POM, Blumberg and Mellor 1987), is used to forecast the ocean processes across the large coastal, estuarine, and riverine NYHOPS domain. As used in NYHOPS v3, the hydrodynamic code includes significant developments not included in the original ECOM/POM, such as wetting-and-drying (W&D) and thin-dam (obstruction grid) formulations, a new dynamically coupled wave module, a new one-way-coupled atmospheric module, and complete Climate and Forecasting Conventions (CF 1.4) compliance of the NetCDF outputs (Georgas 2010). The hydrodynamic NYHOPS v3 model provides forecasts of water level, 3D circulation fields (currents, T, S), significant wave height, and wave period.

The model incorporates the Mellor-Yamada 2.5 level turbulent closure model (Mellor and Yamada 1982).The Smagorinsky constant, HORCON, is set to 0.01, the bottom roughness length Z0 to 0.001m, the minimum bottom drag coefficient CDmin to 0.003, and the molecular diffusivity UMOL to 10-6 m2/s everywhere. No local calibration of the bottom drag coefficient has been performed. However, CDmin is allowed to dynamically adjust based on the presence of the local wave boundary layer computed from the dynamically coupled wave model and Grant-Madsen theory (Georgas 2010). The wave module is based on the GLERL wind-wave momentum model (Donelan 1977, Schwab et al 1984). The GLERL code has been modified with the NYHOPS coastal region in mind to add bottom frictional dissipation (wave friction factor set to 4x10-3), tidally-adjusting depth-induced breaking, unresolved obstructions (thin dams), and open boundary forcing through specification of significant wave height and direction at the oceanic boundary. The empirical fraction of the wind stress that is retained by the waves is set to 2.8% in NYHOPS v3. Added skin friction at the surface uses a coefficient set to 0.7x10-3. More details are found in Georgas 2010.

In OFS forecasting mode, NYHOPS v3 is run daily, to provide a hindcast (-24hrs) and forecast (+48hrs) of the hydrodynamic circulation and wave conditions in the coastal (<200m deep), estuarine, and freshwater zones from coastal Maryland to Cape Cod, Massachusetts (Figure 1). The hydrodynamic model is initiated at 0 hrs local every day, and completes a 24hr hindcast cycle based on observed forcing followed by a 48hr forecast cycle based on forecast forcing. The 72hr NYHOPS v3 daily run code (W&D 3D hydrodynamics with coupled waves and 2D atmospherics) has been compiled with Portland Group’s auto-parallelizable pgf77®. It runs on a Dell Nehalem computer with eight 2.93GHz cores (2 quads with hyper-threading) in about 1.5hrs with a 1sec barotropic (2D) and a 10sec baroclinic (3D) timestep. Coupled chemical kinetics (in particular, fate and transport of chromophoric dissolved organic matter), acoustic transmission loss, and offline data assimilative nowcasting are included in the NYHOPS v3 system of systems, but will not be elaborated here.

The NYHOPS v3 computational domain is discretized on an Arakawa “C” finite-difference grid (147x452 cells, 15,068 of which are designated as water). A high-resolution curvilinear model grid is used to encompass the entire Hudson-Raritan

4

(New York/New Jersey Harbor) Estuary, the Long Island Sound, and the New Jersey and Long Island coastal ocean (Figure 2). The resolution of the grid ranges from approximately 7.5km at the open ocean boundary to less than 50m in several parts of the NY/NJ Harbor Estuary. In order to resolve coastline features that could not be resolved on a grid cell scale, most notably the NJ Atlantic coast barrier islands, 96 cell interfaces across which transport or mixing is disallowed (“thin dams”) have been defined. In the vertical, the model uses a sigma-coordinate system with bathymetrically-stretched sigma layers to permit better representation of bottom topography. The vertical resolution of the grid is 10 sigma layers. NYHOPS has also been tested with 40 sigma layers, but runtimes are currently operationally prohibitive.

Figure 2. High-resolution NYHOPS v3 finite difference grid created with Delft3d RGFGRID®: A) Complete grid, B) NY/NJ Harbor zoom, C) Long Island / Block Island sounds zoom. Contoured bathymetry is in meters [max of 200m offshore]. Some thin dams (explained in the text) are visible (pointed out with TD). Regardless of the effort that went into designing the new higher-resolution grid for NYHOPS v3, it is obvious than any descritization of a continuous field will retain errors that may, in places, be significant. Tidal waves are surface gravity shallow

A B

C

TD TD

TD

5

water waves that propagate based on shallow water physics with celerity highly dependent on depth (e.g. Blumberg and Georgas 2008). Figure 3 depicts a metric for the resolution of the new, greatly improved grid, based on the variation (here, standard deviation) of actual sounding depths found within a grid cell described numerically by a single average depth value. Based on the resolution metric shown in Figure 3, the relative sub-grid variation in tidal propagation, σTP, may be approximated as:

HHgHgHHg HHH

TPσσσ

σ2111

)(±≈−±=

−±= (1)

where, H is the (mean) cell depth, σH is the standard deviation of actual soundings taken from within that cell, and the Maclaurin expansion is used for illustration only. For example, a HH /σ =30% (as in Figure 3) would cause σTP ≈15%. Higher values of σTP may make comparisons of model results to point measurements collected “somewhere within a grid cell” questionable, especially with regard to tidal phase. Figure 3 shows that the resolution of the new grid is excellent in most of the open Atlantic Ocean waters, Long Island Sound, and the NY/NJ Harbor. Represented but less resolved are Delaware Bay, Narragansett Bay, the Peconics, Jamaica Bay, the middle and upper Hudson River, and the embayments behind the NJ Atlantic Coast barrier islands.

Figure 3. Resolution of the new NYHOPS high-resolution grid estimated as percent standard deviation of soundings pooled within a grid cell divided by that cell’s average depth.

6

Open Ocean Boundary (OOB) conditions in NYHOPS v3 are specified from: • Tides; tidal constituents from the East Coast 2001 (EC2001) database based on

the ADCIRC model (Mukai et al 2002), • Storm surge; from observations at a coastal station multiplied by a shoaling factor

similar to Blumberg et al 1999, for the hindcast cycle; forecast cycle from the NOAA/NWS/MDL extra-tropical storm surge model of the U.S. East Coast (http://www.nws.noaa.gov/mdl/etsurge/, Chen et al 1993; Kim et al 1996),

• Historical mean southwestward along-shore flow in the NY Bight through specification of constant cross-shelf elevation tilt at the northeast and west OOB (Blumberg and Galperin 1990, Blumberg et al 1999),

• Long-term, primary annual mode of the steric effect (thermohaline non-Boussinesq expansion/contraction of water volume estimated from the NOS Atlantic City gage; method of Mellor and Ezer 1995, Georgas 2010), and,

• Monthly T/S climatology; from the NOAA Levitus 1998 compiled 100-year historic database, http://www.cdc.noaa.gov/cdc/data.nodc.woa98.html, optimally interpolated on the OOB

During the first high-resolution NYHOPS evaluation year, 2007, wave OOB conditions were extracted from the NOAA/NWS/NCEP regional Western North Atlantic (WNA) WaveWatch-III-type model, which has an approximate 15-minute grid and has been operational since 2000 (Chao et al 2004). As of January 2008, significant wave height and direction at the NYHOPS OOB are taken from MAT4 (ftp://polar.ncep.noaa.gov/pub/waves/develop), an optimally interpolated 4-minute-horizontal resolution, hourly NOAA/NWS/NCEP U.S. East Coast WaveWatch-III-type model, a part of the newly developed NWS multi-grid wave forecast model NMWW3 (Chawla et al 2007).

NYHOPS v3 internal buoyancy forcing is specified for: • 93 primary river systems; hindcast from 74 USGS gages

(http://waterdata.usgs.gov/nwis/rt), forecast from 27 NOAA/AHPS 6-hourly predictions (http://www.nws.noaa.gov/oh/ahps/) or persistence, watershed-area adjusted,

• 146 secondary ungaged watersheds, with estimated hydrologic stream flows from nearby rivers, watershed area adjusted,

• 241 major freshwater discharges; monthly-averaged flows and effluent temperatures compiled from EPA EnviroFacts Warehouse Permit Compliance System Discharge Monitoring Reports (EPA Envirofacts Warehouse PCS DMR, http://www.epa.gov/enviro/html/pcs/pcs_query_java.html), quality controlled,

• 39 major thermal discharges; monthy-averaged flows and thermal intake/effluent input/outputs compiled from EPA EnviroFacts Warehouse PCS DMR, quality controlled.

NYHOPS surface boundary conditions (SBC) for winds and atmospheric heating and cooling are based on the North American Mesoscale model (NAM, ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/nam) run at the National Centers for Environmental Prediction (NCEP). NAM is a Weather-Research-and-Forecasting (WRF)-type model. Each forecast cycle is initiated from an assimilated initial

7

condition (AIC) every 6hrs. Predicted atmospheric fields from each forecast cycle are provided on a sigma-pressure grid with approximate horizontal resolution of 12km every 3hrs. For the NYHOPS v3 24hr hindcast period, a concatenation of timestamps from the most recent NAM runs is used: NYHOPS uses the 0z (AIC) and 3z (forecast) from the 0z NAM run, then the 6z (AIC) and 9z (forecast) from the 6z NAM run, etc. Thus, NYHOPS takes the times closest to the NAM AIC, ensuring the truest representation of the observed and near-term forecasted surface atmospheric fields. 2D NAM winds at 10m elevation are extracted from NAM and used for wind stress forcing and wind wave growth in NYHOPS v3. The 2D surface heat flux variables provided by NAM to NYHOPS v3 are total cloud cover, 2m air temperature, 10m wind, relative humidity, and barometric pressure reduced to mean sea level. Note that barometric pressure has only been used to calculate the latent heat flux component of the air-sea heat fluxes through the calculation of vapor pressure. Barometric pressure load forcing as a free surface adjustment mechanism has been found to reduce operational NYHOPS forecast skill as the NAM 12km resolution does not resolve the Hudson River Estuary (Georgas and Blumberg 2008).

3. NYHOPS v3 Skill Assessment Methodology The two year period from 02/01/2007 to 02/01/2009 was selected for evaluation. Observations from all available Stevens or external (NOS, USGS, CMAN, NDBC, HRECOS, Monmouth University, etc.) stations that had data within the evaluation period were gathered in the Stevens Oceanographic and Meteorological Data Repository to gage the model against. Then, based on the findings of this primary skill assessment period, NYHOPS v3 (with wave-current interactions, 2D heat fluxes, and all other aforementioned improvements) was run in hindcast mode from May 2006 to date, eventually superseding NYHOPS v2 as today’s standard NYHOPS OFS (since June 15 2009). NYHOPS v3 skill assessment was performed for the same two year period (02/01/2007 to 02/01/2009), to prove the advantages of the latest changes. This paper focuses on the NYHOPS v3 results for brevity. NYHOPS v2 skill assessment results will only be used in relative form to illustrate drift of model skill with time separation from forecast initiation.

3.1 Tidal analyses of water level and currents Model evaluation is performed against a standard suite of statistics (Hess et al 2003) that compare the simulated fields against observations. These statistics include: • the series mean (SM), • the root-mean-square-error (RMSE), • the error standard deviation (SD), • the model’s index of agreement (Skill), • central frequency estimates (CF, not to be confused with the Climate and Forecast

conventions: percentage of time the error of the simulated field is below an acceptable value, X; target frequency of 90%),

• positive and negative outlier frequencies (POF/NOF: percentage of time the error in the simulated field is above an unacceptable value, 2X; target frequency 1%),

8

• and maximum duration of positive or negative outliers (MDPO/MDNO: the length of time the simulation error stays above the unacceptable value 2X – target duration N=24-25 hrs).

The acceptable (X) and unacceptable (2X) error bounds are of course field-specific and can be found in tables at the end of the Hess et al 2003 document: • For errors in predicted water levels, in general, as well as for amplitude of high

and low waters (AHW, ALW, respectively), X=15cm, (2X=30cm). • Similar statistics are used for Time of High (Low) Water (THW, TLW), with

X=0.5 hrs (2X=1hr). • For currents, X=26cm/s for the time series as a whole and for the maximum ebb

and flood currents (AFC, AEC), X=0.5hr for the times of slack (start of slack before ebb, TSE, end of slack before ebb, TEE, start of slack before flood, TSF, end of slack before flood, TEF) and maximum currents (TFC for flood, TEC for ebb).

• For current direction, X=22.5 degrees, in general, including during maximum flood (DFC), or ebb (DEC).

For water levels, an additional condition called the worst case outlier frequency (WOF) is also checked: what percentage of time actual water levels turn out to be lower (higher) than astronomical tide, but the model erroneously predicted much higher (lower) water levels; during such events (with a target frequency less than 0.5%) the user would be better off using the astronomical tide water level prediction. Having said that, a good question is how the astronomical tide prediction fairs against all of the above metrics but the last. If it fairs worst that the model, the model should by definition be a better tool than the astronomical tidal prediction alone. This comparison among different forecasts (or different models) can be comprehensively described with one single statistic, the model index of agreement, also known as the Willmott Skill (Willmott 1981; 0-1 with 1 meaning perfect prediction), also included in the method.

The evaluation codes received from Dr. Aijun Zhang were made more robust (esp. in terms of extrema identification), automation was furthered, and Matlab plotting routines were added (Georgas 2010). A significant addition was the explicit calculation of simulated non-tidal water level (“surge”) through the subtraction of simulated tides from the simulated total water level. Simulated tides may include inherent errors that can be large in locations poorely resolved by a model (model resolution deficits, as shown in Figure 3). In terms of total water level prediction, a substitution of the simulated astronomical tidal prediction with the actual astronomical tidal prediction (both calculated a priori) can eliminate errors in the simulated tide level for locations where the astronomical tides are well established based on observations. This adjustment considerably raises the NYHOPS model skill in total water level prediction, as is later shown in Figure 8 under “NYHOPS v3*”. This meta-model processing step (the adjusted model is herein referred to as NYHOPS-v3-adjusted or just NYHOPS-v3*) is used in the Stevens Storm Surge Warning System (SSWS; www.stevens.edu/SSWS) achieving highly accurate total water level predictions for flooding alerts. The expanded version of the model

9

evaluation algorithms is also freely available upon request, and has been successfully used in two Master’s Theses to date (Bhushan 2009 and Di Liberto 2009).

Table 1 lists stations for which water level observations were compared to NYHOPS model results; the locations of the stations are shown in Figure 4. 37 standard NOS tidal constituents at the 33 stations with observations throughout the NYHOPS area were quantified (amplitude and phase) through least-squares harmonic analysis of water level. Available records of currents throughout the 2-yr evaluation period where also harmonically analyzed, but for fewer constituents given Rayleigh criterion (synodic period) limitations of the shorter and segmented current time series. The results of harmonic constituent comparisons were plotted and tabulated. This was carried out for water levels and currents, where possible. Depth-averaged current speed time series were calculated and compared to depth-averaged currents observed at fixed ADCP stations (Figure 5). Vertical profiles of velocity at fixed ADCP stations were compared to NYHOPS predictions. Correct identification of extrema and, in particular, times of extrema, requires Fourier filtering to remove high frequency disturbances (NOAA 2002, Hess et al 2003, Zhang et al 2007). For water levels, a 3hr Fourier Filter was applied, while, for currents, a 90-minute filter was applied. Gaps in the data smaller than 2hrs were filled with cubic spline interpolation, while gaps smaller than 6hrs were filled with singular value decomposition. 3.2 NYHOPS v3 skill in predicting non-tidal events With regard to the ability of NYHOPS to effectively predict meteorological forcing effects on water level and currents, analysis of variance was first performed to separate the total signal in tidal and non-tidal parts. Instead of using Butterworth-type low-pass filtering to remove the tidal effects, a technique that generally underestimates the error in the residual since high-frequency non-tidal errors are also filtered out, the analyzed tidal signal was explicitly reconstituted and removed at observed stations, and model skill was computed for the remaining subtidal signal.

For water level and currents, tides were removed, and root-mean-square-errors (RMSE) for the tidal residual were also calculated. For water level in particular, this calculation is equivalent to the NYHOPS-v3 adjusted RMSE (NYHOPS-v3* as described in the previous section). 3.3 NYHOPS v3 skill in predicting water temperature, salinity, and wave conditions Model predictive skill was also calculated for T, S, and Ho for the fixed stations where data were available (Figure 6 and Table 1). This was done through the expanded NOAA/NOS model evaluation software based on: • For S, X=3.5psu acceptable error, and N=24hrs for MDNO/MDPO calculations. • For T, X=3.0ºC (X decreased from Hess et al 2003, as per Zhang and Wei 2008,

and references therein), and N=24hrs.

10

ID Name Agency Agency ID Latitude Longitude η U T S Ho AC: Atlantic Coast of New Jersey AC-01 Atlantic City, NJ NOS 8534720 39.3550 -74.4183 X X AC-02 Brant Beach, NJ Stevens SBRNG4 39.6143 -74.1959 X

X

AC-03 Little Egg Inlet, NJ USGS 01409335 39.5089 -74.3247 X

AC-04 Ocean Grove, NJ Stevens SOCNG4 40.2088 -74.0040 X X AC-07 Point Pleasant, NJ USGS 1408050 40.1017 -74.0375 X AC-08 Bethany Beach, DE ERDC DE002 38.5400 -75.0400

X

AC-09 Ocean City, MD ERDC MD002 38.3400 -75.0700

X BI: New York Bight waters BI-01 26 NM SE of Cape May, NJ NDBC 44009 38.4636 -74.7019 X X BI-02 33 NM S of Islip, NY NDBC 44025 40.2503 -73.1664 X X BI-03 Entrance to NY Harbor, NY NDBC 44065 40.3690 -73.7030

X

BI-04 Ambrose Light, NY CMAN ALSN6 40.4500 -73.8000 X BI-05 23 NM SE of Montauk Pt,NY NDBC 44017 40.6922 -72.0478 X X BI-06 Buzzards Bay, MA CMAN BUZM3 41.3967 -71.0333 X DB: Delaware Bay DB-01 Brandywine Shoal, DE NOS 8555889 38.9867 -75.1133 X X DB-02 Cape May, NJ NOS 8536110 38.9683 -74.9600 X X DB-03 Lewes, DE NOS 8557380 38.7817 -75.1200 X X DB-05 Bowers Beach, DE USGS 1484085 39.0583 -75.3976 X X DB-06 Lower Delaware Bay, DE NDBC 44054 38.8800 -75.1800

X

DB-07 Central Delaware Bay, DE NDBC 44055 39.1200 -75.2500

X BRS Brown Shoals, DE NOS Db0501

X

HE: NY/NJ Harbor Estuary, (including Hudson, Raritan, and Newark Bay estuarine waters) HE-01 79th Street Boat Basin, NY Stevens SSNBB4 40.7857 -73.9866 X

X

HE-02 Belford, NJ Stevens SBLFD4 40.4339 -74.0793 X X X HE-03 Bergen Point, NY NOS 8519483 40.6400 -74.1467 X X X HE-04 George Washington Br., NJ Stevens SGWBR4 40.8521 -73.9593 X X X HE-05 Hastings-on-Hudson, NY USGS 01376304 40.9878 -73.8875 X X X HE-06 Keansburg, NJ USGS 01407081 40.4492 -74.1475 X

HE-07 Pier 40, NY Stevens SPR404 40.7285 -74.0143 X X X HE-08 Sandy Hook, NJ NOS 8531680 40.4667 -74.0100 X X X HE-09 South Amboy, NJ USGS 01406710 40.4922 -74.2814 X

HE-10 PVSC Plant at Newark, NJ Stevens SPVSS4 40.7129 -74.1230 X X X HE-11 The Battery, NY NOS 8518750 40.7000 -74.0150 X

HE-12 Castle Point, NJ (CPT) Stevens SCPNT4

X

X HE-13 Piermont Pier, NY HRECOS 3B010580 41.0431 -73.8960

X

A30 ATON 30, NY Stevens STATN30

X

NAR Verazzano Narrows Br., NY NOS 8517986

X

HR: Tidal, Freshwater Hudson River HR-01 Albany, NY USGS 01359139 42.6461 -73.7481 X X HR-02 Poughkeepsie, NY (HRP) USGS 01372058

X X X X

HR-03 Schodack Island, NY HRECOS Not Avail. 42.4996 -73.7768 X

HR-04 West Point, NY USGS 01374019 41.3861 -73.9556 X X X LI: Long Island and its Sound LI-01 Bridgeport, CT NOS 8467150 41.1733 -73.1817 X X LI-02 Kings Point, NY NOS 8516945 40.8100 -73.7650 X X LI-03 Montauk, NY NOS 8510560 41.0483 -71.9600 X

LI-04 New Haven, CT NOS 8465705 41.2833 -72.9083 X X LI-05 New London, CT NOS 8461490 41.3550 -72.0867 X

LI-06 Rockaway Inlet, NY USGS 01311875 40.5736 -73.8856 X

LI-10 Point Lookout, NY USGS 1310740 40.5933 -73.5842 X X X LI-12 Central Long Island Snd, NY UCONN 44039 41.1375 -72.6550

X

LI-13 Western Long Island S., NY UCONN 44040 40.9558 -73.5800

X NB: Narragansett Bay NB-01 Conimicut Light, NY NOS 8452944 41.7167 -71.3433 X X NB-02 Fall River, MA NOS 8447386 41.7050 -71.1633 X

X

NB-03 Newport, RI NOS 8452660 41.5050 -71.3267 X

X NB-04 Quonset Point, RI (QPT) NOS 8454049

X X

NJ: New Jersey Back Bays (behind the barrier islands of the NJ Atlantic Ocean coastline)

NJ-01 Margate, NJ USGS 01411330 39.3375 -74.5131 X

NJ-03 Avalon, NJ USGS 01411355 39.1103 -74.7342 X NJ-04 Barnegat Light, NJ USGS 01409125 39.7611 -74.1081 X NJ-05 Cape May Harbor, NJ USGS 01411390 38.9486 -74.8906 X NJ-15 Navesink River, NJ MU 0501 40.3814 -74.0142

X

NJ-16 Branchport Creek, NJ MU 0502 40.3203 -73.9961

X

Table 1. NYHOPS v3 skill assessment stations.

11

Figure 4. Stations used for NYHOPS water level skill assessment laid over the resolution metric defined by Eq. (1). Names are provided in Table 1.

Figure 5. Stations equipped with Doppler Profilers used for NYHOPS currents skill assessment laid over the resolution metric defined by Eq. (1). HRP: USGS Hudson River at Poughkeepsie (HR-02 in Table 1); CPT: Stevens Castle Point (HE-12); A30: Stevens on ATON 30; NAR: NOS under the Verrazano Narrows Bridge; BRS: NOS at Brown Shoals; QPT: NOS at Quonset Point (NB-04).

12

Figure 6. Stations used for NYHOPS water temperature, salinity, and significant wave height skill assessment. Names are provided in Table 1. 4. NYHOPS v3 Skill Assessment Results and Discussion 4.1 Tidal analysis of water level.

An example of harmonic constituent comparisons between the observed and simulated tide in the New York Harbor is provided in Table 2 for the NOS station at the Battery, NY and the two-year NYHOPS evaluation period (2/1/2007-2/1/2009). Comparison of the amplitudes and phases of the listed constituents inferred from the observed tides against the standard NOS tide tables for the Battery, NY reveals only small differences (order of a cm or less for the major constituents). An example of an 8-day tidal time series comparison at the Battery is shown in Figure 7 (left panel). Recall that NYHOPS is forced at the OOB with 7 primary astronomical constituents (M2, N2, S2, K2, K1, O1, and Q1), as well as the zero-frequency component (Z0) and two shallow water overtides (M4 and M6), as provided by the EC2001 database (Mukai et al 2002). These constituents are noted with X (for the 7 primary) and O (for the 2 overtides) in Table 2. Remaining tidal constituents account for only 1.6% of the tidal energy at the Battery, but, regardless, not accounting for them is a systematic truncation error. Figure 7 (right panel) also compares the tide based on the superposition of all 37 observed tidal constituents to the NYHOPS-simulated tide based on the superposition of all simulated tidal constituents at the Battery. The R2

13

value is 0.992 and the root-mean-square-error (RMSE) is only 4.4cm (0.044m), or only 3% of the mean tidal range (1.38m at the Battery). Note, however, that the OFS standard for the central frequency of the total water level error is only 15cm (Hess et al 2003). Practically, this CF(X=15cm)<90% standard, translates to a required RMSE of approximately 9cm for a total water level prediction to meet that standard. Yet, half of that (4.4cm at the Battery) is already lost due to the small, but significant, tidal error, leaving a very small allowance for errors in the non-tidal part of the total water level signal, whether event-driven (hydrological, meteorological) or longer term (e.g. steric) departures from tide. Based on analysis of observed variances at the 33 water level stations listed in Table 1, we estimated that 15% of the pooled-average total water level variance in the NYHOPS region is due to non-tidal processes.

Table 2. The Battery, NY. Amplitudes (m) and phases (degrees GMT) of 9 major NOS tidal constituents (NOS) along with the ones calculated from the 2-year evaluation period from observations (observed) and NYHOPS v3 simulations (NYHOPS v3). X (O) denotes primary (overtide) constituents that NYHOPS is forced by at the OOB.

Figure 7. The Battery, NY. Time series (left, 12/20/08 to 12/28/08) and correlogram (right) comparison of the NYHOPS-simulated versus the observed astronomical tide.

14

Table 3 lists the overall errors in the major NYHOPS-v3-simulated tidal constituents compared to the ones observed at the 33 NYHOPS-resolved stations shown on Figure 4. Mean-Absolute-Errors (MAE) from the Atlantic Coast sub-region of the EC2001 ADCIRC model (Mukai et al 2002) are included, as well as mean cross-station energy (cm2) carried by each constituent as a measure of relative importance. The low-resolution NYHOPS (v1) originated from the Blumberg et al (1999) work on the NYCDEP System Wide Eutrophication Model (SWEM). That older ECOM model, calibrated with bottom drag coefficients varying by two orders of magnitude, had a cross-station error for the M2 constituent of 9.4±5.4% and 16.5±13.9º in the coastal zone (Blumberg et al 1999). For NYHOPS v3, the larger errors are by far in the shallow water overtide constituents M4 and M6, followed by diurnal constituents, while the most energetic semidiurnal constituents carry relatively lower errors; The M2 tide is simulated best. The K1 tide, the most significant diurnal constituent, is the least resolved major constituent in NYHOPS, presumably due to being the least resolved constituent in the EC2001 database (Table 3) which NYHOPS uses for its tidal forcing. Tidal RMSE and correlogram R2 values between NYHOPS-simulated and observed tide – averaged for stations within the NYHOPS subregions listed in Table 1 – are listed in Table 4. Included are the subregion-averaged mean tidal ranges (<MN>). The Atlantic Coast of New Jersey (AC), the Hudson-Raritan Estuary (HE), and the Long Island and its Sound region (LI) are very well represented. The tides in the upper Hudson River (HR) region are simulated the poorest.

Tidal Tidal Energy Amplitude Error (%) Phase Error (degrees) Constituent Across Stations NYHOPSv3 EC2001 NYHOPSv3 EC2001 Symbol (cm2) MAE ± error st.dev. MAE ± error st.dev. M2 4388 5.3±5.1 5.6 4.2±4.1 6.5 S2 150 8.6±8.6 7.5 5.3±5.0 8.4 N2 202 6.5±6.0 6.8 4.3±2.7 4.5 K2 11 8.6±7.0 10.3 6.5±7.2 12.2 K1 98 14.5±10.0 19.0 5.0±4.6 7.4 O1 34 9.4±4.6 7.8 6.4±3.7 7.2 Q1 2 11.7±8.9 14.6 6.8±5.4 9.1 M4 18 38.6±28.9 N/A 35.7±29.7 N/A M6 6 42.0±45.0 N/A 26.0±26.3 N/A

Table 3. Overall errors (mean absolute error, MAE, ± error standard deviation) between the NYHOPS v3 simulation and the estimated tidal harmonics for the 33 tidal stations in Figure 4. Water level time series within the 2-year NYHOPS evaluation period were analyzed. EC2001 results as noted in the text.

Region AC HE LI DB NB HR All 33 R2 0.99 0.99 0.99 0.99 0.97 0.94 0.98

RMSE, cm ±stdev

3.9 ±0.5

5.7 ±2.0

6.0 ±2.0

8.5 ±2.3

7.3 ±2.1

11.5 ±6.5

6.8 ±3.4

<MN>, cm ±stdev

109.9 ±15.7

136.2 ±13.1

149.1 ±66.5

135.7 ±13.9

114.9 ±11.6

121.3 ±31.0

130.3 ±32.6

RMSE/<MN> 3.5% 4.2% 4.0% 6.3% 6.4% 9.5% 5.2% Table 4. Regional grand-mean errors in the simulated tidal signal and the NYHOPS v3 skill in tidal prediction.

15

4.2 Tidal analysis of currents.

Tidal constituents were calculated for all ADCP locations shown in Figure 5 except for the Castle Point (CPT) location for which retrieved current records from three ADPCs were too short (~14 days) to allow for accurate predictions; at CPT only M2 was estimated, while the total current records are discussed in section 4.4. Due to similar Rayleigh-type (synodic period) limitations with all ADCP records, only the following 6 primary NOS tidal constituents were extracted for currents for the rest of the stations: M2, S2, N2, K2, K1, and O1. In addition, it was found that the compass installed on the ATON 30 ADCP (A30, upper New York Harbor, Figure 5) during the evaluation period was damaged, so current directions observed at this ADPC were erroneous. The compass has since been replaced. A30 results throughout this document are reported with a question mark (A30?), and a new skill assessment needs to be performed in the future at that location. Overall, there were 56 individual current time series harmonically analyzed along and across their principal current directions (PCD). PCD statistics and comparisons between NYHOPS and observations are given in Table 5.

Station ID Station Name Observed Simulated A30? ATON 30, Upper NY Harbor

Depth, m below MSL 3.92…<D-A>…12.92

PCD, degrees trigonometric 46…<48>…51 68…<72>…72

R, % 4.6…<4.1>…6.7 2.1…<1.0>…5.5

BRS Brown Shoals, Delaware Bay

Depth, m below MSL 3.27…<D-A>…11.27

PCD, degrees trigonometric 117…<121>…121 115…<116>…116

R, % 1.3…<2.3>…9.7 0.2…<0.2>…1.2

HRP Poughkeepsie, Hudson River

Depth, m below MSL 3.50…<D-A>…16.50

PCD, degrees trigonometric 70…<70>…70 83…<84>…85

R, % 0.4…<0.2>…0.7 0.0…<0.0>…1.3

NAR Verrazanno Narrows

Depth, m below MSL 3.39…<D-A>…15.39

PCD, degrees trigonometric 120…<122>…126 120…<124>…126

R, % 1.1…<0.4>…2.7 0.9…<0.6>…2.2

QPT Quonset Point, Narragansett Bay

Depth, m below MSL 3.10…<D-A>…7.10

PCD, degrees trigonometric 69…<70>…69 87…<87>…88

R, % 5.3…<3.8>…10.3 2.1…<3.0>…5.7

Table 5. Principal Current Component (PCD) comparisons between NYHOPS results and observations. R is the ratio of the across-PCD current variance to the along-PCD current variance, a measure of cross-flow importance. D-A statistics are based on harmonic decomposition of the depth-averaged time series.

16

There was no significant cross-PCD flow found in any of the analyzed bins and locations. The ratio, R, of the cross-PCD current variance to the along-PCD current variance was small in most stations. Since R was much less that 25% everywhere, across-PCD skill evaluation was not required in any of the stations, based on the NOS skill assessment standards (Hess et al 2003). In fact, the vast majority of the cross-PCD tidal constituents were found to be less that 1cm/s, close to the given precision (0.1cm/s for native ADCP, 0.5cm/s for NOS data provided at 0.01 knot increments) and effective accuracy limits of the ADCPs instruments (~0.5cm/s or more). The R ratio calculated based on the depth-averaged current (denoted within brackets <> in Table 5) was most often smaller than the R ratio of any individual ADCP bin at a given station. This finding may imply that small variations in the cross-PCD currents (as well as the along-PCD currents given the light veering of PCD itself in some stations) tend to cancel themselves out when averaged over the vertical, hinting perhaps to secondary circulation (e.g. Georgas and Blumberg 2004) that is averaged out in the barotropic depth-averaged cross-PCD current, abovementioned precision and accuracy errors notwithstanding. Overall, good comparisons were obtained between simulated and observed tidal currents, although maximum NYHOPS errors in the tidal current are larger than ones for tidal water level presented in the previous section.

Region A30? BRS HRP NAR QPT Average R2 0.99 0.97 0.95 0.99 0.87 0.95

RMSE, cm/s ±stdev

8.0 ±1.6

9.6 ±1.2

9.5 ±2.2

7.0 ±1.7

5.6 ±0.9

7.9 ±1.3

2.75*σcur, cm/s ±stdev

105.0 ±24.0

131.3 ±15.7

105.3 ±10.8

130.5 ±18.4

32.1 ±3.2

100.8 ±15.2

RMSE/2.75/σcur 7.6% 7.3% 9.0% 5.4% 17.4% 9.3% Table 6. Errors in the NYHOPS-simulated tidal current signal along-PCD for each station, pooled among vertical bins. 2.75 times the square root of the total tidal variance is used here as an estimate of a representative range of the tidal current, as described in the text.

Finally, Table 6 summarizes the NYHOPS skill in predicting the tidal current at each ADCP station (pooled among bins). Included is a mean tidal current range estimation based on 2.75 times the standard deviation of the tidal signal [the 2.75 factor was based on the average <MN>/stdev(water level) to facilitate comparison with Table 4]. Grid resolution appears to control the quality of simulated currents similarly to the tidal level results of the previous section. The NYHOPS model predicts the tidal current and its profile with varying skill at different regions, with the best predictions obtained at the New York Harbor locations with the best grid resolution (A30? and NAR, R2=0.99), and the worst by far at Quonset Point (R2=0.87) in the more poorly resolved Narragansett Bay. Juxtaposing Tables 4 and 6 in terms of R2 shows that the NYHOPS model explains a greater percent of the variance in the tidal water level (~98%) than the variance in the tidal current (~95%). This is also manifested as a bigger root mean square error in the tidal current as a percent of the mean tidal current range estimate (9.3%, Table 6), compared to the equivalent metric for water

17

level (5.2%, Table 4). Blumberg and Georgas (2008) used first order analysis of variance (FOVA) to show – both analytically and in simulation – that currents are usually more sensitive to bathymetric accuracy than water level is. Bathymetric accuracy in a hydrodynamic model is a function of grid resolution, or, more specifically, a function of the bathymetric deviation within each grid cell illustrated in Figure 3.

Figure 8. RMSE (meters) in total water level predicted by a) astronomical tides alone (PRD; as in the usual NOS tide tables), b) the original NYHOPS version (v1, where available), c) the first high-resolution NYHOPS version (v2), d) the third generation NYHOPS (v3), and e) NYHOPS v3 after post-processing removal of the systematic tidal error as mentioned in the paper (v3*). A 9cm RMSE is practically required to achieve the NOS total water level central frequency OFS standard. 33 stations are shown, as per Table 1. 4.3 NYHOPS v3 prediction skill for total water level

Table 7 shows NOS major skill assessment statistics for total water level. The results of the NYHOPS v3 model are arguably excellent, both in terms of indices of agreement (skill), as well as root-mean-square errors, given the large mean tidal ranges in the region (see RMSE/MN in Table 7). Yet, the strict NOS standards are met in only 4 out of the 33 stations (Atlantic City, NJ; Little Egg Inlet, NJ; George Washington Bridge, NJ; Hastings-on-Hudson, NY) however close some stations might be in achieving them. This is not surprising. For example, none of the four models that were tested by NOAA in the DRB-MEE (Patchen 2008) experiments for

18

Delaware Bay achieved the NOAA/NOS standards for water level, at any station. With the post-processing removal of the aforementioned systematic tidal error (v3*, Figure 8), the NOS standard can be met in 14 out of 33 stations. Average RMSE grows from hindcast to 24hr forecast by 6% and to 48hr forecast by 15% (reaching 81% of the square root of the non-tidal water level variance at 48hr, not shown).

ID

RMSE cm

NOF <1%

CF >90%

POF <1%

MDNO <24hr

MDPO <24hr

WOF <0.5%

SKILL

RMSE/MN

AC-01 8.0 0.1% 94.1% 0.0% 5.2 1.4 0.06% 0.99 5.6% AC-02 11.5 1.7% 82.9% 0.2% 10.6 1.9 0.53% 0.98 8.7% AC-03 7.0 0.0% 96.6% 0.0% 5.0 0.0 0.04% 0.99 5.8% AC-04 9.6 0.5% 89.4% 0.1% 3.1 3.2 0.32% 0.99 6.9% DB-01 12.6 1.0% 75.7% 0.3% 5.9 3.4 0.73% 0.98 8.7% DB-02 12.5 1.0% 76.8% 0.6% 5.8 3.2 0.84% 0.99 8.2% DB-03 9.3 0.2% 89.9% 0.0% 5.1 2.2 0.09% 0.99 6.1% HE-01 10.2 0.5% 86.9% 0.4% 5.0 1.9 0.57% 0.99 8.7% HE-02 10.6 0.5% 85.3% 0.5% 2.9 2.9 0.62% 0.99 8.4% HE-03 12.4 1.3% 79.1% 0.8% 6.1 2.2 1.15% 0.99 8.5% HE-04 9.8 0.1% 90.0% 0.4% 4.5 9.9 0.29% 0.99 6.9% HE-05 8.9 0.2% 91.0% 0.1% 5.3 2.3 0.15% 0.99 9.6% HE-06 13.9 1.7% 75.9% 1.4% 6.5 5.2 2.04% 0.98 7.5% HE-07 10.6 0.3% 85.0% 0.5% 3.3 3.0 0.47% 0.99 8.3% HE-08 10.6 0.7% 85.4% 0.2% 5.3 2.2 0.57% 0.99 8.1% HE-09 12.5 1.2% 79.1% 0.9% 8.2 3.1 1.07% 0.99 10.9% HE-10 16.3 3.0% 65.4% 3.3% 3.3 5.8 3.58% 0.98 14.0% HE-11 10.9 0.8% 84.7% 0.4% 5.3 2.2 0.69% 0.99 24.9% HR-01 22.5 7.0% 46.0% 10.7% 16.9 6.1 8.51% 0.97 25.9% HR-02 11.1 0.6% 82.8% 0.3% 5.4 4.7 0.38% 0.98 9.0% HR-03 21.3 6.0% 50.3% 8.2% 6.8 6.1 8.91% 0.96 10.4% HR-04 9.9 0.3% 87.5% 0.2% 4.6 3.7 0.20% 0.98 55.6% LI-01 13.3 1.8% 75.3% 0.8% 7.2 5.3 1.38% 0.99 9.5% LI-02 15.4 3.1% 69.9% 1.8% 8.6 23.2 2.58% 0.99 20.3% LI-03 10.6 0.9% 85.8% 0.1% 14.7 4.0 0.54% 0.96 9.8% LI-04 12.2 1.3% 79.7% 0.6% 7.7 5.8 0.96% 0.99 10.6% LI-05 9.7 0.6% 89.4% 0.1% 15.5 6.9 0.28% 0.98 7.7% LI-06 10.9 0.9% 84.6% 0.3% 5.5 2.4 0.65% 0.99 15.3% NB-01 14.4 2.6% 72.6% 1.1% 10.5 4.8 2.46% 0.97 10.7% NB-02 16.0 4.2% 67.4% 1.8% 10.6 4.1 4.17% 0.97 10.4% NB-03 11.0 1.0% 85.0% 0.3% 10.4 7.1 0.66% 0.98 17.6% NB-04 11.7 1.1% 81.6% 0.4% 10.0 3.7 0.91% 0.98 10.0% NJ-01 12.4 1.0% 76.9% 0.3% 6.3 3.0 0.70% 0.98 12.2%

Table 7. Water level skill assessment statistics. Station ID as in Table 1. Instances when the NYHOPS v3 model did not meet NOS performance standards are highlighted in bold italics. 4.4 NYHOPS v3 prediction skill for total currents

Table 8 shows NOS major skill assessment statistics for total currents at the six ADCP stations listed in Table 1. 42 out of 56 collected ADCP bins exceeded the NOS standards for both speed and direction, but only two out of five stations at the surface-

19

most bin (recall that the A30 station had a broken compass). From these two stations, the Quonset Point one exceeded the standards due to the relative slow currents measured (mean current magnitude at the surface was only 14.7 cm/s), even though the NYHOPS v3 index of agreement (skill) was the worst there because of low resolution. NOS criteria are not based on skill, but acceptable levels of error. The best model skill is in the better resolved areas, and especially the NY/NJ Harbor estuary.

RMSE NOF<1% CF>90% POF<1% SKILL

ID DEPTH SPD DIR SPD DIR SPD DIR SPD DIR SPD DIR A30? 3.92 16.9 26.4 0.0% 6.8% 88.0% 51.4% 0.5% 0.0% 0.89 0.82

5.92 19.7 32.1 0.0% 17.2% 83.6% 41.9% 2.4% 0.0% 0.82 0.81

7.92 16.1 33.4 0.0% 20.0% 89.5% 41.7% 0.4% 0.1% 0.86 0.78

9.92 12.6 32.2 0.1% 17.4% 95.3% 45.8% 0.0% 0.1% 0.86 0.74

11.92 12.4 32.0 0.1% 18.7% 95.3% 55.1% 0.1% 0.1% 0.80 0.72

D-A 12.2 32.4 0.0% 17.5% 95.5% 40.4% 0.0% 0.0% 0.89 0.77

BRS 3.274 20.2 15.0 0.8% 0.3% 88.5% 89.9% 0.2% 1.5% 0.85 0.85

5.274 15.1 12.6 0.2% 0.1% 94.4% 91.7% 0.0% 1.1% 0.90 0.85

7.274 12.3 11.7 0.1% 0.0% 96.9% 92.7% 0.0% 0.9% 0.91 0.84

9.274 11.6 11.2 0.0% 0.0% 97.9% 93.4% 0.0% 0.8% 0.90 0.81

11.274 11.4 11.1 0.0% 0.1% 97.9% 93.5% 0.0% 0.9% 0.88 0.78

D-A 14.4 11.2 0.1% 0.0% 93.7% 93.1% 0.0% 0.7% 0.88 0.82

CPT 3.623 19.1 10.2 0.2% 0.1% 83.3% 97.3% 0.8% 0.1% 0.93 0.85

5.623 18.0 8.5 0.3% 0.0% 85.0% 98.4% 0.2% 0.0% 0.91 0.86

7.623 15.8 7.5 0.2% 0.0% 90.0% 99.5% 0.1% 0.0% 0.92 0.87

9.623 15.8 7.9 0.2% 0.0% 89.7% 99.7% 0.1% 0.0% 0.89 0.88

11.623 16.0 11.1 0.2% 0.0% 89.3% 98.0% 0.0% 0.0% 0.86 0.83

D-A 12.8 8.5 0.0% 0.0% 95.0% 98.1% 0.1% 0.0% 0.94 0.89

HRP 3.5 12.0 10.9 0.0% 0.0% 96.5% 99.9% 0.0% 0.0% 0.93 0.89

5.5 10.7 11.0 0.0% 0.0% 98.1% 99.9% 0.0% 0.0% 0.94 0.90

7.5 9.7 11.0 0.0% 0.0% 99.0% 99.8% 0.0% 0.0% 0.94 0.91

9.5 8.9 11.1 0.0% 0.0% 99.4% 99.7% 0.0% 0.0% 0.95 0.91

11.5 8.4 11.2 0.0% 0.0% 99.6% 99.6% 0.0% 0.0% 0.95 0.91

13.5 7.8 11.4 0.0% 0.0% 99.8% 99.3% 0.0% 0.0% 0.95 0.90

15.5 7.6 11.4 0.0% 0.0% 99.8% 98.7% 0.0% 0.0% 0.94 0.86

D-A 10.7 10.8 0.0% 0.0% 98.5% 99.9% 0.0% 0.0% 0.92 0.91

NAR 3.386 17.0 9.3 0.3% 0.0% 88.6% 97.0% 0.2% 0.1% 0.92 0.89

5.386 15.6 8.1 0.2% 0.0% 90.5% 98.3% 0.1% 0.0% 0.92 0.93

7.386 14.2 7.6 0.1% 0.0% 93.0% 99.1% 0.1% 0.0% 0.91 0.95

9.386 14.0 7.1 0.1% 0.0% 93.5% 99.7% 0.1% 0.0% 0.90 0.95

11.386 13.7 6.7 0.0% 0.0% 93.9% 99.8% 0.1% 0.0% 0.88 0.94

13.386 13.9 7.4 0.0% 0.0% 93.4% 99.2% 0.1% 0.0% 0.87 0.92

15.386 16.7 9.5 0.0% 0.2% 92.2% 96.2% 0.8% 0.1% 0.81 0.89

D-A 12.8 5.0 0.0% 0.0% 95.2% 100.0% 0.0% 0.0% 0.91 0.95

QPT 3.101 9.5 3.2 0.0% 0.0% 99.3% 99.5% 0.0% 0.0% 0.67 0.52

5.101 8.8 1.2 0.0% 0.0% 99.7% 99.9% 0.0% 0.0% 0.56 0.45

7.101 6.6 0.3 0.0% 0.0% 99.9% 100.0% 0.0% 0.0% 0.57 0.50

D-A 7.8 1.0 0.0% 0.0% 99.8% 99.9% 0.0% 0.0% 0.65 0.46

Table 8. Skill assessment statistics for currents along PCD. Station ID as in Table 1. Depths of individual bins are in meters below MSL (D-A is for the statistics of the depth-averaged current, not the averaged statistics of each station’s ADCP bins). RMSE in cm/s for speed, and degrees for phase (direction). Instances when the NYHOPS v3 model did not meet NOS performance standards are highlighted in bold italics. Only every other bin is listed for brevity.

20

4.5 NYHOPS v3 prediction skill for water temperature

Table 9 shows NOS major skill assessment statistics for water temperature. With regard to the new water temperature standard [CF(error<3.0°)>90%], 26 out of 30 stations exceeded it (<CF>=95%; RMSE=1.4±0.4°), due to a great improvement in the surface heat flux specification in the new NYHOPS v3 (Georgas and Blumberg 2008 and Bhushan et al 2010, this issue). Note that some of the best NYHOPS v3 results for temperature are in the Hudson River, even though the model resolution in the upper tidal Hudson is not great. This is because the significant river flow dominates the water temperature there, and NYHOPS v3 specifies the observed water temperatures from USGS gages. In fact, the only stations for which the drift of water temperature predicting skill with forecasting time is significant are stations near incoming rivers, where observed USGS temperatures are persisted. For the Hudson (HR stations), the low RMSE for water temperature increases by 17% from hindcast to 48hr forecast.

ID RMSE NOF<1% CF>90% POF<1% MDNO<24hr MDPO<24hr SKILL AC-01 1.30 0.0% 96.0% 0.3% 0.7 15.3 0.99 AC-04 1.61 0.0% 93.1% 0.3% 0.8 15.9 0.99 AC-07 1.87 0.4% 91.2% 0.3% 7.3 5.5 0.98 BI-01 1.14 0.0% 99.2% 0.0% 2.8 0.0 0.99 BI-02 1.05 0.0% 99.6% 0.0% 0.0 0.0 0.99 BI-04 1.16 0.0% 99.3% 0.0% 0.0 0.0 0.99 BI-05 1.32 0.0% 96.0% 0.0% 1.9 0.0 0.99 BI-06 1.62 0.0% 94.5% 0.0% 0.0 0.0 0.98 DB-01 0.85 0.0% 99.8% 0.0% 0.0 0.0 1.00 DB-02 1.88 0.0% 91.1% 1.0% 0.9 6.9 0.99 DB-03 1.38 0.1% 95.9% 0.0% 1.9 0.0 0.99 DB-05 1.54 0.1% 93.6% 0.0% 4.0 1.2 0.99 HE-02 1.44 0.0% 95.5% 0.0% 0.0 1.5 0.99 HE-03 1.22 0.0% 99.7% 0.0% 1.1 2.1 0.99 HE-04 1.00 0.0% 99.9% 0.0% 0.0 0.0 1.00 HE-05 1.11 0.0% 99.6% 0.0% 0.4 0.0 1.00 HE-07 1.27 0.0% 99.7% 0.0% 0.0 0.0 0.99 HE-08 1.26 0.0% 99.4% 0.0% 0.2 0.0 0.99 HE-10 1.81 0.0% 90.4% 0.0% 0.1 0.0 0.99 HR-01 0.34 0.0% 99.8% 0.0% 1.1 0.0 1.00 HR-02 0.68 0.0% 100.0% 0.0% 0.0 0.0 1.00 HR-04 0.71 0.0% 100.0% 0.0% 0.0 0.0 1.00 LI-01 1.88 0.0% 89.7% 0.0% 1.1 3.1 0.99 LI-02 1.48 0.3% 96.2% 0.1% 4.7 4.1 0.99 LI-04 2.08 0.0% 81.9% 0.0% 0.0 4.6 0.98 LI-10 2.11 0.4% 84.3% 0.1% 7.8 8.5 0.98 NB-01 1.48 0.0% 96.8% 0.2% 3.0 6.3 0.99 NJ-03 1.10 0.1% 97.7% 0.1% 4.7 3.5 0.99 NJ-04 1.98 1.3% 88.9% 0.0% 6.9 3.2 0.98 NJ-05 1.30 0.5% 97.2% 0.0% 0.0 0.0 0.70

Table 9. Skill assessment statistics for water temperature. Station ID as in Table 1. RMSE in degrees Celsius. Instances when the NYHOPS v3 model did not meet NOS performance standards are highlighted in bold italics.

21

4.6 NYHOPS v3 prediction skill for salinity

Table 10 shows NOS major skill assessment statistics for salinity. Automatic QA/QC of salinity observations based on NDBC methods (NDBC 2003) was used to remove outliers from the salinity records. A 2.75*standard deviation (2.75*σS) column has been added to indicate average local salinity variation (for the astronomical tide, 2.75*ση ≈ MN). As can be seen from the table, only about half of the listed stations meet the NOS salinity standards. The model index of agreement (skill) is best in the NY/NJ Harbor area (HE stations) and worst in Narragansett Bay. The HR stations have low skills, but also very low salinities, as the Hudson River stretch north of Haverstraw, NY is practically fresh for most of the year (<S> < 0.9 psu, Table 10).

ID <S> 2.75*σS RMSE NOF<1% CF>90% POF<1% MDNO<24hr MDPO<24hr SKILL DB-05 19.85 14.19 4.80 11.4% 49.5% 2.5% 35.5 137.2 0.81 HE-01 11.37 14.21 2.40 0.6% 86.0% 0.0% 3.5 0.2 0.95 HE-02 22.46 7.24 1.89 0.0% 94.8% 0.3% 0.0 5.8 0.90 HE-03 18.96 8.66 2.23 0.0% 91.0% 2.0% 0.0 181.5 0.90 HE-04 9.18 12.85 2.01 0.2% 92.7% 0.0% 1.7 0.0 0.96 HE-05 6.08 10.05 2.82 1.0% 77.9% 0.2% 7.0 5.2 0.86 HE-07 17.19 15.12 2.76 0.9% 79.0% 0.0% 3.1 0.0 0.94 HE-08 22.57 7.60 1.92 0.0% 94.3% 0.1% 0.0 3.6 0.90 HE-10 12.79 16.17 3.85 4.9% 65.4% 2.5% 9.0 5.3 0.91 HE-12 12.16 13.98 2.39 0.1% 86.1% 0.3% 1.7 2.6 0.95 HE-13 4.39 8.05 2.53 0.2% 88.1% 1.1% 2.6 5.0 0.80 HR-02 0.13 0.14 0.15 0.0% 100.0% 0.0% 0.0 0.0 0.30 HR-04 0.87 3.16 1.10 0.0% 100.0% 0.0% 0.0 0.0 0.66 LI-10 30.52 2.66 1.80 0.1% 95.9% 0.0% 7.5 0.0 0.45 NB-02 25.77 8.66 3.72 0.0% 74.5% 5.6% 0.0 53.3 0.69 NB-03 29.88 3.59 2.03 0.1% 89.9% 0.1% 7.2 4.6 0.55 NJ-15 22.73 8.72 1.79 0.2% 94.7% 0.0% 3.3 0.0 0.93 NJ-16 21.73 8.20 1.80 0.1% 96.8% 0.0% 4.9 0.0 0.92

Table 10. Skill assessment statistics for salinity. Station ID as per Table 1. Average observed salinity (<S>), standard deviation, σS , and RMSE in psu. Instances when the NYHOPS v3 model did not meet NOS performance standards are highlighted in bold italics.

ID <H0> 2.75*σHo RMSE NOF<1% CF>90% POF<1% MDNO<24hr MDPO<24hr SKILL AC-02 0.77 1.30 0.33 0.0% 100.0% 0.0% 0.0 0.0 0.88 AC-08 0.64 0.98 0.30 0.0% 100.0% 0.0% 0.0 0.0 0.86 AC-09 0.70 1.08 0.34 0.0% 99.8% 0.0% 0.0 0.0 0.85 BI-01 1.26 1.91 0.48 0.0% 98.4% 0.0% 2.1 0.0 0.89 BI-02 1.31 1.99 0.49 0.1% 98.1% 0.0% 3.5 0.0 0.89 BI-03 1.13 1.62 0.42 0.0% 99.4% 0.0% 0.0 0.0 0.87 BI-05 1.39 2.17 0.50 0.1% 98.1% 0.0% 2.7 0.0 0.91 DB-06 0.47 0.77 0.17 0.0% 100.0% 0.0% 0.0 0.0 0.90 DB-07 0.41 0.57 0.15 0.0% 100.0% 0.0% 0.0 0.0 0.86 LI-12 0.50 1.05 0.19 0.0% 100.0% 0.0% 0.0 0.0 0.94 LI-13 0.26 0.73 0.14 0.0% 100.0% 0.0% 0.0 0.0 0.88 NJ-03 0.67 0.99 0.32 0.0% 100.0% 0.0% 0.0 0.0 0.78

Table 11. Skill assessment statistics for significant wave height. Station ID as per Table 1. Average observed wave height, <H0>, standard deviation, σHo, and RMSE in meters. Note that there is no official (NOS) model performance statistic for wave heights. X=1.3m was used as noted in the text.

22

4.7 NYHOPS v3 prediction skill for significant wave heights

An NOS standard did not exist for significant wave heights. As a guide only, we used X=1.3m (4.25ft), a height that could cause a coastal wave run-up on the order of the standard acceptable error (15cm) in water level. Table 11 shows statistics for significant wave heights. The average skill for 12 wave stations is 0.88, with across-station-mean RMSE of 32cm. This level of error in the significant wave height prediction is about 25% of the mean locally experienced wave height range (2.75* σHo in Table 11). For comparison, the average skill for water level is 0.98, the average skill for currents is 0.87, for water temperature it is 0.98, and for salinity it is 0.77. 5. Summary and Conclusions An extensive hydrodynamic model skill assessment based on the standard NOS DRB-MEE methodology (Patchen 2008) has been applied to quantify the hindcasting and forecasting skill of the new NYHOPS v3 OFS. NYHOPS v3 model results are compared to in situ observations of water level, currents, temperature, salinity, and waves from over 100 locations, collected in a 2 year period.

Tides are more important overall within the estuaries of the NYHOPS region, with an average 85% of the total water level variance being tidal. This implies that a very-high-resolution model forced with a complete and perfect tidal signal could be able to forecast an average 85% of the total water level signal for any time in the future (mean sea level rise and other long-term inter-epoch fluctuations notwithstanding), without loss of skill; any loss in forecasting skill would then result from the 15% of the non-tidal variance due to drifting skill in the forecasts of the forcing functions, such as wind, rain and associated hydrologic inputs, etc. The forecast forcing provided to NYHOPS v3 from regional meteorological and hydrological models is enough to create meaningful 48hr forecasts of the tidal residual marine hydrodynamic response, with longer forecasts being of questionable validity and usefulness. NOS model evaluation criteria are not based on local relative model skill but “acceptable” levels of error. These levels of error are set the same for all tidal areas, irrespective of whether the modeled areas are macrotidal or microtidal and can thus become very strict, as in the case of total water level here. The estimated 33-station pooled-average mean tidal range in the NYHOPS region is 130cm ±33cm standard deviation among stations. Given that large a tidal range, the strict NOS central frequency (CF) OFS evaluation metric [CF(e<15cm)>90%, where e=error; an equivalent RMSE of just 9cm] is met only in 4 out of 33 stations for total water levels (4/33 with a pooled average <CF> of 79%; pooled average root mean square errors, RMSE=12±3cm standard deviation among stations). After post-processing removal of systematic tidal errors due to a) incompleteness of the tidal forcing b) remaining grid resolution deficits, and c) remaining uncertainty in physics, as many as 14/33 stations meet the NOS standard (<CF>= 88%; RMSE=10±2cm). With regard to currents magnitude and phase [CF(eM<26cm/s; eP<22.5°)>90%], 42/56 collected ADCP bins meet the standard, though only in 2/5 stations at the surface most bin

23

(<CFM>=91%; RMSEM=13±5cm /s; <CFP>=97%; RMSEP=8.9±4.2°). With regard to salinity [CF(e<3.5psu)>90%], 8/18 stations meet the NOS standard (<CF>=87%; RMSE=2.8±2.0psu). With regard to the temperature standard [CF(e<3.0°)>90%], 26/30 stations meet it (<CF>=95%; RMSE=1.4±0.4°), due to a great improvement in the surface heat flux specification in the new NYHOPS v3. The average index of agreement for 12 wave stations was 0.88 with average RMSE of 0.32cm. Relative model performance is summarized graphically in Figure 9, based on the local RMSE normalized by the apparent mean range, 2.75*σ. For all evaluated parameters, some of the best NYHOPS results are found in the well-resolved NY/NJ Harbor area.

Figure 9. Percent relative RMSE (ratio of local RMSE divided by the local average observed range approximated as 2.75*σ) for total water level (A), wave height (B), water temperature (C), and salinity (D) stations. Circle diameters match the quantile bounds [25%, 50%, 75%, 100%] listed at each panel’s legend for each of the four fields shown. For example, in the upper left panel (panel A), quarter of all water level stations where NYHOPS v3 performance is best (14.63% < RMSE/2.75ση < 21.03%]) are drawn with the largest circles. The 3rd generation NYHOPS model is significantly better than its predecessors overall. For example, results shown in Figure 8 for water level can be ranked as follows: The RMSE of the NOS astronomical tidal prediction (the NOS tide tables, PRD in Figure 8) carries an average RMSE of 18.5cm across a subset of the 33 stations evaluated within the extensive NYHOPS oceanic, estuarine, and freshwater

A B

C D

24

region. In the recent past, the use of skillfully calibrated hydrodynamic models (NYHOPS v1 and its SWEM predecessor) dropped that prediction RMSE for the same stations to 13.7cm. Modelers’ credo has been that skillful calibration has been a necessary substitute for inadequate resolution. Today, NYHOPS v3, enabled by exponentially faster processors, robust data streams, and improvements in the hydrodynamic codes, has dropped the RMSE for the same (v1) set of stations to 11.7cm, without relying on extensive bottom drag coefficient calibrations. This final point is a significant result because it suggests that NYHOPS-like coastal and estuarine operational forecasting systems can quickly be set up nested within large-scale ocean and atmospheric models to provide sufficiently accurate marine environmental predictions to the OFS user base. The NYHOPS v3 observations network and operational modeling system is now part of the integrated, sustained ocean observing system envisioned by the National Oceanographic Partnership Program (NOPP), under the OCEAN.US office, and the Mid-Atlantic Coastal Ocean Observing Regional Association (MACOORA) of the Integrated Ocean Observing System (IOOS). Acknowledgements Dr. Aijun Zhang provided the initial NOS skill assessment software that was adopted and expanded as described in the paper with assistance from Dr. Wei Li of Stevens. Mr. Nicholas Kim from HydroQual, Inc. provided initial versions of the ECOM code. References Bhushan, S., Blumberg, A.F., and N. Georgas (2010, this issue). “Comparison of the

NYHOPS hydrodynamic model Sea Surface Temperature (SST) predictions with satellite observations in the Hudson River tidal, estuarine, and coastal plume region.” Proceedings of the 11th International Conference in Estuarine and Coastal Modeling (ECM11). Spaulding, M.L. 4-6 November 2009, Seattle, WA.

Bhushan, S. (2009): “Validation of the NYHOPS hydrodynamic model SST predictions against in situ data and satellite observations in the Hudson River tidal, estuarine, and coastal plume region.” M.S. Thesis. Stevens Institute of Technology, Hoboken, NJ. 42p.

Blumberg, A.F., and G.L. Mellor (1987): “A description of a three-dimensional coastal ocean circulation model.” Three Dimensional Coastal Models, Coastal and Estuarine Sciences, 4. N. S. Heaps, editor. American Geophysical Union, Geophysical Monograph Board. 1-16.

Blumberg, A.F. and B. Galperin (1990): “On the summer circulation in New York Bight and contiguous estuarine waters.” Coastal and Estuarine Studies. Volume 38. R.T. Cheng (ed.). Residual Currents and Long Term Transport. Springer-Verlag, New York. 451-467.

Blumberg, A. F., L. A. Khan, and J. P. St. John (1999): “Three-Dimensional Hydrodynamic Model of New York Harbor Region.” Journal of Hydraulic Engineering. 125(8): 799-816.

25

Blumberg, A. F. and N. Georgas (2008): “Quantifying Uncertainty in Estuarine and Coastal Ocean Circulation Modeling.” Journal of Hydraulic Engineering. 134(4): 403-415.

Bruno, M.S., Blumberg, A.F., and T.O. Herrington (2006): “The urban ocean observatory - coastal ocean observations and forecasting in the New York Bight.” Journal of Marine Science and Environment. C4: 1-9.

Chao, Y.Y., Burroughs, L.D., and H.L. Tolman (2004): “Wave forecasting for the Western North Atlantic and Adjacent Waters.” Technical Procedures Bulletin No. 495. National Weather Service. NOAA. U.S. Department of Commerce. [online, accessed April 2009]: http://polar.ncep.noaa.gov/mmab/tpbs/operational.tpbs/tpb495/tpb495.htm.

Chawla, A., Cao, D., Gerald V., Spindler T., and H.L. Tolman (2007): “Operational implementation of a multi-grid wave forecasting system.” 10th International Workshop on wave hindcasting and forecasting and coastal hazard symposium. North Shore, Oahu, Hawaii, November 11-16, 2007. 12p. [online, accessed April 2009]: http://www.waveworkshop.org/10thWaves/Papers/Mww3_paper.pdf.

Chen, J., Shaffer, W., and Kim, S (1993): “A forecast model for extra-tropical storm surge.” Advances in HydroScience and Engineering. 1: 1437-1444.

Di Liberto, T. E. (2009): “Verification of a Storm Surge Modeling System for the New York City - Long Island Region.” M.S. Thesis. School of Marine and Atmospheric Sciences. Stony Brook University. Stony Brook, NY 11794-5000. 142 p.

Donelan, M.A. (1977): “A Simple Numerical Model for Wave and Wind Stress Application.” Report. National Water Research Institute. Burlington, Ontario, Canada, 28p.

Georgas, N. and A.F. Blumberg (2004): “The influence of Centrifugal and Coriolis Forces on the Circulation in a Curving Estuary.” Proceedings of the 8th International Conference on Estuarine and Coastal Modeling. Malcolm L. Spaulding, editor. American Society of Civil Engineers. Monterey, CA. 541-558.

Georgas, N., Blumberg A.F., and T.O. Herrington (2007): “An operational coastal wave model for New Jersey and Long Island waters.” Shore and Beach. Spring 2007. 75(2):30-35.

Georgas, N. and A.F. Blumberg (2008): “The New York Bight – Shelf Harbor Dynamic Study: Ocean Forecast Sensitivity to Forecasts of Atmospheric Forcing.” Final Report. ONR grant N00014-06-1-1027. [online, accessed December 2009]: http://www.stevens.edu/ses/documents/fileadmin/documents/pdf/NYBight_ONR08_Stevens.pdf

Georgas, N., Blumberg, A. F., Bruno, M. S. and D. S. Runnels (2009): “Marine Forecasting for the New York Urban Waters and Harbor Approaches: The design and automation of NYHOPS.” 3rd International Conference on Experiments / Process / System Modelling / Simulation & Optimization. Demos T. Tsahalis, Ph.D. July 11 2009. Athens, Greece. 1: 345-352.

26

Georgas, N. (2010). “Establishing Confidence in Marine Forecast Systems: The design of a high fidelity marine forecast model for the NY/NJ Harbor estuary and its adjoining coastal waters.” Ph.D. Dissertation. Stevens Institute of Technology, Hoboken, NJ, 07030. 295p.

Hess, K.W., Gross, T.F., Schmalz, R.A., Kelley, J.G.W., Aikman, F. III, Wei, E. and M.S. Vincent (2003). “NOS standards for evaluating operational nowcast and forecast hydrodynamic model systems.” NOAA Technical Report NOS CS 17, 48pp.

Kim, S.-C., Chen, J., and W.A. Shaffer (1996): “An operational forecast model for extratropical storm surges along the U.S. east coast.” Preprints Conference on Coastal Oceanic and Atmospheric Prediction. Atlanta, GA. American Meteorological Society. 281-286.

Mellor, G.L. and T. Yamada (1982): “Development of a turbulence closure model for geophysical fluid problems.” Review of Geophysics and Space Physics. 20: 851-875.

Mellor, G.L. and T. Ezer (1995): “Sea level variations induced by heating and cooling: An evaluation of the Boussinesq approximation in ocean models.” Journal of Geophysical Research. 100(C10): 20565-20577.

Mukai A.Y., J.J. Westerink, R.A. Luettich Jr., and D. Mark (2002): “Eastcoast 2001: A tidal constituent database for the western North Atlantic, Gulf of Mexico and Caribbean Sea.” Final Report. US Army Corp of Engineers Research and Development Center, Coastal and Hydraulics Laboratory, Technical Report, ERDC/CHL TR-02-24 A337804, September 2002, 201p.

NDBC (2003): “Handbook of Automated Data Quality Control Checks and Procedures of the National Data Buoy Center.” NOAA/NDBC. Stennis Space Center. February 2003.

NOAA (2002): “NOS Experimental Nowcast/Forecast System for the Port of New York/New Jersey (NYEFS): Requirements, Overview, and Skill Assessment.” NOAA Technical Report NOS OSC 15. NOAA/NOS/CSDL/Silver Spring, Maryland. October 2002. 51p.

Patchen, R. (2008): “Establishment of a Delaware Bay Model Evaluation Environment.” Proceedings of the 10th International Conference in Estuarine and Coastal Modeling (ECM10). Malcolm L. Spaulding, editor. 5-7 November 2007. Newport, RI. 783-818.

Schwab, D.J., M.A. Donelan, J.R. Bennett and P.C. Liu (1984): “Application of a simple numerical wave prediction model to Lake Erie.” Journal of Geophysical Research. 89: 3586-3592.

Willmott, C..J. (1981): “On the validation of models.” Physical Geography. 2: 184-194.

Zhang, A., Wenjing, F. and J. Fengying (2007): “The Standards for Skill Assessment of Operational Marine Forecast System.” Chinese Journal of Oceanology and Limnology. 25(1): 27-35.

Zhang, A. and E. Wei (2008): “Delaware River and Bay Hydrodynamic Simulations with FVCOM.” Proceedings of the 10th International Conference in Estuarine and Coastal Modeling. Malcolm L. Spaulding, editor. American Society of Civil Engineers. November 5-7, 2007. Newport, RI. 18-37.