Jeff de La Beaujardière, PhD

Carmel Ortiz

NOAA IOOS Program Office

Integrated Ocean Observing System (IOOS)

• IOOS program overview• IOOS high-level architecture

– Observing Systems– Data Assembly Centers– Data Access Services

• Supported Standards & Formats

– Utility Services– Modeling and Analysis Tools

• IOOS - NextGen WIDB linkages• Schedule & milestones

Outline

2

• IOOS is a system of systems, including:– data assembly centers/data providers– archives– regional associations– service providers

• Add one or more standardized services at (or near) each system– cf. NextGen service adapter alternative approaches

• Implement shared registry/catalog/portal components• Leverage existing infrastructure

– public internet, WMO GTS, NOAAnet?

IOOS is not building a new system

3

8

5

6

11

2

4

10

3

7

9

1

IOOS Partners: Regional Associations (RAs)IOOS

Regional Componen

tA network of 11 regional coastal ocean observing

systems that meet national and regional needs for local ocean

observations, data management, and

modeling

1 national partnership providing sensor

validation/verification1. Alaska Ocean Observing Systems (AOOS)

2. Caribbean Regional Association (CaRA)

3. Central and Northern California Coastal Ocean Observing System (CeNCOOS)

4. Gulf Coastal Ocean Observing System (GCOOS)

5. Great Lakes Observing System (GLOS)

6. Mid-Atlantic Coastal Ocean Observing System Regional Association (MACOORA)

7. Northwest Association of Networked Ocean Observing Systems (NANOOS)8. Northeast Regional Association of Coastal Ocean Observing Systems

(NERACOOS) 9. Pacific Islands Ocean Observing System (PacIOOS)10. Southern California Coastal Ocean Observing System (SCCOOS)11. Southeast Coastal Ocean Observing System Regional Association

(SECOORA)12. Alliance for Coastal Technologies (ACT) {Sensor V & V}

12

Meeting National missions through:– Expanded observations and

modeling capacity

– Connections to users and stakeholders

– Implementation of national data standards

– Sensor validation/verification

– Products transitioned to other regions and to National operations

IOOS Partners: Federal Agencies

5

• Legislation: Integrated Coastal and Ocean Observation System Act• Directs President to establish a National Integrated

Coastal and Ocean Observation System• Identifies NOAA as Lead Federal Agency • Authorizes appropriation of “such sums as are

necessary” through 2013

• Agency partners: closest ties with• NSF (OOI/CI)

• USACE (WL data)

• USGS (WaterML 2.0)

• Navy (METOC)

• EPA (NWQMN)

• IOOS is part of a global framework• We are starting to establish technical linkages to ensure

interoperability– Australia: IMOS (Integrated Marine Observing System)

– Europe: MyOcean– WMO: WMO Information System (desired)

IOOS Partners: International

6

Global Ocean Observing System

GEOSSGOOSIOOS

DMACData

Managementand

Communications

Data Management

Modeling &Analysis

WebPortal

ClientLibraries

AnalysisTools

SystemMonitor

ForecastModels

ObservingSystems

MooredBuoys

FixedStations

SamplesMovingPlatforms

Radar Satellite

DataAssemblyCenters

Real-Time BiologicalArchivesModelOutputs

UtilityServices Visualization

Conversion DataIntegration

Catalog

Registry

Computational Viewpoint from Reference Model for Open Distributed Processing (RM-ODP)

Component Types Needed for IOOS

DataAccessServices

Subscription AlertPullIndividualFeatures

RegularGrids

UnstructuredGrids

UnderseaImagery

…

…

…

…

…

ServiceGateway

Maps &Images

Workflows

Notification

Surveys

GIS

Framework

Observing Systems

8

• Direct support for– in situ sensors at regional associations: buoys, fixed stations, gliders

(AUV: autonomous underwater vehicle)

– Real-time data from RAs onto WMO GTS via NDBC– Regional-scale models– Coastal high-frequency radar (HFR)

• Access to observations from– NWS Met/Ocean buoys– DART (Deep Ocean Assessment & Reporting of Tsunamis)– TAO (Tropical Atmosphere/Ocean)– NWLON (National Water Level Observing Network)– PORTS (Physical Oceanographic Real-Time System)– MODIS ocean color– HFR surface currents

Observing Systems

IOOS Core Variables

9

• List could be expanded based on identified needs...

Observing Systems

IOOS DACs at NOAA

Satellite Ocean Color (Aqua MODIS)

National Environmental Satellite, Data, and Information Service (NESDIS)CoastWatch

National Water Level Observation Network

(NWLON)

Physical Oceanographic Real-

Time System (PORTS)

National Ocean Service (NOS)Center for Operational Oceanographic Products and Services (CO-OPS)

NWSMet/Oc Buoys

IOOS Regional stations

Tropical Atmosphere

Ocean (TAO) Buoys

Deep-Ocean Assessment and

Reporting of Tsunamis (DART)

Surface Currents from High-

Frequency Radar (HFR)

National Weather Service (NWS)National Data Buoy Center (NDBC)

228172 55 47

10

Data Assembly Centers

Data Assembly Centers

IOOS Regional DACs also coming on line

NDBC CO-OPS CoastWatch

NANOOS NERACOOS PacIOOS

CeNCOOSSECOORA AOOS GCOOS

SCCOOS

IOOS Regional Associations - observations and model outputs

NOAA Data Providers

11

Data Access Services

12

In-situ features (buoys, piers, floats, ships, ...)

Regular Grids (radar, satellite, model outputs)

OGC Sensor Observation Service

(SOS)

OpenDAP and/orOGC Web Coverage

Service (WCS)

XML based on OGC Observations and

Measurements (O&M)

NetCDF using Climate and Forecast (CF)

conventions

Images of data(all types)

OGC Web Map Service (WMS)

GeoTIFF, PNG etc.-possibly with

standardized styles

[*OGC = Open Geospatial Consortium]

Data Type Web Service Encoding

Data Access Services

• Sampling feature = discrete location(s) of measurements– Point, Vertical or Horizontal Profile, Trajectory (e.g., ship track)

– …and Time Series or Collections thereof

• GetCapabilities operation: “table of contents”

• GetObservation operation: XML data filecontaining observation values for desired:– Variable(s) of interest– Bounding box

• Or perhaps named geographic feature of interest

• Or perhaps a single sensor

– Time

• DescribeSensor operation:SensorML providing detailed information about a specific sensor(or platform or group of sensors)

SOS for In-Situ DataOGC Sensor Observation Service v1.0

13

Data Access Services

SOS GetObservation ResponseXML Encoding of In-Situ Data

14

XMLExtensible Markup Language

Generic method for structuring text data

OGC GMLGeography Markup Language

XML that can represent any geospatial feature

OGC O&MObservations and Measurements Model

GML that describes the act of measuring real-world phenomena and the result of the measurement

specializedby

specializedby

IOOS v0.6.1 schemaSpecific encoding used by IOOS SOS serversXSD, Examples, XSLT @ http://ioos.gov/dif/

resultingin

Data Access Services

• SOS and O&M specs are fairly general– Need community specialization/restriction

• IOOS adopting, defining or researching practices:– URIs for sensors, stations, networks, CRS, phenomenon

names– HTTP GET request encoding– SensorML metadata– Observation Offerings– Other formats for SOS

• NetCDF/CF• ASCII CSV• KML+JSON (Javascript Object Notation)

– IOOS GML schema evaluation pending

SOS Profile/Best Practices

15

Data Access Services

• Services requested from an OPeNDAP server are specified in a suffix appended to the URL. Depending on the suffix supplied, the server will return one of these response types:

• Data Attribute (.das suffix)– Text file describing the attributes of data quantities in dataset.

• Data Descriptor (.dds)– Text file describing the structure of the variables in the dataset.

• OPeNDAP Data (.dods)– Actual data as binary MIME-typed file.– Constraints can be appended to select a subset of the data.

• ASCII Data (.asc, .ascii)– ASCII representation of the requested data.

• WWW Interface (.html)– HTML form that can be used to construct a data URL.

• Information (.info)– HTML information about the server and dataset.

OPeNDAP for Gridded DataOpen-source Project for a Network Data Access Protocol

16Source: OpenDAP User Guide 1.14

Data Access Services

27 1828 1828 4590 4523 5360 2874 7135 2662 4977

9676 2772 4076 6303 5354 7594 5713 8217 8525 1664

9218 1741 3596 6290 4357 2900 3342 9526 595 6307

2338 2988 753 1952 5101 9011 5738 3418 7930 7021

2447 6146 668 822 6480 168 4774 1185 3742 3454

695 5170 2761 8386 626 1331 3845 8300 752 449

2007 932 8709 1274 4374 7047 2306 9697 7209 3101

6574 6377 2111 2523 8978 4425 569 5369 6770 7854

9879 3163 6889 2300 9879 3127 7361 7821 5424 9992

1936 6803 3182 5288 6939 8496 4651 582 939 2398

1173 123 8197 684 1614 397 198 3767 9320 6832

3287 8250 9819 4558 1530 1756 7173 6133 2069 8112

3515 9888 8519 3458 727 3866 7385 8942 2879 2284

6104 8419 8444 3634 6324 4968 4875 6023 3624 8270

2353 436 9941 8491 4631 4093 4317 3814 3640 5462

167 6839 6424 3781 4059 2714 5635 4906 1303 1072

7041 7189 8610 6873 9696 5521 2671 5468 8957 350

3210 6817 121 56 2788 235 1930 3322 4745 158

5036 6041 6997 3297 2508 8687 6966 4035 5570 7162

7871 3419 5124 6652 103 592 1236 6771 9432 5278

WCS for Gridded DataOGC Web Coverage Service v1.1

• Coverage ~ array of gridded data values– (simplified viewpoint for this discussion – coverage can be more complex)

• GetCapabilities operation: “table of contents”

• GetCoverage operation: data filecontaining header and array(s) ofnumbers customized for:– Variable of interest– User-specified bounding box– User-specified time– File format (e.g, NetCDF,

HDF, floating-point TIFF)

• DescribeCoverage operation:metadata about a specific dataset

(x1,y1)

(x2,y2)

17

Data Access Services

• “Map” = georeferenced picture of data

• GetCapabilities operation: “table of contents” in standardized format

• GetMap operation:image of data customizedaccording to:– Variable(s) of interest– User-specified bounding box– User-specified time– Image size– File format (e.g., PNG,

GetTIFF, JPEG, GIF)

WMS for Images of DataOGC Web Map Service v1.3

18

← width →

← h

eigh

t →

(x1,y1)

(x2,y2)

boundaries water bodiestopography

Layers =

Data Access Services

Data Customers and Sources

CO-OPS

SOS

CoastWatchDAP

CeNCOOSSOS

SECOORASOS DAP WMS

AOOSSOS DAP WMS

NANOOSDAP WMS

NERACOOSDAP WMS

PacIOOSDAP WMS

SCCOOSSOS DAP WMS

GCOOSSOS DAP WMS

NDBC

SOS DAP WMS

Coastal Inundation Harmful Algal Blooms Hurricane IntensityIntegrated Ecosystem

Assessments

Google OceansTsunami Event DB

othercustomers

t.b.d...

Public Internet

otherproviders

t.b.d...

SOS

SOS

DAP DAP DAP

SOS

19

Modeling & Analysis

Additional Services

SOS DAP WMS

SOS DAP WMSSOS DAP WMS

Public Internet

20

IOOS Portal

Catalog

Registry

Web Interface

SOS DAP WMS

ServiceGateway

SOS DAP WMS

VisualizationFormat

Conversion

Utility Services

Scalable translation/visualization service in the cloud (Amazon EC2)

NSF OOI/CI Collaboration

21

SOS

DAP

etc

ERDDAP

CSV

HTML

KML

NetCDF

Figure fromM. Arrott et al.,

Proc. MTS/IEEEOceans 2009

IOOS – NextGen WIDB Linkages

• Does IOOS have data of interest to NextGen/FAA/WIDB Consumers?

– IOOS-supported data was instrumental to the safe rescue of passengers and crew on US Air Flight 1549, which crashed into the Hudson River in January, 2009

– Stevens Institute of Technology’s NYHOPS - New York Harbor Observing and Prediction System (www.stevens.edu/maritimeforecast/) – was used to prepare detailed summary of present water conditions surrounding the crash site and to forecast conditions for the next 48 hours

– Data about swift river currents informed decisions to deploy rescue assets downstream, not upstream and to guide the plane eastward to the Battery area for salvage operations

“Present In the Moment” - Stevens Efforts Regarding the Crash of US Airways Flight 1549. Alan F. Blumberg, Center for Maritime Systems, Stevens Institute of Technology, Castle Point on Hudson . January 19, 2009

Most likely scenario for IOOS sources

IOOS – NextGen WIDB Linkages

• How does IOOS IT Infrastructure integrate with WIDB? – Overall architecture/infrastructure is compatible– IOOS systems would appear as SOA WX Cube sources

• Are supported standards and formats compatible? – Both have basis in OGC standards– Devil will be in the details, but translators or converters can be

employed to bridge the gaps

Schedule and Milestones

Major Capabilities - “Running Stuff” Status

Initial set of standards adopted, encodings and content standards adapted/customized

Standardized data access services implemented atoperational data providers (NDBC, CO-OPS, CoastWatch, etc)

SOS: Point, Profile, Time Series, CollectionsOpenDAP/WCS: Regular Grids

Four active customer projects; NWS customer using IOOS data in operational scenarios

Work In Progress Completion

SOS, WMS, WCS submitted as IOOS standardsNeed to document SOS Profile for ocean observations

Early FY10

Implementing detailed metadata for sensors & platforms Early FY10

Event-specific “features of interest" Early FY10

Expansion towards IOOS/National DMACService types (Registry, Catalog, …)Data types (trajectory, unstructured grid, imagery)Data providers (biological, etc), data customers (climate…)

End FY10

See http://ioos.gov/dif/

Contacts: Jeff.deLaBeaujardiere@noaa.gov

Carmel.Ortiz@noaa.gov

Thank You!

25

Backup Slides

• Adopt open standards & practices

• Federated, service-oriented architecture– Data made accessible via OGC or DAP services

• Not SOAP/WSDL services

– Data stays with data providers– Service adapters for existing systems

Core Principles

27

IOOS DMAC

Cross-cutting Considerations

Observing Systems

Data Assembly Ctrs

Data Access Services

Utility Services

Modeling & Analysis

Cross-cutting Considerations • Governance/Policies• IT Security/Info Assurance• Configuration Management• Metadata Input & Access• Data Quality• Service-Level Agreements• Capability Maturity Levels

29

Federated, Service-Oriented Architecture

IOOS Portal

Catalog

Registry

Web Interface

UtilityServices

Thematic Portal

Registry

Catalog

Obs Syst. or Model

data provider data provider

Data Center

data access service

Obs Syst. or Model

Data Center

data access service

Regional Portal

data access service

RegionalObs Syst. or Model

Registry Catalog

Data Center

data access service

Data Center

data access service

29

IOOS Data and Metadata Types

30

Ocean Properties (Physical, biological, chemical)

Sampling Feature Types (Point, Profile, Trajectory, Reg Grid, Unstructured Grid)

Data Encoding Conventions (GML, KML, O&M, SWEC, CSML, NetCDF/CF)

Collection Types (Time Series, Multi-Station Obs)

Information Viewpoint from Reference Model for Open Distributed Processing (RM-ODP)

Sensor/Platform Metadata (SensorML)

QA/QC Metadata (QARTODS/Q20)

Discovery Metadata (FGDC, ISO 19115/19139)

Service Metadata (OWS Capabilities XML, ISO 19119)

Controlled Vocabularies (CF, MMI, OGC, GCMD, URNs)

THREDDS

IOOS Data and Metadata

31

Properties

Datasets

Information Viewpoint from Reference Model for Open Distributed Processing (RM-ODP)

Registries

QA/QCMetadataDiscovery

Metadata

Services

ControlledVocabularies

Water

Procedures SensorML

Measurements

Catalogs

physical

chemicalbiological

ProcedureMetadata

ServiceMetadata

Data Formats

EPSG

MMI

GCMDCF

GTS

DAP

SOS

ISO 19119

Cap.XML

FGDCISO 19115

has that are

observed by

thatmake

grouped in

accessedthrough

documentedwith

ISO 19115such as describedby

that include

harvested by

published indescribed by

that use

GML/O&M BUFR

CS/W

Sensors

Lab analysisSurveys

KMLNetCDF/CF

and

• Each station (buoy, fixed sensor package) is a separate Offering from the SOS– Allows requests for data from 1 station at a time

• Multi-station Offerings:– “All stations” Offering

• User specifies bounding box instead of station ID

– Soon: program-specific or event-specific Offerings• E.g., “all Hurricane Katrina data”

– Maybe: phenomenon-specific Offerings • E.g., “all temperature data”

• Offering includes ID and English name– gml:name = ID– gml:description = name

• May replace multiple sensor IDs per offering with single station ID

IOOS Practice: Observation Offerings

32

• Using URNs for IDs of sensors, stations, networks (URN = Uniform Resource Name)

• Following “OGC Definition URN” practice– urn:namespace:def:objectType:authority:[version]:code[:otherParams]

• Examples:– urn:x-noaa:def:network:noaa.nws.ndbc::all– urn:x-noaa:def:station:noaa.nws.ndbc::21418– urn:x-noaa:def:sensor:noaa.nws.ndbc::21418:tsunameter0

• Also using URNs for EPGS CRS identifiers• Using URLs for phenomenon names

– Adopting MMI/CF URLs: http://mmisw.org/ont/cf/parameter/sea_water_temperature

– Allow trailing component as abbreviation (sea_water_temperature)

IOOS Practice: Identifiers

33

• Supporting both HTTP POST requests and HTTP GET– HTTP POST defined in spec, GET left out– Mostly following Oceans IE Best Practice for GET

• For Bounding Box, using FOI that could be a BBOX or (in future) a named FOI: featureofinterest=BBOX:minlon,minlat,maxlon,maxlat

IOOS Practice: GetObservation Request

34

IOOS Metadata Linkage Model(Sensors, Stations, Networks, Datasets and Services)

35

Sensor

Sensor Type

instances

Network

type

stationQC procedures

Procedure description

Station

StationTypeinstances

type

group

sensors

members

phenomenonDataset

applicable to

Obs system

ISO 19115

SensorML

SensorML

Service

network

OGC CapXML

Online resource

WSDLISO 19119

station

sensor

THREDDS

(make/model)

IOOS Data Model for Time Series at a Collection of Points

• Collection– Station 1

• Time 1– quantity 1– quantity 2

• Time 2– quantity 1– quantity 2

– Station 2• Time 1

– quantity 1– quantity 2

• Time 2– quantity 1– quantity 2

36

CollectionBounding boxTime range# of stations

Station# of times

Date/TimeTime-dependent metadata# of quantities

QuantityNameUnitsValue

1..*

1..*

1..*

Station MetadataID (URN)NameLocationProcedure IDsStatic metadata

1

Scalable translation service (NSF OOI/CI)

XSLT templates

Format Conversion Tools

37

SensorObservation

Service

CSV

CSVXSLT

HTML

Table of Contents

Data Values

XSLT

KML

SOS

DAP

etc

ERDDAP

CSV

HTML

KML

Spreadsheet

Virtual Globe

Science App

Browser

NetCDF

KML