overview of cuahsi his version 1.1
DESCRIPTION
Overview of CUAHSI HIS Version 1.1. David R. Maidment Director, Center for Research in Water Resources University of Texas at Austin. CUAHSI Biennial Science and Engineering Colloqium Boulder, CO, July 16, 2008. HIS Workshop Overview. Morning: 10AM – 12 noon - PowerPoint PPT PresentationTRANSCRIPT
Overview of CUAHSI HIS Version 1.1
David R. MaidmentDirector, Center for Research in Water Resources
University of Texas at Austin
CUAHSI Biennial Science and Engineering ColloqiumBoulder, CO, July 16, 2008
HIS Workshop Overview
• Morning: 10AM – 12 noon– Overview of CUAHSI HIS and how to use it
• Lunch 12 – 1PM• Afternoon 1PM – 3PM– Building a water data service
• Break 3PM – 3:30PM• Afternoon 3:30 – 4:30PM– Discussion of future plans
HIS Team and Collaborators• University of Texas at Austin – David Maidment, Tim Whiteaker,
Ernest To, Bryan Enslein, Kate Marney• San Diego Supercomputer Center – Ilya Zaslavsky, David Valentine,
Tom Whitenack• Utah State University – David Tarboton, Jeff Horsburgh, Kim
Schreuders, Justin Berger• Drexel University – Michael Piasecki, Yoori Choi• University of South Carolina – Jon Goodall, Tony Castronova• CUAHSI Program Office – Rick Hooper, David Kirschtel, Conrad
Matiuk• WATERS Network – Testbed Data Managers• HIS Standing Committee • USGS – Bob Hirsch, David Briar, Scott McFarlane• NCDC – Rich Baldwin
HIS Overview Report
• Summarizes the conceptual framework, methodology, and application tools for HIS version 1.1
• Shows how to develop and publish a CUAHSI Water Data Service
• Available at:
http://his.cuahsi.org/documents/HISOverview.pdf
Rainfall & SnowWater quantity
and quality
Remote sensing
Water Data
Modeling Meteorology
Soil water
HIS Desktop
• Analytical functions controlled interactively by a human operator to generate information products– Observations Data Model
and Tools– Excel, ArcGIS, Matlab,
Google Earth– Your model or analysis
Services-Oriented Architecture for Water Data
• Links geographically distributed information servers through internet
• Web Services Description Language (WSDL from W3C)
• We designed WaterML as a web services language for water data
• Functions for computer to computer interaction
HIS Servers in the WATERS Network
HIS Central at San Diego Supercomputer Center
Web Services
HIS Central
National Water Metadata
Catalog
WaterML
Get Data
Get Metadata
National Water Metadata
Catalog
Synthesis and communication of the nation’s water data http://his.cuahsi.org
Hydroseek WaterML
Government Water Data Academic Water Data
CUAHSI Point Observation Data Services
1. Data Loading– Put data into the CUAHSI Observations Data
Model
2. Data Publishing– Provide web services access to the data
3. Data Indexing– Summarize the data in a centralized cataloging
system
CUAHSI Point Observation Data Services
1. Data Loading– Put data into the CUAHSI Observations Data
Model
2. Data Publishing– Provide web services access to the data
3. Data Indexing– Summarize the data in a centralized cataloging
system
Data Values – indexed by “What-where-when”
Space, S
Time, T
Variables, V
s
t
Vi
vi (s,t)
“Where”
“What”
“When”A data value
Data Values Table
Space, S
Time, T
Variables, V
s
t
Vi
vi (s,t)
Observations Data Model
Horsburgh, J. S., D. G. Tarboton, D. R. Maidment and I. Zaslavsky, (2008), "A Relational Model for Environmental and Water Resources Data," Water Resour. Res., 44: W05406, doi:10.1029/2007WR006392.
CUAHSI Point Observation Data Services
1. Data Loading– Put data into the CUAHSI Observations Data
Model
2. Data Publishing– Provide web services access to the data
3. Data Indexing– Summarize the data in a centralized cataloging
system
Point Observations Information Model
Data Source
Network
Sites
Variables
Values
{Value, Time, Metadata}
Utah State Univ
Little Bear River
Little Bear River at Mendon Rd
Dissolved Oxygen
9.78 mg/L, 1 October 2007, 5PM
• A data source operates an observation network• A network is a set of observation sites• A site is a point location where one or more variables are measured• A variable is a property describing the flow or quality of water• A value is an observation of a variable at a particular time• A metadata quantity provides additional information about the value
GetSites
GetSiteInfo
GetVariableInfo
GetValues
Locations
Variable Codes
Date Ranges
WaterML and WaterOneFlow
GetSiteInfoGetVariableInfoGetValues
WaterOneFlowWeb Service
Client
Penn State
Utah StateNWIS
DataRepositories
Data
DataData
EXTRACTTRANSFORMLOAD
WaterML
WaterML is an XML language for communicating water dataWaterOneFlow is a set of web services based on WaterML
WaterOneFlow• Set of query functions • Returns data in WaterML
Texas Water Data Services
Using CUAHSI technology for state and local data sources (using state funding)
Ingest Data From Different Sources
Transform Data into Uniform Format with SSIS Scripts
Load Newly Formatted Data into ODM Tables in MS SQL/Server
Wrap ODM with WaterML Web Services for Online Publication
Texas Parks and Wildlife Dept
Texas Water Development Board
Texas Instream Flow Program
Publishing an ODM Water Data Service
TPWD ODM
TWDB ODM
TIFP ODM
Observations Data Model (ODM)
WaterML
TCOONMETADATA
ODM
TCOONDataValues
WaterML
Metadata From:ODM Database in Austin
TCOON Web Site in Corpus Christi
TCOONWater Data
Service
Publishing a Hybrid Water Data Service TCOON Metadata are
Transferred from XML to the ODM
Web Services can both Query the ODM for Metadata and use a Web Scraper for Data Values
Calling the WSDL Returns Metadata and Data Values as if from the same Database
GetSitesGetSiteInfoGetVariableInfo
Get Values from:
http://his.crwr.utexas.edu/tcoonts/tcoon.asmx?WSDL
CUAHSI Point Observation Data Services
1. Data Loading– Put data into the CUAHSI Observations Data
Model
2. Data Publishing– Provide web services access to the data
3. Data Indexing– Summarize the data in a centralized cataloging
system
Data Series – Metadata description
Space
Variable, Vi
Site, Sj
End Date Time, t2
Begin Date Time, t1
Time
Variables
Count, C
There are C measurements of Variable Vi at Site Sj from time t1 to time t2
Series Catalog
Space
Variable, Vi
Site, Sj
End Date Time, t2
Begin Date Time, t1
Time
Variables
Count, C
Vi
Sj
t2
t1
C
CUAHSI National Water Metadata CatalogIndexes:• 50 observation networks• 1.75 million sites• 8.38 million time series• 342 million data values
NWIS
STORET
TCEQ
• Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them
Data Searching
NWIS
NARR
NAWQANAM-12
request
request
request
request
request
requestrequest
request
request
return
return
return
return
return
returnreturn
return
return
Searching each data source separately
Michael PiaseckiDrexel University
Semantic Mediation Searching all data sources collectively
NWIS
NAWQA
NARR
generic request
GetValues
GetValues
GetValues
GetValues
GetValues
GetValuesGetValues
GetValues
GetValues HODM
Michael PiaseckiDrexel University
Hydroseekhttp://www.hydroseek.org
Supports search by location and type of data across multiple observation networks including NWIS and Storet
Bora Beran, Drexel
HydroTaggerOntology: A hierarchy of concepts
Each Variable in your data is connected to a corresponding Concept
Water Quality in Moreton Bay, Brisbane, Australia (Jane Hunter)
National Water Metadata
Catalog
Synthesis and communication of the nation’s water data http://his.cuahsi.org
Hydroseek WaterML
Government Water Data Academic Water Data
Accomplishments
• Observations Data Model (ODM) is robust;• WaterOneFlow web services provide reliable
access to ODM data;• WaterML is a common language for water
observations data from academic and government sources
• National Water Metadata Catalog is the most comprehensive index of the nation’s water observations presently existing.
Limitations
• Focus on observations data measured as time series at fixed point locations;– Needs adaptation for moving sensors, transects,
one-time data collections and field surveys;• Need to work more on– Coverages for weather, climate and remote
sensing– Linking data and models– Linking geographic features WaterML
Observations
Geography
Models
Coverages