Bay/Delta & Tributaries Cooperative Data Management System

Background

How The BDAT/CEDEN Distributed Systems Work

Local and Regional Data Management (Participation)

Strategic and Tactical Benefits

Improving Access to Data

Answer Questions

Participants

Background/Distributed Data Management System

Background/Distributed Data Management System

Data Types

ComprehensiveDatabase

Adult Striped Bass Population Study

Delta Smelt Spawning Season Monitoring

Juvenile Salmon Survival

Midwater Trawl Survey

Mallard SloughMonitoring Program

Predator Dynamics

Resident Fish Survey

North Bay AqueductEntrainment Monitoring

Neomysis/ZooplanktonSurvey

CVPIA CampJuvenile Data

Contra Costa CanalIntake Entrainment Study

Delta AgriculturalDiversion Evaluation

Delta Fish FacilitiesSalvage Monitoring Program

Juvenile SalmonMigration

Hydrodynamic Monitoring

Discrete WaterQuality Sampling(D1485)

Monitoring ofDelta Smelt Abundance andDistribution

San FranciscoBay Monitoring

Research on OtherLife Stages ofDelta Smelt

South Delta TemporaryBarriers Project FisheryEvaluation

Splittail Studies

Summer TownetSurvey

Suisun Marsh SalinityControl Structure(SMSCG) Adult SalmonMigration Evaluation

Suisun Marsh

San Joaquin RiverDissolved Oxygen Monitoring (D1485)

Sturgeon Study

Striped Bass Egg and Larva Survey

Southern Delta EntrainmentMonitoring

TDB

TDB

TDB

TDB

TDB

TDB

TDB

TDB

TDBTDB

TDB

TDBTDB TDB

TDBTDB

TDB

TDB

TDB

TDB

TDB

TDB

TDBTDB

TDB

TDB

TDB

TDB

TDB

TDB

KEY: Data Provider(MS Access Client)

ODBC/FTP Link

QA/QC

Informix TemporaryDatabase (TDB)

QA/QCComprehensive Informix Database

Local Client Databases

ODBC/FTP Link

QA/QC

TransactionDatabase (TDB)

QA/QC

Data Provider(MS Access Client)

Node Database

Cooperative Data Management System

Background-BDAT

Started with IEPExpanded and Used By Other GroupsBecame a Permanent Program in DWR as Part of

the SWP in 2007Still waiting for staff allocation approvalBDAT will now focus on SWP data user needs

Providing necessary maintenance and as resources are available;

Include additional data sets at required frequencySet up additional and revised data delivery systems

Background-BDAT Forecasted simulations in support of O&M operating decisions Recent historical simulations (fingerprinting) and weekly water quality

forecasts in support of O&M operating decisions and in support of the Municipal Water Quality Investigations Program’s weekly Real-Time Data and Forecasting reports sent out to the water contractors

Modeling and data analysis in support of the recent SWRCB Cease and Desist Hearings

Verify continued compliance with the objectives set forth in the State Water Resources Control Board’s Decision 1641 and Suisun Marsh agreements.

Meet requirements for Water Level and Water Quality response plans for Joint Points of Diversion (JPOD) actions.

Determine carriage water costs for completed water transfers and to estimate those costs for planned transfers

Provide data to staff conducting analysis for DAT who help perform decision support for SWP operations

BDAT Background SWP Needs Calibration and Validation of DSM2 Hydrodynamic modeling of recent historical conditions in support of the South Delta

Temporary Barriers program Forecasts of Water Stage in support of the South Delta Temporary Barriers program Particle Tracking Forecast Modeling in support of the Delta Smelt Workgroup Particle Tracking Modeling of historical conditions in support of the Pelagic Organism

Decline research Modeling of hydrodynamics, EC, organic carbon, and taste and odor for the Jones Tract

Levee Failure SWC analysis Aggregating and gaining access to data from many sources for analysis Other future modeling needs include data in the following areas:

Delta Island volumes – to help in flood modeling Other water quality constituents in addition to EC including

Organic Carbon Dissolved Oxygen Temperature

BDAT Background SWP Support

BDAT will continue to Support SWP funded ProgramsThe Interagency Ecological Program (IEP), Bay-Delta Office, Suisun Marsh Compliance and Monitoring, and Operations Control Office

CEDENCooperative Data Management

System

MLMLNode

SM

NodeNDNode

BondersonNode

USGS CSUSNode

EMPNode

OtherNode

BDATNode

SWAMP

AB 1747Compliant

Multiple ClientsOther Standards

ClientAB 1747Compliant

ClientAB 1747Compliant

Web ServicesEDSC Standards

Cal/EPACDX Node

Exchange Network

Multiple ClientsOther Standards

C

E

D

E

N

Enterprise Applications

CEDENCooperative Data Management

System

CEDEN EPA/USGS Coordination Summary

USGS and EPA have begun coordination on developing common web services to serve data out of STORET and NWISServices will follow the WQX SchemaEPA’s services are planned for release Summer ‘07USGS services are planned for release Fall ’07

Services will include Summary level information, Monitoring Stations, and Results

For current access to these data and information on WQX visit:www.epa.gov/storethttp://waterdata.usgs.gov/nwis

CEDENBackground

Easy to implement by other partnersArchitecture-Federated is distributedCan support many types of institutional

participation

Saves on IT budget Saves on monitoring budget Greatly improves on the ability to

perform key business functions Integrates standards, metadata and

architecture Federated distributed approach

CEDENWorking with CEDEN

New Precedence For Interagency Data Systems IT Architecture: CEDEN, BDAT, EPA Exchange

Network … Metadata: CERES Standards: SWAMP Legislation-SB 1070, AB 1404…

DWR will work with the SWRCB on Interagency data systems

Requires Executive leadership and cooperation Dedication to following the software engineering

principals and making the process subservient to water resources planning in California

Executive sponsorship for the project

Working with CEDEN

Cooperative Data Management System

Cooperative Data Management System

Cooperative Data Management System

Cooperative Data Management System

CEDENCooperative Data Management System

CEDENCooperative Data Management System

CEDENCooperative Data Management System

Future Plans BDAT

GET STAFFED Finish Web Services Implementation with the

EPA (OWQX) EPA Data USGS Data NOAA Data?

Complete List of Initial Tasks Provided by SWP Data Users

Work with CEDEN

BDAT Task List• Remove IEP web site.

– Copy/merge IEP data into BDAT– Copy/move IEP interface to BDAT (as-is or upgrade)

• Output options are DSS, ASCII and Excel• Time series graphing capability

– Client/server? Who is going to do it?• Use RKI standard naming convention• Missing data uses same convention• Both time series and grab sample data

– All water quality parameters– Biologics?

• On-line mechanism to notify BDAT of:

– Incorrect data– Missing data– Missing data sources– General comments/feedback

• Metadata:

– Clear indication of original sources of data– Description of QA/QC applied to data, if any– Standardized format (for time-series data)– Summary – Extra:

• Links to QAPP and other original information about the monitoring program.• Photos, other ancillary and anecdotal data

• Data Searches:

– Map based– Text filtered based

• Search by location, collector (agency name), parameter (flow, stage, ec, …), methods, data category, temporal using multiple date ranges, time interval, source (model, fill, observed, etc), average type (mean max min, etc)

– Combine map and parameter-filter searches• GIS Data• Include BDAT in DSM2 Users Group; provide category in DSM2 Users Group Forum.

Future Plans CEDEN

Propagation of Data Standards SWRCBs requirement to be SWAMP comparable

for grant recipients Other Departments?

Grants Data SB 1070 and other legislation TMDL Data Sets Metadata

Catalog Data in CERES

CEDEN/BDATComponents of Data & Information Sharing

Systems

• Data Sharing• &• Management

CEDEN/BDATComponents of Data & Information Sharing

Systems

• Data Sharing• &• Management

Conclusion

BDAT & CEDEN are distributed data systems that will continue to provide more access to ambient monitoring data.

These systems subscribe to a distributed architecture, standards and metadata

We look forward to your input as these systems continue to develop