integrated air quality information system: challenges posed by key community members
DESCRIPTION
Integrated Air Quality Information System: Challenges Posed by Key Community Members. EPA Rich Scheffe, EPA OAQPS Steve Young, EPA OEI Terry Keating, EPA ORD NASA L. Friedl, K. Fontaine, NASA HQ Frank Lindsey, NASA HQ WMO/IGOS Len Barrie, WMO GEOSS. Scheffe Challenge. - PowerPoint PPT PresentationTRANSCRIPT
Integrated Air Quality Information System: Challenges Posed by Key Community Members
EPA• Rich Scheffe, EPA OAQPS• Steve Young, EPA OEI• Terry Keating, EPA ORD
NASA
• L. Friedl, K. Fontaine, NASA HQ• Frank Lindsey, NASA HQ
WMO/IGOS• Len Barrie, WMO
GEOSS
Scheffe Challenge
GEOSS
Eco-informatics
Accountability/indicatorsSIPs, nat.rules
designations
PHASE
PM research
Risk/exposureassessments
AQ forecastingPrograms
NAAQS setting
EPANOAA
NASA
NPS
USDA
DOE
PrivateSector
States/TribesRPO’s/Interstate
Academia
NARSTO
NAS, CAAACCASAC, OMB
Enviros
Organizations
CDC
Supersites
IMPROVE, NCorePM monit, PAMS
CASTNET
Lidarsystems
NADP Satellite data
Intensive studies
PM centers
Other networks:SEARCH, IADN..
Data sources
CMAQGEOS-CHEM
EmissionsMeteorology
Health/mort.records
The Scheffe Challenge: ‘Organizations - Programs – Data Mess’
Info System Challenges:
What’s the overall dependencyInformation FlowForces and Controls on Data FlowCooperation, Competition, Co-OpetitionInformation System of Systems
Air Quality Management System: Components and Functions
Public
AnalyzingInterpreting Evaluating Separating Synthesizing
OrganizingQuality controlFormattingDocumentingDisplaying
DecidingEvaluate optionsMatching goals CompromisingChoosing
Data Manager, Organizer
Technical Analysts, Program Manager
Policy Analysts, Decision Maker
Valu
e Ad
ding
Pr
oces
ses
Human Agents
Decision Support System (DSS)
The primary purpose of data systems is to serve programs/projectsPrograms perform analysis for Orgs., the DSS is within programsThe big decisions of societal importance are Organizations (This needs more wisdom from the practioners)
Relationship BetweenOrganizations - Programs – Data
Version 0.1
Goals $$Info needs, $$
Data need, $$
Judge, Decide, ActAnalyze, Report
Actionable Knowledge
Decision, Action
Public
Measure, Organize
Organized Data
Flow of InformationData systems organize the measurements and models and provide them to programs. Programs analyze the data and provide actionable knowledge to organizations.Organizations evaluate multiple information sources, make decisions and act.
Flow of ControlPublic and special interest groups set up organizations and provides them with funding Organizations develop programs, define their scope, governance and fundingPrograms satisfy their information needs by monitoring or by using other’s dataData sources acquire the data for their parent programs and also expose them for reuse
Information System Components for AQ Programs
Integrated DataDatasets
Std.
In
terf
ace
Data Views
Std.
In
terf
ace
Data
Control
Reports
Reports
Public
Human Information SystemMachine Information System
Obs. & Models Decision Support System
Flow of Data and Usage Control
Flow of ControlReports are commissioned by programs Analysts select, explore, and process data for a report
Flow of DataProviders expose data to analysts who extract the needed subsetThe data is pulled into data exploration or processing software
Integrated DataDatasets
Std.
In
terf
ace
Data Views
Std.
In
terf
ace
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Flow of Data and Usage Control
Data
ControlRequesting Information
Providing Information
Sensors
Acquisition processing User
Agencies
User Progra
msNAAQS SIPs Forecast GEOSS …
Info System
hhh
Integrated DataDatasets
Std.
In
terf
ace
Data ViewsSt
d.
Inte
rfac
e
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Steve Young Challenge
The Steve Young Challenges
Integrated DataDatasets
Std.
In
terf
ace
Data ViewsSt
d.
Inte
rfac
e
Data
Control
Reports
Reports
Obs. & Models Decision Support System
• Real-time monitoring & watch for surprises• Inform decision-makers & assess outcomes• Support for adaptive management
S. Young Challenge #1: Real-time monitoring & watching for surprises
Integrated DataDatasets
Std.
In
terf
ace
Data ViewsSt
d.
Inte
rfac
e
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Choose Data & Tools Browse, Explore Trigger Response
S. Young Challenge #2:Inform decision-makers & assess outcomes
Integrated DataDatasets
Std.
In
terf
ace
Data ViewsSt
d.
Inte
rfac
e
Data
Control
Reports
Reports
Obs. & Models Decision Support System
• This needs more work• Ideas? How could the info-system help here?
AirQuality
AssessmentCompare to GoalsPlan ReductionsTrack Progress
Controls (Actions)
Monitoring(Sensing)
Close the sensory-motor loop!
S. Young Challenge #3. Support for Adoptive AQ Management
From ‘Stovepipe’ to Workflow Software
Integrated DataDatasets
Std.
In
terf
ace
Data ViewsSt
d.
Inte
rfac
e
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Flow of DataFlow of Control
Air Quality Data
Meteorology Data
Emissions Data
Informing Public
AQ Compliance
Status and Trends
Network Assess.
Tracking Progress
Data to Knowledge Transformation
Loosely Coupled Workflow
Terry Keating Challenge
The Terry Keating Challenges
Facilitate data-model comparison, assimilation
Domain ProcessingData Sharing
Std.
In
terf
ace
Gen. ProcessingSt
d.
Inte
rfac
e
Data
Control
Reports
Reporting
Obs. & Models Decision Support System
Include global perspective (observation, model, analysis)
Architechture Challenge
Friedl Fontaine Barrie Lindsey
GEOSS Architecture Mapped to Air QualityThe L. Friedl, K. Fontaine Challenge
Integrated DataDatasets
Std.
In
terf
ace
Data ViewsSt
d.
Inte
rfac
e
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Not part of DataFedPolicy, Management, Personal Decisions
Special Architectural Features:Implied data assimilation into modelsDecision Support black boxFocus on model predictions, not ‘simulation’ Pollutant ‘Characterization’ is implicit
IGOS Architecture Mapped to DataFedThe Len Barrie Challenge
Integrated DataDatasets
Std.
In
terf
ace
Data ViewsSt
d.
Inte
rfac
e
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Not part of DataFedQuality assurance, CalVal?Integrated Data Archives? -
Special Architectural Features:Real-time data assimilation into AQ modelsCharacterization of global pattern
F. Lindsey, NASA:
Air Quality Collaborative Consortium
Goal: Cross-leverage the shared partner resources and activities, while
Maintaining partner’s autonomy in capabilities and activities
Approach:Exchange the resources at the web ‘interfaces’, like portals.
Infuse cutting-edge technologies and resources into Agencies as needed. Use ESIP portal as the linker and the Air Quality cluster as the mediator.
Outcome:Sharing and integration could form next-generation capabilities.
Savings by re-use and better information by broader perspectivesBetter air quality through more informed management
NOAAEPA
PrivDoI
Air Quality ConsortiumData, Tools, Methods
SharedPrivate
NASA
Other FedS
Applications
AQ Policy
Regulation
Research
DataFed Challenge
Value-Adding Processes
Integrated DataDatasets
Std.
In
terf
ace
Data ViewsSt
d.
Inte
rfac
e
Data
Control
Reports
Reports
Obs. & Models Decision Support System
AnalyzingFilter/IntegrateAggregate/FuseAssimilate
OrganizingDocumentStructureFormat
ExploringDisplay BrowseCompare
Valu
e-Ad
ding
Pr
oces
ses
Data Manager Technical Analysts
DB System Analysis System Reporting System
Loosely Coupled Data Access through Standard ProtocolsOGC Web Coverage Service (WCS)
Client request Capabilities
Server returns Capabilities and data ‘Profile’
Client requests data by ‘where, when, what’ query
Server returns data ‘cube’ in requested format
GetCapabilities
GetData
Capabilities, ‘Profile’
Data
Where? When? What? Which Format?
Server
Back End St
d.
Inte
rfac
e
Client
Front EndSt
d.
Inte
rfac
e
Query GetData Standards
Where? BBOX OGC, ISO
When? Time OGC, ISO
What? Temperature CFFormat netCDF, HDF.. CF, EOS, OGC
T2T1
Integrated DataDatasets
Std.
In
terf
ace
Data ViewsSt
d.
Inte
rfac
e
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Web Services and Workflow for Loose Coupling
Integrated DataDatasets
Std.
In
terf
ace
Data ViewsSt
d.
Inte
rfac
e
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Service Broker
Service Provider
PublishFind
BindServiceUser
Web Service Interaction Service Chaining & Workflow
Collaborative Reporting and Dynamic Delivery
Integrated DataDatasets
Std.
In
terf
ace
Data ViewsSt
d.
Inte
rfac
e
Data
Control
Reports
Reports
Obs. & Models Decision Support System
Co Writing - Wiki
Screencast
Analysis Reports: Information supplied by manyNeeds continuous program feedbackReport needs many authorsWiki technologies are for collaborative writing
Dynamic Delivery: Much of the content is dynamicAnimated presentations are compellingMovies and screencasts are for dynamic delivery
The Network Effect:Less Cost, More Benefits through Data Reuse
ProgramPublic
Data Organization
DataData Program
ProgramOrganizatio
nDataData
ProgramData
Orgs Develop Programs
Programs ask/get Data Public sets
up Orgs
Pay only once Richer content
Less Prog. Cost More Knowledge
Less Soc. Cost More Soc. Benefit
Data Re-Use Network Effect
Data are costly resource – should be reused (recycled) for multiple applicationsData reuse saves $$ to programs and allows richer knowledge creationData reuse, like recycling takes some effort: labeling, organizing, distributing
GEOSS Challenge 1: Jose Achache
GEOSS/AMI Domain of ActionsBased on ideas of P. Senge: Architecture of Learning Organizations
Possible DataFed Roles:
1. Guiding Idea: Refine, solidify, evangelize the System of Systems idea
2. Methods and Tools: Develop, promote, implement standards; Coordinate SoS use cases
3. Infrastructure: Maintain the DataFed middleware for distributed data access; Supply Agency champions with SoS ‘sales’ material
P. Senge et. al, 1994: Fifth Discipline Fieldbook (Link)
Guiding Idea: System of Systems
Methods, Tools: Standards, Use Cases
Infrastructure: Web 2.0, Local Champs
Domain of ActionOrganizational Architecture
Key Agencies for Air Quality Management and Science
Flexible NAAMS
Advanced Monitoring InitiativeProgrammatic Path to GEOSS
Advanced Monitoring InitiativeProgrammatic Path to GEOSS
• Goal: Advance air quality model-observation complex to level of meteorological FDDA systems
• Build Organizational frameworks : IGACO-EMEP efforts• Data base, IT standards: data unification center? Practical LRTAP task?• Standardization/QAQC Reference material, or method (aerosols)• Adopt model evaluation/fusion as a design principle• Support linkage of ground based and satellite observation platforms through
development of a sustainable vertical profiling system (aircraft and ground based lidar)
• Address integration of disparate data bases– QA/QC: provide requirements for data standards/metadata descriptions– Data base unification
• Harness the communities around major tools, platforms and programs. – Air quality modeling platforms (GEOS-chem, MOZART, CMAQ,….more)– Satellite Instruments (MODIS, OMI)– Existing routine surface (AIRNow) and aircraft programs– Integration efforts (AEROCOM)
The Architecture of SoS Networks
• SoS Architecture – Form and Function Different aspects of the network
•
Right Level of Networking?
• One of the key factors dictating the achievement of system-of-systems configurations that support network centric operations is the availability of mechanisms that promote information sharing among systems. Direct system-to-system linkages presuppose that the systems know a priori about each system that might benefit from any other system. Many combat situations have disproved this assumption.
• Emergent behavior in a network that is only created by the combination, not by the individual members.
• Emergent behavior: novel and coherent (logically connected, purposeful, meaningful) structures, patterns and properties arising from self-organization (mashers are self-organizers) in complex systems Wikipedia
• Emergent behavior is created by mashing • Mashing occurs when open on both ends of network.- diffusion
– Input side – grow by assimilation– Output side – Harvest (i.e. DataFed harvests datasets)
Stages of Self-Organization
• Stage 1: Expose goods. Sharing was doable, but lots of work. Need glue, duct tape, shop-based approach. Value Shop.
• Burden: user pays • Stage 2: Mediated sharing. DataFed,
where wrappers/adapters used. Loose coupling automate the wrapper and mediator
• Burden: Mediator pays. Users gain. • Stage 3: Creating new things together,
service chaining, mashing, collaboration – Emergent behavior where network produces value, non-linearly.
• Burden: creators• Tim Burners-Lee: The web allows
creating new things together. • Howard Rheingold: The computer as
mind amplifiers. • The ultimate goal of all this is to both
amplify the mind of individuals and also connect the minds.