Integration of Multi-Sensory Earth Observations for Characterization of Air Quality Events
using Service Oriented Architecture
E. M. RobinsonAdvisor, R. B. Husar
2010 M.S. ThesisSt. Louis, MO, Nov. 3, 2010
Illustrate the use of multi-sensory dataTechnical Challenge: Characterization
• PM characterization requires many sensors, sampling methods and analysis tools
• Each sensor/method covers only a fraction of the 7-Dimensional PM data space.
– Spatial dimensions (X, Y, Z) – Temporal Dimensions (T)– Particle size (D)– Particle Composition ( C ) – Particle Shape (S)
• Most of the 7 Dim PM data space is extrapolated from sparse measured data
• Others sensors integrate over time, space, chemistry, size etc. .
Satellite-IntegralSatellites, have high spatial resolution but integrate over height H, size D, composition C, particle shape
Kansas Agricultural Smoke, April 12, 2003
Fire Pixels PM25 Mass, FRM65 ug/m3 max
Organics35 ug/m3 max
Ag Fires
SeaWiFS, Refl SeaWiFS, AOT Col AOT Blue
Networking Multiplies Value Creation
ApplicationData
1 User Stovepipe Value = 1 1 Data x 1 Program = 1
Enclosed Value-Creating Process - ‘Stovepipe’
“The user cannot find the data;
If he can find it, cannot access it;
If he can access it, ;
he doesn't know how good they are;
if he finds them good, he can not merge them with other data”
The Users View of IT, NAS 1989
Service Oriented ArchitectureActions: Publish – Find – Bind
Applications
Data
Broker
The data reuse is possible through the service oriented architecture
ApplicationData
Application
Application
Application
Application
Stovepipe
1 User Stovepipe Value = 1 1 Data x 1 Program = 1
5 Uses of Data Value = 5 1 Data x 5 Program = 5
Networking Multiplies Value Creation
Merging data may creates new, unexpected opportunities
Not all data are equally valuable to all programs
1 User Stovepipe Value = 1 1 Data x 1 Program = 1
5 Uses of Data Value = 5 1 Data x 5 Program = 5
Open Network Value = 25 5 Data x 5 Program = 25
Data
Data
Data
Data
Data
StovepipeApplication
Application
Application
Application
Application
Networking Multiplies Value Creation
Dataset Description
Convergence Protocols
GetCapabilities
GetData
Capabilities, ‘Profile’
Data
Where? When? What? Which Format?
Server
Back End S
td.
Inte
rface
Client
Front End
Std
. In
terf
ace
Query GetData Standards
Where?
BBOX OGC, ISO
When? Time OGC, ISO
What? Temperature CF
Format netCDF, HDF.. CF, EOS, OGC
T2T1
Standards needed for Distributed Data Access
ScientistScience
DAACs
Info UsersData Providers Info System
AIRNowPublicAIRNow
ModelCompliance
Manager
‘Stovepipe’ and Federated Usage Architectures Landscape
• Data are accessed from autonomous, distributed providers• DataFed ‘wrappers’ provide uniform geo-time referencing• Tools allow space/time overlay, comparisons and fusion
DataFed: Over 100 Federated Datasets
Near Real Time Data IntegrationDelayed Data Integration
Surface Air Quality AIRNOW O3, PM25 ASOS_STI Visibility, 300 sitesVIEWS_OL 40+ Aerosol ParametersMETAR Surface Visual Range
SatelliteMODIS_AOT AOT, Idea ProjectOMI AI, NO2, O3, Refl. TOMS Absorption Indx, Refl.SEAW_US Reflectance, AOT
Model OutputNAAPS Dust, Smoke, Sulfate, AOTWRF Sulfate
Emissions InventoriesNEI Point, Area, MobileEDGAR SO2,NOx,CO2
Fire DataHMS_Fire Fire PixelsMODIS_Fire Fire Pixels
Web Services: Building Blocks of DataFed
Programming
Access, Process, Render Data by Service Chaining
NASA SeaWiFS Satellite
NOAA ATAD Trajectory
OGC Map Boundary
RPO VIEWS Chemistry
Data Access
Data Processing
Layer OverlayLAYERS
Web Service Composition
Exceptional Event Rule: An air quality exceedance that would not have occurred but for the presence
of a natural event.
Transported Pollution
Transported African, Asian Dust; Smoke from Mexican
fires & Mining dust, Ag. Emissions
Natural Events
Nat. Disasters.; High Wind Events; Wild land Fires;
Stratospheric Ozone; Prescribed Fires
Human Activities
Chemical Spills; Industrial Accidents; July 4th; Structural
Fires; Terrorist Attack
Evidence for Flagging Exceptional Events• A. Establish a site is in potential violation of the PM2.5 standard.
Gather qualitative or quantitative evidence showing that the violation could have been caused by a source that is not reasonably controllable or preventable
Consoles: Data from diverse sources are displayed to create a rich context for exploration and analysis
Viewer: General purpose spatio-temporal data browser and view editor applicable for all DataFed datasets
May 2007 Georgia FiresAn actual Exceptional Event Analysis for EPA
May 5, 2007
May 12, 2007
Observations Used: OMI AI, Airnow PM2.5
DataFed WMS layers overlaid on Google Earth
Evidence for Flagging Exceptional Events
• B. Demonstrate a clear causal relationship between the measured exceedance value and the exceptional event.
CATT: Combined Aerosol Trajectory Tool for the browsing backtrajectories for specified chemical conditions
Sulfate Organics
Evidence for Flagging Exceptional Events
• C. The measured high value is in excess of the normal, historical values.
-
=
Actual Day 84th Percentile
Difference
Evidence for Flagging Exceptional Events
• D. The exceedance occurred but for the contribution of the exceptional source qualify for EE flag.
Social Media and Air Quality
Social Media Listening for Air Quality
Air Twitter Aggregator
RSS Feeds
Air Twitter Filter
ESIPAQWG
Air Twitter – Event Identification
August 2009, Los Angeles Fires
Normal Weekly Trend
Air Quality EventSpacesEventSpaces are community workspaces on the ESIP wiki that are created to describe the Event
Science Data
Social Media
Google Analytics Results: August LA Fires
580 Views
Google Analytics Results: August LA Fires
Future Work: GEOSS
MD_Metadata
+ fileIdentifier [0..1]: CharacterString (O)+ contact [1..*] : CI_ResponsibleParty (M)+ dateStamp : Date (M)+ metadataStandardName [0..1]: CharacterString (O)+ metadataStandardVersion [0..1]: CharacterString (O)+ metadataLanguage [0..1]: CharacterString (C)+ characterSet [0..1]: MD_CharacterSetCode = "utf8“ (C)
+identificationInfo 1..*
MD_DataIdentification
+ citation : CI_Citation (M)+ abstract : CharacterString (M)+ extent: EX_Extent (C)+ pointOfContact [0..*] : CI_ResponsibleParty (O)+ language [1..*] : CharacterString (M)+ characterSet: [0..*] : MD_CharacterSetCode = "utf8“ (C)+ topicCategory [1..*] : MD_TopicCategoryCode (M)+ spatial RepresentationType: [0..*] : MD_SpatialRepresentationTypeCode (O)+spatialResolution [0..*]: MD_Resolution
CI_Citation
+ title : CharacterString (M)+ date [1..*] : CI_Date (M)
CI_ResponsibleParty
+ individualName [0..1] : CharacterString+ organisationName [0..1] : CharacterString+ positionName [0..1] : CharacterString+ contactInfo [0..1] : CI_Contact+ role : CI_RoleCode
<<Abstract>>EX_GeographicExtent
+ temporalElement [0..*] (O)+ verticalElement [0..*] (O)
EX_GeographicBoundingBox (C)
+westBoundingLongitude: Decimal+eastBoundingLongitude: Decimal+southBoundingLatiitude: Decimal+northBoundingLatiitude: Decimal
MD_Format
+ name: CharacterString (O)+ version: CharacterString (O)
MD_Distribution
EX_GeographicDescription (C)
+geographicIdentifier: MD_IdentifierISO 19115 CoreM = mandatoryO = optionalC = mandatory under certain conditions
MD_DigitalTransferOption
+ CI_OnlineResource (O) Access InformationContact Information
Spatial/Temporal Extent
Dataset Description
Metadata Description