powerpoint (4.4 mb)
Post on 27-May-2015
505 Views
Preview:
TRANSCRIPT
Ocean Color Data Processing System:Status
2 November 2006
NASA’s Goal
To make available the highest quality ocean color (and sst) data to the broadest user community in the most timely and efficient manner possible.
•
• Expertise:
Requirements for Success
• Expertise: internal
- highly integrated project structure with all elements co-located - continuous
communication.
Requirements for Success
• Expertise: internal and external
- highly integrated project structure with all elements co-located - continuous
communication.
- Strong links with mission-specific expertise (MCST) and research community (algorithms, validation data, new products)
Requirements for Success
• Expertise• Infrastructure
- flexible data processing system that constantly upgrades procedures, technologies and equipment
Requirements for Success
• Expertise• Infrastructure
- flexible data processing system that constantly upgrades procedures, technologies and equipment
SCIENCE drives the system rather than the SYSTEM driving the science
Requirements for Success
Processing Cluster34 processing nodes 1.5 TB
9 storage nodes 17.1TB
Ingest Servers2 SeaSpace groundstations 78 GB9 storage nodes (1xx,2xx) 16.6 TB
Distribution Servers (ftp)6 storage nodes (2xx) 25.5 TB
1 server 250GB Distribution Servers (web)Production: large server 3 TB 3 processing nodes (2xx, 3xx)
46 storage nodes (2xx, 25x) 303 TBDevelopment: 2 superstore nodes 10 TB
Testing Cluster8 test nodes (1xx) 480GB
7 processing nodes (2xx) 365GB2 storage nodes 2.4 TB
Network Support Systems
Database ServerProduction: 1 large server 876 GB
Development: 1 superstore node 5.8TB
Backup Servers1 large server 876 GB+2 TB tape
13 storage nodes 15.6 TB
Extreme NetworksBlack Diamond 6816
Gigabit Ethernet switch280 ports
User Desktops
Cal/Val & QC Systems
Mission Operations Systems
Ocean Color Processing System Overview
Development Servers1 processing node
2 storage nodes 2.4 TB
Ingest queuetable
L0Ingest
process
L1A-L1B(MOD_PR02)
L1B-L2(MSl12)
MODISA L2table
Archive - DistribServer
L3BIN(l2bin)
MODISA L3-bintable
Archive - DistribServer
MODISA L3-maptable
Archive - DistribServer
Operational MODIS-Aqua Data Flow (NRTPE)Terra and Aqua
Software process
Hardware system
Database table
Ancillary input data
SensorCAL
Sensorattribs
Atmcorr
BrowserCGI /httpd
Ocean ColorWeb Server
UserCommunity
MySQLDB
Ozone
L3MAP(smigen)
MET
MET, Ozone, and OISST data
are dynamically selected for each L1A granule
Product meta data are
populated from production DB
October 30, 2006
OISST
Ocean ColorIngest
MODISA L0table
Archive - DistribServer
ATT/EPHIngest
process MODISA attephtable
Archive - DistribServer
L0-L1A(MOD_PR01)
Geo-Location(MOD_PR03)
MODISA L1table
Archive - DistribServer
Full-resolution day- and nighttime via SEN (~60 GB per
day/mission)
NOAA RealtimeSystem
NASA EDOSSystem
MissingL0
process MODAPS L0Archive
MODAPS L0Archive
Granules not received from
NOAA realtime flow
• Expertise• Infrastructure• Data
- Most efficient and ultimately most cost effective when source data is available online for all needs including processing, reprocessing, evaluation testing, distribution.
Requirements for Success
CZCS: Volume = 582.72 GB1978-1986 Files = 15499OCTS: Volume = 137.62 GB 1996-1997 Files = 7641SeaWiFS GAC: Volume = 935.48 GB 1997-present Files = 51077SeaWiFS MLAC: Volume = 6.70 TB Files =
39257SeaWiFS LAC: Volume = 358.90 GB
Files = 42231
Current Source Data Holdings
MODIS/AQUA Level 1A Subset 7/02 - now
Volume = 19 TB Files = 449823MODIS/AQUA Level-0 (approx 20TB / year)
Volume = 58 TB Files = 332765
MODIS/TERRA Level 1A Subset 1/05 - now Volume = 8 TB Files = 183623
MODIS/ TERRA Level-0 - since August 2006 Volume = 5 TB Files = 27374
Current Source Data Holdings
• Expertise• Infrastructure• Data• Communication
Requirements for Success
OceanColor Web
oceancolor.gsfc.nasa.go
v
Consolidated data access, information,
services andcommunityfeedback
• Expertise• Infrastructure• Data• Communication• Intuitive and efficient data
distribution
Requirements for Success
• Expertise: internal and external• Infrastructure• Data• Communication• Intuitive and Efficient data
distribution • Community access to data
processing/analysis tools
Requirements for Success
1- Extensive user support withover 500 sites, active online Forums. > 3000 posts
2- SeaDAS ported to the Macintosh OS X includingnew Intel architecture. Alsoruns on Linux, Sun, SGI
3- Redesigned GUI and website are now much more user-friendly
4- New simple online installer and four SeaDAS ftp mirrors (Australia, Brazil, Japan, UK)
5- SeaDAS-lite option for display/analysis only (~50MB download)
6- Modis Direct BroadcastAnd High Resolution processing module.
7- User training workshops
top related