esdis project status 11/29/2006 dan marinelli, science systems development office
TRANSCRIPT
ESDIS Project Status ESDIS Project Status
11/29/200611/29/2006
Dan Marinelli, Science Systems Dan Marinelli, Science Systems Development OfficeDevelopment Office
EOSDIS System EvolutionEOSDIS System Evolution
• ESDIS was directed to evolve the systems under its budget to accommodate vision identified by a joint EOSDIS Elements Study/Technical team
• Key vision elements include:
– Improve access and processing services, ensure available expert knowledge, reduce operational costs, ensure safe stewardship, maintain IT currency
EOSDIS System EvolutionEOSDIS System Evolution
• Top 3 cost drivers contribute to approx. 50 % of total budget:– EMD/ECS– GES DAAC– LaRC DAAC
• Factors that contribute to top 3 cost drivers:– Operating multiple systems (ECS, V0/V1, LaTIS, etc.) – DAAC-unique capabilities and science community
support beyond specific operation of ECS/SDPS – Providing sustaining engineering for ECS/SDPS at the
four ECS DAACs
ESDIS Evolution PathESDIS Evolution Path
• Approval has been given to embark down an evolution path
• GES DAAC and ASDC DAAC to evolve away from ECS SDPS at their sites
• MODAPS to evolve towards archive and distribution of all MODIS products
• ECS SDPS footprint to be reduced greatly in terms of hardware and custom code
• Summary of the plan can be found at http://eosdis-evolution.gsfc.nasa.gov/
EOSDIS TodayEOSDIS TodayEOSDIS provides
– A production capability for standard science data products from EOS instruments– An “active archive” of Earth science data from EOS and other past and present missions – A distributed information framework (data centers, SIPS, networks, interoperability, other system
elements) with partners supporting EOS investigators and other users in science, government, industry, education, and policy
EOSDIS_Today_11222006. xls
Non-ECS (reporting)ECSEOSDIS Metrics for FY2006
41.171.7Total Archive Products at the End of FY06 (in millions)
0.943.67Total Archive Volume at the End of FY06 (in PB)
4.165.6End User Distribution Products (in millions)
0.162.86End User Daily Distribution Volume (in TB)
0.652.40Daily Archive Growth with Deletion (TB/day)
0.653.21Daily Archive Growth without Deletion (TB/day)
2743,033Daily Ingest Volume (in GB)
Non-ECS (reporting)ECSEOSDIS Metrics for FY2006
41.171.7Total Archive Products at the End of FY06 (in millions)
0.943.67Total Archive Volume at the End of FY06 (in PB)
4.165.6End User Distribution Products (in millions)
0.162.86End User Daily Distribution Volume (in TB)
0.652.40Daily Archive Growth with Deletion (TB/day)
0.653.21Daily Archive Growth without Deletion (TB/day)
2743,033Daily Ingest Volume (in GB)
3,291,397Distinct Users over FY06 at DAACs
37.8M (est.)Number of Accesses at DAACs
3,362Unique Data Products
50System Interface Control Docs (ICDs)
EOSDIS SystemsEOSDIS Overall Metrics (FY2006)
3,291,397Distinct Users over FY06 at DAACs
37.8M (est.)Number of Accesses at DAACs
3,362Unique Data Products
50System Interface Control Docs (ICDs)
EOSDIS SystemsEOSDIS Overall Metrics (FY2006)
10 of 14SIPS
8DAACs
No.ESDIS Funded Entities
10 of 14SIPS
8DAACs
No.ESDIS Funded Entities
15International
8U.S.
No.Partnerships
15International
8U.S.
No.Partnerships
70Instruments Supported
26Archiving and Distribution
10Science Data Processing
No.Missions
70Instruments Supported
26Archiving and Distribution
10Science Data Processing
No.Missions
EOSDIS Mission ProfileEOSDIS Mission Profile
file name: EOS missions vs time6.xls
2011 2012 2013 20142007 2008 2009 20102003 2004 2005 20061999 2000 2001 20021995 1996 1997 19981991 1992 1993 1994
2011 2012 2013 20142007 2008 2009 20102003 2004 2005 20061999 2000 2001 20021995 1996 1997 19981991 1992 1993 1994
10/1984 - 2 years planned mission life
10/1978 - 10 yrs m life
ERBS (SAGE II)
ERS-1
NIMBUS-7 (TOMS I)
Meteor 3 TOMS (TOMS II)
Planned decay 2010UARS
TOPEX/Poseidon
JERS-1
OrbView-2 (SeaStar)
ERS-2
Earth Probe TOMS (TOMS III)
RadarSat 1
ADEOS I (Midori)
ADEOS II (Midori
Jason-1
ACRIMSAT
Meteor 3M
EOS Aqua
ICESat
EOS Aura
SORCE
3 yrs
Landsat 7
QuikSCAT
EOS Terra
3 yrs
TRMM CERES data only Heritage Missions
EOS Missions
Extended Mission Life
Planned Mission Life
4 yr Data Access Period(includes 3 yr reprocessing)
No Planned EOL
KEY
3 yr Reprocessing Pd
Archive Volume TrendArchive Volume Trend
FY03 FY04FY05 FY06
GSFCEDC
LARC
NSIDC
100
200
300
400
500
600
700
Archive Volume (TB)
Granules into Archive TrendGranules into Archive Trend
FY03FY04
FY05FY06
GSFCEDC
LARC
NSIDC
0
2,000,000
4,000,000
6,000,000
8,000,000
10,000,000
12,000,000
14,000,000
16,000,000
Archive Granules
Distribution Volume TrendDistribution Volume Trend
FY03 FY04 FY05 FY06
GSFCEDC
LARCNSIDC
0
50
100
150
200
250
300
DAAC Distribution Volume (TB)
Distribution Granules TrendDistribution Granules TrendDistribution Granules TrendDistribution Granules Trend
FY03 FY04 FY05 FY06
NSIDCEDC
LARCGSFC
0
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
7,000,000
8,000,000
Number of Granules Distributed
User Access StatisticsUser Access StatisticsDAAC Fiscal Year 2006 Access Statistics for ECS and Non-ECS Data
Number of Distinct Users Accessing DAACs
TITLE ASF EDC GHRC GSFC JPL LARC NSIDC ORNL SEDAC TOTAL
ECS Orders/Subscriptions 5,535 3,117 463 811 9,926
Non-ECS Orders/Subscrip 127 0 152 284 41 399 414 119 0 1,536
WWW (users) 25,563 0 174,764 294,271 86,862 279,545 644,943 170,943 814,770 2,491,661
FTP (users) 56 0 181 2,884 1,545 0 530 1,296 556 7,048
Off-line (users) 0 0 26 419 224 981 611 791 376 3,428
Datapool (users) 0 2,463 0 2,831 0 688 512 0 0 6,494
Total (users) 25,746 7,998 175,123 303,806 88,672 282,076 647,821 173,149 815,702 2,520,093
EOSDIS 2006 Customer EOSDIS 2006 Customer Satisfaction SurveySatisfaction Survey
EOSDIS 2006 Customer EOSDIS 2006 Customer Satisfaction SurveySatisfaction Survey
• EOSDIS’ third survey, about 2800 responders
• Survey has changed slightly each time, but the standard questions for measuring satisfaction are the same
• 2006 survey addressed product search, selection and order, distribution, quality, documentation and customer support
Respondent BackgroundRespondent BackgroundRespondent BackgroundRespondent Background
2%
3%
4%
6%
9%
9%
10%
11%
16%
17%
20%
22%
21%
22%
22%
25%
27%
26%
33%
37%
44%
47%
1%
3%
4%
5%
8%
8%
10%
11%
15%
15%
16%
19%
21%
21%
21%
23%
23%
25%
32%
36%
41%
45%
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%
Flight dynamics
Space Weather
Sun-Earth Connections
Socioeconomics
Solid Earth
Other
Cryosphere
Planning
Carbon Cycle
Geolocation
Radiance
Education
Resources
Natural Hazards
Weather
Water and Energy
Oceans
Agriculture
Ecosystems
Atmosphere
Climate/Climate Change
Land Cover/Land Use
2005 2006
Q8. For which disciplines do you need or use Earth science data? (n=2,857)*
Customer Survey Product Quality Customer Survey Product Quality
Q27. In what format were
your data products
provided to you?
Q28. What format would
/ do you prefer?
HDF-EOS 37% 21%HDF 30% 21%NetCDF 3% 8%Binary 6% 7%ASCII 6% 9%TIFF or GeoTIFF 10% 23%JPEG, GIF, PNG 3% 4%OGC Web services 1% 2%Other 3% 5%
Summary of HDF-related comments to the CFI Survey
(Informally assessed)
Summary of HDF-related comments to the CFI Survey
(Informally assessed)Need tools for data handling 20
More useful for various user types (GIS/educational/science) if in some other than HDF format. 19
Make the distribution format fit what the user wants (or at least add more flexibility) 14
Would prefer NetCDF 8
Need to pay attention to usability of data products 3
Need documentation to extract from HDF-EOS (or HDF) file 2
Need program to extract or convert HDF-EOS Data 2
Hard for nonprogrammers to handle HDF 1
HDF-EOS is a bad format for multi-spectral/multi-angle data sets 1
Need windows tool for HDF 1
No response to help 1
We read HDF-EOS data as HDF5 1
How May We Help You?How May We Help You?
• The data gleaned from the survey leads us to conclude that the ESDIS Project needs to examine solutions for the areas of:– Data handling support software– Preprocessed/flexibly-formatted data access
paths– NetCDF– GeoTIFF when applicable