The SAFARI 2000 Data Distribution System

Jeffrey L. PrivetteNASA’s Goddard Space Flight Center

Many others from NASA, Wits, NDMC, ORNL

28 August 2001

SAFARI 2000 Data Group

Ad hoc, voluntaryObjective: Develop and implement a system to:

Allow registered participants to access data in timely fashionProtect rights of data collectors/providers and students

• Including appropriate acknowledgement

Ensure complete SAFARI data archive is permanently housed in southern Africa

ParticipantsUniversity of WitwatersrandNational Disaster Management CenterNASA’s Goddard Space Flight CenterOak Ridge National Laboratory’s DAACUniversity of Virginia

“Registered SAFARI User”

Full participation is available to anyone who conducts relevant research and who agrees to the terms of the Data Policy

Registration via official SAFARI WWW siteSAFARI Steering Committee confirms that applicant meets above requirements

Approved registrant is provided with accounts on SAFARI data systems (Regional Data Centre and a Mercury host center)

• Added to email “list server” (Newsletters, meeting information, invititations to publish, project information)

Data System Components

Regional Data Centre(National Disaster Management Centre)

• Participant Registration

• Mercury Metadata Editor Host

• Permanent Archive

ORNL DAAC• Mercury Metadata Editor Host

• Mercury Host

NASA’s GSFC• Remote Sensing Data

Acquisition

• CD-ROM Production

Associated Data

Centers

Associated Data

Centers

Associated Data

Centers Associated Data

Centers

Associated Data

Centers

Participant

Data Archives

Regional Data Centre (RDC)

Core of SAFARI Data System

Cooperative effort of the University of the Witwatersrand and the National Disaster Management Centre

NDMV provides hardware and software infrastructure to host and distribute data

Wits provides research input, strategic guidance and management assistance

RDC Support of SAFARI

Host official SAFARI 2000 WWW site Up-to-date project information Downloadable software tools

Provide participant registration systemHost Mercury Metadata EditorHost remote sensing data sets

NASA’s EOS Terra, Aircraft, etc.

Final repository for all SAFARI data

RDC ResourcesHardware

Main web application serverIBM 4xPentium III 750 MHz, 5x18 GB HDD RAID, 750 MB RAM, Windows NT

Database serverCompaq Pro-Liant 5500, 2x Pentium III 650 MHz, 4x18GB HDD RAID, 780 MB RAM, with HP Intelligent RAID Rack 1.3 TB HDD system, Windows NT

GIS serverCompaq Pro-Liant 5500, 2x pentium III650 MHz, 4x18GB HDD RAID, 780 MB RAM, Windows NT

Maths / Statistical server, Compaq Pro-Liant 5500,2x pentium III 650 MHz, 4x18GB HDD RAID, 780 MB RAM,Red Hat Linux 6.2 (7.1)

E-mail server, Compaq Pro-Liant 5500, pentium III200 MHz, 2x18GB HDD, 512 MB RAM

Personal Computers (6)

Data System Components

Regional Data Centre(National Disaster Management Centre)

• Participant Registration

• Mercury Metadata Editor Host

• Permanent Archive

ORNL DAAC• Mercury Metadata Editor Host

• Mercury Host

NASA’s GSFC• Remote Sensing Data

Acquisition

• CD-ROM Production

Associated Data

Centers

Associated Data

Centers

Associated Data

Centers Associated Data

Centers

Associated Data

Centers

Participant

Data Archives

ORNL DAAC… What’s That?

Oak Ridge National Laboratory’s (ORNL)Distributed Active Archive Center (DAAC)

Located in Oak Ridge, Tennessee (near Knoxville)

1 of 7 DAACs created in the U.S. to provide data management for NASA’s Earth Observing System (EOS)

ORNL DAAC’s Focus: Biogeochemical Cycles Provides strong support of the EOS Validation Program and major field campaigns (e.g., SAFARI, LBA)

ORNL DAAC Support of SAFARI

Support Mercury Metadata Editor Software to allow easy metadata submission

Internet online and “standalone” MELite

Host Mercury to allow SAFARI participants to search and access data

Harvests metadata from RDC and ORNL to create comprehensive SAFARI data catalog

Single best starting point to find any/all SAFARI data• Ground, aircraft, satellite, climatological, historical

• Provides information and pointer (URL, name/email) for each data set

Data and documentation are downloaded directly from data provider

User links to dataprovider’s server

Mercury collects the metadata file and adds

to SAFARI Catalog

Participants query the Catalog and retrieve full metadata, including

WWW links to data provider

SAFARIMetadataCatalog

ORNL

Data anddocumentation

Data provider uses Metadata Editor to create file containing general information and links to

the data and documentation

Participant

Mercury OverviewMercury Overview

Data Provider

1.2.

4.

3.

5.

Mercury Data Pathways

Mercury SystemInvestigator Data SiteInternet ConnectionCourier / Air Mail

1. Fielded Search(choose from list of

parameters, persons, etc.)

2. Spatial Search

3. Temporal Search

Data System Components

Regional Data Centre(National Disaster Management Centre)

• Participant Registration

• Mercury Metadata Editor Host

• Permanent Archive

ORNL DAAC• Mercury Metadata Editor Host

• Mercury Host

NASA’s GSFC• Remote Sensing Data

Acquisition

• CD-ROM Production

Associated Data

Centers

Associated Data

Centers

Associated Data

Centers Associated Data

Centers

Associated Data

Centers

Participant

Data Archives

NASA’s GSFC Support of SAFARI

Edit SAFARI Newsletter

Create and publish CD-ROMs

Stream MODLAND Product subsets to RDC in real-time

Manage requests for tasked NASA satellite data e.g., Landsat 7, IKONOS

SAFARI 2000Participant

NASA’s GSFCSAFARI Team

(Maryland)

Landsat 7Science Team/

EROS Data Center(South Dakota)

Task

Request

Acquire

Archive/

Distribute

“Associated Data Centers”The Landsat 7 Example

SAFARI 2000Participant

NASA’s GSFCSAFARI Team

(Maryland)

Landsat 7Science Team/

EROS Data Center(South Dakota)

Task

Request

Acquire

Archive/

Distribute

TRFIC Data Center(Michigan State Univ.)

“Associated Data Centers”The Landsat 7 Example

SAFARI 2000Participant

NASA’s GSFCSAFARI Team

(Maryland)

Landsat 7Science Team/

EROS Data Center(South Dakota)

Regional Data Centre(S. Africa)

Task

Request

Acquire

Archive/

Distribute

TRFIC Data Center(Michigan State Univ.)

“Associated Data Centers”The Landsat 7 Example

SAFARI Remote Sensing Data Pathways to Region: Current

Investigator Data SiteInternet ConnectionInternet or Air Mail, direct to user

Landsat, IKONOS

MODLAND Products(Core Sites)

Start your data search in the ORNL

DAAC Mercury System

SAFARI Remote Sensing Data Pathways to Region: Current

Investigator Data SiteInternet ConnectionInternet or Air Mail, direct to user

Landsat, IKONOS

MODLAND Products(Core Sites)

Start your data search in the ORNL

DAAC Mercury System

CD-ROMs

SAFARI 2000 CD-ROM SeriesSiavonga: Volume 1 Release Party!

CD-ROM Architect:

Dave Landis, NASA’s GSFC

The CD-ROM's HTML Interface

Uses an HTML/Web-browser interface that works the same on all computers (Windows, UNIX, Macintosh)

Allows access to data, documentation, and WWW links to additional resources.

Can automatically copy and decompress the image files from the CD-ROMs to the user’s hard drive

Help available through on-disc tutorials

Data Contents of Volume 1

Stratification by data type and volumeHydrology Data (5 data sets) -- 2 megs

Land Cover Data (5 data sets) -- 32 megs

Meteorological Data (4 data sets) -- 30 megs

Satellite Images (6 data sets) -- 860 megs

Vegetation & Soils Data (4 data sets) -- 0.5 megs

Documents and Software -- 190 megs

CD-ROM Satellite Images

Data Product Data FormatData Volume

(Megs)

Satellite Images (SAT)AHVRR Regional Images -- 6km; 2 week averages; July 1998-Oct 2000

binary 204.80

AVHRR Site Extract Images -- 1.5 km; 2 week averages; July 1998-Oct 2000

binary 63.20

AVHRR Site Extract Images -- 1.5 km; daily; Feb,Mar,Aug,Sep 2000

binary 160.20

SeaWiFS Regional Images 1999-2001 hdf, zipped 353.40SeaWiFS Site Extract Images 2000-2001 hdf, zipped 283.30MeteoSat-7 Images (movies) jpg 8.00

* AVHRR Parameters -- Day, LST, NDVI, PHI, R1, R2, TS, TV

CD-ROM Documents and Software

Data ProductData

Format

Data Volume (Megs)

Which CD-ROM

SAFARI 2000 DocumentsEOS Newsletter May-J une 1999 pdf 1.40 1EOS Newsletter May-J une 2000 pdf 0.56 1EOS Newsletter September-October 2000 pdf 0.46 1S2K Dry Season Plan pdf 2.70 1S2K ER-2 Summary pdf 5.50 1S2K Final Plan pdf 0.24 1S2K Lithograph pdf 0.26 1S2K Operations Document 2000 pdf 0.26 1

Software

OME Lite Metadata Editor1

Netscape Software 59.20 1Image Viewing Software (MultiSpec & Osiris) 14.40 1 Web Winds HDF Viewing Software 86.70 1Zip Software 3.10 1

Next Step Involves You

Volume 1: first in a series Satellite and climatological data were ready now

Forthcoming volume(s) will host participants’ data sets

Opportunity to branch out, collaborate, coauthorHonor the Data PolicyCD-ROM includes many reminders of required acknowledgementsTo be included: You must provide your metadata to the SAFARI Data Catalog!

• Mercury Metadata Editor (online or standalone/email)

Thanks

Received from publisher last Wednesday

140 CD-ROMs brought to Siavonga by…Tom Eck

Michael King

Steve Platnick

Tim Suttles

Jeff Privette