satella: tool for peer to peer landsat data sharing bruno margerin (ssai) john owens and dr. chris...
Post on 30-Dec-2015
218 Views
Preview:
TRANSCRIPT
Satella: Tool for Peer to Peer Landsat
Data Sharing
Bruno Margerin (SSAI)
John Owens and Dr. Chris Justice (UMd)
RationaleHow can scientists share data easily ? we need a way let data users know - who has what data. we need a tool to access and provide data. we want to minimize the burden from those sharing data
We need to reduce the costs of managing NASA data
We need to promote reduce obstacles to increased use of NASA data
The internet provides a mean for electronic data exploration and delivery
Can we take advantage of IT currently being developed in the private sector and apply these advances to our science data needs ?
NASA Data Needs: An exampleNASA has purchased large volumes of Landsat data to support the science community Landsat Pathfinder (Landsat 5) Regional Data Buys Earthsat Data Buy 1990 Landsat 7 Data Purchases for R and A (LCLUC) Landsat 7 Earthsat Data Buy Data 2000
These data are held by individual PI’s – most of whom are willing and able to share data with other scientistsOnce purchased by NASA, these Landsat data can be shared openly for science use We need a way to facilitate data sharing and reduce costs of dataWe need to promote free and open sharing of science dataWe need to move the burden of data management away from centralized systems to more distributed systemsOther opportunities – MODIS Data, ASTER Data, Landsat Derived Products, Model Output
Requirements for the System
A unified method of access to heterogeneous, distributed archives
Better utilization of Internet bandwidth, storage
Low management and maintenance
Easy to install and use
Highly scalable performance
The Satella Concept
A scalable, robust system for data search and distribution over a large collection of heterogeneous providers
A means for individual scientists to obtain and share satellite data
Satella Conceptualization
A distributed, self-organizing network based on peer-to-peer topology
Integrating the data available in individual archives
Through a freely available client software for search and transfer of the data
Satella: Peer 2 Peer SystemPrivileges Local Peer Interactions.Replicates information towards ConsumerDiscover and Adapt dynamically to new information sources.Build complex services by combining simple ones.Increase performance as system ages and more peer participates. Protect data by replicating informationBuild reliability from interchangeable peers
Based on JXTA P2P Protocols and Implementation
Open Network Programming platform for P2P ComputingSet of Protocols for interoperating (XML based)Language and network Agnostic Technology.Open Source Technology: http://www.jxta.orgVirtual network overlayGeneralize the P2P problemSolve key technical challenge (security, stability, scalability ...)Interoperable, Platform independance, ubiquity136,000 Downloads, 40+ Projects, 7500 Members (4/25/2001 – 12/8/2001)
From http://www.jxta.org/community
Software Architecture 1/2Prototype Service software allowing sharing and search for Landsat 7 ETM datasets.
2 Set of Service: Root Peer Service:
Maintains and provides secure access to Satella Peer Group. Provide initial topology information of Net Peer Group and Satella Peer
Group Provide Relay and Rendez Vous capability for Satella Peer Group and
Net Peer Group. Data Peer Service:
Provide P2P File Sharing capability Automated Landsat 7 Metadata creation Search and Download of files based on their metadata.
Fully Open Source
N-Pier Architecture
UDDI/WSDL/SOAP Web Service Ready Architecture.
Software Architecture 2/2
Data Peer Service
Root Peer Service
XML XML
Oth
er P
2P D
ata
& R
oo
t S
vc
Comm. ObjectsRMI Interface
Tomcat Web Server
MVC: Turbine + VelocityRMI + Web Based GUI
RMI
Web Browser
HTTP
Interact
Data Peer Service
Root Peer Service
XML XML
Oth
er P
2P S
vc
Comm. ObjectsWSDL/SOAP Interface
Other Web Service
SOAP
Associate GUIInteract
UD
DI
Satella Development to Date
Phase I:Internal Proof of concept and feasibility. AWT/SWING based GUI Search on File Name Only Basic, download capability. Partially Multi-Threaded.
Phase II: N-Pier Architecture Service Based Architecture 3- Pier Model View Controller (MVC) Web Based GUI Automatic Landsat 7 Metadata Generation Improved Download capability. Search on Landsat 7 Metadata (in progress).
Satella Caveats
JXTA is one competing web services platform (there are other competitors e.g. Microsoft Hailstorm), will it be the eventual winner? Mitigated since Other platforms are less mature especially in the P2P Architecture.Distributed archive depends on participation by potential users/contributors. Success of Napster/Gnutella music file sharing networks illustrate people willing to give as well as take.Programmatic guidance could encourage PIs and providers to play an active role. Satella would not replace the need for a long-term archiving of satellite data but will share the load and reduce the cost of data distribution
Satella: The Way Forward
Future Improvements:Continued improvements for Satella
network performance and reliabilityEvolution to a UDDI/WSDL Web ServiceEnhanced Semantic, distributed searchDownloading of a single file from multiple
available peersOther improvement dictated by the user
community.
Satella project Contributions to the Open Source Community
Involvement (Committer or Contributor) on 9 JXTA projects
Provided tutorial for the creation of a Secure Peer Group on JXTA’s Java implementation build 34c (10-12-2001)
Provided segment “Creating a Secure Peer Group” to “Project JXTA: Java[tm] Programmers Guide”.
In collaboration with Robin Johnson, Steer the architectureof the development of Metadata Support into CMS (Content Management System), JXTA File Sharing engine.
A look at the futureThe costs of data distribution are becoming unaffordable – DAAC’s are struggling to manage the large data volumes - new and innovative solutions are needed to improve data access and reduce costs to NASA and the PI – we need to invest in new approaches and long term solutionsData distribution is low compared to the size of the archives and the potential user community – we need to stimulate data useNASA Science requires regional time-series analysis i.e. large volume data sets giving an increased burden to existing data systems already proving to be unaffordableThe NASA Earthsat Data Buys provide an important asset for the global change research community – need to ensure their widest possible use – extensive data sharing is now feasibleFuture Landsat Continuity Missions will hopefully allow free and open sharing of 30m dataThe user community can and is willing to play an increasing role in data management – we need to develop sustainable ways of doing businessIn the context of the NEWDISS program NASA needs to support innovative use of advances in information technology to solve its data management challenges
Credits:John Owens and Dr. Chris Justice From University of Maryland, Dept of Geography.
Bernard Traversat and Juan C. Soto From Sun Microsystems
All participating JXTA members.
Satella Phase I: SnapShots
Contact Information
Bruno Margerin: bruno@turbodog.gsfc.nasa.gov
John Owens: jowens@umd.edu
Dr. Chris Justice: justice@hermes.geog.umd.edu
Web Site (to come): http://satella.geog.umd.edu
top related