open science and geoss: the cloud sandbox enablers
Post on 14-May-2015
213 Views
Preview:
DESCRIPTION
TRANSCRIPT
THEME[ENV.2011.4.1.3-1]: Inter-operable integration of shared Earth Observation in the
Global Context Duration: Sept. 1, 2011 – Aug. 31, 2014
Total EC funding: 6,399,098.00 € Project Web Site: www.geowow.eu
EC Grant Agreement no. 282915
GEOSS interoperability for Weather, Ocean and Water
Hervé Caumont Terradue
herve.caumont@terradue.com
Open Science & GEOSS: the Cloud Sandbox enablers
GEO-X Plenary Geneva, January 14th, 2014
Digital Earth Communities
A long-term vision for GEOSS - GCI evolution … …considering feedback from all the stakeholders___
to engage with more user categories
data providers, data specialists, scientists, decision makers
within a more flexible architecture community components, resource enablers, cloud services
the GEOWOW Vision oo ooo
2 GEO-X Plenary
Digital Earth Communities
14/01/2014 3 GEO-X Plenary
CREATING THE
CONDITIONS
Digital Earth Communities
The concept of Cloud Sandbox enablers
4
Connect web resources into an experiment apparatus
Data assembly APIs in the Cloud
Data usage rights globally registered for scientific use
Exchange and reuse of a scientist’s workspace
14/01/2014 GEO-X Plenary
http://en.wikipedia.org/wiki/ATLAS_experiment
Digital Earth Communities
The present situation for most GEO partners
5
A discovery and download modus operandi Dataset file selections To reach out a multitude of fragmented project environments
14/01/2014 GEO-X Plenary
Digital Earth Communities
A look at the Future through Cloud Sandboxes
6
Repeatable environments for reuse of scientific work Data as a Service within federated environments, with usage metrics Shared resources across Cloud Computing clusters
14/01/2014 GEO-X Plenary
Digital Earth Communities
Matching the Open Science goals
Open Source
Open Data
Open Access
Open Notebook
Transparency in experimental approach & collection of observations
Public availability & reusability of scientific data
Public accessibility of peer-reviewed scientific communication
Shared versioning environment to facilitate progress in science
29/11/2013 7 GEO-X Plenary
Digital Earth Communities
14/01/2014 8 GEO-X Plenary
LOOKING FOR
WHAT’S INSIDE ?
Digital Earth Communities
Sandbox instance
Tools (Python, Libraries, ...)
Cloud Sandbox Enablers
• Cloud Appliances Marketplace: a ‘VM Store’ to manage user Sandboxes • Platform as a Service (PaaS): an algorithms integration environment • Data staging tools: access and manage dataset slices required by applications
Sandboxes
Digital Earth Communities
Apache Hadoop Streaming programming model
Scale up MapReduce Jobs from single servers to thousands of computing nodes
Automate failure handling at the application layer
The PaaS environment
Digital Earth Communities
Cloud Sandbox enablers
Cloud Sandbox UI dashboard - VM Information, App Descriptor, App runs workflow
status, VM monitoring, Run invocation, Support tools
Cloud Applications linked to GitHub repositories – Automated Code Versioning & Collaborative
developments – A new paradigm: a distributed service for dissemination
Data Casting enablers for data preparation - Get the atomic, scalable, data slice units - Validate a processing job in Sandbox simulation mode - Scale out on a cluster (e.g. by time slices)
11 14/01/2014 GEO-X Plenary
Digital Earth Communities
Tutorials & Training material
Digital Earth Communities
14/01/2014 13 GEO-X Plenary
THEY DARED
TO TEST IT
Digital Earth Communities
Developers on Cloud Sandboxes
On-boarding GEOSS partners: Dec’12: UNESCO Apr’13: INPE May’13: ECMWF Sep’13: ESA
14 14/01/2014 GEO-X Plenary
Digital Earth Communities
Catalyzing resources
Support researchers to compute indicators for policy makers
Feed resources to Computing Clusters to run global & regional marine ecosystems assessments
Take-up for future commercial applications (under ad-hoc conditions)
Go through research oriented, not for profit, uses of TIGGE-LAM data in order to spread innovation
15
Expand uses of ESA satellites data
Development of new applications leveraging innovative uses of earth observations
Improve natural resources management and evolve policies
Handle large Earth Observation Temporal Series for Land Change Events Detection
14/01/2014 GEO-X Plenary
Digital Earth Communities
Capacity Building
Ability to leverage ESGF’s CMIP5 Climate projections data, slice it and process it
Support reproducibility of scientific experiments & open science
Explore and visualize ECMWF data from integrated Cloud appliances
Improve accessibility of key TIGGE data for a wide user community and better support ECMWF partner users
16
Experiment with ENVISAT SCIAMACHY, global measuring of trace gases in the troposphere and in the stratosphere
Towards integration of Earth Explorers missions: catalogs & processors on Cloud Sandboxes
Streamline the processing of large EO data Temporal Series
Experiment flexible access to compute-intensive resources
14/01/2014 GEO-X Plenary
Digital Earth Communities
Data Sharing
Ocean Biogeographical Information System (OBIS)
Transboundary Waters Assessment Programme (TWAP) indices and indicators
TIGGE and TIGGE LAM, THORPEX Interactive Grand Global Ensemble Limited Area Model
ERA Re-Analysis archives
17
ESA Missions data (archived and Earth Explorer)
Earth System Grid Federation – Model Intercomparison Project Phase 5 (CMIP5) data
14/01/2014 GEO-X Plenary
Digital Earth Communities
PERSPECTIVES Developer Cloud Sandboxes – What’s next
18
IN THE NEWS
Digital Earth Communities
What we can have for GEOSS
Smooth deployments of Cloud environments for users.
Lean on-boarding process for custom user needs.
Data casting services for data intensive computing needs.
EO data processing applications shared and reused.
19 14/01/2014 GEO-X Plenary
Digital Earth Communities
Bringing in the lean approach
Build Monitor Learn
Digital Earth Communities
Coming next from GEOWOW
Open Science ready
21 14/01/2014 GEO-X Plenary
Best practices Success stories
Hands-on tutorials
Getting users from all GEOSS communities on Cloud Sandboxes
Online demos
THEME[ENV.2011.4.1.3-1]: Inter-operable integration of shared Earth Observation in the
Global Context Duration: Sept. 1, 2011 – Aug. 31, 2014
Total EC funding: 6,399,098.00 € Project Web Site: www.geowow.eu
EC Grant Agreement no. 282915
GEOSS interoperability for Weather, Ocean and Water
Open Science & GEOSS: the Cloud Sandbox enablers
GEO-X Plenary Geneva, January 14th, 2014
top related