Summary and future directions

Bob Jones, CERNHelix Nebula Review

26 June 2014

This document produced by Members of the Helix Nebula consortium is licensed under a Creative Commons Attribution 3.0 Unported License.Permissions beyond the scope of this license may be available at http://helix-nebula.eu/. The Helix Nebula project is co-funded by the European Community Seventh Framework Programme (FP7/2007-2013) under Grant Agreement no 312301

Bob Jones - CERN 2

Main achievementsAt the end of a two year pilot phase, the Helix Nebula EC FP7 project has fulfilled its goal of enabling a federated cloud service across Europe through:

The successful deployment of several scientific flagship applications The establishment of a production platform providing a basis for future expansionTracing the path for an ecosystem to transform scientific data into valuable informationExpanding the consortium from 20 to 41 members and adapting the governance framework to the public-private marketplace

The CERN, ESA and EMBL flagship use cases drove the transition from pilot phase to production of HNX:

Launched at the Helix Nebula public event from cloud active to cloud productive ‐ ‐hosted by CERN on 14 May 2014

The European cloud service providers supporting HNX (Atos/Canopy, CGI, CloudSigma, EGI, Interoute, SixSq, The Server Labs and T-Systems) declared their intention to deliver easy access to a range of commercial Cloud Services http://hnx.helix-nebula.eu/index.html

Bob Jones - CERN 3

HNX & business generated

Similar services offered to other sectors:Atos: national research, electronics, aerospace, oil and gas, automotive manufacturing T- Systems: Insurance & pharmaceuticalsCloudSigma:

Hosting the Sentinal-1 data (Earth Observation SAR data)Partnered with CloudEO (HN Adopter) to deliver innovative Info-as-a-Service capabilities & other SMEs looking to use cloud infrastructureBuilt a solid work relationship with PICNICCAcquired 10Gbps connection to GEANT via Switch

Bob Jones - CERN 4

Investments and costs

Atos:Creating and running a PoC environment, 2012 – 2013: total cost estimated at €1.8mHNX direct contribution/developments & running costs 2014 = €441k

CloudSigma: Personnel + Indirect costs = €33k

Bob Jones - CERN 5

Maintaining the FP7 project artifacts Trust-IT will maintain and update the HN website until the end of 2014EMBL will maintain the alfresco collaboration site until the end of 2014EGI.eu will continue offering the mailman tool, which hosts 21 Helix Nebula email lists, for two more yearsCERN will maintain the open repository for all Helix Nebula documents on the CERN Doc. System (CDS) for a minimum of 2 years

Future Directions

This document produced by Members of the Helix Nebula consortium is licensed under a Creative Commons Attribution 3.0 Unported License.Permissions beyond the scope of this license may be available at http://helix-nebula.eu/. The Helix Nebula project is co-funded by the European Community Seventh Framework Programme (FP7/2007-2013) under Grant Agreement no 312301

Bob Jones - CERN 7

The initial 3 flagshipsESA:

Carry on with the SSEP InfoaaS stimulus project:Initialising the marketplaceIntroducing cash-flow throughout the value chains

EMBL:Continue testing the HNX EC2 bridge to support the ongoing maturation of this critical APIContinue our deployment testing of the EMBL large-scale genome analysis platform towards full-scale automation of on-demand HPC provisioning on HNX federated cloud resources

CERN:Test the hybrid cloud model linking the Helix Nebula suppliers with EGI FedCloud & GEANT (CERN ATLAS flagship)Work on the procurement of IT services for the LHC computing grid

Bob Jones - CERN 8

Port d’Informació Científica (PIC), BarcelonaReduce costs and improve speed of delivery, increase volume and accuracy for NeuroimagingRequirements entered into the online toolAlready deployed directly with CloudSigma

EC DIGITEUROPA web-based application under review

New Flagships

Bob Jones - CERN 9

Helix Nebula and the ECPMembers of Helix Nebula are active in the Select Industry Groups:

Service Level AgreementsCode of Conduct Certification

HN presented at/attended the Cloud for Europe consultation meetings

with a view to taking part in the tender process for new innovations

HN contributed to the Trusted Cloud Europe document

Includes references to Helix Nebula as a contender to be a pilot implementation

Bob Jones - CERN 10

Public e-infrastructure interoperability

EGI.eu and selected EGI Federated Cloud providers will continue testing the integration with SlipStream towards production stateGÉANT offered to:

Support the Helix Nebula production service through to the end of 2014, so as to integrate the HN suppliers into the GÉANT framework Support the HN Supplier-Supplier traffic during the next 6 months and measure its impact on the utilisation of GÉANTEstablish with the HN suppliers a cloud service “GN3 Plus Support to Cloud”

Bob Jones - CERN 11

The Helix Nebula Initiative

T-Systems committed to support the governance framework implementation

ESA will host the next general assembly in November 2014, where the revised governance model is foreseen to be adopted

Bob Jones - CERN 12

Procurement Background

Effort and costs of developing the HNX platform were under estimated by the suppliersThis FP7 project was a CSA which excluded funding for technical developmentsFuture developments could be a good match for a PCP scheme

ConsequenceProposal made to the EC under call ICT35 (f) to set up a procurement network of public research organisations - PICSE (Procurement Innovation for Cloud Services in Europe)

Goal

Create a reference framework for the public procurement of cloud services in which a European multi-tenant ‘Open Market Place for Science’ composed of data, scientists, funding bodies, SMEs and downstream industry meet to work towards common interestsBeyond PICSE – lead to the creation of PCP/PPI actions focused on expanding the cloud services marketplace

Bob Jones - CERN 13

Information as a Service

Science domain interfacing with private sector supporting business model “Information as a Service”

A number of pilot projects have been proposed to ESA, making use of their Earth Observation data

ESA Thematic Exploitation Platforms (TEP) ITT has been launched, to create an environment where those projects can be accommodated, but which would also provide an ideal vehicle for further dissemination of the data from the newly-launched Copernicus Sentinel satellitesContinuation with other FP7 projects on Supersites to establish a homogeneous ecosystem (Iceland, Italian Volcanoes)

Bob Jones - CERN 14

Information as a Service: Science interfacing with private sector

Helix Nebula Marketplace

Customer

Customer

Customer

€

€

€

€

• Lower upfront investment

• Fast access to EO and other geodata resources

• Disruptive technology

• Risk and profit sharing

Satellite data(ESA/Copernicus)

Data

Content Provider

(Research)

Knowledge

GEO-IT & AppProvider (SME)

Tools

Service Provider (Industry)

Computing & IT

Customer

Information

• Sustainability

Bob Jones - CERN 15

The e-Infrastructure commons MarketplaceScience workflows can be facilitated for all researchers through an e-Infrastructure commons Marketplace bringing together publicly funded e-infrastructures and data providers with commercial service providers:

E-Infrastructure: Encompass Generic, Domain and Community specific servicesCommons: Open but governed to protect demand and supply side actorsMarketplace: Pay for use mechanism for public and private sector suppliers

Provide an innovation platform to reduce barriers in the publicly funded research sector

Evolution of HNX

This document produced by Members of the Helix Nebula consortium is licensed under a Creative Commons Attribution 3.0 Unported License.Permissions beyond the scope of this license may be available at http://helix-nebula.eu/. The Helix Nebula project is co-funded by the European Community Seventh Framework Programme (FP7/2007-2013) under Grant Agreement no 312301

Bob Jones - CERN 17

Evolution of the HNX platform

Enhancements: Develop platform delivery as a service:

Unified contracting and billing structure

Improve EGI and GEANT integrationSmart Cities for regional engagement Innovation Vouchers

Structured release plan for the upcoming months and yearsApplication made to the EC for further funding to develop the required facilities:

Project proposal XZELCloudFocus on real-world use cases such as Open City servicesTo be revised by supply-side

Bob Jones - CERN 18

Serv

ices

HNX

Infrastructure

Blue Box• Blue Box(es)

• Processing• Storage• Network

IaaS

Applications

InfoaaSAppCrowdOpenCity

• SaaS applications• AppStores• Discovery• Composition• Collaboration• etc.

Middleware

• Databases• Metadata management• Directories, FedIM• SOA framework• etc.

Sector-specific

frameworks

Time

Funct

ionalit

y

PaaS

Helix Nebula GA, CERN Geneva, May 2014 19

Roadmap: HNX 1.13Q 2014

Blue Box SlipStream 2.2

API Fully documented

Commercial Agreement Updates

SLAs IaaS Provider SpecificMulti-Provider SLA Support

Support Additional Brokerage Functions

IaaS CSPs Additional Suppliers (+1-2)

Networking Advanced internet options (e.g. inter-cloud routing)

EGI EGI FedCloud Integration

Catalogue Updates, alignment of units, T-Shirt sizing

Other New user-level managementAdvanced Metering (e.g. Reservation/Burst)Enhanced usage and cost overviewOnline access to provider invoice detail incl. credits, discounts etc. (based on Fed IdM)

Helix Nebula GA, CERN Geneva, May 2014 20

Interoperability wish-list developed as D6.2

• 17 activities identified for future possible implementation

Helix Nebula GA, CERN Geneva, May 2014 21

Roadmap: HNX 1.24Q 2014

Blue Box SlipStream 2.3

API HNX APIBridge Updated for HNX API

Commercial Electronic Contracting

SLAs IaaS Provider SpecificMulti-Provider SLA Support

Support Additional Brokerage Functions

IaaS CSPs Additional Suppliers (+3-4)

Networking GEANT Agreement Finalised

EGI EGI FedCloud Integration

Catalogue Broker IaaS Performance Profiling

Other PaaS functionsProvisions for Information aaS

Helix Nebula GA, CERN Geneva, May 2014 22

HNX v1.2 (Q4 2014)

GÉANT / Internet

EC2 BridgeIaaS

Broker

Atos CloudSigma Interoute T-SystemsEGI

FedCloud

Marketplace Operator

API

NewProvider

New Provider

new functions

PaaS