Helix Nebula Science Cloud:Getting commercial suppliers andresearch organisations together

29 November 2017Bob Jones

CERNBob.Jones <at> cern.ch

Helix Nebula – The Science Cloud

Helix Nebula – The Science Cloud with Grant Agreement 687614 is a Pre-Commercial Procurement Action funded by H2020 Framework Programme

HNSciCloud Joint Pre-Commercial Procurement

Procurers: CERN, CNRS, DESY, EMBL-EBI, ESRF, IFAE, INFN, KIT, STFC, SURFSaraExperts: Trust-IT & EGI.eu

The group of procurers have committed• Procurement funds• Manpower for testing/evaluation• Use-cases with applications & data• In-house IT resources

Resulting services will be made available to end-users from many research communities

Co-funded via H2020 Grant Agreement 687614

Total procurement budget >5M€

12/1/2017

What is being procuredA hybrid cloud platform for the European research community

3

HNSciCloudPCP

Sou

rce:

Clo

ud

Co

mp

uti

ng

for

Go

vies

, DLT

So

luti

on

s,D

avid

Bla

nke

nh

orn

, V

an R

ista

uan

d C

aro

n B

eesl

ey

Combining services at the IaaS level to support science workflows

The R&D services to be developed are to be integrated withResources in data centres operated by the Buyers Group

GEANT network

HNSciCloud project phases

Preparation

•Analysis of requirements, current market offers and relevant standards

•Build stakeholder group

•Develop tender material

Implementation & Sharing

Jan’16 Dec’18

Each step is competitive - only contractors that successfully complete the previous step can bid in the next

4 Designs 3 Prototypes2 Pilots

Call-off Feb’17

Call-off Dec’17

Tender Jul’16

We are here

€ €€ €€€

Access to cloud service capacity

Bob Jones, CERN5

100C

10TB

2,000C

200TB

3,500C

350TB

3,500C

350TB

Jun

2017

Dec

2017

Feb

2018

5,000C

500TB

10,000C

1PB

Dec

2018

prototype x3 pilot x2

Functional

Testing

Scalability

Testing

End User Access

What is needed to ensure alignment of the EOSC with user’s needs and enable users to actively participate in service portfolio management?

Requirement gathering process engaging users/procurers and service providers

Users identify requirements and prioritise them

Service providers associate risk to each requirement

Collectively converge of what is achievable

All must be aware of the cost of the services

Ensures users focus on high priority requirements and are ready to verify they are satisfied

01/12/2017

7

Prioritization grid: value vs. risk

Must haves This could make

the true difference

Off-the-shelf Avoid if possible

Risk(Innovation potential from

technological/market point-of-view)

Value(Innovation potential

from end-user

perspective)

0 10 100

100

10

0

Innovative for end-user

+

Technological/market uncertainty

8

Planning Poker - Value

The cards:

Planning poker is a ‘best practice’ methodology to estimate e.g. added value, level of complexity, required

implementation effort, risk. The technique is based on domain expert evaluation and attaining consensus.

The interpretation when assessing value:

0-2 Could not care less, not needed.

3-5 OK, why not, could perhaps bring some value

13 Must have. Solution should have this requirement

20-40 WOW, home run, this would really shift the boundaries !

? Don’t know, no experience with this subject..

9

Planning Poker - Risk

The cards:

Planning poker is a ‘best practice’ methodology to estimate e.g. added value, level of complexity, required

implementation effort, risk. The technique is based on domain expert evaluation and attaining consensus.

The interpretation when assessing risk:

0-2 No worries, off-the-shelf standard solutions exist.

3-5 A frequent problem, certainly solvable.

13 Absolutely not a standard problem. A solution requires

important choices, thorough elaboration, and specific expert

effort. Success can be achieved with significant time and effort.

100 Impossible, requires breakthroughs of physical laws.

? Don’t know, no experience with this subject..

Results of risk/value assessment

Published report on the HNSciCloud Open Market Consultation

https://doi.org/10.5281/zenodo.254052

01/12/2017

Challenges

Innovative IaaS level services integrated with procurers in-house resources and public e-infrastructure as part of a hybrid cloud to support a range of scientific workloads

Compute and Storagesupport a range of virtual machine and container configurations including HPC working with datasets in the petabyte range

Network Connectivity and Federated Identity Managementprovide high-end network capacity via GEANT for the whole platform with common identity and access management

Service Payment Modelsexplore a range of purchasing options to determine those most appropriate for the scientific application workloads to be deployed

1112/1/2017

What criteria should services of the EOSC meet in order to be part of the service portfolio?

Technical – defined in the tenderRefer to relevant international stds whenever possible

Legal – defined in the tenderExclusion, selection & compliance criteria including legislation/jurisdiction under which services are operated

Signature of a common cloud services agreement by all contractors and procurers

Financial – M.E.A.T Most Economically Advantageous Tender

01/12/2017

How can certification be organized to ensure that services of the EOSC service portfolio meet a list of community-defined requirements?

Testsuite assembled by procurers to verify technical conformance

Monitoring of SLA criteria linked to payment

service availabilitysupport responsivenessperformance

etc.

01/12/2017

HNSciCloud test suite

14

CERN

• Edugain AAI

• CERN Benchmark Suite

• S3 Endpoint

• PerfSONAR Network

• VM Provisioning and Personalisation

• Grid Jobs in Docker

• Dockerized deployment of EOS+CERNBOX+SWAN

CNRS

• IAAS access via EduGain and local accounts

DESY

• HDF5_IO

EMBL

• CLOUD_BENCHMARK

• ELIXIR_AAI

• data_transparency

ESRF•FDMNESIFAE•monitoring-accounting•image creation and contextualizationINFN•BelleII_SIM•COMPASS/TGEANT_IO•INFN/GPFS_AFM•DynfarmKIT•static-batch-extension•dynamic-batch-extension•AFM-deployment-testSURFsara•BBMRi_data•dCache_io•kubertest

01/12/2017 – DI4R – Brussels