implementation of a european e infrastructure century · physics, space science and biology to...

CER

N-O

PEN

-201

3-01

925

/10/

2013

EIROforumITWorkingGroup25thOctober2013

This document produced by Members of the EIROforum and is licensed under a Creative Commons Attribution 3.0 Unported License.�Permissions beyond the scope of this license may be available at http://www.eiroforum.org/

Implementation of a European e‐Infrastructure for the 21st Century

Executive Summary Thisdocumentproposesanimplementationplanforthevisionofane‐infrastructureasdescribedin“AVisionforaEuropeane‐Infrastructureforthe21stCentury”.Theobjectiveof the implementation plan is to put in place the e‐infrastructurecommons that willenable digital scienceby introducing ITasa service to the public research sector inEurope.The rationale calls for a hybrid model that brings together public and commercialservice suppliers to build a network ofResearchAcceleratorHubs offering a range ofservicestoawideuserbase.Theexploitationplatformwillmakeuseofandcooperatewith existing European e‐infrastructures by jointly offering integrated services to theend‐user. This hybridmodel represents a significant change from the status‐quo andwill bring benefits for the stakeholders: end‐users, research organisations, serviceproviders(publicandcommercial)andfundingagencies.ResearchAcceleratorHubscanbeownedandoperatedbyamixtureofcommercialcompaniesandpublicorganisations.Theirportfolioofservices,usingasastartingpointthose listedbye‐IRGandtheHighLevelExpertGrouponScientificData,willbemadeavailableunderasetoftermsandconditions that are compliant with European jurisdiction and legislationwith servicedefinitionsimplementingrecognisedpoliciesfortrust,securityandprivacynotablyfordataprotection.Afundingmodelengagingallstakeholdergroupsisdescribed.Theabilitytofullyexploitthepotentialforknowledgeandjobcreationthatislocked‐upin the datasets and algorithms to be hosted by the Research AcceleratorHubs willrequirethenurturingofanewgenerationofdatascientistswithacoresetofICTskills.Amanagementboardwhereall theResearchAcceleratorHubs operatingorganisationsarerepresentedwillprovidestrategicandfinancialoversightandauserforum,throughwhich the end‐users themselves, in a cross‐disciplinary body collaborate to definerequirementsandpoliciesfortheservices.Apilot service isproposed that canbe rapidly establishedbybuildingon theexistinginvestments. The pilot service will demonstrate the feasibility of the network ofResearchAcceleratorHubsmodel for a range of scientific disciplines and evaluate thesuitabilityfortheESFRIResearchInfrastructures,thatarecurrentlyunder‐developmentand represent Europe’s future “bigdatafactories”. Implementationwill start in 2014,initiallyofferingalimitedsetofservicesatseveralprototypeResearchAcceleratorHubs.



Contents

ExecutiveSummary............................................................................................................................1

Introduction..........................................................................................................................................3

Rationale.................................................................................................................................................4

Benefitsofthehybridmodel.....................................................................................................7

Architectureofthee‐infrastructurecommons......................................................................8

PilotService...........................................................................................................................................9

PortfolioofServices...................................................................................................................10

TrainingandEducation.................................................................................................................10

Integrationofpublicandcommercialservicesuppliersintoahybridmodel.......11

GovernanceModel...........................................................................................................................11

RelationshiptoexistingEuropeane‐Infrastructures.......................................................12

GEANT..............................................................................................................................................12

PRACE...............................................................................................................................................12

EGI......................................................................................................................................................13

VolunteerComputing................................................................................................................13

EUDATdataservices..................................................................................................................13

Engagementoffundingagencies...............................................................................................14

PrototypeResearchAcceleratorHubs....................................................................................16

EMBL‐EBIasaResearchAcceleratorHub........................................................................16

CERNandWignerasaResearchAcceleratorHub........................................................17

FundingModelsforthePrototypeResearchAcceleratorHubs..............................20

Timeline...............................................................................................................................................21

ThisdocumenthasbeenpreparedbyCERN’sITdepartmentonbehalfoftheEIROforumITworkinggroup.



Introduction Thisdocumentproposesanimplementationplanforthevisionofane‐infrastructureasdescribed in a separate document1“AVisionforaEuropeane‐Infrastructureforthe21stCentury”.Theobjectiveoftheimplementationplanistoputinplacethee‐infrastructurecommons that will enable digital scienceby introducing ITasa service to the publicresearchsectorinEurope.EIROforum3partnersareintergovernmentalresearchorganisations–CERN,ESA,EMBL,EFDA,ESO,EuropeanXFEL, ILLandESRF–coveringdisciplinesranging fromparticlephysics, space science and biology to fusion research, astronomy, and neutron andphotonsciences.ThepartnerorganisationshaveatrulyEuropeangovernance,fundingandremit,andinmanycasesshareaglobalengagement.Theyareworldleadersinbasicresearch, as well as in managing and operating large research infrastructures andfacilities. The EIROforum collaboration is helping European science reach its fullpotential through exploiting its unparalleled resources, facilities and expertise. Bycombining international facilities and human resources, EIROforum exceeds theresearchpotential of the individual organisations, achievingworld‐class scientific andtechnological excellence in interdisciplinary fields. EIROforum works closely withindustrytofosterinnovationandtostimulatethetransferoftechnology.Thisimplementationplanestablishesapilotserviceaimedataddressingthechallengesahead to capture,manage, andprocess the vast amounts of data tobe generated, notonly by the established fundamental research domains but in a growing range ofscientific disciplines, large and small. This pilot service is seen by the researchorganisationsasanexploratoryactivitytobeoperatedandfundedinadditiontotheirbaselineplansfortheproductionservicestheyoffertotheirusercommunities.

1 http://cds.cern.ch/record/1550136/files/CERN-OPEN-2013-018.pdf 3 http://www.eiroforum.org/



Rationale E‐infrastructureshavemaderemarkableprogressoverthelastfewyears.Movingfrombespokeinstallationsforsingleapplicationstodistributedstructuresservingmanyusercommunities.Inordertofulfilthevisionforthefuturethesee‐infrastructuresmustbebrought together to provide integrated solutions to awider range of users during allstages of the research life‐cycle. The justification, or rationale, to push for theimplementation of this vision is that the impact of combining the e‐infrastructures toform an e‐infrastructure commons will generate far greater added value for all thestakeholders. This rationale has been inspired by the Helix Nebula initiative’spublication “HelixNebula –The Science Cloud:A catalyst for change inEurope”4fromwhichextractshavebeentaken.HelixNebulawasconceivedasawayofbringingcoherence toahighly fragmented ITservices industry through the vision of a federated ‘science cloud’ integrated withpublicly‐fundedscientifice‐Infrastructures.TwofundamentalstepsremainifthevisionofHelixNebulaistobecomereality. Oneistobringtogetheracriticalmassofsupply‐sideinterestsandtheotheristodothesameforthedemand‐side.The work of Helix Nebula on an architecture model5has produced a number ofscenarios exploring the means by which publicly funded infrastructures caninteroperatewithcommercialcloudservices6.Suchhybridsystemsareintheinterestofthe users of publicly funded infrastructures and funding agencies because they willprovide ‘freedom and choice’ over the source of resources to be consumed and themanner inwhichtheycanbeobtained,facilitatingthegoalof ‘makingeveryresearcherdigital’.Hybridsystemsservetheinterestsofsuppliersbyencouragingtheadoptionofcloud services by more researchers and thus creating a larger and more vibrantmarketplaceinwhichtheycanoffertheirservices.This integrationwill allow thepublic infrastructure users to strike their ownbalancebetween publicly funded resources and commercial cloud services while taking intoaccount aspects of policy and cost. Private sector service providers can supplyresources/services and additional capacity not available in the public fundedinfrastructures. Research and innovation activities that have the potential forcommercial exploitation can work with the private sector to unlock that potential,demonstratefeasibilityandmarketacceptance.Itisexpectedthatserviceswillmigratefrom publicly funded infrastructures to commercial suppliers as suitable offers,addressing cost and policy aspects, become available on themarket. In thismanner,HelixNebulais implementingwhatthee‐InfrastructrueReflectionGroup(eIRG)referstoasthee‐InfrastructureCommonsinits2012Roadmappaper7.

4 http://cds.cern.ch/record/1537032/files/HelixNebula-NOTE-2013-003.pdf 5 http://cdsweb.cern.ch/record/1478364/files/HelixNebula-NOTE-2012-001.pdf 6 http://cds.cern.ch/record/1548323/files/HelixNebula-D6_1.pdf 7 http://www.e-irg.eu/images/stories/publ/e-irg_roadmap_2012-final.pdf



Thishybridcloudhasthepotential tobringthepublicandprivatesectorstogethersothat the public sector has something to offer the private sector more than a simpledemand for cloud services. Thecreationof a continuumof services across thepublicandprivatesector, insteadofasimplesupply‐demandmodelwillalleviatethecurrentsituationwhichisinhibitingtheestablishmentofhybridcloudinfrastructures.Itisalsoinhibiting the establishment of big‐data services since there are a number ofsensitivities,explainedbelow,thatneedtobeaddressedbeforeamodelcanbeenvisagewherealldatasetsarehostedoncommercialcloudservices.

Business models need to be developed that ensure the Data Stewards8, which arefrequentlypubliclyfundedresearchcentres,retaincontroloverthecurationandusageof the datasets and the associated intellectual property. Researchers also needguarantees about the long‐termavailability of the cloud services and thedata. Such ahybrid model would address these points and allow the progressive migration ofservicesbetweenpublicandprivateproviders.Furthermore,itisofupmostimportancethatdatacatalogueservices,suchasthosebeingdevelopedbytheEUDATproject9,areavailabletoallresearchcommunities’datasetshostedbybothpublicandprivatecloudservices.

There exist several institutes in the biomedical sciences with well‐established datainfrastructures,suchastheEuropeanBioinformaticsInstituteandtheSwissInstituteofBioinformatics.Thenew,distributedinfrastructureELIXIRwilltakedatamanagement,curation and interoperability to the next, Europe‐wide level. ELIXIR in turn iscoordinating theFP7‐fundedBioMedBridgesproject,whichaims to integratedataandthus construct computational ‘data and service’ bridges between the new biomedicalsciencesinfrastructuresontheESFRIroadmap,includingBBMRI,INSTRUCTandECRIN.Together these span a wide range of disciplines from the biological, medical,translational and clinical domains. Their diverse datamanagement needs can be bestcateredforbyacombinationofdomain‐specificservicesthatareintegratedintoawiderecosystem of domain‐agnostic services. Other community specific research datainfrastructures have been established in recent years, such as The Catalog of Lifeindexing the world’s known species10, iMARINE11and GENESI‐DEC12, which haveproducedvaluable data curation tools and expertise, alongwith data sharingpolicies.Being able to interface thesenewdata e‐infrastructures into a hybrid cloudmodel asproposedherewillallowalargeruserbasetoexploitthedata,biggeropportunitiestocontribute scientific data to multi‐disciplinary research, and provide sustainabilitymodelsfortheircontinuedexistence.

8 http://en.wikipedia.org/wiki/Data_governance 9 http://www.eudat.eu/ 10 http://www.catalogueoflife.org/ 11 http://www.i-marine.eu/Pages/Home.aspx 12 http://www.genesi-dec.eu/



Initial investigations on the potential impact of cloud services in the researchcommunitysuggest that thecommercialpubliccloudservicesare likely tobeadoptedinitiallyforthe‘longtailofscience’conductedbyresearchersthatdonothaveaccesstosignificant in‐house computing resources and skills. Effortsmust bemade to simplifyaccesstocommercialcloudservicesforsuchgroupsthatgenerallyhavestraightforwardrequirementsandfrequentlydonothavesufficientin‐houseITexpertisetomanageandoperatetheirowncomputingresources.

Conversely,itwillrequirefurtherreductioninthecostsofcommercialservicesforlargeresearch userswith important in‐house computing capacity to find commercial cloudservices as financially attractive as is the case for small scale users. As one industryrepresentativeputit ‘whywouldIhireacaronadailybasisifIknowinadvanceIwilluseiteverydayfor3years–itwillbecheapertobuymyown’13.

It isrecognisedthatnotallpublicly fundedresearchcentresare inapositiontomakeaccurateestimationsof the total costofownershipof in‐house IT servicessince somecontributingcostsarebornebydifferentdepartments.Butinorderforthedemand‐sideuserstobeencouragedtopurchasecloudcomputingservices,theservicesofferedmustbeeconomicallyadvantageouscomparedtoothermeansofprocuringITservices.

These alternatives include purchasing and operating IT equipment internally whichrequirescapitalinvestmentandITexpertisebutremainseconomicallyattractiveforIT‐intensiveapplicationswithasustainedandpredictableusage.

13 Similar cost arguments are made in this article: http://www.theregister.co.uk/2013/04/26/cloud_magic_number/



Benefits of the hybrid model Thehybridmodeloutlinedintherationaleaboverepresentsachangefromthestatus‐quobut isnecessary tokick‐startnewandemergingsciencesuseofe‐infrastructures.Theidentifiedbenefitsofthismodelforthestakeholdersinclude:Users

Awiderangeofintegratedservicesanddataresourcesprovidingamorecompleteworkingenvironmentsupportingthefulllifecycleofresearchactivities

Easymeanstopublishtheirownresearchresultsanddata Freedomandchoiceoverwhichservicestouseaccordingtopreferencesfor

functionality,policyandbudget Anenvironmentintowhichtheirownbespokeservicescanbeintegrated Readilyavailabletrainingmaterialandsupport Timetofocusontheirresearchobjectivesratherthanhavingtolearnaboutand

dedicatetimetooperateITinfrastructure Keepcontrolovertheirresultsanddatawiththeassurancethattheywillbe

availableoverthelong‐term Haveavoiceonhowtheservicestheyusearegoverned

ResearchOrganisations

Ameanstoexposetheirresearchproductstoawidercommunity Anexploitationplatformthroughwhichresearchorganisationscanbemore

closelyconnectedtotheirusersandreactmorequicklytotheirchangingneeds Abilitytoprocureadditionalresourceson‐demand Groupednegotiationofserviceprocurementwithotherresearchorganisations Aready‐madechannelforimprovingknowledgeandtechnologytransferwith

industryServiceproviders(publicandcommercial)

Creationofamarketwithanextensiveuserbaseintowhichproviderscanoffertheirservices

Privilegedcommunicationchannelwithend‐usersandpurchasingorganisations Ameansofestimatingthescaleandsophisticationofservicesthatwouldbe

suitableforthepublicresearchsector Reducedriskandup‐frontinvestmentduringthedevelopmentandintroduction

ofinnovativeservicesFundingagencies

Ameansofencouragingtheuptakeofcloudservicesbythepublicsector Creationofanpublic‐privateinnovationplatformforICT Asimplemeansforimplementingopenaccesspoliciesofpublicationsanddata Democratisationofaccesstodataandcomputeresourcessotheyareavailable

toanyusers,laboratoryorproject,regardlessofsizeandlocation Achannelforincreasingtheimpactandexploitationofpubliclyfundedresearch Ameanstomonitortheimpactofpublicinvestments



Architecture of the e‐infrastructure commons Thee‐infrastructurevisiondocumentproposesacommonplatformforthe futurethatbuilds on the experience of the last decade and is flexible enough to adapt totechnological and service innovations. Such a platformmust provide the underlyinglayers of common services, butmust be adaptable to the very different and evolvingneedsoftheresearchcommunities.Theproposalhas3distinctlayersofservices:

1. European and international networks; services for identity management andfederation across all European research and education institutions andintegratedwithotherregionsoftheworld;

2. A small number of facilities toprovide cloud anddata servicesof general andwidespreadusage.

3. Software services and tools to provide value‐added abilities to the researchcommunities,inamanagedrepository:

a. The tools to provide those research communities that have access tolarge sets of resources the ability to federate and integrate thoseresourcesandtooperatethemfortheircommunity,potentiallysharingwithothercommunities;

b. Tools to help build applications: e.g. tools to manage data, storage,workflows,visualisationandanalysislibraries,etc.

c. Toolsandservices toallowresearchers to integrateeverydayactivitieswiththee‐Infrastructure;

d. Toolstohelpresearchcommunitiesengagethegeneralpublicascitizenscientists.

Figure1below showshow the layersof services arebrought together intoanoverallarchitecture. Akeyelement is theResearchAcceleratorHub(ReAcH)which isacentrethatoffersasetofmanagedITservicestailoredtotheneedsoftheirusers.TogethertheResearchAcceleratorHubs form a network of interoperating centres that can supportworkflows addressing the full research life‐cycle. The underlying high‐performancenetworkbringstogethertheResearchAcceleratorHubswiththeotherelementsofthee‐infrastructure commons. The high‐performance network extends to scientificinstruments which act as big‐data factories and additional data providers hostingimportantdatasetsaroundtheworld.Super‐computingsitesconnectedtothenetworkhave adual role: like scientific instruments they canbe seenasbig‐data factoriesbuttheyalsooffercapability‐styledataanalysis facilities fordatasetsheldat theResearchAccelerator Hubs or provided by external data providers. Volunteer computingstructures are an integral part of the e‐infrastructure commonsproviding significantcomputingcapacity aswell as channels for engagingcitizen scientistsand the generalpublic. All the elements of the e‐infrastructure commons share a federated identitymanagementsystemenablingsingle‐sign‐onaccessforuserstoanyresourceforwhichthey have authorisation. A common digital object identifier service promotes thecitationandre‐useofanypublicationordatasetinthee‐infrastructurecommons.



Figure1e‐infrastructurecommonsarchitecture

Pilot Service Inordertoverifythefeasibilityofthisvisionandrationale,apilotserviceisproposedthatcanberapidlyestablishedbybuildingontheexistinginvestmentstodemonstrateadded‐valuetotheresearchcommunityandthesustainabilityoftheapproach.Implementationwillstartin2014,initiallyofferingalimitedsetofservicesatprototypeResearch AcceleratorHubs. This ramp‐up phase will include a roadmap for how toextend the e‐infrastructure to a group of independently operated and cooperatingResearchAcceleratorHubs offering a richer set of layered services and higher totalcapacity to serve the European Research Area. Such facilities will be ResearchAcceleratorHubsprovidingcapacitystyleHighPerformanceComputing(HPC)14servicesfordatacentricapplicationsandwillbefederatedintoanetworkofsuchcentresacrossEurope. TheResearchAcceleratorHubswill not be duplicates but rather offer a set ofservicestailoredtotheirusers’specificneedsandbusinessmodels.

14 For a definition of HPC capacity computing refer to the ETP4HPC Strategic Research Agenda document http://www.etp4hpc.eu/wp-content/uploads/2013/06/ETP4HPC_book_singlePage.pdf



Portfolio of Services TheResearchAcceleratorHubswill offer an initialportfolioof services taken from thelist documented by eIRG in its blue paper of 201015with the technical characteristicsidentifiedbytheHighLevelExpertGrouponScientificDataintheir“RidingtheWave”reportfromthesameyear16.WhileeachResearchAcceleratorHubwilloffertheirownsetofservices,allsuchserviceswill incorporate state‐of‐the‐art security measures, including host‐based protections,systemloggingandaccesstraceability.Allserviceswillmakeuseofafederatedidentitymanagement system addressing authentication and authorization to ensure seamlessandsecureaccesstotheservicesanddata.WhiletheResearchAcceleratorHubswilltaketheroleandresponsibilitiesofDataCustodians17,theStewardsofthedata‐setshostedby the ResearchAcceleratorHubswill be responsible for defining and managing theaccess policy for their data. To ensure a trustful relationship is established with theusers,itwillalwaysbepossibleforStewardstoextractorremoveanymaterialthathasbeendepositedwiththeResearchAcceleratorHubs.Brokering services will be deployed so that the full portfolio of services from allResearchAcceleratorHubsisvisibletoallusersandcanbecomparedandaccessed.

Training and Education Thedesign,creationandoperationofe‐infrastructureservicesareessentialtoolsinthedevelopmentofskillsandcompetenciesfortheEuropeanmarket.Theservicesofferedby the network of ResearchAcceleratorHubswill provide a focus for the creation ofpublic‐privateteamsofskilledpersonnelacrossthecooperatingresearchorganisations,servicesuppliersandusers,enabling thesharingofknowledge todevelopaco‐designapproach18to services and applications. The ability to fully exploit the potential forknowledgeandjobcreationthatislocked‐upinthedatasetsandalgorithmstobehostedbythecentrewillrequirethenurturingofanewgenerationofdatascientistswithacoreset of ICT skills. To ensure the training and education services impact as large anaudienceaspossiblewhilecontrollingthecostsofofferingsuchservices,massiveopenonline course (MOOC19) techniqueswill be employedwhichwill be supported by theResearchAcceleratorHubsmodelitself via the digital library services. The EIROforumorganisations have core competences in training and education20which can form thebasis of this activity. Since its formation in 2002, EIROforum joint initiatives havealreadyresultedinanimportant impact intheareasofOutreachandEducation.High‐profile activities have involved thousands of European science teachers and students,thus emphatically supporting Europe‐wide efforts to raise interest in science andtechnology and to secure a sound recruitment base for European R&D efforts in thefuture. Such collaboration in the area of human resources also have a bearing on theEuropeanResearchArea:byattractingandretainingworld‐classresearchersinEurope,

15 http://www.e-irg.eu/images/stories/eirg_bluepaper2010_final.pdf 16 http://cordis.europa.eu/fp7/ict/e-infrastructure/docs/hlg-sdi-report.pdf 17 http://en.wikipedia.org/wiki/Data_governance 18 http://echallenges.org/e2010/outbox/eChallenges_e2007_ref_195_doc_3562.pdf 19 http://en.wikipedia.org/wiki/Massive_open_online_course 20 http://www.scienceinschool.org/



through technological exchange and common development, and by organising multi‐disciplinaryscientificconferences.

Integration of public and commercial service suppliers into a hybrid model The investigations within Helix Nebula have revealed that with current pricing ofcommercial services many public organisations will continue to find it financiallyattractivetoprovidetheservicesthemselvesin‐house.Hencethisimplementationplanforesees that ResearchAcceleratorHubscan be owned and operated bya mixture ofcommercial companies and public organisations. Each ResearchAcceleratorHubwillincludeapay‐per‐usageelementinitsfundingmodelbuttheoperatingorganisationisfreetodefineitsownpricingstrategyandimplementation.TheportfolioofserviceswillbemadeavailableunderasetoftermsandconditionsthatarecompliantwithEuropeanjurisdictionandlegislationwithservicedefinitionsimplementingrecognisedpoliciesfortrust,securityandprivacynotablyfordataprotection.By integrating commercial and publicly operated Research Accelerator Hubs into ahybrid model it will be possible to migrate workloads to commercial suppliers onceacceptable pricing and terms and conditions have been negotiated. In this way theResearch AcceleratorHubmodelwill bolster the creation of a market to which thecommercialsupplierscanoffertheirservices.

Governance Model Thegovernancemodelforeseenforthepilotservicewillbebasedontwobodies:

A management board where all the ResearchAcceleratorHubs operators arerepresentedandthestrategicandfinancialaspectsof theResearchAcceleratorHubmodelareaddressed.PolicyaspectsimpactingtheoperationofthenetworkofResearchAcceleratorHubswillbeaddressedbythemanagementboard.Theintroduction of a new Research Accelerator Hub into the network will beapprovedbythemanagementboardsubject toverificationof thepolicies(seeIntegrationofpublicandcommercialservicesuppliers)underwhichitoperatesandthequalityoftheservicesprovided.

A user forum21, through which the end‐users themselves, in a cross‐disciplinarybodycollaborateonrequirementsandpoliciesconcerningthee‐infrastructureservices.TheUserForumwillalsoactasaconsultationbodyformanagementboard.

The pilot service will demonstrate the feasibility of the e‐infrastructure ResearchAcceleratorHubsmodel for a range of scientific disciplines. Thepilot servicewill also

21 David Foster, Bob Jones, “Science Strategy and Sustainable Solutions; A Collaboration on the Directions of e-Infrastructure for Science”, CERN-OPEN-2013-017, http://cds.cern.ch/record/1545615/files/CERN-OPEN-2013-017.pdf



provide a means for the ESFRI Research Infrastructures that are currently under‐development and represent Europe’s future “big data factories”, to evaluate thesuitabilityoftheservicesprovidedbyResearchAcceleratorHubs.Theintegratedserviceportfolio offered by the network of Research Accelerator Hubs will enable multi‐disciplinary research across the Research Infrastructures and contribute to ensuringexcellenceinscience.

Relationship to existing European e‐Infrastructures TheResearchAcceleratorHubswillmakeuseofandcooperatewithexistingEuropeane‐infrastructures to build the e‐infrastructure commons by jointly offering integratedservices to theend‐user.Therelationshipwith theseexistinge‐infrastructureswillbedefinedviawrittenagreementsandisdescribedbelow.

GEANT The ResearchAcceleratorHubswill be connected to the GEANT network in order toprovide high performance access for the whole European Research Area and ensureintegrationwith other regions of theworld so Europe can be a leadingparticipant inglobal research challenges. Building on the experience gathered through the HelixNebulainitiativewhichisextendingGEANTconnectivitytocommercialdatacentreswillgreatly simplify the establishment of a hybrid model engaging both public andcommercialserviceproviders.

PRACE The capacity style HPC services for data centric applications offered by theResearchAcceleratorHubswillbecomplimentarytothecapabilitystyleHPCservicesprovidedbyPRACE. The expertise developed by PRACE and related projects in efficient parallelprogrammingparadigmsandoptimisingsoftwareforarangeofarchitecturesisdirectlyrelevant to the e‐Infrastructure Research Accelerator Hubs and application/servicedevelopers. The European Technology Platform for High Performance Computingproject22recentlypublishedaStrategicResearchAgenda forachievingHPC leadershipin Europe23which specifically highlights the upcoming big‐data challenges for leadingresearchactivitiesandtherelevanceofcloudservices:

“Europe is inauniquepositiontoexcel intheareaofHPCUsageandBigDataowingtotheexperiencelevelofcurrentandpotentialusers(andtherecognitionoftheimportanceofdatabysuchusersasCERN,ESA,andbiologicaldatabanks)andthepresenceof leading ISVs for large‐scalebusinessapplications.Europe shouldexploit that knowledge to create competitive solutions for big‐data businessapplications, by providing easier access to data and to leading‐edge HPCplatforms,bybroadentheuserbase(e.g.,throughCloudComputingandSoftwareasaService(SaaS),andbyrespondingtonewandchallengingtechnologies.”

22 http://www.etp4hpc.eu/ 23 http://www.etp4hpc.eu/wp-content/uploads/2013/06/ETP4HPC_book_singlePage.pdf



So aswell assharing expertise, the services offered by theResearchAcceleratorHubsandthePRACETier‐0andTier‐1centresshouldbeintegratedtoformpartoftheoveralle‐infrastructure ecosystem. Thiswill require the PRACEHPC centres to participate inthe federated identity management scheme and data sharing services if the PRACEcentresaretobefullyintegratedasserviceproviderswithinthismodel.

EGI The experience gathered by EGI in managing a federated grid infrastructure will bedirectlyrelevanttothenetworkofResearchAcceleratorHubsmodel.EGIhasalsobeenevaluating cloud technologies via the EGI federated cloud24. It is proposed that aconsolidated setofEGI sitesbecomeResearchAcceleratorHubs. Thiswill give theEGIdistributed computing infrastructure a clear direction for how to contribute itsexperience andmake a larger portfolio of services accessible to its existing user‐basewhileintroducingtheinnovationpotentialcreatedbytheuptakeofcloudcomputinginresearchandbusinesssectors.

Volunteer Computing Volunteer computing initiatives acrossEuropehave establishedproduction structureswhich serve a range of research communities. Such structures allow research andeducationorganisationsasawellasindividualsandcitizenscientiststocontributeandparticipate in research activities. Significant computing resources are assembled bystructures such as the International Desktop Grid Federation26that can support agrowing range of application types with very modest operational and coordinationoverheads. It is important that such structures become an integral part of the e‐infrastructurecommons.

EUDAT data services It is expected that data services currently under development by various projects,notablyEUDAT,willprovidecandidatesforfutureservicesandpotentiallyadditionale‐infrastructureResearchAcceleratorHubs. A goalwill be to introduce services that canprofit from theco‐locationofdataandcomputeservices tosupportmulti‐disciplinaryresearch. Metadata and indexing facilities across the set of services in all the e‐infrastructure ResearchAcceleratorHubsare seen as being particularly relevant. It isessential that new services are fully integratedwith existing services to preserve thedata and compute continuum of the exploitation platform and support the e‐infrastructurecommons.

24 https://wiki.egi.eu/wiki/Fedcloud-tf:UserCommunities 26 http://desktopgridfederation.org



Engagement of funding agencies Theimplementationplanandassociatedfundingmodelhavebeendesignedsothatthee‐infrastructure Research Accelerator Hubs can be sustained by their operatingorganisations according to a continuum of financial models ranging from sponsoredresources for peer‐reviewed scientific cases to communities who would pay for theservicestheyreceive.Additionalresourceswillberequiredinorderfortheseservicestobe expanded and to serve a wider range of users. The European Commission andnationalfundingagencieswillbeinvitedtobecomestakeholdersandcontributetotheexpansionofthee‐infrastructureResearchAcceleratorHubmodel.Theguidingprincipleis that funding from such stakeholderswill be focused on innovation of services anduptakebynewusercommunitiesandbusinessactorswhiletheoperationalcostswillbeborne by the operating organisations and the users themselves. Below is a non–exhaustivelistofareaswherefundingagenciesmaycontributetotheexpansionoftheResearchAcceleratorHubmodel:

Developmentofnewservicestobedeployedonthee‐infrastructure.Significanteffort will be required to co‐develop scalable services that can operate in adistributedenvironmentandserveawiderangeofusers.

Financial incentive scheme to increase adoption of services (both public andprivate)byusersincluding‘long‐tailofscience’researchgroupsandSMEs.

Engagingtheuseoftheservicesbynewresearchcommunities(e.g.curationofdata‐sets,connectionofidentityfederations,deploymentofcommunityspecificservices,trainingfornewusers,etc.)

Develop training and educational activities building on the e‐infrastructureservicestomaximisetheirimpact.Thiscanalsoincludeexpansionofservicestosupport for volunteer computing so that researchers can build citizen‐cybersciencecommunitiesandfurtherengagethegeneralpublicinscience.

Organisation of user forum events aswell as outreach and dissemination to arangeofaudiencesandproductionofmaterialforpolicyrelatedactivities.

International collaboration (beyond Europe) through interoperation withequivalentstructuresinotherregionsoftheworld.

ExpansionofthenetworkoftheResearchAcceleratorHubsacrosstheEuropeanmemberstatestoaddressnationalandthematicneeds.

Manyresearchorganisationsthatoperateresearch infrastructuresdonothavethe mandate to provide e‐infrastructure services to their users for themanagementandprocessingoftheirexperimentaldata.Thisrepresentsagapinthe scientific lifecycle and a missed opportunity to highlight the results andimpact of public funded research. These research organisations will requireassistancetobridgethisgapbysupportingtheiruserssotheycanmakeuseofe‐infrastructuresservicestomanageandprocesstheirexperimentaldata.

TherecentlycompletedeInfraNetproject27broughttogethernationalfundingagencies28fromacrossEurope to explore the economics of e‐Infrastructuresand alignEuropean

27 http://e-infranet.eu/ 28 http://e-infranet.eu/partners/



policies with the needs of the member states. In its final report29the projectrecommended:

“The use of standards‐based private cloud infrastructuresmay also provide theneeded flexibility in case of jobs that exceed available resources through cloudbursting (the use of external public clouds for the exceeding capacity) or bypooling individual institutional private clouds into a federated cloud that canprovide the necessary resources in a federated way, following the extremelysuccessfulapproachalreadyinuseforGRIDs.”

Themodel foranetworkofResearchAcceleratorHubsdescribed in thisdocumentcanserve as a blueprint to be expanded across Europe. The establishment of ResearchAcceleratorHubscould be supported by national funding agencies to address local orthematicneedsandprovideafocusforinnovationine‐Infrastructures.Suchnationalorthematic Research AcceleratorHubs should be integrated with existing European e‐Infrastructures(seeRelationship to existing European e‐Infrastructures) andwill be in a position to hostdata,informationandservicesthatforpolicyreasonsneedtoremainwithinanationalcontext. As these Research Accelerator Hubs come online they must be federatedtogethertoprovideaEuropeandimensiontothee‐infrastructurecommons.

29 Cloud Computing Economics: An evidence-based approach for Research Applications http://e-infranet.eu/wp-content/uploads/2013/02/Cloud-Computing-Economics-An-evidence-based-approach-for-Research-Applications1.pdf



Prototype Research Accelerator Hubs Several ResearchAcceleratorHubs will be required to satisfy the needs of the publicresearch sector. Here we outline two EIROforum prototypes that can act as“pathfinders”fortheoverallmodelandpilotservice.Itisproposedtousetheresourcesinstalled by CERN at theWignerResearch Centre for Physics inBudapest,Hungary30,and by EMBL‐EBI at Telecity’s Powergate site in London, UK, to develop prototypeResearchAcceleratorHubsforthepilotservice.

EMBL‐EBI as a Research Accelerator Hub EMBL‐EBIwilloperateapilotResearchAcceleratorHub toserve thebroad lifesciencecommunity based on its successful Embassy cloud, which it has been piloting since2011. EMBL‐EBI is Europe’s principal provider of life science services, and providesmanywell‐used resources, includingUniProtKB, Ensembl, PDBe andENA.The largestdatasets, such as the ENA’s DNA sequence archive requires around 2 Petabytes ofstorage, doubling every year, and many other datasets are growing rapidly. TheEmbassy cloud offers Infrastructure as a Service to other organisations (described as“tenant”organisations)using theprovenVMWare technology.Theservice isoperatedfromEMBL‐EBI'sresourceswithinitstier‐3Londondatacentres.The principal benefit offered by the Embassy cloud is the potential of having virtualmachines running on the same high performance network as the substantial publicbiology data resources held by EMBL‐EBI. This allows high bandwidth, low latencyoperations to successfully occur, for example, mounting file systems directly to thevirtual machines. This eliminates the requirement to replicate such data within thetenant's own remote infrastructure. Tenant organisations build and administer theirownvirtualisedinfrastructure,bothmachinesandnetworkswithaccesstoEMBL‐EBI'sportfolioofdatasets.There isno fundamental changeofdataaccess rightsbybeingatenant organization; the benefit is removing both the need to duplicate data and thecomplexdatamanagementofkeepingaremotecopyinsyncwithaEBIdataset.As of June 2013 there are 8 tenant organisations participating in the Embassy Cloudtrial, fromboththeacademicandprivatesectors.Theprivatesectorwillbepaying“atcost”forthemachineusage;currentlyacademicusersareselectedbytheirusecase.Usecaseshaveincludedtestingdirectaccesstothe1000Genomesdatacollection,aswellaslivepublicservicesoperatedbythirdpartiessuchasaEuropeanmirrorforOMIM.EMBL‐EBIplanstocontinuetoevolvetheEmbassymodelfrom2014.Thiswillinvolveasignificantscale‐upincapacitytoincludehigh‐performancecompute,providingamoreefficientcombinedpoolforinternalandtenantorganisations.Plannedusecasesincludelarge‐scale analysis of genomic data via partnerships with the International Cancer

30 http://press.web.cern.ch/press-releases/2013/06/cern-and-wigner-research-centre-physics-inaugurate-cern-data-centres



Genome Consortium and the metagenomics dataset of the high profile Tara Oceansproject. Further developments will target other life science use cases in medicine,agriculture and ecosystems, integration into a wider ecosystem of federated cloudinfrastructures, and further adoption of open technologies as they become available.Manyof these activitieswill be basedupon the outcomes of EIROforumHelixNebulainitiative,aswellasrelatedELIXIRPilotActions.

CERN and Wigner as a Research Accelerator Hub TheWigner centrewas contracted by CERN following a competitive call to tender towhich approximately 20 sites around Europe responded. Wigner is operated as anextension to the CERN Data Centre (hosted on the CERN site in Switzerland) andconsequentlyCERNcanassurethebusinesscontinuityforthecriticalsystemsincaseofmajor problems. The Wigner data centre is connected to CERN via multiple 100Gbsnetwork links. CERN is integrated with many of the world leaders in internationalnetworking,includinghostingaGEANTPointofPresence(PoP)aswellasbeingamajorcenter for commercial networking for both telecoms suppliers as well as InternetService Providers (ISPs). Additional network options provided by public and privatebodies are welcome according to the needs expressed by the users, ranging fromcommodityIPservicestodedicatedcircuitsdirectlytoWignerorviaCERN.The following initial portfolio of serviceswill be provided by this prototypeResearchAcceleratorHub:

Avirtualmulti‐tenantcomputeenvironmenttoprovisionandmanagenetworksofvirtualmachineson‐demand.

A‘dropbox’styleserviceforsecurefilesharingovertheinternet. A point‐to‐point reliable, automated file transfer service for bulk data

transfers31. An openAIRE compliant open access repository 32 for publications and

supportingdataallowinguserstocreateandcontroltheirowndigitallibraries.Persistent digital object identifiers will be assigned to all publically availableuploads so as to make them citable and permit the creation of a digital datacontinuumspanning fromexperimentaldata through topublications includinglinkstocommercialpublishers.

Along‐termarchivingservice33. Integrated Digital Conferencing 34 tools allowing users to manage their

conferences,workshopsandmeetings. Training services: Online training material35will be made available and re‐

enforced with advanced training via brief residential programmes andsecondments,suchasCERN’sopenlabsummerstudentprogramme36,whichhas

31 http://information-technology.web.cern.ch/services/file-transfer 32 http://www.zenodo.org/features 33 http://castor.web.cern.ch/ 34 http://indico-software.org/ 35 http://indico.cern.ch/categoryDisplay.py?categId=88 36 http://openlab.web.cern.ch/news/cern-openlab-summer-student-programme-invites-2013-applications



recentlybeenexpandedwith the engagementofEIROforumorganisationsandmorecompanies.

The services will be accessible in a single sign‐on (SSO) manner supported by afederatedidentitymanagementsystem. Theseserviceswillbe implementedusingtheOpenStack37open source software suite accessible via the most popular interfaces(includingEC2,S3andpotentiallyOCCI).OpenStackhasbeenselectedastheframeworkfor the e‐infrastructure services offered by the prototype Research AcceleratorHubbecauseithasemergedasaclearleaderinthevibrantcloudmanagementmarket.CERNhas extensively tested the software and has adopted it for the management of itsproduction resourcesat itsdata centre inGeneva.OpenStack is fullyopen sourceanddistributedundertheApache2licensesupportedbyagrowingconsortiumofmorethan180 private companies and public organisations with a well‐defined and transparentgovernance structure representing a global community of more than 9,000 peopleacross87countries.Morethan800developersarecontributingtothesoftware.AlloftheseattributesmeansOpenStackdoesnotrequirepublic fundstobemaintainedandoffersoneofthebestenvironmentsforinnovation.RecentdevelopmentsleadCERNtoconcludethattheopensourcesoftwaretechnologywill shortly be available tomake this federation of a network ofResearchAcceleratorHubs technically feasible. Federation is stated as an objective of the OpenStackFoundation38:

“The vision of federation across deployed OpenStack clouds is critical to theOpenStack vision. At the core OpenStack is a framework for managing andprovisioning compute, network and storage. From that core many things arepossible. But,we hope andwill push for the OpenStack core to remain robust,consistentand stable.Yes,everydeploymentwillhave itsownconfigurationandtechnicalspecifications,butaslongasthecoreconceptsandnativeAPIsarerobustandconsistent,thepossibilitytodriverealinteroperabilitywillexistandthevaluefromthisisahugeadvantageoftheopencloudconcept”.

Long term sustainability, governance and collaborationwith industry are achievedbybuildingtheprototypeCentreofExcellenceontheexistingopensourcecommunityandmeansnationalandEuropeanfundingcanbefocusedoninnovation.The services provided by the prototype ResearchAcceleratorHubwill not be madeavailable on a commercial basis. CERN rents space in the Wigner data centre andcommercial suppliers can discuss directly with the local authorities if they areinterested in locating their own services inWigner. This co‐locationmodel is alreadyusedinanumberofcentresaroundEuropeofferingservicestothebusinessandpublicresearchsectors,includinganinternetexchangepointatCERN’sdatacentreinGeneva39andtheEMBL‐EBIEmbassycloudthatenablesexternalorganisationstoperformsecuredataanalysesonlargedatasetsusingvirtualmachines.

37 http://www.openstack.org/ 38 http://www.rackspace.com/blog/an-open-letter-to-the-openstack-community/ 39 The CERN Internet eXchange Point, http://cixp.web.cern.ch/



The proposed fundingmodel for the pilot service is that CERNwill enter into formalagreementswithpartnerresearchorganisationsthatwishtojointlydevelopandusetheservices offered by the prototype Research Accelerator Hub. CERN will takeresponsibility foroperating theservicesat theResearchAcceleratorHubandcoverthecosts during the first year of operation. Beyond this first year, each partner researchorganisation engages to fund the cost of the services their registered users consumeaccording to a pay‐per‐usage model where the detailed cost model will be jointly‐developedduringthefirstyear.Shouldapartnerresearchorganisationwithdrawfromthepilotservice thenaccess to theservicesof theprototypeResearchAcceleratorHubfortheirregistereduserswillbedisabledfollowingagraceperiodafterwhichthehostorganisation will no longer guarantee the integrity of any material under itsstewardship. The costs of operating the Research Accelerator Hub at the capacityforeseenforCERN’sownusagewillcontinuetobepaidbyCERNitself.CERNwillchargeto the partner research organisations the additional costswhich are incurredwhenincreasing the site’s capacity for the purpose of the ResearchAcceleratorHub. Theseadditional costs will include processors, storage, network switches, cabling, energy,software licensing costs and operational staff. CERN will operate the services on thesamehardwareconfigurations40andatthesamelevelofqualityasthoseofferedtoitshigh energy physics user community. As theDataCustodian, CERNwill not exert anyownership or intellectual property rights over the material that has been deposited.Toolsformonitoringtheresourcesconsumedandthequalityofservicesprovidedwillbemadeavailable.Theserviceswillbeintegratedinabottom‐upmanner,startingwithbasic IaaS services and allowing each partner research organisation to integrate andmanagetheirownhigher‐levelservicesandportals.Anynewproposedservicewillonlybe introduced if there is at least one identified partner research organisation that isprepared to contribute to the costs of operating the service. Support for the cloudservices will be integrated into CERN existing support structures (e.g. the standardservice portal41). Effort will be required from each partner research organisation tocurate their data‐sets, connect their identity federations, deploy their communityspecific services and portals as well as manage the interaction with their registeredusersandassociatedsupportactivities.

40 Technical Description Servers and Storage for Physics Data Processing, Acquisition and Control, CERN, MS-3903/IT, http://cds.cern.ch/record/1472671/files/MS-3903_Technical_Description.pdf 41 https://cern.service-now.com/service-portal/



Funding Models for the Prototype Research Accelerator Hubs ToensurethesustainabilityoftheprototypeResearchAcceleratorHubsfundingmodelsareenvisagedwhereallthestakeholdersparticipate.Sustainabilityisseenasaprocessratherthanafixedsolutionandhencethefundingmodelswillevolveovertime.The rationalisation offered by the prototype Research Accelerator Hubs and theirintegrationintoahybridmodelwillprovideanumberofadvantages:

Provide a clear example of the scale and nature of the services thatwould berequiredfromcommercialsuppliers.

Offer “user aggregation” of procurement since the prototype ResearchAcceleratorHubswillbeinapositiontonegotiatewithsuppliersonbehalfofallthepartnerresearchorganisations.Thiswillensurethatthemostcosteffectivemeansofprovidingaservicewillbeavailableatanymomentintime.Thiswillalso simplify the contractual and administrative aspects of procuring servicesfromcommercialsuppliersforthecooperatingresearchorganisations.

Theprocurementanddeploymentcycleforthecentrewilltakeseveralmonthsandsotheintegrationwithcommercialcloudserviceproviderswillbeusedtoprovide additional capacity in the interim. This will bring added value to thecooperating research organisations and their users by making it simpler forthemtousecommercialcloudservicesin‘bursts’toaccessadditionalresourcestofulfilanyshort‐termrequirements.

The technical aspectsofmigratinguserworkloads to commercial serviceswillbesimplifiedandaccelerated.

TheprototypeResearchAcceleratorHubswillmakeitpossibletohostdata‐setsonacost‐recoverybasiswhileofferingopenaccesssotheycanbeexploitedviacommercial services. This will remove the need for commercial suppliers tomake investments to host data‐sets without a clear estimation of how muchbusinesstheywillgenerate.

Thearchivingservicewouldofferameansofensuringthatdata‐setshaveasafeback‐upcopy.

The prototype Research Accelerator Hubswill, in relations to Europe’s HPCstrategy,addresstheneedformoreflexible,easier‐to‐use,moreproductiveandmorecost‐effectiveHPCsystems.



Timeline Thetimelinebelowshowsthemajormilestonesforeseenfromconceptionthroughtotheestablishmentofthepilotservice.FourthQuarter2013:

DiscussimplementationplanwithrelevantECprojects RefineimplementationplansfortheprototypeResearchAcceleratorHubswith

furtherdetailsofeachserviceoffered,SLA,termsandconditionsofservice Firstmeetingoftheuserforum

Firstquarter2014:

Firstmeetingofmanagementboard Initialportfolioofservices(seePortfolioofServices)madeavailablefortesting

byasmallnumberofresearchpartnerorganisation(s)viatheprototypeResearchAcceleratorHubs

SecondQuarter2014:

ManagementboardmeetstoapproveopeningofthepilotserviceThirdQuarter2014:

Startofpilotservice

implementation of a european e infrastructure century · physics, space science and biology to...

Documents