complete dynamic multi-cloud application management€¦ · • scalability, reliability, and...

CompleteDynamicMulti-cloudApplicationManagement

Projectno.644925

InnovationAction

Co-fundedbytheHorizon2020FrameworkProgrammeoftheEuropeanUnion

Callidentifier:H2020-ICT-2014-1

Topic:ICT-07-2014–AdvancedCloudInfrastructuresandServices

Startdateofproject:January1st,2015(36monthsduration)

DeliverableD6.4

SummaryofProvidedBrokering,Deployment,andManagementFeatures

Duedate: 30/09/2017

Submissiondate: 27/10/2017

Deliverableleader: TUB

Editorslist: DirkThatmann(TUB),C.Loomis(SixSq)

DisseminationLevel PU: Public PP: Restrictedtootherprogrammeparticipants(includingtheCommissionServices) RE: Restrictedtoagroupspecifiedbytheconsortium(includingtheCommissionServices) CO: Confidential,onlyformembersoftheconsortium(includingtheCommissionServices)

H2020-ICT-644925–CYCLONED6.4:SummaryofProvidedBrokering,Deployment,andManagementFeatures

CYCLONE_D6.4_feature_summary-v1.1.docx Page2of36

ListofContributors

Participant Short Name Contributor

Interoute S.P.A. IRT Domenico Gallico

SixSq Sàrl SIXSQ C. Loomis, K. Skaburskas, L. Schaub, K. Basbous

QSC AG QSC

Technische Universitaet Berlin TUB Dirk Thatmann, Mathias Slawik

Fundacio Privada I2CAT, Internet I Innovacio Digital A Catalunya

I2CAT

Universiteit Van Amsterdam UVA Alexéy Ilyushkin

Centre National De La Recherche Scientifique

CNRS



Changehistory

Version Date Partners Description/Comments

0.0 13/09/2017 SixSq Table of contents

0.5 19/10/2017 SixSq Initial draft for use case input and feedback.

0.6 23/10/2017 TUB Feedback on initial draft.

0.9 24/10/2017 SixSq Final draft for internal review.

1.0 27/10/2017 SixSq Final document.

1.1 02/11/2017 SixSq Fix missing EDL17 reference.



TableofContents

ListofContributors..................................................................................................................2

Changehistory........................................................................................................................3

ListofTables...........................................................................................................................5

ListofFigures..........................................................................................................................6

ExecutiveSummary.................................................................................................................7

1. Introduction........................................................................................................................8

2. ApplicationCuration...........................................................................................................9

3. AuthenticationandAuthorization......................................................................................113.1.AuthenticationMethods.................................................................................................................113.2.AccessControl.................................................................................................................................12

4. CloudApplicationDeploymentandManagement..............................................................144.1. ServiceCatalog...............................................................................................................................144.2.DeploymentEngine.........................................................................................................................144.3.ApplicationScalability....................................................................................................................154.3.1. RiemannScaling......................................................................................................................154.3.2. PegasusScaling.......................................................................................................................16

4.4.Multi-CloudSupport.......................................................................................................................174.5. Features..........................................................................................................................................18

5.Monitoring.........................................................................................................................20

6. Scalability,Reliability,andUsability...................................................................................22

7. Summary...........................................................................................................................23

References.............................................................................................................................25

Glossary.................................................................................................................................27

AppendixA Requirements.................................................................................................28



ListofTables

Table1:SlipStreamCloudConnectors............................................................................................................18Table2:NuvlaCloudServiceProviders...........................................................................................................18Table3:SummaryofAvailableandPlannedFeatures....................................................................................24Table4:ImplementationStatusofRequirements..........................................................................................28Table5:CollectedRequirements....................................................................................................................29



ListofFigures

Figure1:SlipStream(Nuvla)AuthenticationServices.....................................................................................12Figure2:PegasusAutoscalingwithSlipStream...............................................................................................17Figure3:OldInterfaceforCloudUsageInformation......................................................................................21



ExecutiveSummary

AtthebeginningoftheCYCLONEproject,SlipStreamwasalreadycapableofmanagingthebasiclifecycleofcloud applications. With input and feedback from the application developers of the targeted use cases(WP3),potentialimprovementswereidentifiedandrequirementsdefined.

Over the last three years, the project has significantly enhanced the brokering, deployment andmanagementfeaturesformulti-cloudapplicationsbyextendingSlipStream.TheusecasesfromWP3haveprovided guidance on the features to be implemented and have validated those features with Nuvla,SixSq’scommercialSlipStreamservicethatisacomponentofCYCLONE’stestbed.ThefactthatNuvlawasupdated fortnightly with CYCLONE updates is further evidence that the developed features are ofproductionqualityandusefultousersfromdiversedomains.

Thisdocumentsummarizesthebrokering,deployment,andmanagementfeaturesthatareavailablefromSlipStream, the core service within the CYCLONE project for cloud application management. Theimplementedfeaturesinclude:

• Application curation: Portability, Automated Deployment, Component Coordination, ApplicationParameterization,SharedApplications,andStockComponents.

• Authenticationandauthorization:AuthenticationbyUsername/Password,APIKey/Secret,GitHub,eduGAIN,ElixirAAI;AuthorizationbyUserorbyRole/Group;andRole/Groupdefinition.

• Deployment and management: Offer-Based Provisioning, Multi-Cloud Deployments, HorizontalScaling,VerticalScaling,RankingbyCost,andPolicyConstraints.

• Monitoring:Benchmarking,CurrentUsage,HistoricalUsage,andQuota.

• Scalability,reliability,andusability:PythonAPI,LibcloudDriver,andClojure(Script)APIinadditiontoageneralmovetowardsmicro-servicestoimprovethescalabilityandreliability.

Overall, therewere56requirementsofwhich33 (59%)were fully implemented,11 (20%)remainontheshort-termroadmap,12(21%)willnotbeimplementedforvariousreasons.

As SlipStream is a commercial solution at the core of SixSq’s portfolio, the evolution of SlipStreamwillcontinueaftertheendofCYCLONE.Asmentionedabove,11oftherequirementsremainontheshort-termSlipStreamroadmapandwilllikelybeimplementedwithinthenextsixmonths.Theseinclude:

• Applicationcuration:Services,HierarchicalApplications,ModuleIsolation,andGitHubIntegration.

• Authenticationandauthorization:CompletionofCIMIMigrationandResourceSegmentation.

• Deploymentandmanagement:ExtendedPre-Filtering,User-SpecifiedRanking,ScalingwithOffers,ScalingthroughtheUI,andDynamicApplicationTopology.

• Monitoring:ExtendedResourceCoverageandUser-DefinedQuotas.

• Scalability,reliability,andusability:ContinuedMigrationtoMicro-Services.

One largearea that remains tobecovered is the inclusionofdatamanagement.Experimentshavebeenconducted with the European Space Agency to see how data resources can be defined in the ServiceCatalogandwithHNSciCloudtounderstandhowtointegratescientificdatamanagementservices, inthiscaseOnedatafromtheIndigoDataGridproject.



1. Introduction

AtthebeginningoftheCYCLONEproject,SlipStreamwasalreadycapableofmanagingthebasiclifecycleofcloud applications. With input and feedback from the application developers of the targeted use cases(WP3),anumberofimprovementswereidentified.Previousdocumentshavedescribedtherequirementsassociatedwiththoseimprovementsandplansforimplementation.

Thefeaturesareorganizedintothefollowingcategories:

• Applicationcuration,

• Authenticationandauthorization:

• Deploymentandmanagement,

• Monitoring,and

• Scalability,reliability,andusability.

Ashortsummaryofthefeaturesisprovidedattheendofthedocument.TheappendixcontainsdetailedcommentaryonindividualrequirementsdefinedinthepreviousthreedeliverablesofWP6.

Thefeedbackfromtheusecases (WP3)anddevelopments fromtheotherCYCLONEworkpackageshavebeen instrumental in defining the direction of the CYCLONE developments, prioritizing changes, andvalidatingthosethathavebeenimplemented.DetailscanbefoundintheWP3documents.Althoughsomerequested features have not yet been implemented, they are, in most cases, part of the SlipStreamroadmapthatwilldrivetheevolutionofSlipStreamthroughtheremainderoftheprojectandafterwards.



2. ApplicationCuration

Running applicationson a cloud system requiresmanagementof virtualmachine images. Inmany cloudmanagement systems, users generate these machine image files and then upload them to their cloudserviceprovider(CSP).Themanagementoverheadassociatedwiththetransport,conversion,andevolutionoftheseimagesdiscouragestheuseofmultipleCSPs.

SlipStreamtakesadifferentapproach:usersspecifytheresourcerequirements,placementconstraints,andthesoftwareinstallationandconfigurationprocedures.SlipStreamthenusesthisinformationtotransformexisting,minimalimages(optimizedforeachcloudprovider)intothecustomizedVMrequestedbytheuser.Thishastwoadvantages:1)theimagedescriptionsareportableandcanbeusedforanycloudsupportedbySlipStreamand2)allknowledgeabouttheapplicationiscapturedandmanaged.

Oncloudsthatsupportcustomizeduserimages,binaryimagefilescanbeproduced(“built”)bySlipStreamtoreducethestartuplatency.Usersmustexplicitlyrequestthebuildofthesebinaryimagefiles,butonceproduced,SlipStreamwillusethemautomatically.ThismaintainscloudportabilitywhileallowinguserstoshortenstartuplatenciesforparticularCSPs.

SlipStreamprovidesa“workspace” inwhichusersmanagetheirapplicationandcomponentdescriptions.Thesedescriptionscanbesharedwithotherusers. Inaddition, systemadministratorscanpublishvettedapplicationsintoan“AppStore”tomakethemvisibletoallSlipStreamusers.ApplicationsintheAppStorecanbefoundeasilyandlaunchedwitha“singleclick”.

Withinthecomponentdefinitions,theresourcerequirementandplacementconstraintsdirectlyaffecttheCSPsthatarechosen.Theplacementconstraintscanincludesecurity,location,availability,andothernon-functionalrequirementstosupportthedefinitionandenforcementofSLAs.

ThefollowingsubsectionshighlightfeaturesthatarecommonlyusedbyCYCLONEprojectparticipantsandwithintheportedusecases.Theyalsoidentifypointswheretheplatformcouldbefurtherimproved;thisfeedbackhelpsdefinetheroadmapforSlipStreamduringtheremainderoftheprojectandafterwards.

ImplementedfeaturescommonlyusedforCYCLONEplatformcomponentsandusecases:

• Portability. All the CYCLONE components have been designed for a multi-cloud (hybrid cloud)context. Consequently, all these components make use of the cloud portability features ofSlipStream. The use cases similarly take advantage of cloud portability, with most use casesdemonstratedonmultiplecloudinfrastructures.

• AutomatedDeployment.Users caneasilydefine complexapplications containinganynumberofcomponents. SlipStream fully automates the deployment of such applications, allowing users toconcentrate on managing the application as a whole, rather than the numerous individualmachines. This feature saves time by freeing people from the tedious details of a complexdeploymentbutalsobyavoidingmistakesthatoccurwithmanualdeployments.

• Component Coordination. In any complex application, there are dependencies between thevariouscomponents.Forexample, ina typical3-tieredwebapplication thebusiness logiccannotstart before the underlying database is ready, and of course, the business logicmust know theendpoints of the database. SlipStream provides a simple mechanism for passing information



betweenthecomponentsandcritically, forwaitingonrequired informationtobepublished.Thisfeatureisusedforallcomponentandapplicationdefinitions.

• Application Parameterization. The same feature that allows coordination between applicationcomponents also allows the application to take values for input parameters from the persondeployingtheapplication,promotinggeneralizationofapplicationsandreuse.Thisfeature isalsoused by application developers to feed information (e.g. randomized passwords or serviceendpoints)backtotheapplicationowner.AlltheCYCLONEcomponentsusethisparameterizationto provide reusable components. Most of the use cases also use this feature to allow forcustomizeddeployments,forexample,varyingtheinputdataforascientificanalysis.

• Sharing Applications. Having the ability to reuse component definitions and parameterizeapplicationsisof limiteduseunlesstheycanalsobesharedwithothers.Onthedeveloper’sside,thismeanssharing theworkofbuildingandmaintaininganapplicationbetweenteammembers.On theuser’s side, thismeansbeingable topublishapplications forusebyotherpeopleon theplatform. The SlipStream access control allows the developer to define the visibility of anapplicationbyothers.VettedapplicationscanalsobepublishedintheAppStore.AlltheCYCLONEcomponentsarevisible toallusersof theplatform.Theapplications forusecasesarecommonlysharedbetweenateamofdevelopersandatargetedsetofusers.

• Stock Components.Developers can takeadvantageof a largenumberof stock components thatare maintained by SixSq and published in the App Store. These range fromminimal operatingsystemimagestoscalablecontainerinfrastructuressuchasKubernetesorDockerSwarm.CYCLONEhastakenthesameapproachwithmostofthenetworking,security,andAAIcomponentsavailableas“bricks”toincorporatemorecomplexapplications.

Features identified through the CYCLONE activities that would further enhance the utility of the cloudapplicationmanagementplatform:

• Services.AlthoughmanyapplicationshaveasinglelifecycleandbenefitdirectlyfromSlipStream’sautomated deployments, many applications have subsystems with independent lifecycles. Forexample, operators of a 3-tieredweb applicationmaywant to operate the database separatelyfromthebusinesslogic.Nonetheless,theywouldstillliketobenefitfromSlipStream’scoordinationinfrastructuretoallowinformation(e.g.endpointsoraccesscredentials)tobepassedbetweenthedifferent“service”deployments.

• HierarchicalApplications.Allowinganoperator todecideatdeploymenttimewhether todeployanapplicationinitsentiretyortodeployitasasetofcooperatingserviceswouldprovidefurtherflexibility. To support this, it would be useful to be able to define SlipStream applicationshierarchically,allowingapplicationdefinitionstoincorporateotherapplicationsdirectly.(Currentlyapplicationscanonlyincorporatesinglemachinecomponents.)

• Module Isolation.TheSlipStreamworkspacehasaroot that issharedbyallusersof thesystem.Althoughthismakessharingabiteasier, itraisesthepossibilityofconflictswithtop-levelprojectnames and “pollutes” a user’s workspace with the visible projects of others. Removing thiscommon root while providing the ability to search for shared components would improve theusabilityoftheplatform.

• GitHub Integration.SlipStreamallowscomponentandapplicationdefinitionstobemanagedandversioneddirectlywithintheplatform.However,manydeveloperswouldpreferinsteadtobeabletomanage their SlipStream recipes next to their source code. Allowing SlipStream to integratedirectlywithGitHub (oranyotherpopular sourcecodemanagementsystem)would improve theusabilityoftheplatformandappealtoalargergroupofdevelopers.



3. AuthenticationandAuthorization

WhenCYCLONEstarted,theAuthenticationandAuthorizationInfrastructure(AAI)forSlipStreamwasbasic.Onlyauthenticationwithusernamesandpasswordsdefinedwithin the internaldatabasewassupported.SlipStream access control was limited to Unix-like permissions, allowing rights to be defined for owner,group(asanexplicitlistofusers),andothers.

TheSlipStreamAAIhas improvedconsiderablyover the lifetimeof theproject,expanding the supportedauthentication sources basedonwork from the securitywork package (WP4) and implementing amoreflexibleresourceauthorizationbasedonagenericAccessControlList(ACL).

3.1. AuthenticationMethods

To allownewauthenticationmethods to be addedmore easily, the SlipStreamauthentication codewassignificantlyrefactored.Addingdirectsupportfornewauthenticationmethodsnowonlyrequiresaddingacouple resources compatible with the Cloud Infrastructure Management Interface (CIMI) from DMTF[CIMI16] and a function to handle the interactions with the (external or internal) service that provideauthenticationinformation.Figure1showstheinternalandexternalauthenticationmethodsthatarenowsupportedbySlipStream(Nuvla).

The refactored authentication code contains “plugins” for handling the GitHub (OAuth2) [OAUTH] andOpenID Connect (OIDC) [OIDC] protocols for external authentication. The project’s Keycloak server,allowingaccesstotheeduGAINidentityfederationthroughtheGermanNRENDFN,isaccessedthroughtheOIDC protocol by SlipStream. As Keycloak supports awide variety of identity providers, sources such asLDAP,Twitter,Facebook,etc.canbesupportedviastraightforwardconfigurationchanges.

SixSq has deployed and certified a second Keycloak server to support the Helix Nebula Science Cloud(HNSciCloud) project. This second Keycloak server allows access to the eduGAIN [EDUGAIN] identityfederationthroughSWITCH,theSwissNREN.Inaddition,accesstotheidentityfederationforElixir[ELIXIR],a European flagship project for bioinformatics, has been configured. This deployment is an excellentvalidationoftheworkdoneonKeycloakwithinWP4,showingthegeneralityoftheserviceanditsutilityforlargescientificcollaborations.



Figure1:SlipStream(Nuvla)AuthenticationServices

SlipStream still supports authentication via usernames and passwords stored in SlipStream’s internaldatabase. In addition, SlipStream also allows users to generate separate API key/secret pairs. The APIkey/secretpairscanoptionallybecreatedwithanexpirationdate;anAPIkey/secretpaircanberevokedatany time. A different API key/secret pair can be generated for each client, allowing more fine-grainedcontroloverauthorizationforclientsaccessingSlipStreamviathecommandlineorAPI.

Insummary,theauthenticationmethodsthataresupportedare:

• Username/Password.SimplecredentialstiedtoauseraccountandstoredinSlipStream’sinternaldatabase.

• APIKey/Secret.CredentialsthatcanbegeneratedbyuserstoallowclientstoaccessSlipStreamviathecommandlineorAPI.Thepairscanberevokedindependentlyofoneanother;theycanalsobecreatedwithanexpirationdate.

• GitHub(OAuth2).DirectsupportforauthenticationviatheOAuth2protocol,configuredonNuvlaforGitHub.

• eduGAIN(OIDC).SupportfortheeduGAINidentityfederationviatheCYCLONEandSixSqKeycloakservers.ThisallowsauthenticationthroughtheidentityprovidersofmostacademicinstitutionsinEurope.

• Elixir(SAML2).SupportfortheElixirAAIfederationviatheSixSqKeycloakserver.• Group/RoleDefinition.WhenusingexternalauthenticationmethodsthroughtheKeycloakserver,

it ispossibletodefinegroupsandrolesandassignthesetousers.Thisallowscollaborationsthatspan multiple institutes to define common attributes independently of each institute’s identityprovider.

Theabovefeaturescovercompletelythedefinedusecasesandtherearenoauthenticationenhancementson the current SlipStream roadmap.However, therehavebeen some tentative requests for direct LDAPsupporttoallowCOmanage(fromtheInternet2initiative)tobeusedinsteadofKeycloak.

3.2. AccessControl

As described in previous deliverables, the SlipStream code base is migrating from one that provides acustom API to one based on the CIMI standard. The authorization models for the two cases differsignificantly.

Theauthorizationmodelassociatedwithresourcesinsidethe“customAPI”usesUnix-likesemantics.Usersmayassignalimitedsetofrightstothe“owner”,“group”and“others”.Thegroupisdefinedasanexplicit



list of users. This model only recognizes one group and does not allow the rights to be assigned toexternallydefinedgroupsorroles.Althoughthereissomelimitedinheritanceofgroups,groupdefinitionscannotbeeffectivelyreused.Overall, thisrigidmodel istoo limitedtoprovideeffectiveauthorizationformulti-institutecollaborations.

TheauthorizationmodelfortheCIMI-basedresourcesismuchmoreflexible.TheAccessControlList(ACL)foraresourceconsistsofanowner(whichmaybeauser,group,orrole)thatalwayshasfullcontroloftheresourcesandanoptionallistofrules.Aruleconsistsofaprincipal(user,group,orrole)andanassociatedright. This is a significant improvement over the previous authorization model because rights can beassociatedwithexternalgroupandroleinformation.OneintentionallimitationoftheimplementedACListhatitallowsonlypositivedefinitionofrights,thatisrulesthatdenycertainrightsarenotsupported.Thislimitation allows fast evaluation of the ACL and allows pre-filtering within the database, ensuring goodperformanceandscalabilityoftheSlipStreamservice.

The implementationof theACLs is completely generic and is automatically added to a resourcewhen itmigrates to the CIMI-based framework. Consequently, the features of the new authorizationmodel arevisibletousersassoonasaresourcehasbeenmigrated.Theresourcesthatremaintobemigratedaretheuser,deployment,andmoduleresources.Theuserresourceiscurrentlybeingmigratedandthemigrationofthedeploymentresourceisontheshort-termroadmap.

TheauthorizationfeaturesthatareavailablewiththeCIMI-basedframeworkare:

• AuthorizationbyUser.Allowstherights foraresourcetobeassignedbasedonthe identityofauser.Differentrightscanbeassignedtodifferentusers.

• AuthorizationbyRole/Group.Allows rights fora resource tobeassociatedbasedonagrouporrole,whichisassignedindependentlyoftheACL.

ThefollowingfeaturesrelatedtoauthorizationappearontheSlipStreamroadmap:

• CompleteMigration.Completethemigrationoftheremainingresources(user,deployment,andmodule)sothattheybenefitfromthenewauthorizationmodel.

• Resource Segmentation. The ability to segment thedefined resources intometadata, data, andACLandcontroltheauthorizationseparatelyforeachsegment.

Theresourcesegmentationwillbeimplementedasarichersetofrightsthatcanbeassignedtousers. Infact,thisrichersetofrightshasbeenimplementedbutisnotyetappliedseparatelytothethreesegmentsofaresource.



4. CloudApplicationDeploymentandManagement

IntheinitialSlipStreamdeploymentmodel,applicationdevelopershadtospecifythevirtualmachinesize(“flavor”)foreverycloudandusershadtospecifythetargetclouddirectly.Thisdeploymentmodelworkswellforsmallnumbersofcloudsbutquicklybecomestediousasthenumberofcloudsgrows.Moreover,thismodeldoesnotallowfor“specialoffers”,likereservedinstancesorspotinstances,tobeselected.

Because of these limitations, a deployment model based on “Service Offers” was designed near thebeginningoftheproject.The implementationconsistsofaServiceCatalogandanenhancedDeploymentEngine.Inaddition,theapplicationscalingprocesshasbeenenhanced.

4.1. ServiceCatalog

The Service Catalog is a collection of resources to allow people to understand the offers available fromcloud providers and to select offers appropriate for a given cloud applications. The Service Catalogresourcesare:

• Service Offer: The primary resource containing information about a specific offer from a cloudprovider. For instance, thismay be an offer for a particular flavor of a virtualmachinewith anassociatedpriceand/orqualityofservice.

• ServiceBenchmark:Asecondary resource thatallowsanyuser topostperformanceor reliabilityinformationaboutcloudresources.Thisisprimarilyintendedforpre-filteringandrankingofcloudinfrastructures.

• Service Attribute: A human-readable description of a particular attribute that can appear in aServiceOffer.Thisprimarilydocumentsthesemanticsofanattribute.

• Service Attribute Namespace: An association between an attribute prefix and URI. This is atechnicalrequirementtoensureuniquenessofattributes.

Togethertheseresourcesprovidea“database”thatcanbequeriedbyhumansandmachinealike,toselectappropriatecloudresources.

4.2. DeploymentEngine

Thedeploymentprocedureconsistsofthefollowingsteps:

1. Pre-filtering.Selectionofthecloudinfrastructurestobeconsideredforthedeployment.Removefromconsiderationoffersfromineligibleclouds.

2. Offer Filtering. Based on resource requirements and policy constraints (e.g. jurisdiction) for theapplication,filterthosethatdonotmeettheapplicationrequirements.

3. Ranking.Basedonarankingalgorithm,ordertheeligibleoffersfrommosttoleastattractive.

4. Selection.Choosetheserviceoffertouseforeachcomponentofanapplication.Thiscaneitherbedonemanuallybytheuserorautomaticallybasedontheranking.



5. Provisioning.Allocatetheselectedresourcesandthenruntheapplication’s“recipes”tobringtheapplicationintoarunningstate.

6. Scaling. As the loadon the application changes over time, scale the applicationby changing theresources allocated to the application. When adding resources, the same resource selectionprocedureshouldbeusedasfortheinitialstartup.

7. Termination.Stoptheapplicationandfreeallallocatedresources.

Note that theprovisioning isdoneper-component, so thatamulti-componentapplicationcaneasilyuseresourcesfrommultiplecloudsinthesameapplicationdeployment.

Thisnewdeploymentmodelhasbeenprogressivelyimplementedoverthecourseoftheproject.CurrentlytheSlipStreamdeploymentengineisentirelybasedonserviceoffers,althoughtherearesomelimitationsatthevariousstageslistedabove.Theselimitationsinclude:

• Pre-filteringislimitedtothelistofconfiguredcloudsforagivenuser.Theabilitiestopre-filteroncloudperformancemetricsorthelocationofrequireddataareplanned,butnotyetimplemented.

• The ranking algorithm is limited to cost. The ability to allow users to define ranking algorithmsbasedonothercriteriaisplanned.

• The scaling of an application uses resources from the same cloud provider(s) that are currentlyused.Thescalingprocesswilleventuallyre-evaluatetheoffersforeachnewresourceallocation.

TheselimitationswillberemovedaspartoftheongoingSlipStreamdevelopmentroadmap.

4.3. ApplicationScalability

The loadofacloudapplicationwillvaryovertime. Tomaintainperformance,theresourcesallocatedtotheapplicationmustriseandfall intandemwiththeload.SlipStreamprovidesamechanismtoautomatethescalingprocess.

SlipStreamprovidesbothhorizontalandverticalscalingthroughtheAPI.Theprimary limitationsarethatthe topology of an application cannot be changed at run time and that the scaling actions cannot betriggeredthroughthebrowserinterface.

SixSqprovidesdocumentationforhowtotriggerscalingthroughtheSlipStreamAPI.SixSqalsoprovidesanexampleapplicationthatdemonstrates“auto-scaling”basedonapplication-definedmetrics.UvAhasalsoworkedtoshowthatthemechanismsaregeneralenoughtobeadaptedtodifferentscalingalgorithms.

4.3.1. RiemannScaling

The example auto-scaling application from SixSq bases its implementation on the Riemann plugin ofcollectdanda custompublisherwritten inPython conforming to theRiemannclient libraryAPI. The fulldetailshavebeendescribedinSections4.7and4.8ofapreviousdeliverable[D6.3]andarenotrepeatedhere.

Theexampleauto-scalingapplicationcontainsthefollowingcomponents:

• webapp: a stateless web application that takes requests, synchronously performs a moderatelyintensivecomputation(calculatingπupto100digits),andreturnstheresult,

• nginx:aloadbalancerbasedontheNginx[NGIN16]webserverthatdistributesclientrequeststothesetofstatelesswebservers,

• client:atestclientbasedonLocust[LOC16]thatsimulatesavaryingnumberofclients,and • autoscaler:StandardSlipStreamautoscalercomponentthatmakesscalingdecisions.



The application can be found in the AppStore onNuvla; its source code is in the “client-nginx-webapp”moduleintheGitHubrepository.Theapplicationchangesthenumberof“webapp”machinesbasedontheobservedresponsetimeforrequests;the“autoscaler”componentinteractswithSlipStreamtotriggertheadditionorremovalof“webapp”machines.

4.3.2. PegasusScaling

To ensure that the scaling mechanisms in SlipStream are independent of technologies chosen in theexample autoscaling application, UvA implemented an elastic deployment of the Pegasus workflowmanagement system [PEG17] in SlipStream, called ElasticPegasus. The provided setup allows to executecomplex scientific workflows in an elastic environment equipped with an autoscaler, which permitsadjustingthenumberofallocatedresources(thus,horizontalscaling)tomeetSLAsautomatically.

ThePegasusworkflowmanagementsystemisbasedonHTCondor[CON17]workloadmanagementsystemfor compute-intensive jobs. Pegasus facilitates the execution of complex (scientific) workflows indistributed computing environments. There are many popular workflow types which are used in manydifferent fields from astronomy to bioinformatics [EDL17]. Pegasus uses HTCondor to transfer the filesusing either a shared file systemor a so-called condoriomode inwhichHTCondor is responsible for filetransfers between the tasks/computing nodes. In ElasticPegasus, the condoriomode is used because itsimplifiesthesetup,avoidingtheinstallationandconfigurationofanetworkfilesystem.

ForElasticPegasus,anelasticSlipStreamapplicationwithasingleheadnodeandasetofworkernodesisused.ThewholesetuprunsanadditionalsingleorchestratorVM,automaticallyinstantiatedbySlipStreamduringthedeploymentoftheapplication.TheorchestratorisneededtomakethedeploymentscalableandthisisarequirementoftheSlipStreamplatform.

The head node acts as an HTCondor Central Manager, which maintains the HTCondor pool and it isresponsible for job submissions. The worker VMs only execute HTCondor jobs. Using the componentparametermapping,SlipStreamguaranteesthattheheadVMbootsbeforealltheotherVMsandthattheIPaddressoftheheadVMisprovidedtoalltheworkerVMs.WorkerVMsautomaticallyjointheHTCondorpoolusingtheprovidedIPaddressoftheheadVM.

Pegasus is only installed on the head VM, where it provides commands to generate workflow DirectedAcyclic Graphs (DAGs), plan the workflow execution, and to submit new workflows. It also allowsmonitoringtheexecutionofworkflows,collectsresults,andrunsawebinterfacetomonitortheprogressofthesubmittedworkflows.

The head VM also runs a small provisioning controller daemon and a RESTful autoscaling service. Theprovisioning controller periodically (by default every 30 seconds) invokes the autoscaler. The RESTfulinterface requires two input arguments: current demand and service rate, and returns as a prediction anumberofVMstoprovisionorrelease.Theprediction isusedbytheprovisioningcontroller tostartandstop VMs. We specify current demand as the number of currently eligible (with satisfied precedenceconstraints) and runningworkflow tasksofall theworkflows in the system.Tocalculate the service rateparameter,theprovisioningcontroller,measureshowmanyworkflowtasksareprocessedperautoscalingintervalbyasingleVM.Theprovisioningcontrollerisalsoresponsibleforissuingprovisioningandreleasecommands to SlipStream. For that, it keeps track on the VM load information and decides which VMsshouldbestopped.ItonlystopsidleVMs,startingwithVMsthathavebeenidlethelongest.

ElasticPegasushastwoautoscalingpolicies:ReactandAdapt[ILY17].Bothpoliciesareworkflow-agnostic,whichmeansthattheycanalsobeusedtocontrolotherapplicationtypes.

• React: Chieu et al. [CHI09] presented a dynamic scaling algorithm for automatedprovisioningofVM resources based on the number of concurrent users, the number of active connections, the



number of requests per second, and the average response timeper request. The algorithm firstdetermines the current web application instances with active sessions above or below a givenutilization.Ifthenumberofoverloadedinstancesisgreaterthanapredefinedthreshold,newwebapplication instances are provisioned, started, and then added to the front-end load-balancer. Iftwo instances are underutilized with at least one instance having no active session, the idleinstance is removed from the load-balancer and shutdown from the system. In each case, thetechniquereactstotheworkloadchange.

• Adapt:Ali-Eldinetal.[ALI12]proposeanautonomouselasticitycontrollerthatchangesthenumberofVMsallocatedtoaservicebasedonbothmonitoredloadchangesandpredictionsoffutureload.Thepredictionsarebasedon the rateofchangeof the requestarrival rate, i.e., theslopeof theworkload,andaimsatdetectingtheenvelopeoftheworkload.Thedesignedcontrolleradaptstosudden load changes and prevents premature release of resources, reducing oscillations in theresource provisioning. Adapt tries to improve the performance in terms of number of delayedrequests, and the average number of queued requests, at the cost of some resource over-provisioning.

Figure2showstheprimarycomponentsoftheElasticPegasusapplication.TheapplicationcanbeselectedanddeployedthroughtheSlipStreambrowserinterfaceorAPI.Runningthisapplicationwithvaryingloadsshow that the autoscalingmechanismswork as expected. These autoscalers showed good performancewhenusedfortheautoscalingofworkflowapplications.

Figure2:PegasusAutoscalingwithSlipStream

4.4. Multi-CloudSupport

TheCYCLONEplatformeasilyintegratesmultiplecloudserviceproviderstocreateauniforminterfacetoahybrid(andusuallyheterogeneous)cloud.Thebroker’s“connectors”actasanabstraction layerbetweenthebroker(SlipStream)andthevariousAPIsofthecloudserviceproviders.Inthisway,usersseeauniforminterface for provisioning and managing cloud resources independently of the underlying cloud API.SlipStream provides connectors for popular open-source and commercial cloud APIs. Table 1 lists theavailable connectorsand the requiredSlipStreamEdition. TheCommunityEdition is availableunder theApache2license;theEnterpriseEditionrequirespurchaseofacommerciallicense.

SixSq’s commercial SlipStream service, called Nuvla, is the broker used to bind cloud infrastructurestogethertocreatetheCYCLONEtestbed.Table2showsthecloudinfrastructuresthatareaccessiblefromNuvla.CloudinfrastructuresoperatedbyInteroute,CNRS-LAL,andQSCcontributeofficiallytotheCYCLONEtestbed.ProjectparticipantshavealsousedcloudsoperatedbyAmazon,Exoscale,andIFBtorunvariousapplicationsandtovalidatecomponentsoftheCYCLONEplatform.

Worker VM n Worker VM 2

Worker VM 1

User

Head VM

HTCondor Central Manager

Pegasus

Autoscaler

Provisioning Controller + −

HTCondor Runner



Table1:SlipStreamCloudConnectors

SlipStreamEdition

CloudAPI Community Enterprise

AmazonEC2

Azure

Cloudstack Exoscale

Openstack NuvlaBox OpenTelekomCloud(OTC) OpenNebula

SoftLayer

StratusLab

Table2:NuvlaCloudServiceProviders

CYCLONECloudServiceProvider CloudAPI Countries Regions Testbed

Advania Openstack SE 1 AmazonWebServices(AWS) EC2 EU+ 10 ≈AtosITER StratusLab ES 1 CESNET OpenNebula 1 CloudFerro Openstack 1 CNRS-LAL Openstack FR 1 EBIEmbassy Openstack 1 Exoscale Exoscale CH 2 ≈IBMSoftLayer SoftLayer IT 1 InstitutFrançaisBioinformatique Openstack FR 6 ≈Interoute(IRT) Openstack IT 1 MicrosoftAzure Azure NL 1 NuvlaBox NuvlaBox CH,FR,IT 18 OpenTelekomCloud(OTC) OTC DE 1 QSC Openstack DE 1 SCISSOR(H2020) OpenNebula FR,IT 3 TiedeHPC OpenNebula ES 1

4.5. Features

Thefeaturesthathavebeenimplementedare:

• Offer-BasedProvisioning.Theabilitytoprovisioncloudresourcesbasedonspecificserviceoffers,includingcharacteristicslikeavailableresources,location,andqualityofservice.

• Multi-Cloud Deployments. Developers can create cloud applications that simultaneously useresourcesfromdifferentclouds.

• Scaling.Allowinganapplicationtoscalehorizontally(moreorfewermachines)orvertically(morelocalresources)toaccommodateachangeinload.

• HorizontalScaling.Addingorremovingvirtualmachinesfromarunningcloudapplication.



• VerticalScaling.Addingmoreresources(CPU,RAM,disk)toasinglevirtualmachine.

• RankingbyCost.Theability to rankappropriateserviceoffersbasedon theapproximatecostofthoseresources.

• Policy Constraints. The possibility for application developers to define policy constraints for anapplication,forexample,tolimitwhereanapplicationcanberun.

• Benchmarking.Allowingallusersoftheplatformtopublishbenchmarkinformationaboutserviceofferswithin theServiceCatalog, toallowusers tomakebetter choices concerning theavailablecloudresources.

Thefullprovisioningchainisavailabletousers,buttherearesomelimitations:

• Extended Pre-filtering. Allow the cloud pre-filtering to include benchmarking criteria and otherconstraints,likethelocationofrequireddatasets.

• User-Specified Ranking. Allow the users to specify their own ranking algorithms to identify themostattractiveoffersfromtheServiceCatalog.

• ScalingwithOffers.UpdatethescalingprocessinsideofthedeploymentenginetouseoffersfromtheServiceCatalog.

• ScalingthroughUI.Extendthebrowser-basedUItoallowscalingactionstobetriggered.

• Dynamic Topology. Application developers must describe how many clouds will be used in anapplication in its definition. The distribution of components over clouds cannot currently bechangeddynamically.

All these limitations appear in the SlipStream development roadmap and will be implemented inaccordancewithuserrequirements.



5. Monitoring

The SlipStreammonitoring system is intended to provide information about both current and historicalresourceutilization.Theexisting implementation (seeFigure1)at thestartof theprojectwasextremelylimited.Itcouldnotbeextendedtoresourcesotherthanvirtualmachinesandevenforvirtualmachinesitwasnotpossibletoassociatetheusagewithgroupsandroles.Atthetechnical level,theimplementationwouldalsonotscaletoalargenumberofusers.

For these reasons, the system was redesigned and re-implemented. The system now uses a separateservicetomaintainthecurrentglobalstateofresourceutilizationacrossallclouds.Toallowforhistoricalviews,asnapshotofthecurrentglobalstateistakeneveryminute.Withthisnewdesign,userscanobtaindetailedreportsofusageoveranytimeperiodandfiltertheusagebygroup,role,application,oranyotherattributeofthemeteredresource.Becausethecostisaddedduringthesnapshot,approximatebillingcanalsobeprovided.

The quota enforcement mechanism has also been re-implemented to rely on the current global state.Becauseofthegeneralityofthesolution,quotascanbeplacedonanyresourcewhichappearsintheglobalstate.Thesequotascanalsobeappliedbyuser,group,role,orotherattributes.

ThenewimplementationhasjustrecentlybeenincorporatedintotheproductionversionofSlipStream,sothereisnotwidespreaduseofthesenewfeatures:

• Current Usage. The ability to see a “near” real time view of a user’s (or group’s, etc.) resourceutilization.

• HistoricalUsage.Theabilitytoseeauser’s(orgroup’s,etc.)resourceutilizationoveragiventimeperiodwithcostinformation.

• Quota. The ability to limit resource utilization to a predefined value and to prevent furtherresourceallocationsthatwouldviolatethislimit.

All of these features are available through theRESTAPI, but havenotbeen integrated into SlipStream’sbrowserinterface.Acommandlike:curl -X PUT -H 'content-type:application/x-www-form-urlencoded' \ https://nuv.la/api/metering \ --data-urlencode '$filter=deployment/user/href="user/'${user_id}'"' \ -d '$filter=snapshot-time>="2017-10-01T00:00:00.000Z"' \ -d '$filter=snapshot-time<="2017-10-08T00:00:00.000Z"' \ -d '$last=0' \ -d '$aggregation=count:id' \ -d '$aggregation=sum:price' \ -d '$aggregation=sum:serviceOffer/resource:vcpu' \ -d '$aggregation=sum:serviceOffer/resource:ram'

Canbeusedtorecovertheresourcesusedoveraperiodoftime.Thiscommandwouldprovidethetotalnumberofmachinesrunning, thecost, totalCPU ·minute,andRAM·minuteusedover theperiod foragivenuser.

Theforeseenextensionsofthemonitoringfeaturesinclude:

• Extended Resource Coverage. Currently only virtual machines are monitored. The monitoringneedstobeextendedtostorage,networking,andotherresources.



• User-DefinedQuotas.OnlytheSlipStreamadministratorcandefinequotas.Individualusersshouldbeabletodefinequotasforthemselvesandgroupmanagersshouldbeabletodothesameforthegroupandformembersofthegroup.

Asfortheotherdefinedlimitations,worktoremovethelimitationsappearsintheSlipStreamroadmap.

Figure3:OldInterfaceforCloudUsageInformation



6. Scalability,Reliability,andUsability

The scalability, reliability, and usability are critical characteristics for any production service. There havebeenanumberofimprovementstoSlipStreamintheseareasoverthecourseoftheproject.

ToimprovethescalabilityandreliabilityoftheSlipStreamplatform,importantchangeshavebeenmadetotheoperationsmodel:

• Micro-services.TheSlipStreamservicedeploymenthasshiftedtowardsmicro-servicestoimproveits performanceand resilience.A SlipStreamdeploymentnowconsistsof the customAPI server,theCIMIAPI server, the “Pricing andRanking Service”, themetering service, and the “collector”(formaintainingglobalstate).Thisallowsabetterviewofresourceconsumptionwithinthesystemandmorecontrolovertheindividualprocesses.

• Redundancy. Newer micro-services within the SlipStream ecosystem have been designed to be“stateless”. This allows for easy replication and scaling of the individual services, which in turnimprovestheoverallreliabilityoftheplatform.

• DistributedDatabase.Onthebackend,adistributedElasticsearchdeploymentisusedtoprovideareliable database for themajority of the SlipStream platform. A single HSQLDB database is stillused for the deployment and module resources, but it will disappear as those resources aremigratedtowardsCIMI.

• Reduced Polling. Foundational changes have been made that will allow the use of Server SentEvents (SSE) when interacting with the SlipStream services. This provides a straightforwardmechanismtoavoidpollingbyclients,reducingtheloadonboththeclientandserver.

Changeshavealsobeenmadetoimprovetheusabilityoftheplatformfromthebrowserandprogramminginterfaces:

• RefactoredPythonAPI.Thecommandlineinterface(CLI)forSlipStreaminitiallyhadaninternalAPIforSlipStreamthatwasnotexposedtoendusers.ThisinternalAPIwasseparatedfromtheCLIandimproved so that Python programmers now have a clean, native interface to the SlipStreamplatform[SSPY].

• LibcloudDriver.ManyPythonprogrammersprefertousetheLibcloudAPI [LIBCLOUD]tocontrolcloud-basedworkloads. In addition to thenative SlipStreamPythonAPI, a Libclouddriver is alsoavailable[SSLC].

• Clojure(Script)API.NewerservicesontheSlipStreamplatformarewritteninClojure,aLISPdialectrunning over the JVM. A complete Clojure API is available. This API can be used (with minimalwrapping)fromJava.Moreover,thisAPIiscompatiblewithClojureScript,allowingitsusefromtheJavaScriptecosystem.[SSCLJ]

In addition to the above changes, significantwork has been done to replace the browser interface. ThenewerdesignreliesheavilyonmatureJavaScripttechnologiestoprovideanimprovedandmoreresponsiveexperiencethroughthebrowser.Asthisinterfacebecomesmoremature,itwillreplacetheolderinterfacethatreliesonamixtureofclientandserversiderendering.



7. Summary

This work package has over the last three years, significantly enhanced the brokering, deployment andmanagementfeaturesformulti-cloudapplicationsbyextendingSlipStream.TheusecasesfromWP3haveprovided guidance on the features to be implemented and validated those features with Nuvla, SixSq’sproductionSlipStreamserverthatispartofCYCLONE’stestbed.ThefactthatNuvlawasupdatedfortnightlywithCYCLONEupdatesisfurtherevidencethatthedevelopedfeaturesareofproductionqualityandusefultoawidediversityofusers.

The project has made significant improvements in the following areas: application curation, AAI,deployment&management,monitoring, and scalability, reliability,& usability. Table 3 lists the featuresthat were implemented in each category. In addition, Appendix A contains the full list of identifiedrequirements, their states,anddetailedcommentary. Overall, therewere56 requirementsofwhich33(59%) were fully implemented, 11 (20%) remain on the short-term roadmap, 12 (21%) will not beimplementedforvariousreasons.

As SlipStream is a commercial solution at the core of SixSq’s portfolio, the evolution of SlipStreamwillcontinueaftertheendofCYCLONE.Asmentionedabove,11oftherequirementsremainontheshort-termSlipStreamroadmapandwill likelybe implemented in thenextsixmonths.Table3also lists theprimaryCYCLONEfeatures(derivedfromtherequirements)thatremainontheroadmap.

One largearea that remains tobecovered is the inclusionofdatamanagement.Experimentshavebeenconducted with the European Space Agency to see how data resources can be defined in the ServiceCatalogandwithHNSciCloudtounderstandhowtointegratescientificdatamanagementservices, inthiscaseOnedatafromtheIndigoDataGridproject.



Table3:SummaryofAvailableandPlannedFeatures

ImplementedFeatures Roadmap

ApplicationCuration Portability ServicesAutomatedDeployment HierarchicalApplicationsComponentCoordination ModuleIsolationApp.Parameterization GitHubIntegrationSharingApplications StockComponents

AAI Username/Password CompleteCIMIMigrationAPIKey/Secret ResourceSegmentationGitHub eduGAIN ElixirAAI Group/RoleDefinition AuthorizationbyUser Authz.byRole/Group

Deployment&Management Offer-BasedProvisioning ExtendedPre-filteringMulti-CloudDeployments User-SpecifiedRankingHorizontalScaling ScalingwithOffersVerticalScaling ScalingthroughUIRankingbyCost DynamicTopologyPolicyConstraints

Monitoring Benchmarking ExtendedResourceCoverageCurrentUsage User-DefinedQuotasHistoricalUsage Quota

Scalability,Reliability&Usability PythonAPI ContinuedMigrationtoMicro-ServicesLibcloudDriver Clojure(Script)API



References

[D6.1] ComplexApplicationDescriptionSpecificationhttp://www.cyclone-project.eu/assets/images/deliverables/Complex%20Application%20Description%20Specification.pdf

[D6.2] SpecificationofInterfacesforBrokering,Deployment,andManagementhttp://www.cyclone-project.eu/assets/images/deliverables/Specification%20of%20Interfaces%20for%20Brokering,%20Deployment,%20and%20Management.pdf

[D6.3] SolutionsforNon-functionalAspectsofCloudComputinghttp://www.cyclone-project.eu/assets/images/deliverables/Solutions%20for%20Non-functional%20Aspects%20of%20Cloud%20Computing.pdf

[ALI12] AhmedAli-Eldin,JohanTordsson,andErikElmroth.2012.AnAdaptiveHybridElasticityControllerforCloudInfrastructures.InIEEENOMS.

[CHI09] T.C.Chieuandothers.2009.DynamicScalingofWebApplicationsinaVirtualizedCloudComputingEnvironment.InIEEEICEBE.

[CIMI16] DistributedManagementTaskForce,Inc.,CloudInfrastructureManagementInterface(CIMI)ModelandRESTfulHTTP-basedProtocol,2016.https://www.dmtf.org/sites/default/files/standards/documents/DSP0263_2.0.0.pdf

[COMANAGE] COmanagehttps://www.internet2.edu/products-services/trust-identity/comanage/

[DFN] DFN-Verein(GermanNREN)https://www.dfn.de/en/

[EDL17] E.Deelman,etal.“Thefutureofscientificworkflows”.TheInternationalJournalofHighPerformanceComputingApplications,2017.

[EDUGAIN] eduGAINhttps://www.geant.org/Services/Trust_identity_and_security/Pages/eduGAIN.aspx

[ELIXIR] ElixirAAI.https://www.elixir-europe.org/services/compute/aai

[EDUGAIN] eduGAINhttps://www.geant.org/Services/Trust_identity_and_security/Pages/eduGAIN.aspx

[ELIXIR] ElixirAAI.https://www.elixir-europe.org/services/compute/aai

[ILY] A.Ilyushkin,etal.AnExperimentalPerformanceEvaluationofAutoscalingPoliciesforComplexWorkflows.Proceedingsofthe8thACM/SPEConInternationalConferenceonPerformanceEngineering,2017.

[LIBCLOUD] ApacheLibcloudhttps://libcloud.apache.org/

[LOC16] Locust,Locust,2016.http://locust.io/

[NGI16] Nginx,Nginx,2016.https://www.nginx.com/

[OAUTH] OAuth2.0



https://oauth.net/2/

[OIDC] OpenIDConnect.https://openid.net/connect/

[PEG] PegasusWorkflowManagementSystem,2017.https://pegasus.isi.edu/

[RIES16] Riemann,Riemann,2016.http://riemann.io

[SSLC] SlipStreamLibclouddriverdocumentation.https://slipstream.github.io/slipstream-libcloud-driver/

[SSCLJ] SlipStreamClojure(Script)documentation.https://slipstream.github.com/SlipStreamClojureAPI

[SSPY] SlipStreamPythonAPIdocumentation.https://slipstream.github.io/SlipStreamPythonAPI/

[SSDOC] SlipStreamdocumentation.http://ssdocs.sixsq.com

[SSAPI] SlipStreamAPIdocumentation.http://ssapi.sixsq.com

[SWITCH] SWITCH(SwissNREN)https://www.switch.ch/



Glossary

AAI AuthenticationandAuthorizationInfrastructureACL AccessControlListAPI ApplicationProgrammingInterfaceCIMI CloudInfrastructureManagementInterfaceCLI CommandLineInterfaceCSP CloudServiceProviderDAG DirectedAcyclicGraphIaaS Infrastructure-as-a-ServiceJVM JavaVirtualMachineNREN NationalResearchandEducationNetworkingorganizationOIDC OpenIDConnectPaaS Platform-as-a-ServiceSaaS Software-as-a-ServiceSAML SecurityAssertionMarkupLanguageSP ServiceProviderSSE ServerSentEventsWP WorkPackage



AppendixA Requirements

Thisappendixcollects therequirementsdefined in thepreviousWP6deliverablesandprovidesasummaryof thecurrent implementationstatus.Thoserequirementsmarkedwithacheckmark(�)arefullyimplemented;thosewithapproximatelyequalsign(≈)indicaterequirementsthatareplannedtobeimplementedintheshorttermorhavebeenimplementedinanalternateway;andthosewithacross(�)indicatethosethathavenotbeenimplemented.Detailedcommentsonthestatusareprovidedforalloftherequirements.Asummaryoftheoverallimplementationstatusisprovidedinmaintextofthisdocument.

Table4:ImplementationStatusofRequirements

No.ofRequirements PercentageofRequirementsFullyimplemented(�) 33 59%Plannedoralternateimplementation(≈) 12 21%Notimplemented(�) 11 20%TOTAL 56 100%



Table5:CollectedRequirements

Doc. ID Title Level Description CommentD6.1 1 � Machine

readabilityMUST Theapplicationdescriptionswillbeprocessed

primarilybytheCYCLONEcomponents.ThechosenformatMUSTbeeasilyreadablebymachinefromawidevarietyofdifferentprogramminglanguages.

TheapplicationdescriptionsarenotyetmanagedviatheCIMIinterfaceandconsequentlyarestillexposedprogrammaticallyinXML.TheXMLismachinereadablewithaconsistentschema.Becausetheseresourcesareusednearlyexclusivelybythemicro-servicesandclientswithintheSlipStreamecosystem,theportabilityofthisapplicationisnotamajorconcern.

D6.1 2 � Humanreadability

SHOULD Applicationdevelopersmayneedtoreadortocreatethedescriptionsdirectlyfordebugging,testing,ordevelopmentoftooling.ThechosenformatSHOULDbeeasilyreadablebyhumans.

AstheapplicationdescriptionisstillinXML,itisnotfriendlyforhumanprocessing.WhenthesedescriptionsmigratetoCIMI,theJSONrepresentationwillbemuchmoreaccessibletohumans.MigratingtheseresourcesisintheSlipStreamroadmapbuttheywillbethelasttobemigrated.

D6.1 3 � Easilyunderstandable

SHOULD TheapplicationdescriptionsSHOULDbeeasilyunderstandablebyhumans,meaningthattheschemashouldhavelimitedcomplexityandbewelldocumented(e.g.withtutorials,examples,APIs,specifications,etc.).

AstheapplicationdescriptionisstillinXML,itisnoteasyforhumanstounderstand.OncetheseresourcesaremigratedtoCIMI(andJSONformat),theywillbeeasierforhumanstoworkwithdirectly.MigratingtheseresourcesisintheSlipStreamroadmapbuttheywillbethelasttobemigrated.

D6.1 4 ≈ Extensible MUST TheapplicationdeploymentformatMUSTbeextensibletoallowforcharacteristicsorfeaturesspecifictoCYCLONEcomponents.

TheformatisextensiblebutrequiressupportingmodificationswithintheSlipStreamserversandclientsforeachchange.SomeCYCLONEenhancementshavebeenaddedtothedescriptionsoverthecourseoftheproject.TheextensibilitywillbegreatlyimprovedoncethefullmigrationtoCIMIiscomplete.

D6.1 5 � Maturity SHOULD TheapplicationdeploymentformatSHOULDhaveproductionadoptionwithinthewiderITcommunity,demonstratingthatitisappropriateforgeneraladoption.

ThematureSlipStreamapplicationdescriptionformathasbeeninproductionuseformorethan3years.TheCIMItranslationoftheapplicationdescriptionwillcontainthesameinformationbutbeeasiertoprocessandforhumanreadability(beingbasedonJSONratherthanXML).

D6.1 6 � Applicationsummary

MUST ThedescriptionformatMUSTallowtheapplicationdevelopertoprovideadetailed,human-readablesummaryoftheapplication’sfeatures,characteristics,andlimitations.

Theapplicationdescriptionformatallowsthedevelopertoprovidedetailedmetadataconcerningthecomponentorapplication.



Doc. ID Title Level Description CommentD6.1 7 � Singlemachine

apps.MUST Manyapplicationsandservicescanbecontained

withinasinglevirtualmachine.ThedescriptionformatMUSTbeabletodescribesimple,single-VMapplications.

ForSlipStream,asingleVMapplicationorserviceiscalleda“component”.Thesecomponentscanbefullydescribedbydevelopersandeasilydeployedbyoperatorsorendusers.

D6.1 8 � Multiplemachineapps.

MUST Manyapplicationsrequireanumberofdifferentservicesdeployedondifferentvirtualmachines.ThedescriptionformatMUSTbeabletodescribeapplicationscomposedofmultiplemachines.

ForSlipStream,amulti-machinecloudapplicationiscalledan“application”andiscomposedofSlipStream“components”.SlipStreamhandlesthecoordinationbetweentheconstituentcomponentsandallowsthefullapplicationtobemanagedasawhole.

D6.1 9 � Hierarchicalcomposition

SHOULD Complexapplicationsareoftenbuiltfromreusablecomponentsthatarethemselvescomplex,multi-machineservices.ThedescriptionformatSHOULDallowahierarchicalcompositionofcomponentstofosterreuseofdescriptions,minimizingdevelopereffort.

ThisisnotcurrentlysupportedbutisanimportantcomponentoftheSlipStreamroadmap.Theimplementationofthisispendingbecauseofadependencyonthemoveofthe“deployment”resourcetotheSlipStreamCIMIserver.This“deployment”migrationisexpectedtohappenattheendof2017andwillunblocktheimplementationofthisfeature.

D6.1 10 � Virtualmachines MUST ThedescriptionformatMUSTbeabletodescribethealloftheparametersfortheprovisioningofvirtualmachinesonacloudinfrastructure.

Thedescriptionallowsthedevelopertoprovidetheresourcerequirementsofavirtualmachine,includingCPU,RAM,anddisk.Withtheshifttoprovisioningbasedonserviceoffers,allcharacteristicsofaresource(location,price,etc.)canbeusedforresourceselection.

D6.1 11 ≈ Containers SHOULD Useof(Linux)containersisbecomingmorewidespreadasanalternativetofullvirtualmachines.ThedescriptionformatSHOULDbeabletodescribealloftheparametersfortheprovisioningofacontainer.

AnumberofSlipStreamcomponentsandapplicationshavebeenprovidedintheAppStorethatallowuseofcontainers,includingDocker,DockerCompose,DockerSwarm,Kubernetes,andMesos.FurtherintegrationofcontainersintoSlipStream’scloudapplicationmanagementisplannedbutnotyetavailable.

D6.1 12 � CPUspecification

MUST ThedescriptionformatMUSTallowtheapplicationdeveloper(oruser)todefinethecharacteristicsoftheCPU(s)required.

TheminimumnumberofvCPUsforacomponentcanbedefined.

D6.1 13 � RAMspecification

MUST ThedescriptionformatMUSTallowtheapplicationdeveloper(oruser)todefinetheamountofRAMrequired.

TheminimumRAMrequiredforacomponentcanbedefined.

D6.1 14 � Diskspecification

MUST ThedescriptionformatMUSTallowtheapplicationdeveloper(oruser)todefinetheamountofdiskspaceavailableontheapplication’svirtualmachines.

Theminimumdiskspacerequiredforacomponentcanbedefined,onthecloudplatformsthatsupportthis.Inaddition,userscanspecify“extradisks”whentheunderlyingcloudallowsadditionaldiskstobeallocated.



Doc. ID Title Level Description CommentD6.1 15 ≈ Persistent

storageMAY ThedescriptionformatMAYallowthe

applicationdeveloper(oruser)tospecifythecharacteristicsofpersistentstorage(type,amount,location,etc.).

ManagementofstorageresourcesisnotintegratedintoSlipStream.However,Onedataservices(fromtheIndigoDataCloudproject)areavailablefromtheAppStore.ThisallowsuserstocreatetheirowndatamanagementplatformandcontrolitthroughSlipStream.

D6.1 16 � Multi-cloud SHOULD ThedescriptionformatSHOULDallowtheapplicationdevelopertoindicatewhichpartsofanapplicationcanbedeployedondifferentcloudinfrastructures.

TheSlipStreamapplicationdescriptionformatandprovisioningsystemfullysupportmulti-cloudapplications.

D6.1 17 � Placementpolicies

MAY ThedescriptionformatMAYallowtheapplicationdeveloper(oruser)tospecifypoliciesfortheplacementofapplicationcomponents.

Developerscanspecifyplacementpoliciesonindividualcomponentsthatidentifyappropriateserviceoffers.ThepoliciesaredefinedintherichCIMIfilteringlanguageandcanuseanycharacteristicsdefinedintheserviceoffers.

D6.1 18 ≈ Networkconnectivity

MUST ThedescriptionformatMUSTallowtheapplicationdevelopertodefinethenetworkconnectivitybetweenapplicationcomponentsandbetweentheuserandtheapplication.Thisconcernstheaccessibilityofportsontheapplication’sVMs.

TheCYCLONEnetworkingfeaturesaredeliveredthroughtheCNSMOcomponentthatisavailablefromtheSlipStreamAppStore.Theconnectivity(firewall)isdefinedthroughtheCNSMOcomponentparameters.

D6.1 19 ≈ Networkisolation

SHOULD ThedescriptionformatSHOULDallowtheapplicationdevelopertospecifythecharacteristicsofisolatednetwork(s)tobecreatedforagivenapplication.ThisnetworkisolationMAYbeextendedtomachinesoutsideoftheapplication,suchasauser’sworkstation.

TheCYCLONEnetworkingfeaturesaredeliveredthroughtheCNSMOcomponentthatisavailablefromtheSlipStreamAppStore.Thenetworkisolation(VPN)isdefinedthroughtheCNSMOcomponentparameters.

D6.1 20 � Parameterization MAY ThedescriptionformatMAYallowanapplicationorapplicationcomponenttobeparameterized,allowinginformationtobepassedintooroutofanapplicationatdeploymentorruntime.

ASlipStreamcomponentcanbeparameterizedwithinputandoutputparameters.DeveloperscanwireparametersfromdifferentcomponentstogetherwhendefiningaSlipStreamapplication.

D6.1 21 � Metrics SHOULD ThedescriptionformatSHOULDallowtheapplicationdeveloper(oruser)tospecifywhatstandardmetrics(e.g.CPUloadorRAMutilization)shouldbecollectedalongwiththeirfrequency.

Becausethemetricsandtheperformancetrade-offsvarygreatlybetweenapplications,metricscollectionhasnotbeenintegrateddirectlyintotheSlipStreamdeploymentandapplicationmanagementprocesses.Insteadexampleshavebeenprovidedtoshowhowapplicationdeveloperscanimplementmetriccollection,analysis,andactionsusingSlipStream.



Doc. ID Title Level Description CommentD6.1 22 � Dynamicmetrics MAY ThedescriptionformatMAYallowthe

applicationdeveloper(oruser)tomodifythecollectedmetricswhiletheapplicationisrunning.

SeecommentforD6.1/21.

D6.1 23 � Externalservices SHOULD ThedescriptionformatSHOULDallowtheapplicationdeveloper(oruser)specifyoutsideservicestobeusedinconjunctionwithadeployedapplication(forexample,externalauthenticationdatabasesforsinglesign-on).


D6.1 24 � Actions MAY ThedescriptionformatMAYallowactionstobetakeninresponsetothevaluesofcollectedmetricsorprovidedkeyperformanceindicators(KPIs)tobedefined.


D6.1 25 � SLA MAY ThedescriptionformatMAYallowservice-levelagreementstobedefinedasconstraintsonserviceplacementand/ortriggerstodefinedactions.


D6.1 26 � Credentials MUST ThedescriptionformatMUSTbeabletoprovidecredentialsforconfiguringaccesstodeployedapplicationservices(e.g.SSHpublickeysforremoteshellaccess).

ThedescriptionformatcanspecifywhatSSHkeysshouldbeusedforanapplicationdeployment.SlipStreamnowprovidesageneralcredentialresourcethatsupportsSSHKeysandAPIkeys/secretsandcanbeextendedtoothercredentialsinthefuture.Theapplicationdescriptionformatandtheuserresourcearebeingextendedtoincludereferencestothesecredentials.

D6.1 27 � Tooling SHOULD Tooling(e.g.IDE)thatfacilitatesthecreationanduseoftheapplicationdescriptionsWOULDbeadvantageous.

SlipStreamitselfprovidesgoodtoolingforcreatingtheapplicationdescriptions.

D6.1 28 � Lifecycleactions MUST ThedescriptionformatMUSTallowthecloudapplicationdevelopertodefineactionsassociatedwithstatetransitionsintheapplicationlifecycle.

Applicationdeveloperscandefineactionsforanyofthecloudapplicationlifecyclestatetransitions.

D6.1 29 � Executionenvironment

MUST TheapplicationmodelanddescriptionMUSTnotimposeconstraintsontheexecutionenvironmentoftheapplication,forexample,byexcludingtheuseofcertainoperatingsystems(e.g.Windows).

ThetoolsusedtocontrolresourcesfromSlipStreamarewrittentobeaslightweightandportableaspossible.AllLinuxandWindowsplatformsaresupported.Otherplatforms,ifrequired,mayrequiresmalladaptationsoftheSlipStreamintegrationtools.



Doc. ID Title Level Description CommentD6.2 1 � Machine

ReadableServiceDescriptionFormat

MUST CYCLONEconsistsofasetofinteroperatingtoolsthatwillproduceandprocesstheservicedescriptions.Consequently,theformatMUSTbeastandardformat(XML,JSON,etc.)thatcanbeeasilyprocessedinawidevarietyofprogramminglanguages.

Theservicedescriptions(“serviceoffers”)areprovidedinJSONformat.ThedocumentscanbeconsumeddirectlyorviaoneofthesupportedprogrammingAPIs—Clojure(Java),ClojureScript(JavaScript),andPython.Thesyntaxofthedescriptionsisessentiallyakey/valueformat,wherethekeysarenamespacedtoavoidcollisions.

D6.2 2 � HumanReadableServiceDescriptionFormat

SHOULD Humanswillneedtoreadtheservicedescriptionstounderstandproviderofferingsandtoprepareappropriatequeries(policies).TheservicedescriptionformatSHOULDbeeasilyreadablebyhumans.

Theserviceoffersareextensiblekey/valuepairsprovidedinJSONformat.Inaddition,supportingresourcesthatprovideinformationaboutkeynamespacesandsemanticinformationabouteachkeyareprovided.ThesealsouseaJSONformat.

D6.2 3 ≈ EndorsementFormat

MUST Theformatusedforserviceendorsements(e.g.certifications,availabilitymetrics,etc.)MUSTbethesameformatastheservicedescriptionformat.

Separateendorsementresourceshavenotbeenprovidedasthesehavenotbeenrequiredbytheuserstodate.ThesewilllikelybeaddedaspartofanotherH2020project(EU-SEC).However,aBenchmarkresourcethatpermitsuserstoprovideperformanceinformationaboutserviceoffershasbeenadded.

D6.2 4 � MachineReadableServiceQueryFormat

MAY Theservicequeryformat(policy)maybecreatedoralteredbytheCYCLONEtools.ThepolicyformatMAYbeeasilyreadablefromawidevarietyofprogramminglanguages,eitheritcanbeawell-supportedstandardorwrittenaccordingtoawell-definedgrammar.

Thequerylanguageusedfortheservicecatalog(andindeedallCIMI-basedresources)istheCIMIfilteringlanguage.ThegrammarisdefinedintheCIMIAPIspecification.

D6.2 5 � HumanReadableServiceQueryFormat

MUST Humansaswellastoolswillbecreatingtheservicequeries(policies),sotheformatandgrammarMUSTbeeasilyreadablebyhumans.

ThefilteringlanguagedefinedintheCIMIspecificationwasdesignedtobeeasilyusedbyhumans.Thesyntaxisquitenaturalforanyonewhohasprogrammedorusedadatabase.

D6.2 6 � FlexibleServiceDescriptionSchema

MUST Thecharacteristicsoftheunderlyingcloudservicesmayvaryfromservicetoservice(e.g.onebasedonvirtualmachinesandanotherbasedoncontainers)andovertimeasnewservicesappear.Consequently,theschemaoftheservicedescriptionMUSTbeflexible.

Theschemaoftheserviceoffers(andbenchmarks)isanopencollectionofkey/valuepairs.Anyappropriateattributescanbeaddedtotheserviceoffers.Theonlyrequirementisthattheattributenamespace(s)mustberegisteredwithSlipStreamtoavoidcollisions.



Doc. ID Title Level Description CommentD6.2 7 � Graceful

HandlingofMissingorIncorrectInformation

MUST Formanyreasons(e.g.afieldisn’tappropriateforaservice,atool/humancreatesanerroneousentry),theservicedescriptionsmaynotcontaininformationexpectedbyaparticularqueryormayhaveprovidedtheinformationinanincompatibleformat(e.g.stringinsteadofaninteger).ThematchmakingprocessMUSTrespondtomalformedqueries/descriptionsinawell-definedmannerandMUSTacceptthemwithoutcrashing.

Theimplementationwillignoremissinginformationorincorrectinformation.Inaddition,theimplementationextendstheCIMIfilteringlanguagetoallowuserstosearchforserviceoffersthatdonotdefineaparticularkey(throughsupportofa“null”value).

D6.2 8 ≈ JoinServiceDescriptionsandEndorsements

MUST Formatchingonnon-functionalcharacteristics,thequerylanguageMUSTbeabletojoininformationfromservicedescriptionsandfromthird-partyendorsements.

Theabilitytoperforma“join”betweenalldocumentswithintheservicecatalogwasimpossibletoimplementwhilemaintaininggoodgeneralqueryperformance.Consequently,thejoinwillbeimplementedasastagedselectionprocessinstead.Apre-filteringstagewilleliminateoffersthatdonotconformtoperformance,security,orotheruser-levelrequirements.Thiswillthenbefollowedbythefilteringofoffersbasedonapplicationrequirements.

D6.2 9 � OrderingofResults

SHOULD Aparticularservicequerymayreturnmorethanonethatmatchesthegivenrequirements.ThequerylanguageSHOULDallowmultiplematchestoberankedorordered.Thisallowsuserpreferencestobetakenintoaccount.

Theresultsofanyquerycanbeordered(ascendingordescending)bythevalue(s)ofanykey(s)intheserviceoffer.

D6.2 10 � LogicalOperations

MUST ThequerylanguageMUSTsupportlogicalANDandORoperations.Additionallogicaloperations(e.g.XOR)MAYalsobesupported.

ThequerylanguagesupportsbothANDandORlogicaloperations.

D6.2 11 ≈ ArithmeticOperations

MUST ThequerylanguageMUSTsupportbasicarithmeticoperations(add,subtract,multiply,divide)forintegerandfloatingvalues.OtherarithmeticoperationsMAYalsobesupported.

Thequerylanguagesupportsrelationaloperationsoninteger,string,anddatevalues.ExtensionstotheCIMIfilteringlanguagealsoallowaggregations(sums,averages,statistics,etc.)tobeperformedoverasetofserviceoffers.Generalarithmeticoperationshaven’tbeenrequestedbythecurrentusers.

D6.2 12 � StringMatching MUST ThequerylanguageMUSTsupportstringequalityandinequalityoperations.

Bothequalityandinequalityofstringsaresupported.Inaddition,a“starts-with”(^=)operatorhasbeenprovided.



Doc. ID Title Level Description CommentD6.2 13 � StringRegular

ExpressionsSHOULD ThequerylanguageSHOULDsupportstring

matchingbasedonregularexpressions.Stringmatchingbasedon“globbing”wouldbefairlystraightforwardtoimplement;fullregularexpressionsupportwouldbedifficult.Inanycase,thisfunctionalityhasnotbeenrequiredbycurrentusers.

D6.2 14 ≈ ArrayOperations SHOULD ThequerylanguageSHOULDsupportarrayoperations,forexample,beabletodefinefiltersontheinclusionandexclusionofvalueswithinanarray.

Thisfeaturehasnotbeenrequestedbycurrentusersandisnotprovided.However,allCIMIresourcesallowuserstodefineasetofproperties(key/valuepairs)thatcanbeusedtotagresources.SearchingfordocumentswithcertainpropertiesisfullysupportedbytheCIMIfilteringsyntaxandthecurrentSlipStreamimplementation.

D6.2 15 � ServiceLevelAgreement

MUST ThebrokeringimplementationsmustfacilitatetheexchangeofServiceLevelAgreementsinstandardorcustomaryformats.

TheimplementationdoesnotdirectlysupporttheimportorexportofSlipStreamplacementpoliciesoroffersasServiceLevelAgreements.

D6.3 1 � ScalingonLoad MUST Triggeringofscalingactionsofanapplicationbasedonapplicationmetricsusingsimple,predefinedalgorithms(e.g.addingnodebasedonmachineload).

TheSlipStreamAPIallowsbothhorizontal(e.g.anotherdatabasenode)andvertical(e.g.moreRAM)scalingactions.Theseactionscanbetriggeredbyusersorbyapplications.Anexampleauto-scalingapplicationisprovidedintheAppStorethatdemonstrateshowthiscanbeaccomplished.

D6.3 2 � ScalingonApplicationMetric

MUST Triggeringofscalingactionsofanapplicationbasedonapplicationmetricsdefinedbythedeveloperoftheapplication.


D6.3 3 � PublishingBenchmarks

MUST Abilitytopublishapplication-specificbenchmarksofcloudprovidersintotheServiceCatalogorOpenServiceCompendium.

ABenchmarkresourcethatpermitsuserstoprovideperformanceinformationaboutserviceoffershasbeenadded.

D6.3 4 � StaticPlacement MUST Placementbasedonstaticcharacteristics(e.g.geographicallocation)ofacloudserviceprovider.

Applicationdeveloperscanspecifyplacementpoliciesbasedonthecharacteristicsinaserviceoffer.

D6.3 5 ≈ DynamicPlacement

SHOULD PlacementbasedondynamicVMmonitoringinformationfromSlipStreamitself.

SeecommentforD6.2/8.Thisdynamicinformation(e.g.availability)willeventuallybeprovidedasbenchmarking(orsimilar)resources.

D6.3 6 � PlacementwithExternalInformation

MUST PlacementbasedonexternalinformationpushedintotheSlipStreamServiceCatalogorOpenServiceCompendium.

Theinformationforserviceofferscanbegatheredfromanysource,includingexternalones.Onceavailableinaserviceoffer,thestandardquery/filteringmechanismsareavailabletousers.

D6.3 7 ≈ JoinedPlacement

SHOULD Placementbasedonthejoinofallinformationassociatedwitha�givencloudserviceprovider.




Doc. ID Title Level Description CommentD6.3 8 � PriceRanking MUST Rankingofselectedcloudserviceprovidersbased

onpredefined�algorithms(e.g.price).SlipStreamalreadyranksselectedserviceoffersbasedonthecalculatedprice(forserviceofferswithpriceinformation).

D6.3 9 � User-DefinedRanking

SHOULD Rankingbasedonalgorithmsprovidedbytheapplicationdeveloperand/ortheapplicationoperator.

ThisfeatureisnotscheduledintheSlipStreamroadmap.

D6.3 10 � Notifications SHOULD Abilitytotriggernotifications/alertsthroughSlipStream.

Thisfeatureisplannedbutconsideredlowpriority.Thiswillprobablybemadeavailableneartheendof2017.

D6.3 11 � Auto-scaling MUST Abilitytotriggerscalingactionsfromwithintheapplication.

Anexampleauto-scalingapplicationhasbeenprovidedintheNuvlaAppStore.Thisshowshowtocollectandanalyzeapplication-definedmetricsandthentoperformscalingactionsbasedonthosemetrics.

D6.3 12 � ServiceOfferSearch

MUST AbilitytosearchtheServiceCatalogandOpenServiceCompendiummanuallytoseetheresultsfromvariouspoliciesandtoideallythenassociatethosepolicieswithapplications.

SearchingthecatalogofserviceoffersisfullysupportedthroughtheSlipStreamCIMIAPI.Inaddition,aprototypeweb-basedUIhasbeendeveloped.TheprototypeisavailableonNuvla,althoughnotyetafully-supported,officialfeature.

complete dynamic multi-cloud application management€¦ · • scalability, reliability, and...

Documents