cloudbroker platform integration into ws-pgrade/guse
DESCRIPTION
CloudBroker Platform integration into WS-PGRADE/gUSE. Zoltán Farkas MTA SZTAKI LPDS [email protected]. Outline. Motivation WS-PGRADE/gUSE CloudBroker Platform Integration: Architecture, features DCI Bridge modifications WS-PGRADE modifications WFI modifications - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/1.jpg)
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI-283481
CloudBroker Platform integration into WS-PGRADE/gUSE
Zoltán FarkasMTA SZTAKI LPDS
![Page 2: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/2.jpg)
Outline
• Motivation• WS-PGRADE/gUSE• CloudBroker Platform• Integration:
– Architecture, features– DCI Bridge modifications– WS-PGRADE modifications– WFI modifications
• SaaS vs IaaS execution modes
![Page 3: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/3.jpg)
Motivation• There are many user communities who would like to access
compute resources in a transparent way through science gateways
• Cloud resources are getting more and more popular• Clouds are more reliable than grids• Additionally, cloud resource can provide a vast amount of
resources (flexible resource allocation)• A customizable science gateway with cloud support is capable
of satisfying compute needs of complex scientific computations
• Aim: provide access to cloud resources in a transparent way
![Page 4: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/4.jpg)
WS-PGRADE/gUSE in SCI-BUS
![Page 5: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/5.jpg)
The CloudBroker Platform• Easy, scalable, secure, integrable and pay-per-use access to scientific and
technical applications in the cloud• High performance computing application store and marketplace with
accounting and billing• Deployment and execution of applications in the cloud with the same
parameters and files as for local execution• Accessible through the internet via a web browser UI• Integration possibilities into third party tools via web service-based APIs• Public version available under https://platform.cloudbroker.com
• Using infrastructure as a service (IaaS) from cloud providers• Offering platform as a service (PaaS) for software vendors• Providing software as a service (SaaS) to end users• Available compute resources: Amazon, IBM, OpenStack, OpenNebula• Available storage resources: Amazon, Walrus, Rados S3
![Page 6: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/6.jpg)
CloudBroker Platform Architecture
CloudBroker PlatformCloudBroker Platform
AmazonCloud
IBMCloud
…Cloud
ChemistryAppli-
cations
BiologyAppli-
cations
MedicineAppli-
cations
WebBrowser
UI
Engineering Appli-
cations
Web Service API
GenericWorkbenches CloudBroker Integration
CloudBroker Integration
Domain-Specific Gateways
R&D End Users and Software Vendors
CLI
…Appli-
cations
![Page 7: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/7.jpg)
Integrated architecture
![Page 8: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/8.jpg)
Integration features I.
• Slight modification in the fully featured UI• Also available in every customization level,
completely hiding the fact that cloud is being used:– ASM– Remote API– End-user view– DCI Bridge
WS-PGRADEWF
DeveloperUI
gUSE DCI Bridge
DCI 1
DCI 2
DCI n
ApplicationSpecific
User Interface
ExistingApplicationSpecific UI
WS-PGRADEEnd-User
UI
Remote API
BES interface
ASM API
A
B
C
D
E
BES interface
![Page 9: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/9.jpg)
Integration features II.
• Support for commercial clouds with costs (prices configured in CloudBroker Platform):– Estimated job cost before submission– Actual job and workflow cost after execution
![Page 10: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/10.jpg)
Integration features III.IaaS model SaaS model
Enables users to run their own executables Yes No
What has to be pre-deployed in the virtual machine image
A single wrapper application Every application that users would like to use
What has to be configured in the CloudBroker Platform
Only the wrapper application Executables for the different application
Level of security Low: user can run anything High: users can run only pre-registered, tested apps
Ease of use (user’s perspective) Very easy: very similar to existing WS-PGRADE/gUSE interface
Very easy: user simply selects from the pre-defined Software
Ease of use (portal administrator’s perspective)
Easy: only two additional properties have to be set in the DCI Bridge configuration (see previous slide)
Very easy: no need to set additional properties
Ease of use (CloudBroker administrator’s perspective)
Easy: only one application has to be configured for each cloud resource
Hard: a number of applications have to be configured
Easy of use (cloud administrator’s perspective)
Easy: only one VM image has to be deployed Hard: either a number of VM images have to be deployed or one VM image must be updated a number of times.
![Page 11: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/11.jpg)
DCI Bridge modifications I.
• New Configuration tab for CloudBroker
![Page 12: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/12.jpg)
DCI Bridge modifications II.• New CloudBroker plugin to manage CloudBroker jobs• Using slightly extended JSDL
<JobDefinition xmlns="http://schemas.ggf.org/jsdl/2005/11/jsdl" xmlns:ns2="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix" xmlns:ns3="uri:MBSchedulingDescriptionLanguage" xmlns:ns4="extension.dci" xmlns:ns5="http://cloud.extension.jsdl" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://schemas.ggf.org/jsdl/2005/11/jsdl"> <JobDescription> <JobIdentification> <JobName>Workflow Test Applications 1.0</JobName> <JobAnnotation>1f803ac0-8cfd-47de-9b9a-718f2a287f20</JobAnnotation> </JobIdentification> <Application> <ApplicationName>Workflow Test Applications 1.0</ApplicationName> <ApplicationVersion>Workflow Test Applications 1.0 ARInew.sh</ApplicationVersion> <ns2:POSIXApplication_Type> <ns2:Executable>Workflow Test Applications 1.0 ARInew.sh</ns2:Executable> <ns2:Argument>A</ns2:Argument> <ns2:Output>job.out</ns2:Output> <ns2:Error>job.err</ns2:Error> <ns2:UserName>10196</ns2:UserName> <ns2:GroupName>http://192.168.152.73:8080/wspgrade</ns2:GroupName> </ns2:POSIXApplication_Type> </Application> <Resources> <ns5:CloudResources_Type> <ns5:CloudResource> <ns5:ResourceName>OpenNebula MTA SZTAKI SZTAKI</ns5:ResourceName> <ns5:RegionName>OpenNebula MTA SZTAKI SZTAKI SZTAKI OpenNebula</ns5:RegionName> <ns5:InstanceTypeName>OpenNebula MTA SZTAKI SZTAKI SZTAKI OpenNebula x86_64</ns5:InstanceTypeName> </ns5:CloudResource> </ns5:CloudResources_Type> </Resources>
![Page 13: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/13.jpg)
WS-PGRADE modifications I.
• CloudBroker Authentication Portlet
![Page 14: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/14.jpg)
WS-PGRADE modifications II.
• IaaS and SaaS-based configuration
![Page 15: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/15.jpg)
WFI modifications
• Produce JSDL document for DCI Bridge based on workflow XML containing CloudBroker-specific settings
![Page 16: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/16.jpg)
Accessible Cloud Resources
• Access provided by the CloudBroker Platform• Commercial:
– Amazon EC2– IBM
• OpenSource/Free:– OpenStack– OpenNebula
• Currently, within SCI-BUS accessible:– MTA SZTAKI OpenNebula (400 cores)– BIFI OpenStack (50 cores)
![Page 17: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/17.jpg)
Example applications - IaaS
• University-level cloud resource access• Run any application
![Page 18: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/18.jpg)
Example applications - SaaS
• Predeployed application execution• Docking portal: AutoGrid, AutoDock,
AutoDock Vina (should be deployed)
![Page 19: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/19.jpg)
Summary
• WS-PGRADE/gUSE offers production-level cloud access through the CloudBroker Platform
• Mainstream cloud resources can be used (both IaaS and SaaS model is supported)
• Slight UI changes (can be completely hidden for end-users)
• Commercial exploitation available
![Page 20: CloudBroker Platform integration into WS-PGRADE/gUSE](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813d5c550346895da727fe/html5/thumbnails/20.jpg)
Questions?
?