cloudbroker platform integration into ws-pgrade/guse
Post on 06-Jan-2016
40 Views
Preview:
DESCRIPTION
TRANSCRIPT
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI-283481
CloudBroker Platform integration into WS-PGRADE/gUSE
Zoltán FarkasMTA SZTAKI LPDS
zoltan.farkas@sztaki.mta.hu
Outline
• Motivation• WS-PGRADE/gUSE• CloudBroker Platform• Integration:
– Architecture, features– DCI Bridge modifications– WS-PGRADE modifications– WFI modifications
• SaaS vs IaaS execution modes
Motivation• There are many user communities who would like to access
compute resources in a transparent way through science gateways
• Cloud resources are getting more and more popular• Clouds are more reliable than grids• Additionally, cloud resource can provide a vast amount of
resources (flexible resource allocation)• A customizable science gateway with cloud support is capable
of satisfying compute needs of complex scientific computations
• Aim: provide access to cloud resources in a transparent way
WS-PGRADE/gUSE in SCI-BUS
The CloudBroker Platform• Easy, scalable, secure, integrable and pay-per-use access to scientific and
technical applications in the cloud• High performance computing application store and marketplace with
accounting and billing• Deployment and execution of applications in the cloud with the same
parameters and files as for local execution• Accessible through the internet via a web browser UI• Integration possibilities into third party tools via web service-based APIs• Public version available under https://platform.cloudbroker.com
• Using infrastructure as a service (IaaS) from cloud providers• Offering platform as a service (PaaS) for software vendors• Providing software as a service (SaaS) to end users• Available compute resources: Amazon, IBM, OpenStack, OpenNebula• Available storage resources: Amazon, Walrus, Rados S3
CloudBroker Platform Architecture
CloudBroker PlatformCloudBroker Platform
AmazonCloud
IBMCloud
…Cloud
ChemistryAppli-
cations
BiologyAppli-
cations
MedicineAppli-
cations
WebBrowser
UI
Engineering Appli-
cations
Web Service API
GenericWorkbenches CloudBroker Integration
CloudBroker Integration
Domain-Specific Gateways
R&D End Users and Software Vendors
CLI
…Appli-
cations
Integrated architecture
Integration features I.
• Slight modification in the fully featured UI• Also available in every customization level,
completely hiding the fact that cloud is being used:– ASM– Remote API– End-user view– DCI Bridge
WS-PGRADEWF
DeveloperUI
gUSE DCI Bridge
DCI 1
DCI 2
DCI n
ApplicationSpecific
User Interface
ExistingApplicationSpecific UI
WS-PGRADEEnd-User
UI
Remote API
BES interface
ASM API
A
B
C
D
E
BES interface
Integration features II.
• Support for commercial clouds with costs (prices configured in CloudBroker Platform):– Estimated job cost before submission– Actual job and workflow cost after execution
Integration features III.IaaS model SaaS model
Enables users to run their own executables Yes No
What has to be pre-deployed in the virtual machine image
A single wrapper application Every application that users would like to use
What has to be configured in the CloudBroker Platform
Only the wrapper application Executables for the different application
Level of security Low: user can run anything High: users can run only pre-registered, tested apps
Ease of use (user’s perspective) Very easy: very similar to existing WS-PGRADE/gUSE interface
Very easy: user simply selects from the pre-defined Software
Ease of use (portal administrator’s perspective)
Easy: only two additional properties have to be set in the DCI Bridge configuration (see previous slide)
Very easy: no need to set additional properties
Ease of use (CloudBroker administrator’s perspective)
Easy: only one application has to be configured for each cloud resource
Hard: a number of applications have to be configured
Easy of use (cloud administrator’s perspective)
Easy: only one VM image has to be deployed Hard: either a number of VM images have to be deployed or one VM image must be updated a number of times.
DCI Bridge modifications I.
• New Configuration tab for CloudBroker
DCI Bridge modifications II.• New CloudBroker plugin to manage CloudBroker jobs• Using slightly extended JSDL
<JobDefinition xmlns="http://schemas.ggf.org/jsdl/2005/11/jsdl" xmlns:ns2="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix" xmlns:ns3="uri:MBSchedulingDescriptionLanguage" xmlns:ns4="extension.dci" xmlns:ns5="http://cloud.extension.jsdl" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://schemas.ggf.org/jsdl/2005/11/jsdl"> <JobDescription> <JobIdentification> <JobName>Workflow Test Applications 1.0</JobName> <JobAnnotation>1f803ac0-8cfd-47de-9b9a-718f2a287f20</JobAnnotation> </JobIdentification> <Application> <ApplicationName>Workflow Test Applications 1.0</ApplicationName> <ApplicationVersion>Workflow Test Applications 1.0 ARInew.sh</ApplicationVersion> <ns2:POSIXApplication_Type> <ns2:Executable>Workflow Test Applications 1.0 ARInew.sh</ns2:Executable> <ns2:Argument>A</ns2:Argument> <ns2:Output>job.out</ns2:Output> <ns2:Error>job.err</ns2:Error> <ns2:UserName>10196</ns2:UserName> <ns2:GroupName>http://192.168.152.73:8080/wspgrade</ns2:GroupName> </ns2:POSIXApplication_Type> </Application> <Resources> <ns5:CloudResources_Type> <ns5:CloudResource> <ns5:ResourceName>OpenNebula MTA SZTAKI SZTAKI</ns5:ResourceName> <ns5:RegionName>OpenNebula MTA SZTAKI SZTAKI SZTAKI OpenNebula</ns5:RegionName> <ns5:InstanceTypeName>OpenNebula MTA SZTAKI SZTAKI SZTAKI OpenNebula x86_64</ns5:InstanceTypeName> </ns5:CloudResource> </ns5:CloudResources_Type> </Resources>
WS-PGRADE modifications I.
• CloudBroker Authentication Portlet
WS-PGRADE modifications II.
• IaaS and SaaS-based configuration
WFI modifications
• Produce JSDL document for DCI Bridge based on workflow XML containing CloudBroker-specific settings
Accessible Cloud Resources
• Access provided by the CloudBroker Platform• Commercial:
– Amazon EC2– IBM
• OpenSource/Free:– OpenStack– OpenNebula
• Currently, within SCI-BUS accessible:– MTA SZTAKI OpenNebula (400 cores)– BIFI OpenStack (50 cores)
Example applications - IaaS
• University-level cloud resource access• Run any application
Example applications - SaaS
• Predeployed application execution• Docking portal: AutoGrid, AutoDock,
AutoDock Vina (should be deployed)
Summary
• WS-PGRADE/gUSE offers production-level cloud access through the CloudBroker Platform
• Mainstream cloud resources can be used (both IaaS and SaaS model is supported)
• Slight UI changes (can be completely hidden for end-users)
• Commercial exploitation available
Questions?
?
top related