servicess, a new programming model for the cloud daniele lezzi, rosa m. badia, jorge ejarque, raul...

ServiceSs, a new programming model for the

CloudDaniele Lezzi, Rosa M. Badia, Jorge Ejarque, Raul Sirvent, Enric Tejedor

Grid Computing and Clusters Group Barcelona Supercomputing Center

OGF-Europe COS on Software Development Tools for Distributed Computing

OGF28, Munich 15 March 2010

COS on Software Development Tools for Distributed Computing


Outline

• StarSs programming model

• COMP Superscalar framework

• COMP Superscalar towards SOA and Clouds

• ServiceSs

2



...for (i=0; i<N; i++){ T1 (data1, data2); T2 (data4, data5); T3 (data2, data5, data6); T4 (data7, data8); T5 (data6, data8, data9);}...

Sequential Application

T10 T20

T30

T40

T50

T11 T21

T31

T41

T51

T12

…

Resource 1

Resource 2

Resource 3

Resource N

.

.

.

Task graph creation

based on data

precedence

Task selection +

parameters direction

(input, output, inout)

Scheduling,

data transfer,

task execution

Synchronization,

results transfer

Parallel Resources(cluster, grid)

Star Superscalar Programming Model

3



StarSs programming model

• GRIDSs, COMPSs

• Tailored for Grids or clusters

• Data dependence analysis based on files

• C/C++, Java

• SMPSs

• Tailored for SMPs or homogeneous multicores

• Altix, JS21 nodes, Power5, Intel-Core2

• C or Fortran

• CellSs

• Tailored for Cell/B.E. processor

• C or Fortran

• GPUSs (Next session talk by Rosa M. Badia)

• NestedSs

• Hybrid approach that combines SMPSs and CellSs



COMPSs

• Componentisation of the GRID superscalar runtime

• Each component in charge of a functionality

5

Base technologies:

• Java as programming language

• ProActive:

• Reference implementation of the GCM model

• Used to build the components

• JavaGAT

• API that provides uniform access to different kinds of Grid middleware

• Used for job submission and file transfer



initialize(f1);

for (int i = 0; i < 2; i++) {

genRandom(f2);

add(f1, f2);

}

print(f2);

Java application

COMPSs Programming model – Application + interface

public interface SumItf {

@ClassName(“example.Sum")

@MethodConstraints(OSType = "Linux")

void genRandom(

@ParamMetadata(type = Type.FILE, direction = Direction.OUT)

String f

);

@ClassName(“example.Sum")

...

}

Task constraints

Parametermetadata

Implementation

Java interface

6



Custom Java Class Loader

Java app code

COMPSs runtime

Annotated

interface

Javassist

insertscalls to

Custom Loader

uses

input

C/C++ app code

COMPSs runtime

Interface

insertscalls to

Stubs Generator

input

JNI



COMPSs and XtreemOS

• COMPSs has been ported in the XtreemOS framework by means of the Simple API for Grid Applications (SAGA).

• COMPSs benefits from SAGA by delegating to it the job execution including resource allocation and data transfers.



Integration in a Service-Oriented and Cloud infrastructure

•Main idea: moving the COMPSs runtime from the client side to a

server SOA platform

•Characteristics of this environment:

• Execution of application tasks offered as services

• N applications can be served simultaneously

• Several COMPSs can be deployed, to serve the tasks from one or more

applications

• Resource provisioning brought by a Cloud

9



COMPSs and EMOTIVE Cloud



COMPSs and EMOTIVE Cloud

•The Virtualization Resource Management and Monitoring (VRMM) layer wraps each virtualized node and monitors its state

•Within the VRMM layer, the Virtualization Manager (VtM) provides an application-specific virtual machine (VM) for each application; the application is given full control of its execution environment without introducing risk to the underlying system or other applications

•The VMs are created on demand, according to application requirements, and are consolidated in the provider’s physical resources to optimise their use

•A Resource Monitor (RM) monitors task and resource status, including historical information

•SERA (Semantically-Enhanced Resource Allocator): Resources allocation using semantics and agents



ServiceSs envisioned architecture

12

COMPSs API

COMPSs API

COMPSs API

JavaApp

WS

Co

nta

iner

Ru

nti

me

Man

ag

er

COMPSsruntime

instance 1

CloudScheduler

COMPSsruntime instance N

WorkerVM 1

WorkerVM 1

WorkerVM 1

WorkerVM M

COMPSsApplication

Side

JavaApp

JavaApp

Cloud

WS

Co

nta

iner

UserSide

App

App

User

WS

Co

nta

iner

WS

Co

nta

iner

WorkerVM 1

ExternalWS

WorkerFP7 OPTIMIS Project(Optimized Infrastructure Services)



COMPSs and EMOTIVE Cloud – Step 1

VM1 VMnVM2

1. The Task Scheduler asks SERA for a pool of VMs

2. COMPSs executes tasks on these VMS

3. COMPSs requests the creation of more or “bigger” VMs (memory, CPU, etc)

VMn+1



Next steps

1. Extension of the COMPSs API to support the new features

2. Extension of the Runtime Manager

3. Tools for deployment of the core elements and composite applications.

COMPSs API

COMPSs API

COMPSs API

JavaApp

WS

Co

nta

iner

Ru

nti

me

Man

ager

COMPSsruntime

instance 1

CloudScheduler

COMPSsruntime

instance N

WorkerVM 1

WorkerVM 1

WorkerVM 1

WorkerVM M

COMPSsApplication

Side

JavaApp

JavaApp

Cloud

WS

Co

nta

iner

UserSide

App

App

User

WS

Co

nta

iner

WS

Co

nta

iner

WorkerVM 1

ExternalWS Worker

• www.bsc.es/grid• www.emotivecloud.net

servicess, a new programming model for the cloud daniele lezzi, rosa m. badia, jorge ejarque, raul...

Documents

grid superscalar

software development

distributed computing

distributed computingogf28

enric tejedorgrid computing

new programming model

raul sirvent

jorge ejarque