APSCC 2011, 13 Dec 2011, Jeju, Korean 1

Elastic High Performance Applications – a Composition Framework

Tran Vu Pham1, Hong-Linh Truong2, Schahram Dustdar2

1Faculty of Computer Science and EngineeringHo Chi Minh City University of Technology

2Distributed Systems Group, Vienna University of Technology

truong@infosys.tuwien.ac.athttp://www.infosys.tuwien.ac.at/Staff/truong

APSCC 2011, 13 Dec 2011, Jeju, Korean 2

Outline

Motivation

Elastic components and elastic high performance applications

Prototypes and experiments

Conclusions and future work

APSCC 2011, 13 Dec 2011, Jeju, Korean 3

Background

APSCC 2011, 13 Dec 2011, Jeju, Korean 4

Motivation (1)

HPC cloud markets: Infrastructure Providers, Software Providers, Service

Vendors, and End-user Diverse types of components:

HPC programs, libraries, operating systems, virtual machine images, Web services, SaaS, PaaS, and IaaS,

Different costs and licensing

Complex application requirements: quality, elastic time and money, scale in/out different

cloud environments

APSCC 2011, 13 Dec 2011, Jeju, Korean 5

Motivation (2)

Existing resolving software dependency and compatibility

→ deal wih software dependencies and compatibilities around a fixed OS

Workflow composition

→ deal with matching service input/output eHPA:

→ conflicting diverse types of components within and among cloud-based environments dynamically

Few existing solutions deal with resource elasticity only

→ there are multi-dimensional elasticity (see “Principles of Elastic Processes – IEEE Internet Computing 15, 5 (September 2011), 66-71”)

APSCC 2011, 13 Dec 2011, Jeju, Korean 6

eHPA components and relationships

APSCC 2011, 13 Dec 2011, Jeju, Korean 7

Multiple elasticity dimensions for eHPA

Resource elasticity: software (os/library/middleware/service) on multiple clouds

Non-functional parameters elasticity: quality, available time, right of uses

Pricing/Rewarding/Incentive elasticity: cost

eHPA elasticity

These elasticity metrics are simplified for the sake of brevity

See multiple elasticity dimensions at: http://www.slideshare.net/linhsolar/principles-of-elastic-processes-on-clouds-and-some-enabling-techniques

APSCC 2011, 13 Dec 2011, Jeju, Korean 8

Requirements and elastic properties

Properties End user/Service Vendor Software/Infrastructure Provider

Functions + +

Dependencies +

Conflicts +

Properties End user/Service Vendor Software/Infrastructure Provider

Resource +

Cost + +

Quality + +

Time + +

Rights of Use + +

Functional requirements

Elastic properties

APSCC 2011, 13 Dec 2011, Jeju, Korean 9

Elastic High Performance Application (eHPA)

Elastic component Functional description Elastic properties

Elastic Component and eHPA

Component properties and dependencies are modeled using ontology

APSCC 2011, 13 Dec 2011, Jeju, Korean 10

Right of Uses

Available Time

Quality

Component cost Aggregated cost

Resource for components Internal dependency External dependency

Elastic measurements and aggregation

APSCC 2011, 13 Dec 2011, Jeju, Korean 11

Composition algorithms (1)

Requested partitions of components and elastic requirements

eHPA Compostion – functionality aspect Resolve

dependencies, check conflicts, and form partitions

EHPA composition – elastic requirement Check elastic

requirements

APSCC 2011, 13 Dec 2011, Jeju, Korean 12

Composition Algorithms (2)

APSCC 2011, 13 Dec 2011, Jeju, Korean 13

Prototype

APSCC 2011, 13 Dec 2011, Jeju, Korean 14

Experiments – application

Star3D Solving Euler equations in the cases of 3D flows

Based on MPI

APSCC 2011, 13 Dec 2011, Jeju, Korean 15

Experiments – modeling Star3D

Elastic properties Resources: 32

processes Subjective rank:

2-4 Free of charge Unlimited time Academic license

APSCC 2011, 13 Dec 2011, Jeju, Korean 16

Experiments – possible solutions

75 components in knowledge based Star3D on EC2 with linux 12 different solutions

Four groups of solutions, different component external and internal dependencies

APSCC 2011, 13 Dec 2011, Jeju, Korean 17

Experiments – examples of solutions

An eHPA solution using free a Fortran compiler (solution 8)

An eHPA solution using Portland Fortran compiler, licensed foruse up to 256 MPI processes (solution 4).

APSCC 2011, 13 Dec 2011, Jeju, Korean 18

Discussion and future work

Complex HPAs in clouds Deal with complex software dependencies and conflicts

Determine and characterize elastic properties Multi-dimensional elasticity metrics: resource, quality, cost, available time and

right of uses

We propose modeling and composition techniques We use simple elastic properties but they can further modeled into

sub-dimensions our first step toward multi-dimensional elasticity for HPAs

Current we do not consider dependencies among these properties

Our future work Integrate with TOSCA (www.open-tosca.org)

Work on elasticity tradeoff

Develop runtime packaging and deployment

APSCC 2011, 13 Dec 2011, Jeju, Korean 19

Thanks for your attention!

Hong-Linh TruongDistributed Systems GroupVienna University of TechnologyAustria

truong@infosys.tuwien.ac.athttp://www.infosys.tuwien.ac.at/staff/truong