elastic processes on clouds of hybrid services: principles, enabling techniques and approaches

SummerSOC 12, 3 Jul 2012 1

Elastic Processes on Clouds of Hybrid Services: Principles, Enabling Techniques

and Approaches

Hong-Linh Truong Distributed Systems Group,

Vienna University of Technology

[email protected]://www.infosys.tuwien.ac.at/staff/truong

mailto:[email protected]

2

Acknowledgement: some results from joint works with: Kamal Bhattacharya, Schahram Dustdar, Muhammad Zuhri Catur Candra, Aditya Ghose, Lam-Son Le, Frank Leymann, Vitaliy Liptchinsky, Tran-Vu Pham, Michael Reiter, Mirela Riveni, Ognjen Scekic, Quang-Hieu Vu, and Rostyslav Zaboloznyi

SummerSOC 12, 3 Jul 2012

3

An example of elastic processes

Recap: Schahram‘s presentation yesterday


4

Hybrid services in the cloud


5

How to realize elasticity in hybrid systems?

Attention: ongoing work not everything is well tested !


6

Outline

Vienna Elastic Computing Model Incorporating humans into programming

paradigms Programming hybrid services in the cloud Modeling elasticity constraints Reasoning elasticity constraints Conclusions and future work


7

VIENNA ELASTIC COMPUTING MODEL



Multi-dimensional Elasticity

Units can be networks, middleware, SaaS, PaaS


Vienna Elastic Computing Modelmulti-dimensional elasticity, service computing models and cloud provisioning models

Schahram Dustdar, Hong-Linh Truong, "Virtualizing Software and Humans for Elastic Processes in Multiple Clouds - a Service Management Perspective" International Journal of Next-Generation Computing, To appear, 2012


INCORPORATING HUMANS INTO PROGRAMMING PARADIGM

11

Incorporate humans into a programming paradigm

• Abstracting human compute units as program elements• Extending programming languages to support human

compute units• Data/control flows via extensible APIs

Programming languages

• Shared memory (e.g., human –software – human), message passing (human-to-human), artifact-centric, etc., via APIs working atop the compute unit abstraction layer

Multiple programming

models

• Computing capability /profile management: human computing power, reputation and incentive models

• Monitoring: monitoring and enforcing incentives/rewards, quality of results, availability

• Communication: between human-middleware, among ICU/SCU , within SCU for exchanging artifacts and comprehensing social tasks

Execution environment


12

Incorporate humans into a programming paradigm

Volunteers ProfessionalsTeamIndividual

Service-based Middleware

CommunicationMonitoringCapability/Profile

Management

Provisioning/Negotiation/Execution API

Abstraction of Human Compute Units

SCU

ICU

ICU ICU SCU

Program languages and programming models

Program elements

Software Compute

Units

program human actions and dependenciesprogram incentive condition and rewarding actionprogram result evaluation method

Human-to-middleware interfaces:• visualization of collective

tasks• embedding of common

forms• mobile app


13

Existing technologies are not adequate


14

PROGRAMMING HYBRID SERVICES IN THE CLOUD

Hong-Linh Truong, Schahram Dustdar, Kamal Bhattacharya, Programming Hybrid Services in the Cloud, Working paper,2012


15

HBS Communication Interface

Based on well known technologies for integrating human requests/responses into SBS Emails, web interfaces, Web services, etc.

Some special characteristics The same protocol can be used for multiple layers

(middleware, application, OS) -- unlike in SBS Only asyschronous communication and unknown

upper-bound delay

Can support message passing and shared memory models hbsi hbsj or hbsi sbs hbsj


16

Human Power Unit (HPU)

Humans are determined via skills/skill levels A particular HBS cloud can define its own sets of

skills (CS) and skill levels (SK) Similar to Amazon defines its own EC unit Different ways to make sure skill and skill levels

declared in a cloud consistent (e.g., via testing and monitoring).

HPU Definition: HPU is a value describing the computing power of an HBS measured in an abstract unit. A cloud of HBS has a pre-defined basic power unit, hpuθ, corresponding to the baseline skill bsθ of the cloud.”


17

Human Power Unit (HPU) – our starting point

Given an HBS, its theoretical HPU can be larger or smaller than its real HPU – unlike SBS

HPU for a particular (skill, skill level)

HPU for a set of (skill, skill level), no weighted factors

Decomposition/composition of HPU for a shared HBS


18

HBS Instances Provisioning

• Individual Compute Unit (ICU)• Social Compute Unit (SCU

Types of services

• iICU(CS, HPU, price, utilization,location, APIs)

• iSCU(CS,HPU, price, utilization,connectedness, location, APIs)

• Other (traditional) NFPs

Instances Description

s

• utilization• offering communication APIs• connectedness

Pricing factors



Cloud APIs for Provisioning Hybrid Services

APIs hide low-level platforms and utilize low level HBS communication interfaces

APIs for HBS information and management

•listSkills();listSkillLevels();•listICU();listSCU()•negotiateHBS()•startHBS()•suspendHBS ()•resumeHBS ()•stopHBS()•reduceHBS()•expandHBS()

APIs for HBS execution and

communication •runRequestOnHBS ()•receiveResultFromHBS()

•sendMessageToHBS() •receiveMessageFromHBS()

20

Prototype (simulated environment)

Combined with Jcloud/boto for real SBS



Framework for Utilizing Hybrid Services

IT ManagementSmart City Management

22

Modeling HPU-aware Task Dependency Graphs

Link management skills and compliance constraints to tasks required HBS

Can be specified in TOSCA!Modeled in dependency graphs


23

Programming a combination of HBS and SBS

e.g., preparing/managing inputs/outputs for HBS using SBS


24

Forming iSCUs Done by consumers or cloud providers

SCU Formation Algorithms

Business As Usual

Corrective Action

HBS Constraints


25

Configuring iSCU

Establish „connectedness“ based on compliance constraints and network topology Addional cost might occur!

Program SBS and HBS for the iSCU to have a complete working environment.

Different connectedness E.g., Ring-based, star-based, and master-slave

topologies


26

Selecting HBS: Some algorithms

Several algorithms can be built based on existing team formation algorithms which do not consider dependency graphs

Different weighted factors can be consideredSummerSOC 12, 3 Jul 2012

27

Forming iSCU by minimizing cost and considering no direction



Example of star-based iSCU using Dropbox as a communication hub

29

Change Model for Task Graph’s Human Power Unit


30

MODELING ELASTICITY CONSTRAINT


31

Elasticitiy modeling

• Beyond traditional QoS metrics and for only services

• Modeling elasticity constraints• Attaching elasticity constraints to processes at

the design time for composition and runtime elasticity


32

Beyond QoS metrics (1)

And many more metrics!


33

Beyond QoS metrics (2)

Data marketplaces and DaaSElastic dynamic properties associated with:Service APIs, data APIs and data assets

Quang Hieu Vu, Tran Vu Pham, Hong-Linh Truong, Schahram Dustdar, Rasool Asal, DEMODS: A Description Model for Data-as-a-Service, The 26th IEEE International Conference on Advanced Information Networking and Applications (AINA-2012)


34

Elastic Profile Using Rules

process

process fragment

activity


35

Declarative constraints based first-order logic and set theory

Lam-Son Le, Hong-Linh Truong, Adytia Ghose, Schahram Dustdar, On Elasticity and Constrainedness of Business Services Provisioning, 9th International Conference on Services Computing (SCC 2012)



REASON THE ELASTICITY CONSTRAINTS


Runtime quality, cost and benefits evaluation and reasoning principles

Part A Part B Part C

model i model j model k

Runtime: Elastic processes

Quality, cost and benefits

38

Composable cost evaluation

Hong Linh Truong, Schahram Dustdar: Composable cost estimation and monitoring for computational applications in cloud computing environments. Procedia CS 1(1): 2175-2184 (2010)

Tran Vu Pham, Hong-Linh Truong, Schahram Dustdar "Elastic High Performance Applications - A Composition Framework", The 2011 Asia-Pacific Services Computing Conference (IEEE APSCC 2011), (c) IEEE Computer Society, December 12 - 15, 2011, Jeju, Korea



Runtime evaluation of quality of data in workflows

Michael Reiter, Uwe Breitenbuecher, Schahram Dustdar, Dimka Karastoyanova, Frank Leymann, Hong-Linh Truong, A Novel Framework for Monitoring and Analyzing Quality of Data in Simulation Workflows, (c)IEEE Computer Society, The 7th IEEE International Conference on e-Science, 5-8 December, 2011, Stockholm, SwedenMichael Reiter, Hong-Linh Truong, Schahram Dustdar, Dimka Karastoyanova, Robert Krause, Frank Leymann, Dieter Pahr, On Analyzing Quality of Data Infuences on Performance of Finite Elements driven Computational Simulations, EuroPar 2012

40

Incentive Monitoring

Ognjen Scekic, Hong-Linh Truong, Schahram Dustdar, "Modeling Rewards and Incentive Mechanisms for Social BPM",, Short paper, 10th International Conference on Business Process Management (BPM2012), (c)Springer-Verlag, September 3-6, 2012, Tallinn, Estonia


41

Put things together -- IDE for elastic processes


Put things together – from high-level languages to programmable units

42

Statelet: a declarative coordination language for collaboration processes

Statelet statements can be mapped to APIs for programmable units

Vitaliy Liptchinsky, Roman Khazankin, Hong Linh Truong, Schahram Dustdar: Statelets: Coordination of Social Collaboration Processes. COORDINATION 2012

Provisioning/Negotiation/Execution API

Abstraction of Human Compute Units

SCUICUICU ICU SCU


43

Conclusions and Future Work

Novel models and APIs are needed for integrating humans into program paradigms

We need to address the whole stack Elasticity modeling, monitoring and reasoning Mapping high level languages to APIs of hybrid services

Future works Several issues mentioned here are ongoing work Declarative languages for specifying elasticity Methods for evaluation and experiments

Some open questions will be addressed in FP7 CELAR


44

Thanks for your attention

Hong-Linh TruongDistributed Systems GroupVienna University of [email protected]://www.infosys.tuwien.ac.at/staff/truong


mailto:[email protected]

elastic processes on clouds of hybrid services: principles, enabling techniques and approaches

Education

human software multiple

programming languages

programming paradigmsummersoc

artifactprogramming

generation computing

hybrid systems

action program

program elementsprogramming