denis caromel, et al. proactive.objectweb oasis team

Denis Caromel1

Denis Caromel, et al.http://ProActive.ObjectWeb.org

OASIS TeamINRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis, IUF

Strong Programming Model for Strong Weak Mobility:

1. Introduction to ProActive2. PM: Active Objects + Groups3. Mobile Agents4. Strategies for Localization 5. GUI (Video)6. Applications: Load Balancing

Denis Caromel2

ProActive Parallel Suite (1)

Open

Source

+

PROFESSIONAL

SUPPORT

Denis Caromel3

ProActive Parallel Suite: GUI

Denis Caromel4

ProActive Parallel Suite: GUI

Denis Caromel5

Denis Caromel6

ProActive Parallel Suite: Deploy

Denis Caromel7

ProActive Parallel Suite: Deploy

Denis Caromel8

Deploy on Various Kinds of Infrastructures

Internet

Clusters

ParallelMachine

LargeEquipment

Internet

Job management forembarrassingly parallel application (e.g. SETI)

Internet

Internet

Servlets EJBs Databases

Denis Caromel9

Abstract Deployment Model

Problem:

Difficulties and lack of flexibility in deployment

Avoid scripting for: configuration, getting nodes, connecting, etc.

A key principle: Virtual Node (VN) + XML deployment file

Abstract Away from source code, and Wrapping code:

Machines

Creation Protocols

Lookup and Registry Protocols

Protocols and infrastructures:

Globus, ssh, rsh, LSF, PBS, SGE, IBM Load Lever, …

Web Services, ...

Data management: File transfer

Denis Caromel10

Scheduler and Resource Manager:User Interface

Denis Caromel11

Scheduler: User Interface

Denis Caromel12

ProActive Parallel Suite: Program

Denis Caromel13


Denis Caromel14


Denis Caromel15 15

2. Distributed and Parallel

Objects

ProActiveProgramming

Denis Caromel16 16

A

ProActive : Active objects

Proxy

Java Object

A ag = newActive (“A”, […], VirtualNode)V v1 = ag.foo (param);V v2 = ag.bar (param);...v1.bar(); //Wait-By-Necessity

V

Wait-By-Necessity

is a

Dataflow

Synchronization

JVM

A

JVM

Active Object

Future Object Request

Req. Queue

Thread

v1v2 ag

WBN!

Denis Caromel17 17

ProActive: Inter- to Intra- Synchronization

Sequential Multithreaded Distributed

Synchronizations, Behavior: not dependent upon

the physical location (mapping of activities)

Denis Caromel18 18

ProActive : Explicit Synchronizations

Explicit Synchronization:- ProActive.isAwaited (v); // Test if available- .waitFor (v); // Wait until availab.

Vectors of Futures:- .waitForAll (Vector); // Wait All- .waitForAny (Vector); // Get First

A ag = newActive (“A”, […], VirtualNode)V v = ag.foo(param);... v.bar(); //Wait-by-necessity

Denis Caromel19 19

ProActive : Intra-object synchronization

Explicit control: Library of service routines:

Non-blocking services,...serveOldest ();serveOldest (f);

Blocking services, timed, etc.serveOldestBl

();serveOldestTm

(ms);Waiting primitives

waitARequest(); etc.

class BoundedBuffer extends FixedBuffer implements RunActive

{ // Programming Non FIFO behavior

runActivity (ExplicitBody myBody) { while (...) { if (this.isFull()) serveOldest("get"); else if (this.isEmpty()) serveOldest ("put"); else serveOldest ();

// Non-active wait waitArequest (); }}}

Implicit (declarative) control: library classese.g. : Blocking Condition Abstraction for concurrency control:

doNotServe ("put", "isFull");

Denis Caromel20 20

ProActive: First-Class Futures

Sequential Multithreaded Distributed

Synchronizations, Behavior: not dependent upon

the physical location (mapping of activities)

Denis Caromel21 21

Wait-By-Necessity: First Class Futures

ba

Futures are Global Single-Assignment Variables

V= b.bar ()

c

c

c.gee (V)

v

v

b

Denis Caromel22 22

Standard system at Runtime:No Sharing

NoC: Network On ChipProofs of Determinism

Denis Caromel23

Calculus

ASP: Asynchronous Sequential Processes

Denis Caromel24

Proofs in GREEK

ASP Confluence and Determinacy Future updates can occur at any time, Mobility does not change behavior

Denis Caromel25 25

TYPED

ASYNCHRONOUS

GROUPS

Denis Caromel26 26

A

Creating AO and Groups

Typed Group Java or Active Object

A ag = newActiveGroup (“A”, […], VirtualNode)V v = ag.foo(param);...v.bar(); //Wait-by-necessity

V

Group, Type, and Asynchrony

are crucial for Cpt. and GRID

JVM

Denis Caromel27 27

Broadcast and Scatter

JVM

JVM

JVM

JVM

agcg

ag.bar(cg); // broadcast cgProActive.setScatterGroup(cg);ag.bar(cg); // scatter cg

c1 c2c3c1 c2c3

c1 c2c3c1 c2c3

c1 c2c3

c1 c2c3

s

c1 c2c3

s

Broadcast is the default behavior Use a group as parameter, Scattered depends on rankings

Denis Caromel28 28

Dynamic Dispatch Group

JVM

JVM

JVM

JVM

agcg

c1

c2

c3

c4

c5

c6

c7

c8c0

c9c1

c2

c3

c4

c5

c6

c7

c8c0

c9

c1

c2

c3

c4

c5

c6

c7

c8c0

c9

Slowest

Fastest

ag.bar(cg);

Denis Caromel29

ProActive Parallel Suite

Denis Caromel30

ProActive Parallel Suite

Denis Caromel31 31

3. Mobile Agents

Denis Caromel32 32

ProActive : Migration of active objects

Migration is initiated by the active object itself through a primitive: migrateTo

Can be initiated from outside through any public method

The active object migrates with:• all pending requests• all its passive objects • all its future objects

Automatic and transparent forwarding of:• requests (remote references remain valid)• replies (its previous queries will be fullfilled)

Denis Caromel33 33

Principles and optimizations

Same semantics guaranteed (RDV, FIFO order point to point, asynchronous)

Safe migration (no agent in the air!)

Local references if possible when arriving within a VM

Tensionning (removal of forwarder)

Denis Caromel34 34






Denis Caromel35 35






direct

Denis Caromel36 36






direct

direct

Denis Caromel37 37






direct

direct

forwarder

Denis Caromel38 38






direct

direct

forwarder

Denis Caromel39 39






direct

direct

forwarder

Denis Caromel40 40






direct

direct

forwarder

Denis Caromel41 41

ProActive : API for Mobile Agents

Mobile agents (active objects) that communicate

Basic primitive: migrateTo

public static void migrateTo (String u)

// string to specify the node (VM)

public static void migrateTo (Object o)

// joinning another active object

public static void migrateTo (Node n)

// ProActive node (VM)

Denis Caromel42 42

ProActive : API for Mobile AgentsMobile agents (active objects) that communicate// A simple agentclass SimpleAgent implements runActive, Serializable { public SimpleAgent () {}

public void moveTo (String t){ // Move upon requestProActive.migrateTo (t);

}

public String whereAreYou (){ // Repplies to queriesreturn (“I am at ” + InetAddress.getLocalHost ());

}

public runActivity (Body myBody){ while (… not end of itinerary …){

res = myFriend.whatDidYouFind () // Query other agents …

} myBody.fifoPolicy(); // Serves request, potentially moveTo

} }

Denis Caromel43 43

ProActive : API for Mobile AgentsMobile agents that communicate

Primitive to automatically execute action upon migrationpublic static void onArrival (String r)

// Automatically executes the routine r upon arrival

// in a new VM after migration

public static void onDeparture (String r)

// Automatically executes the routine r upon migration

// to a new VM, guaranted safe arrivalpublic static void beforeDeparture (String r)

// Automatically executes the routine r before trying a migration

// to a new VM

Denis Caromel44 44

ProActive : API for Mobile Agents Itinerary abstraction

Itinerary : VMs to visitspecification of an itinerary as a list of (site, method)

automatic migration from one to another

dynamic itinerary management (start, pause, resume, stop, modification, …)

API:myItinerary.add (“machine1’’, “routineX”); ...

itinerarySetCurrent, itineraryTravel, itineraryStop, itineraryResume, …

Still communicating, serving requests:itineraryMigrationFirst ();

// Do all migration first, then services, Default behavioritineraryRequestFirst ();

// Serving the pending requests upon arrival before migrating again

Denis Caromel45 45

Host 1 Host 2

Host 3

A

Home

Destination Methods

Host 1 echo

Host 2 callhome

Host 3 processData

Migration

Dynamic itineraries

Migration

Host 4 foo

Host 4

Migration

Migration

A A

A

A

Denis Caromel46 46

4. Localization

Denis Caromel47 47

Forwarders

Migrating object leaves forwarder on current site

Forwarder is linked to object on remote sitePossibly the mobile object

Possibly another forwarder => a forwarding chain is built

When receiving message, forwarder sends it to next hop

Upon successful communication, a tensioning takes place

Denis Caromel48 48

Other Strategy: Centralized (location Server)

S

Host A

A

Host B Host C Host D

S : SourceA : Agent

referenceServer

Denis Caromel49 49

Centralized Strategy (2)

S

Host A

Host B

A

Host C Host D

S : SourceA : Agent

reference

Migration

Server

Server Update

A migrating object updates the server

Denis Caromel50 50


S

Host A


S : SourceA : Agent

reference

Message

MigrationA

Server

UpdateFailed

A migrating object updates the server

Denis Caromel51 51


S

Host A


S : SourceA : Agent

référence

A

ServerAsk for a new

reference

Response

Message

But the AO might have moved again in the meantime … just play again.

!

The source get a new reference from the server

Request

Denis Caromel52 52

Location Server vs Forwarder

ServerNo fault tolerance if single server

Scaling is not straightforward

Added work for the mobile object

The agent can run away from messages

ForwardersUse resources even if not needed

The forwarding chain is not fault tolerant

An agent can be lost

What about performance?

Denis Caromel53 53

0

20

40

60

80

100

120

140

Forwarder (1s per site) Server (1s per site)Forwarder (1/5s per site) Server (1/5s per site)

Forwarder vs. Server LAN (100 Mb/s)

1 2 3 4 5 6 7 8 9 10 11

Response time (ms) vs. Communication rate

Server better on a LAN

Denis Caromel54 54

0

400

800

1200

Forwarder (1s per site) Server (1s per site)Forwarder (1/5s per site) Server (1/5s per site)

Forwarder vs Server MAN (7 Mb/s)Response time (ms) vs. Communication rate

1 2 3 4 5 6 7 8 9 10 11

Forwarders sometimes better on a MAN

Denis Caromel55 55

Formal Performance Evaluation of Mobile Agents:Markov Chains

Together with Fabrice Huet and Mistral Team

Objectives:Formally study the performance of Mobile Agent localization mechanismInvestigate various strategies (forwarder, server, etc.)Define adaptative strategies

Modeling:Frequency of:– Message from source– Agent migrationTime for:– Message transmission– Agent migration

Denis Caromel56 56

Modeling of Forwarder Strategy

No message

One In-transitmessage

Denis Caromel57 57

Modeling of Server Strategy

Denis Caromel58 58

TTL-TTU mixed parameterized protocol

TTL: Time To Live + Updating Forwarder:After TTL, a forwarder is subject to self destruction

Before terminating, it updates server(s) with last agent known location

TTU: Time To Update mobile AO:After TTU, AO will inform a localization server(s) of its current location

Dual TTU: first of two events:maxMigrationNb: the number of migrations without server update

maxTimeOnSite: the time already spent on the current site

5 s.

10

5 s.

Denis Caromel59 59


S

Host A

A


S : SourceA : Agent

referenceServer

Denis Caromel60 60


S

Host A

Host B

A

Host C Host D

S : SourceA : Agent

reference

Migration

Server

Server Update

FTTL

TTU

Denis Caromel61

5. IC2DInteractive Control & Debug for Distribution

Eclipse GUI

for the GRID

Denis Caromel62

GUI in ProActive Parallel Suite

Denis Caromel63

GUI in ProActive Parallel Suite

Denis Caromel64

Programmer Interface for

MonitoringDebugging Optimizing

Denis Caromel65

Denis Caromel66

IC2D

Denis Caromel67

Video 1: IC2DMonitoring, Debugging, Optimizing

Denis Caromel68

Ongoing Work: 3D View in IC2D

Denis Caromel69

6. Example of

ProActive

Applications

Denis Caromel70 70

Load Balancing

Denis Caromel71 71

Load Balancing using Mobility (1)

Denis Caromel72 72

Load Balancing using Mobility (2)

Deals with heterogeneous machines,

applications, and network.

Denis Caromel73

Sylvain Cussat-Blanc, Yves Duthen – IRIT TOULOUSESylvain Cussat-Blanc, Yves Duthen – IRIT TOULOUSE

Artificial Life GenerationArtificial Life Generation

Application J+1 J+5 J+6 J+7

Version ProActive

251300 CPUs

Initial Application

1 PC 56h52 => Crash!

ProActive Version

300 CPUs 19 minutes

Developpement of artificial creatures

Denis Caromel74

Sylvain Cussat-Blanc, Yves Duthen – IRIT TOULOUSESylvain Cussat-Blanc, Yves Duthen – IRIT TOULOUSE

Artificial Life GenerationArtificial Life Generation

Denis Caromel75

JECS : 3D Electromagnetism Radar Reflection on Planes

Denis Caromel76

Code Coupling : Vibro Acoustic (courtesy of EADS)

Denis Caromel77

P2P:AO Overlay

Network

Denis Caromel78

Denis Caromel79

Denis Caromel80

Summary

Denis Caromel81

Multi-Core to DistributedMulti-Core to Distributed

Concurrency + Mobility + Parallelism

Multi-Cores + Distribution

Denis Caromel82

Conclusion: Why does it move ?

Thanks to a few key features:

Connection-less, RMI+JMS unified

Messages rather than long-living interactions

Denis Caromel83

Conclusion: Why does it

Communicate and Behave ?Thanks to a few key features:

Because it Scales: asynchrony !

Because it is Typed: RMI with interfaces !

First-Class Futures: No unstructured Call Backs and Ports

Denis Caromel84 84

Conclusion on Mobile Active ObjectsActive Objects = a good unit of Computational MobilityWeak Migration OK (even for Load Balancing)Both Actors and Servers

Ensuring communications: several strategies to choose from: Location ServerForwardersMixed: based on TTL-TTU

Primitive + Higher-Level abstractions:migrateTo (location)onArrival, onDepartureItinerary, etc.

Application to Mobile Telecoms: DLP, NH Avatar (NHA), Shared Space for Session Management

Denis Caromel85

Summary-Perspective: Mobility at the Core

Denis Caromel86

ProActive/GCMSpecifications

for

ComponentsServicesSLAQoS

Open the way to Soft.+Serv. EU Industry with Clouds &Utilities, DAAS

ProActive/GCMSpecifications

for

ComponentsServicesSLAQoS

Open the way to Soft.+Serv. EU Industry with Clouds &Utilities, DAAS

Denis Caromel87

Denis Caromel88 88

Parallel, Distributed, Hierarchical

3. ComponentsComposing

Denis Caromel89 89

Objects to Distributed Components (1)

Typed Group Java or Active Object

ComponentIdentity Cpt = newActiveComponent (params);A a = Cpt … .getFcInterface ("interfaceName");V v = a.foo(param);

V

AExample of

component

instance

JVM

Truly

Distributed

Components

IoC:InversionOf Control(set in XML)

Denis Caromel90

GCM: Grid Component ModelGCM Being defined in the NoE CoreGRID

(42 institutions)

Open Source ObjectWeb ProActive

implements a preliminary version of GCM

Service Oriented: NESSI relation

GridCOMP takes:GCM as a first specification,

ProActive as a starting point, and

Open Source reference implementation.

The vision: The vision: GCMGCM to be the IT Service to be the IT Service GSMGSM

Denis Caromel91

GridCOMP Partners

http://www.wmin.ac.uk/

http://www.ibm.com/us/

http://www.atosorigin.com/en-us/Services/Industries/Major_Events/Olympics/

http://www.unimelb.edu.au/

http://www.uchile.cl/uchile.portal

04/20/23

Denis Caromel

GCMScopes and Objectives:

Grid Codes that Compose and Deploy

No programming, No Scripting, … No Pain

Innovation:

Abstract Deployment

Composite Components

Multicast and GatherCast

MultiCastMultiCast GatherCastGatherCast

Denis Caromel93

Pies for Analysis and Optimization

Denis Caromel94

Pies for Analysis and Optimization

Denis Caromel95

With Summary Report

Denis Caromel96

Ongoing Work: Integration with JMX JConsole:Monitoring Heap, Threads, CPU Over 60 Hours

denis caromel, et al. proactive.objectweb oasis team

Documents