Transcript
Page 1: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Building Interoperable Grid-aware ASSIST Applications via Web Services

ParCo 2005Malaga, Spain

M. AldinucciM. Danelutto, A. Paternesi,R. Ravazzolo, M. Vanneschi

ISTI - CNR, Pisa, ItalyUniversity of Pisa, Italy

Page 2: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Building Interoperable Grid-aware ASSIST Applications via Web Services

ParCo 2005Malaga, Spain

M. AldinucciM. Danelutto, A. Paternesi,R. Ravazzolo, M. Vanneschi

ISTI - CNR, Pisa, ItalyUniversity of Pisa, Italy

Page 3: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

• Motivating ...

• high-level programming for the grid• application adaptivity for the grid

• ASSIST basics

• WS as transport in ASSIST apps

• Grid.it components & WS ports

• Concluding remarks

Outline

2

Page 4: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

The grid“... coordinated resource sharing and problem

solving in dynamic, multi institutional virtual organizations.”

Foster, Anatomy

“1) coordinates resources that are not subject to centralized control …”“2) … using standard, open, general-purpose protocols and interfaces”“3) … to deliver nontrivial qualities of service.”

Foster, What is the Grid?

Page 5: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

The grid“... coordinated resource sharing and problem

solving in dynamic, multi institutional virtual organizations.”

Foster, Anatomy

“1) coordinates resources that are not subject to centralized control …”“2) … using standard, open, general-purpose protocols and interfaces”“3) … to deliver nontrivial qualities of service.”

Foster, What is the Grid?

Moreover, since this is not SeqCo, I assume applications we are focusing on should be parallel (and hopefully high-performance).

Page 6: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

// progr. & the grid

• concurrency exploitation, concurrent activities set up, mapping/scheduling, communication/synchronization handling and data allocation, ...

• manage resources heterogeneity and unreliability, networks latency and bandwidth unsteadiness, resources topology and availability changes, firewalls, private networks, reservation and jobs schedulers, ...

Page 7: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

// progr. & the grid

• concurrency exploitation, concurrent activities set up, mapping/scheduling, communication/synchronization handling and data allocation, ...

• manage resources heterogeneity and unreliability, networks latency and bandwidth unsteadiness, resources topology and availability changes, firewalls, private networks, reservation and jobs schedulers, ...

... and a non trivial QoS for applications

Page 8: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

// progr. & the grid

• concurrency exploitation, concurrent activities set up, mapping/scheduling, communication/synchronization handling and data allocation, ...

• manage resources heterogeneity and unreliability, networks latency and bandwidth unsteadiness, resources topology and availability changes, firewalls, private networks, reservation and jobs schedulers, ...

... and a non trivial QoS for applications

not easy leveraging only on middleware

Page 9: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Grid

Abstract

Machine

Application Manager

(AM: non-functional aspects & QoS control)

ASSIST applications

Abstraction of the basic services:

resource management & scheduling,

monitoring, ...

standard middleware tools

(Globus, WS, CCM, ...)

Ap

p. d

ep

en

de

nt

co

mp

iler g

en

era

ted

ASSIST idea

“moving most of the Grid specific efforts needed while

developing high-performance Grid applications from

programmers to grid tools and run-time systems”

ASSIST is a high-level programming environment for grid-aware // applications. Developed at Uni. Pisa within several national/EU projects.

First version in 2001. Open source under GPL.

Page 10: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

app = graph of modules

6

P2 P3

P4P1

input output

Page 11: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

app = graph of modules

6

P2 P3

P4P1

input output

Page 12: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

app = graph of modules

6

P2 P3

P4P1

input output

Sequential or parallel module

Typed streamsof data items

Programmable, possibly nondeterministic input behaviour

Page 13: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

native + standard

7

P2 P3

P4P1

ASSIST native or wrap (MPI, CORBA, CCM, WS)

TCP/IP, Globus,IIOP CORBA,HTTP/SOAP

Page 14: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

ASSIST native parmod

8

VP VP

VP VP

VP VP

Page 15: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

ASSIST native parmod

8

VP VP

VP VP

VP VP

An “input section” can be programmed in a CSP-like way

Page 16: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

ASSIST native parmod

8

VP VP

VP VP

VP VP

An “input section” can be programmed in a CSP-like way

Data items can be distributed (scattered,

broadcasted, multicasted) to a set of

Virtual Processes which are named accordingly to a

topology

Page 17: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

ASSIST native parmod

8

VP VP

VP VP

VP VP

An “input section” can be programmed in a CSP-like way

Data items can be distributed (scattered,

broadcasted, multicasted) to a set of

Virtual Processes which are named accordingly to a

topology

Data items partitions are elaborated by VPs, possibly in

iterative way

while(...) forall VP(in, out) barrier

data is logically shared by VPs (owner-computes)

Page 18: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

ASSIST native parmod

8

VP VP

VP VP

VP VP

An “input section” can be programmed in a CSP-like way

Data items can be distributed (scattered,

broadcasted, multicasted) to a set of

Virtual Processes which are named accordingly to a

topology

Data items partitions are elaborated by VPs, possibly in

iterative way

while(...) forall VP(in, out) barrier

data is logically shared by VPs (owner-computes)

Data is eventually gathered accordingly to

an user defined way

Page 19: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

ASSIST native parmod

8

VP VP

VP VP

VP VP

An “input section” can be programmed in a CSP-like way

Data items can be distributed (scattered,

broadcasted, multicasted) to a set of

Virtual Processes which are named accordingly to a

topology

Data items partitions are elaborated by VPs, possibly in

iterative way

while(...) forall VP(in, out) barrier

data is logically shared by VPs (owner-computes)

Data is eventually gathered accordingly to

an user defined way

Easy to express standard paradigms(skeltons), such as

farm, deal, haloswap, map, apply-to-all,

forall, ...

Page 20: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

parmod implementation

9

inputmanager

VP VP

VP manager (VPM)

VP VP

VP manager (VPM)

inputmanager

VP VP

VP manager (VPM)

processes VP Virtual Processes

Page 21: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

// matrix mul

10

send1

send2

recv

matrix_mul

A

B

C=AB

Page 22: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

// matrix mul

11

1 parmod matrix_mul (input_stream long M1[N][N], long M2[N][N]2 output_stream long M3[N][N]) {3 topology array [i:N][j:N] Pv;4 attribute long A[N][N] scatter A[*ia][*ja] onto Pv[ia][ja];5 attribute long B[N][N] scatter B[*ib][*jb] onto Pv[ib][jb];6 stream long ris;7 do input_section {8 guard1: on , , M1 && M2 {9 distribution M1[*i0][*j0] scatter to A[i0][j0];

10 distribution M2[*i1][*j1] scatter to B[i1][j1];11 } } while (true)12 virtual_processes {13 elab1 (in guard1 out ris) {14 VP i, j { f_mul (in A[i][], B[][j] output_stream ris); }}}15 output_section {16 collects ris from ALL Pv[i][j] {17 int elem; int Matrix_ris_[N][N];18 AST_FOR_EACH(elem) {19 Matrix_ris_[i][j]=elem;20 }21 assist_out(M3, Matrix_ris_);22 }<>; } }23 proc f_mul(in long A[N], long B[N] output_stream long Res)24 $c++{ register long r=0;25 for (register int k=0; k<N; ++k)26 r += A[k]*B[k];27 assist_out(Res,r); }c++$

Page 23: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

// matrix mul

11

1 parmod matrix_mul (input_stream long M1[N][N], long M2[N][N]2 output_stream long M3[N][N]) {3 topology array [i:N][j:N] Pv;4 attribute long A[N][N] scatter A[*ia][*ja] onto Pv[ia][ja];5 attribute long B[N][N] scatter B[*ib][*jb] onto Pv[ib][jb];6 stream long ris;7 do input_section {8 guard1: on , , M1 && M2 {9 distribution M1[*i0][*j0] scatter to A[i0][j0];

10 distribution M2[*i1][*j1] scatter to B[i1][j1];11 } } while (true)12 virtual_processes {13 elab1 (in guard1 out ris) {14 VP i, j { f_mul (in A[i][], B[][j] output_stream ris); }}}15 output_section {16 collects ris from ALL Pv[i][j] {17 int elem; int Matrix_ris_[N][N];18 AST_FOR_EACH(elem) {19 Matrix_ris_[i][j]=elem;20 }21 assist_out(M3, Matrix_ris_);22 }<>; } }23 proc f_mul(in long A[N], long B[N] output_stream long Res)24 $c++{ register long r=0;25 for (register int k=0; k<N; ++k)26 r += A[k]*B[k];27 assist_out(Res,r); }c++$

Page 24: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

// matrix mul

11

1 parmod matrix_mul (input_stream long M1[N][N], long M2[N][N]2 output_stream long M3[N][N]) {3 topology array [i:N][j:N] Pv;4 attribute long A[N][N] scatter A[*ia][*ja] onto Pv[ia][ja];5 attribute long B[N][N] scatter B[*ib][*jb] onto Pv[ib][jb];6 stream long ris;7 do input_section {8 guard1: on , , M1 && M2 {9 distribution M1[*i0][*j0] scatter to A[i0][j0];

10 distribution M2[*i1][*j1] scatter to B[i1][j1];11 } } while (true)12 virtual_processes {13 elab1 (in guard1 out ris) {14 VP i, j { f_mul (in A[i][], B[][j] output_stream ris); }}}15 output_section {16 collects ris from ALL Pv[i][j] {17 int elem; int Matrix_ris_[N][N];18 AST_FOR_EACH(elem) {19 Matrix_ris_[i][j]=elem;20 }21 assist_out(M3, Matrix_ris_);22 }<>; } }23 proc f_mul(in long A[N], long B[N] output_stream long Res)24 $c++{ register long r=0;25 for (register int k=0; k<N; ++k)26 r += A[k]*B[k];27 assist_out(Res,r); }c++$

Page 25: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

// matrix mul

11

1 parmod matrix_mul (input_stream long M1[N][N], long M2[N][N]2 output_stream long M3[N][N]) {3 topology array [i:N][j:N] Pv;4 attribute long A[N][N] scatter A[*ia][*ja] onto Pv[ia][ja];5 attribute long B[N][N] scatter B[*ib][*jb] onto Pv[ib][jb];6 stream long ris;7 do input_section {8 guard1: on , , M1 && M2 {9 distribution M1[*i0][*j0] scatter to A[i0][j0];

10 distribution M2[*i1][*j1] scatter to B[i1][j1];11 } } while (true)12 virtual_processes {13 elab1 (in guard1 out ris) {14 VP i, j { f_mul (in A[i][], B[][j] output_stream ris); }}}15 output_section {16 collects ris from ALL Pv[i][j] {17 int elem; int Matrix_ris_[N][N];18 AST_FOR_EACH(elem) {19 Matrix_ris_[i][j]=elem;20 }21 assist_out(M3, Matrix_ris_);22 }<>; } }23 proc f_mul(in long A[N], long B[N] output_stream long Res)24 $c++{ register long r=0;25 for (register int k=0; k<N; ++k)26 r += A[k]*B[k];27 assist_out(Res,r); }c++$

Page 26: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

// matrix mul

11

1 parmod matrix_mul (input_stream long M1[N][N], long M2[N][N]2 output_stream long M3[N][N]) {3 topology array [i:N][j:N] Pv;4 attribute long A[N][N] scatter A[*ia][*ja] onto Pv[ia][ja];5 attribute long B[N][N] scatter B[*ib][*jb] onto Pv[ib][jb];6 stream long ris;7 do input_section {8 guard1: on , , M1 && M2 {9 distribution M1[*i0][*j0] scatter to A[i0][j0];

10 distribution M2[*i1][*j1] scatter to B[i1][j1];11 } } while (true)12 virtual_processes {13 elab1 (in guard1 out ris) {14 VP i, j { f_mul (in A[i][], B[][j] output_stream ris); }}}15 output_section {16 collects ris from ALL Pv[i][j] {17 int elem; int Matrix_ris_[N][N];18 AST_FOR_EACH(elem) {19 Matrix_ris_[i][j]=elem;20 }21 assist_out(M3, Matrix_ris_);22 }<>; } }23 proc f_mul(in long A[N], long B[N] output_stream long Res)24 $c++{ register long r=0;25 for (register int k=0; k<N; ++k)26 r += A[k]*B[k];27 assist_out(Res,r); }c++$

Page 27: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

// matrix mul

11

1 parmod matrix_mul (input_stream long M1[N][N], long M2[N][N]2 output_stream long M3[N][N]) {3 topology array [i:N][j:N] Pv;4 attribute long A[N][N] scatter A[*ia][*ja] onto Pv[ia][ja];5 attribute long B[N][N] scatter B[*ib][*jb] onto Pv[ib][jb];6 stream long ris;7 do input_section {8 guard1: on , , M1 && M2 {9 distribution M1[*i0][*j0] scatter to A[i0][j0];

10 distribution M2[*i1][*j1] scatter to B[i1][j1];11 } } while (true)12 virtual_processes {13 elab1 (in guard1 out ris) {14 VP i, j { f_mul (in A[i][], B[][j] output_stream ris); }}}15 output_section {16 collects ris from ALL Pv[i][j] {17 int elem; int Matrix_ris_[N][N];18 AST_FOR_EACH(elem) {19 Matrix_ris_[i][j]=elem;20 }21 assist_out(M3, Matrix_ris_);22 }<>; } }23 proc f_mul(in long A[N], long B[N] output_stream long Res)24 $c++{ register long r=0;25 for (register int k=0; k<N; ++k)26 r += A[k]*B[k];27 assist_out(Res,r); }c++$

Page 28: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

// matrix mul

11

1 parmod matrix_mul (input_stream long M1[N][N], long M2[N][N]2 output_stream long M3[N][N]) {3 topology array [i:N][j:N] Pv;4 attribute long A[N][N] scatter A[*ia][*ja] onto Pv[ia][ja];5 attribute long B[N][N] scatter B[*ib][*jb] onto Pv[ib][jb];6 stream long ris;7 do input_section {8 guard1: on , , M1 && M2 {9 distribution M1[*i0][*j0] scatter to A[i0][j0];

10 distribution M2[*i1][*j1] scatter to B[i1][j1];11 } } while (true)12 virtual_processes {13 elab1 (in guard1 out ris) {14 VP i, j { f_mul (in A[i][], B[][j] output_stream ris); }}}15 output_section {16 collects ris from ALL Pv[i][j] {17 int elem; int Matrix_ris_[N][N];18 AST_FOR_EACH(elem) {19 Matrix_ris_[i][j]=elem;20 }21 assist_out(M3, Matrix_ris_);22 }<>; } }23 proc f_mul(in long A[N], long B[N] output_stream long Res)24 $c++{ register long r=0;25 for (register int k=0; k<N; ++k)26 r += A[k]*B[k];27 assist_out(Res,r); }c++$

Page 29: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

12

Compiling & Running

ASSISTcompiler

Page 30: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

12

Compiling & Running

QoScontract

ASSISTprogram

ASSISTcompiler

resourcedescription

XML

executablecode

(linux, mac,uindoz)

Page 31: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

12

Compiling & Running

QoScontract

ASSISTprogram

ASSISTcompiler

resourcedescription

XML

executablecode

(linux, mac,uindoz)

launch

query new resources

reco

nf

com

man

ds

Managers

AM+MAMs

Grid execution

agent (GEA)

ISM OSM

VPM

seqseq

Network of processes

Run

Page 32: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

WS as transport

• ASSIST can use WS as transport for streams

• provide interoperability with standards

• automatically generated• helps in dealing with firewalls

• on the whole, an ASSIST app with WS can be considered a composite service with distributed orchestration

13

Page 33: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

BPEL distributed orchestration

14

Nanda et al. (IBM)Decentralizing Execution

of Composite Web ServicesOOPSLA 2004

Page 34: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

BPEL distributed orchestration

14

Nanda et al. (IBM)Decentralizing Execution

of Composite Web ServicesOOPSLA 2004

Page 35: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

// matrix mul

15

send1

send2

recv

matrix_mul

A

B

C=AB

Page 36: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

WS composition

16

1 <ComponentConfiguration>2 <Assembly>3 <ComponentSection>4 <Component name="send1" com="ws" kind="xml" file="./xmls/send1.xml"> </Component>5 <Component name="send2" com="ws" kind="xml" file="./xmls/send2.xml"> </Component>6 <Component name="matrix_mul" com="ws" kind="xml" file="./xmls/matrix_mul.xml"> </Component>7 <Component name="recv" com="ws" kind="xml" file="./xmls/rec.xml"> </Component>8 </ComponentSection>9 <ConnectionSection>

10 <Connection>11 <Output component="send1" interface="Matrix1"/>12 <Input component="matrix_mul" interface="Matrix1"/>13 </Connection>14 <Connection>15 <Output component="send2" interface="Matrix2"/>16 <Input component="matrix_mul" interface="Matrix2"/>17 </Connection>18 <Connection>19 <Output component="matrix_mul" interface="Matrix_ris"/>20 <Input component="recv" interface="Matrix"/>21 </Connection>22 </ConnectionSection>23 </Assembly>24 </ComponentConfiguration>

Page 37: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

WS composition

16

1 <ComponentConfiguration>2 <Assembly>3 <ComponentSection>4 <Component name="send1" com="ws" kind="xml" file="./xmls/send1.xml"> </Component>5 <Component name="send2" com="ws" kind="xml" file="./xmls/send2.xml"> </Component>6 <Component name="matrix_mul" com="ws" kind="xml" file="./xmls/matrix_mul.xml"> </Component>7 <Component name="recv" com="ws" kind="xml" file="./xmls/rec.xml"> </Component>8 </ComponentSection>9 <ConnectionSection>

10 <Connection>11 <Output component="send1" interface="Matrix1"/>12 <Input component="matrix_mul" interface="Matrix1"/>13 </Connection>14 <Connection>15 <Output component="send2" interface="Matrix2"/>16 <Input component="matrix_mul" interface="Matrix2"/>17 </Connection>18 <Connection>19 <Output component="matrix_mul" interface="Matrix_ris"/>20 <Input component="recv" interface="Matrix"/>21 </Connection>22 </ConnectionSection>23 </Assembly>24 </ComponentConfiguration>

Page 38: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

WS composition

16

1 <ComponentConfiguration>2 <Assembly>3 <ComponentSection>4 <Component name="send1" com="ws" kind="xml" file="./xmls/send1.xml"> </Component>5 <Component name="send2" com="ws" kind="xml" file="./xmls/send2.xml"> </Component>6 <Component name="matrix_mul" com="ws" kind="xml" file="./xmls/matrix_mul.xml"> </Component>7 <Component name="recv" com="ws" kind="xml" file="./xmls/rec.xml"> </Component>8 </ComponentSection>9 <ConnectionSection>

10 <Connection>11 <Output component="send1" interface="Matrix1"/>12 <Input component="matrix_mul" interface="Matrix1"/>13 </Connection>14 <Connection>15 <Output component="send2" interface="Matrix2"/>16 <Input component="matrix_mul" interface="Matrix2"/>17 </Connection>18 <Connection>19 <Output component="matrix_mul" interface="Matrix_ris"/>20 <Input component="recv" interface="Matrix"/>21 </Connection>22 </ConnectionSection>23 </Assembly>24 </ComponentConfiguration>

Page 39: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

WS composition

16

1 <ComponentConfiguration>2 <Assembly>3 <ComponentSection>4 <Component name="send1" com="ws" kind="xml" file="./xmls/send1.xml"> </Component>5 <Component name="send2" com="ws" kind="xml" file="./xmls/send2.xml"> </Component>6 <Component name="matrix_mul" com="ws" kind="xml" file="./xmls/matrix_mul.xml"> </Component>7 <Component name="recv" com="ws" kind="xml" file="./xmls/rec.xml"> </Component>8 </ComponentSection>9 <ConnectionSection>

10 <Connection>11 <Output component="send1" interface="Matrix1"/>12 <Input component="matrix_mul" interface="Matrix1"/>13 </Connection>14 <Connection>15 <Output component="send2" interface="Matrix2"/>16 <Input component="matrix_mul" interface="Matrix2"/>17 </Connection>18 <Connection>19 <Output component="matrix_mul" interface="Matrix_ris"/>20 <Input component="recv" interface="Matrix"/>21 </Connection>22 </ConnectionSection>23 </Assembly>24 </ComponentConfiguration>

Page 40: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Generating WS

17

matrix_mul

Page 41: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Generating WS

17

matrix_mul

WS

exposesend_a(A)

Page 42: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Generating WS

17

matrix_mul

WS

exposesend_a(A)

WS

exposesend_b(B)

Page 43: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Generating WS

17

matrix_mul

WS

exposesend_a(A)

WS

exposesend_b(B) stub

client callinginvoke(send_x(X)),where X is decided

at wiring time

generated at wiring timestarting from the WSDL of X

Page 44: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Generating WS

17

matrix_mul

WS

exposesend_a(A)

WS

exposesend_b(B) stub

client callinginvoke(send_x(X)),where X is decided

at wiring time

generated at wiring timestarting from the WSDL of X

expose a “config” methodto overwrite “address” field

of the WSDL

Page 45: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Generating WS

17

matrix_mul

WS

exposesend_a(A)

WS

exposesend_b(B) stub

client callinginvoke(send_x(X)),where X is decided

at wiring time

generated at wiring timestarting from the WSDL of X

expose a “config” methodto overwrite “address” field

of the WSDL

WS are implementedby means of gSOAPembedded server

Page 46: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

A simple farm with WS

18

W1 W2 Wn...

Distr

Coll

Page 47: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

A simple farm with WS

18

W1 W2 Wn...

Distr

Coll

N. of Worker Modules (n)

Tim

e (

ms)

WS (gSOAP)

raw TCP/IP

ideal

Page 48: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

A simple farm with WS

18

W1 W2 Wn...

Distr

Coll

N. of Worker Modules (n)

Tim

e (

ms)

WS (gSOAP)

raw TCP/IP

ideal30-50%communication

slowdown

Page 49: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Composite WS & ASSIST

• Differently from BPEL, we start from the graph

• not a big deal, however not fully compliant with pragmatic approach to WS

• we also would like to provide full interoperability with RPC WS

19

Page 50: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

ASSIST & components• moving towards component approach

• an ASSIST module of a graph of them may be defined as Grid.it component

• stream ports (use/provide)• RPC ports (use/provide)

• component wiring through ASSIST native, HTTP/SOAP, IIOP/CORBA (for CCM components)

20

Page 51: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

By the way ...

21

Page 52: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

By the way ...

• is a WS a component?

• following the Szyperski’s definition

21

Page 53: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

By the way ...

• is a WS a component?

• following the Szyperski’s definition

• Dennis Gannon said yes!

• Europar 2004 invited talk• we also believe that

21

Page 54: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

22

!

"#$%&$'()(*$&)+,,%$-+.$/'0

+'&),/*.+%01

!"#!$$%&'()#*'+,-#&"#.&/$&","0+1

2,3#4,%56(,+#'"-#2&%789&:#;&&9+<

&2)(344546)"2),7378

,92&2)0:;<84:=1)02)>?@=94346)%2)A34B6)(2>34<3=C3DE6)F2)0@DD9346

+2 075D@4=G@

$4<@343)#4@H8?=@:E!!

""#$%&'(&)*)+,$-,$.*/$0*123%*,

4 #$)-+51-6$*7+*),3&)$&8$+9*$'&:*6;

< =-%9$>1&23:*,$(&1+$%-)$/*$-$%&'(6*+*$?*/,*123%*

< @,*,$(&1+,$/*%&'*$?*/$,*123%*$A%63*)+$,+5/,BC

4 #$"&))*%+3&)$3,$+9*)$-$/3):3)D$/*+?**)$-%63*)+$,+5/$-):$+9*$.0EF$8&1$+9*$,&'*(1&23:*,$(&1+C

4 G"#HI$3'(6*'*)+,$+93,$8*-+51*C

< @,*,$(J+9&)$-,$+9*$,%13(+3)D$6-)D5-D*C

4 #6,&$/-,*:$&)$+9*$KL0M$,+-):-1:C

!"

#$%&'$()*+$,&-,&../&+01203$34,

5 .-3&6$&7,$&48$&900:;$&6$%&,$()*+$&-,&-&../

+01203$34<

5 =$,,-:$&0(*$34$>&-3>&304&?.@&%-,$>A

B '$3>&-&1$,,-:$&40&48$&,$()*+$

5 C07&1-D&:$4&-&($,203,$&0(&D07&1-D&304A

B E$2$3>,&7203&48$&,$()*+$&,$1-34*+,A

5 F0&+03+$24&0G&H7,$,&20(4IA

B J06$)$(&,01$&,$()$,&:$3$(-4$&1$,,-:$,&*3&($,203,$

40&1$,,-:$,&,$34A

B #$%&,$()*+$&->>($,,*3:&-;;06,&-&($2;D&40&%$&G0(6-(>$>

40&-&H"(>&2-(4DI&($+$*)$(A

!

"#$%&'(&)*+,$-$#$+,(.#/0

1 2.+%$3&-$#+4$(($#+(56)+#$637(&/$+8#$%&'(&.)09 :6(6+/&)&)*+.;+3&-$+&)0(#</$)(+0(#$6/0+6)%+5&0(.#&'630(.#/+/$(6%6(6

9 =$><&0&(&.)+36#*$+'./8<(6(&.)63+#$0.<#'$0+.)+%$/6)%(.+0(6#(+6+36#*$+)</4$#+.;+0&/<36(&.)01 ?&)$+0&/<36(&.)+.<(8<(0+(.+0$$+@5&'5+(#6'A+#$63+0(.#/+$-.3<(&.)B

1 =$;&)$+0'$)6#&.0+(56(+/6('5+&)'./&)*+%6(6B

9 ?6C+D$$%+(.+#$><&0&(&.)+46)%@&%(5+(.+/6A$+(5$+)$$%$%%6(6+6)63C0&0+8.00&43$B

9 ?6C+#$><&#$+#$637(&/$+#$763&*)/$)(+.;+&)0(#</$)(0B

9 E.#A;3.@0+/6C+#<)+;.#+6+3.)*+(&/$+6)%+(5$C+/<0(+4$6%68(&-$+6)%+-$#C+%C)6/&'

Page 55: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

M2

M1

M3

legacyWS

legacy CCM component

HTTP/SOAP

Corba/IIOPHTTP/SOAP

(or ASSIST native)

client WS

ASSIST module (seq)

ASSIST module (parmod)

client WSASSIST streams

AC2

AC1

GRID.it components

Components: the big picture

Page 56: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Components: the big picture

M2

M1

M3

AC2

AC1

!

A method invocation arrives

to RPC provide port of AC2!

Page 57: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

M2

M1

M3

AC2

AC1

!"

#

"

Gateway

Components: the big picture

A method invocation arrives

to RPC provide port of AC2!

Parameters are injected in a input streamtogether with an unique id

Computation is performed

As soon as a matching id arrive to the same port a reply message is prepared

!

"

#

Page 58: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

M2

M1

M3

AC2

AC1

!

Components: the big picture

A method invocation arrives

to RPC provide port of AC2!

Parameters are injected in a input streamtogether with an unique id

Computation is performed

As soon as a matching id arrive to the same port a reply message is prepared

!

"

#

The method invocation is finalizedwith a result message

!

Page 59: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Conclusions• WS extension is not rocket science, but being

compliant to standards may make the difference for real applications

• ASSIST provide high-level programming for grid• dynamic adaptivity, autonomic QoS control, fault-tolerance

(ongoing), ... and this is rocket science

• interoperability with WS & CCM• transparent to the programmer, automatic generation of the

needed adaptors

• distributed orchestration of workflows

• the run-time exploits standard middleware (POSIX/TCP, Globus, ...), it provide to the programmer higher-level view of it

Page 60: Building Interoperable Grid-aware ASSIST Applications via ...calvados.di.unipi.it/storage/talks/2005_WS_parco_talk.pdfBuilding Interoperable Grid-aware ASSIST Applications ... R. Ravazzolo,

Thank you

26


Top Related