Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

1

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

EPOS and Myrinet: Effective Communication Support for

Parallel Applications Running on Clusters of Workstations

Antônio Augusto Fröhlich

Gilles Pokam Tientcheu

Wolfgang Schöder-Preikschat

mailto:guto@first.gmd.de

http://www.first.gmd.de/~guto

May 2000

2

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Outline

Introduction Application-orientation systems Component-based systems

EPOS Design Framework Generation Tools

Conclusion

3

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

high performanceX

all purpose, global, generic

Introduction

There is no best “general” solution Each (class of) application has particular demands

4

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Applicaiton-orientation

Each application deserves its own run-time support system

Set of applications that will run on top of the operating system and their requirements Isn’t known until run-time => dynamic adaptation Can be determined before run-time => static

configurationParallel and embedded applications

Adaptability and configurability Component-based?

5

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Component-orientation

componentgranularity

many simple components customizability

reusabilityperformance“a queue link”

“a file system”usability

maintainabilitya few complex components

6

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gyThe Gap to Applications

(Motivation for EPOS)

componentgranularity

many simple componentsWhat is this component for?

What is the difference to that other one?How do I use it?

Do I have to take this junk as well?The component I need is not there!a few complex components

7

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gyEPOS: Embedded Parallel

Operating System EPOS operating system

Application-driven assemblage of system components

EPOS components Statically configurable, application ready system

abstractions EPOS and the real world

Intelligent visual tools for configuration Invisibility

Standard libraries (Posix files, libc, libstdc++, libm)Standard APIs (Posix threads, MPI)

8

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

System Abstraction

system micro-components

EPOS System Abstractions

Application ready components

Independent from execution scenario

Examples: A thread on a given

scheduling policy NOT a thread for one

processor, or for multi-tasking

9

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Scenario Adapters

EPOS Scenario Adapters

Adapt existing system abstractions for a given execution scenario

Adapt system micro-components to grant the semantics dictated by a given execution scenario

Examples: An SMP adapter A secure remote

invocation adapter

10

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

task

memory segment

thread

file

synchronizer

communicator

Inflated Interfaces

EPOS Inflated Interfaces

Export system abstractions to applications

Well-known to application programmers

Comprehensive Promote requirement

analysis Examples:

thread communicator synchronizer

Not a fat interface if intersection => subset

11

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

realization

<<interface>>

partially realize selectively realize

key

realization 1

<<interface>>

realization 3realization 2

1

2

3

Partial and Selective Realization Relationships

System configuration is basically restricted to the setting of selective realize keys

12

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Wrapped No allocation or sharing

control No cross-domain

invocation Always embedded in

the application

Controlled Allocation and sharing

control is possible Cross-domain invocation

is possible Embedded in the

application or packed in a kernel

EPOS Framework

Static metaprogrammed Metaprograms run at compile-time (Almost) no run-time overhead Implemented as C++ templates

System abstraction implementation

13

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

inflated interface

scenario adapter<<select>>

wrapper realization<<select>>

Application

EPOS FrameworkWrapped System Abstraction

14

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

basic stub

message exchange stubprocedure call stubInflated interface

<<select>>stub

<<select>>server

<<select>>realization

basic server

message exchange serverprocedure call server

handle

<<select>>id

<<select>>message

<<select>>id

state

Application

System

EPOS FrameworkControlled System Abstraction

<select>>scenario adapter

15

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Automatic Generation

Application refers to inflated interfaces Requirement analyzer parses the application

searching for inflated interfaces references Which interfaces are referred to? How they are referred to?

Expert system select the realizations that better match the referred inflated interfaces

An application-oriented operating system is compiled

16

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

syntacticanalyzer

tasks

segments

threadsmailboxes

mutexfiles

complex application program

code = new Segment(buffer, size);task = new Task(code, data);thread = new Thread(task, &entry_func, priority, SUSPENDED, arguments);mutex->entry();Mailbox mailbox;mailbox >> message; File file(name);file << mailbox;

Requirement Analysis

17

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Requirement Analysis

flat memorylinks

syntacticanalyzer

real application program (MPI)

Channel link(destination);link << mesage;

18

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Context Information

We need context informationnew Thread(task, &func) => location?mailbox << message => protocol? Absence of Task => single-task?

Buildup databases Scenario dependencies Target machine description

19

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Execution Scenario

execution scenario

+ =tasks

segments

threadsmailboxes

monitorsfiles

required Inflated interfaces

20

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Application-oriented EPOS

application

interfaces

scenario adapters

system micro-components system abstractions

21

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Conclusions

EPOS doesn’t grant the tailored OS to be optimal When several realizations fulfill the requirements,

the selection is arbitrary Profiling could help

EPOS framework overhead is close to zero Tools are being implemented System abstraction repository is growing First real applications are being negotiate

22

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Titel:send.epsErstellt von:fig2dev Version 3.2 Patchlevel 1Vorschau:Diese EPS-Grafik wurde nicht gespeichertmit einer enthaltenen Vorschau.Kommentar:Diese EPS-Grafik wird an einenPostScript-Drucker gedruckt, aber nichtan andere Druckertypen.

23

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Titel:bandwidth.epsErstellt von:gnuplot 3.7 patchlevel 0.2Vorschau:Diese EPS-Grafik wurde nicht gespeichertmit einer enthaltenen Vorschau.Kommentar:Diese EPS-Grafik wird an einenPostScript-Drucker gedruckt, aber nichtan andere Druckertypen.

24

Ger

man

Nat

iona

l Res

earc

h C

ente

r for

Info

rmat

ion

Tech

nolo

gyR

esea

rch

Inst

itute

for C

ompu

ter A

rchi

tect

ure

and

Softw

are

Tech

nolo

gy

Titel:latency.epsErstellt von:gnuplot 3.7 patchlevel 0.2Vorschau:Diese EPS-Grafik wurde nicht gespeichertmit einer enthaltenen Vorschau.Kommentar:Diese EPS-Grafik wird an einenPostScript-Drucker gedruckt, aber nichtan andere Druckertypen.