Distributed ComputingDistributed Computing

Software based solutions to Software based solutions to Parallel ComputingParallel Computing

Distributed ComputingDistributed Computing

Cluster based computingCluster based computing Network of WorkstationsNetwork of Workstations

• Duality of standalone workstation Vs. Duality of standalone workstation Vs. parallel computing environment.parallel computing environment.

Distributed ApplicationsDistributed Applications• RC5 ClientsRC5 Clients• SETI ClientsSETI Clients

Issues of Distributed Issues of Distributed ComputingComputing

Homogeneous networksHomogeneous networks• Networks of similar machines. Requires only Networks of similar machines. Requires only

one version of executables. one version of executables. Heterogeneous networksHeterogeneous networks

• Different Architectures of machines. Different Architectures of machines. (Alpha,x86]. Each architecture requires its (Alpha,x86]. Each architecture requires its own version of executables.own version of executables.

Issues of data representation, system Issues of data representation, system level interface differences, and suitability.level interface differences, and suitability.

Support ModelSupport Model

Application support is critcal.Application support is critcal. Three basic modelsThree basic models

• Application is built with full awareness Application is built with full awareness and supportand support

• User land facilities are provided to User land facilities are provided to facilitate awareness and supportfacilitate awareness and support

• Kernel level supportKernel level support

Application SupportApplication Support

Application modeled as self-contained Application modeled as self-contained entity capable of client-server, peer-entity capable of client-server, peer-peer cooperation models.peer cooperation models.• RC5 clientRC5 client• SETI clientSETI client

Support can be streamlinedSupport can be streamlined Loss of transparency, load Loss of transparency, load

balancing,lower QoS.balancing,lower QoS.

User Level SupportUser Level Support

Provide basic parallel mechanismsProvide basic parallel mechanisms• Message/Data passingMessage/Data passing• SynchronizationSynchronization• Process HandlingProcess Handling

– Rough controlRough control

Provides greater transparencyProvides greater transparency• Less dependence on architecture, Less dependence on architecture,

transparency supports heterogeneous transparency supports heterogeneous solutionssolutions

Kernel SupportKernel Support

Provides system level supportProvides system level support• SchedulingScheduling

– Load BalancingLoad Balancing– Process MigrationProcess Migration

• Resource AllocationResource Allocation Doesn’t work well in heterogeneous Doesn’t work well in heterogeneous

configurations.configurations. Process Migration increases Process Migration increases

communication costscommunication costs

Three SolutionsThree Solutions

PVM - Parallel Virtual MachinePVM - Parallel Virtual Machine• User Land Parallel SolutionUser Land Parallel Solution• Message PassingMessage Passing

MPI - Message Passing InterfaceMPI - Message Passing Interface• StandardStandard• Several ImplementationsSeveral Implementations• User Land SupportUser Land Support

Three Solutions - Three Solutions - continuedcontinued

MOSIX - Linux Kernel ExtensionsMOSIX - Linux Kernel Extensions• Provide dynamic schedulingProvide dynamic scheduling• process migrationprocess migration• system level integrationsystem level integration

PVM - Parallel Virtual PVM - Parallel Virtual MachineMachine

Message Passing ParadigmMessage Passing Paradigm Heterogeneous ComputingHeterogeneous Computing

• Deals effectively with data Deals effectively with data representation issuesrepresentation issues

Supports variety of systems including Supports variety of systems including MPP’s, SMP’s, and vector machinesMPP’s, SMP’s, and vector machines

Uses a daemon to provide parallel Uses a daemon to provide parallel facilitiesfacilities

PVM - continuedPVM - continued

Tracks tasks using a unique Tracks tasks using a unique system assigned ID called a TIDsystem assigned ID called a TID

Supports grouping, and group level Supports grouping, and group level activities.activities.

MPI or LAMMPI or LAM

Message Passing InterfaceMessage Passing Interface MPI is the standard, several MPI is the standard, several

implementations exist. One is LAM implementations exist. One is LAM and is maintained by Notre Dame.and is maintained by Notre Dame.

Similar to PVMSimilar to PVM• HeterogeneousHeterogeneous• Uses a daemon, also includes a peer-Uses a daemon, also includes a peer-

peer mode.peer mode.

MPI - LAM - continuedMPI - LAM - continued

LAM environment must be started LAM environment must be started explicitlyexplicitly

Provides compiler shells to handle Provides compiler shells to handle program compilationprogram compilation

Nodes are dynamicNodes are dynamic Suite of utilities to maintain the Suite of utilities to maintain the

message passing virtual machine.message passing virtual machine. The parallelism is explicitly programmedThe parallelism is explicitly programmed

MPI - LAM - continuedMPI - LAM - continued

MPI addresses resource limitations. MPI addresses resource limitations. Uses a property called Guaranteed Uses a property called Guaranteed Envelope Resources to maintain Envelope Resources to maintain integrity of processes.integrity of processes.

MOSIX and Linux 2.2.7MOSIX and Linux 2.2.7

Linux, not much more to say.Linux, not much more to say. Mosix is developed to extend Mosix is developed to extend

several unix operating systemsseveral unix operating systems HomogeneousHomogeneous Transparent and Preemptive Transparent and Preemptive

process migrationprocess migration Dynamic process reassignmentDynamic process reassignment

Mosix - continuedMosix - continued

Global resource assignmentGlobal resource assignment Ideal for cluster based computingIdeal for cluster based computing Offers memory usheringOffers memory ushering

• prevents thrashingprevents thrashing Maintains home systemMaintains home system Requires that the nodes of the Requires that the nodes of the

cluster be networked wellcluster be networked well

Mosix - ContinuedMosix - Continued

Mosix maintains a given process’ Mosix maintains a given process’ association with a home node.association with a home node.• Unique home nodeUnique home node

Implements a bi-level approach to Implements a bi-level approach to maintain this association through maintain this association through migrationmigration• User Context (remote) - system User Context (remote) - system

independent contextsindependent contexts

Mosix - continuedMosix - continued

• Deputy - node dependent interface. Remains Deputy - node dependent interface. Remains when process migrates from home node.when process migrates from home node.

All system dependent calls are routed All system dependent calls are routed through remote to deputy at home through remote to deputy at home systemsystem• gettimeofdaygettimeofday

Initial assignment needs to be done by Initial assignment needs to be done by PVM or MPI. Mosix doesn’t handle this PVM or MPI. Mosix doesn’t handle this aspect.aspect.

MOSIX - Migrated ProcessMOSIX - Migrated Process

Bad MosixBad Mosix

Overhead is increasedOverhead is increased• Delayed system callsDelayed system calls• I/O sockets (file, network) don’t route I/O sockets (file, network) don’t route

wellwell• Migration time adds overheadMigration time adds overhead

Needs further work in migratable Needs further work in migratable sockets and filessockets and files

You’re a good audienceYou’re a good audience

Are you asleep yet.Are you asleep yet. But I tried!But I tried!

Thank you,Thank you, Any Questions?Any Questions?