parallel and distributed systems
DESCRIPTION
Parallel and Distributed Systems. Peter Brezany Institu te for Software Science Universit y of Vienna. Typical One-Processor Architecture (SISD Architecture). SISD : Single Instruction stream Single Data stream. Array Processor (SIMD Architecture). SIMD: Single Instruction - PowerPoint PPT PresentationTRANSCRIPT
Institute for Software Science – University of Vienna
P.Brezany
Parallel and Distributed Systems
Peter Brezany
Institute for Software Science
University of Vienna
Institute for Software Science – University of Vienna
P.Brezany2
Typical One-Processor Architecture (SISD
Architecture)
SISD : Single Instruction stream Single Data stream
Institute for Software Science – University of Vienna
P.Brezany3
Array Processor (SIMD Architecture)
SIMD: Single Instructionstream Multiple Data streams
Institute for Software Science – University of Vienna
P.Brezany4
Loop Parallelizing for SIMDSsA typical scientific program spends approx. 90% of its execution
timein loops.Example in Java:
float A[1000], B[1000]; for (int i = 1; i < 1000; i++) { A[i-1] = B[i]; }
The above loop can be expressed in Fortran 95 in the following way:
A(0:998) = B(1:999)
This statement can be directly mapped onto a SIMD processor.
There is an initiative to extend Java by similar constructs.
Institute for Software Science – University of Vienna
P.Brezany5
Parallel Multi-Processor-Hardware
(MIMD Architectures)• Distributed-memory machines (DM Multiprocessors, DM MIMDS)– Each processor has local memory and disk– Communication via message-passing– Hard to program: explicit data distribution– Goal: minimize communication
• Shared-memory machines (SM Multiprocessors, SM MIMDs, SMPs)– Shared global address space and disk– Communication via shared memory variables– Ease of programming– Goal: maximize locality, minimize false sharing
• Current trend: Cluster of SMPs
Institute for Software Science – University of Vienna
P.Brezany6
Distributed Memory Architecture
(Shared Nothing)
LocalMemory
LocalMemory
LocalMemory
LocalMemory
CPU CPU
Interconnection Network
CPUCPU
Institute for Software Science – University of Vienna
P.Brezany7
DMM: Shared Disk Architecture
LocalMemory
LocalMemory
LocalMemory
LocalMemory
CPU CPU CPU CPU
Interconnection Network
Global Shared Disk Subsystem
Institute for Software Science – University of Vienna
P.Brezany8
Loop Parallelizing for DM MIMDs
Example in Java:
float A[10], B[10];
for (int i = 1; i < 10; i++) { A[i] = B[i-1]; }
For two processors, P1 and P2, a straightforward solution would be:
Institute for Software Science – University of Vienna
P.Brezany9
Loop Parallelizing for DM MIMDs (2)
Institute for Software Science – University of Vienna
P.Brezany10
Loop Parallelizing for DM MIMDs (3)Code on P1: float A[5], B[5]; float temp; for (int i = 1; i < 6; i++) { if ( i == 5 ) { receive message from P2 into temp; A[i-1] = temp; { else A[i-1] = B[i]; } Code on P2: float A[5], B[5]; float temp; for (int i = 0; i < 5; i++) { if ( i == 0 ) { temp = B[0]; send temp to P1; { else A[i-1] = B[i]; }
Institute for Software Science – University of Vienna
P.Brezany11
Shared Memory Architecture(Shared Everything, SMP)
CPU CPU
Interconnection Network
CPUCPU
Global Shared Memory
Institute for Software Science – University of Vienna
P.Brezany12
Loop Parallelizing for SMPsExample in Java:
float A[1000], B[1000]; for (int i = 1; i < 1000; i++) { A[i-1] = B[i]; }
If we have, e.g. two processors, P1 and P2, a straightforward
(non-optimal) solution would be:
Code on P1: for (int i = 1; i < 500; i++) { A[i-1] = B[i]; } Code on P2: for (int i = 500; i < 1000; i++) { A[i-1] = B[i]; }Data elements of A and B are stored in the shared memory.
Institute for Software Science – University of Vienna
P.Brezany13
Cluster of SMPs
CPU
Interconnection Network
CPU
CPU CPU
4-CPUSMP
CPU CPU
CPU CPU
4-CPUSMP
CPU CPU
CPU CPU
4-CPUSMP
CPU CPU
CPU CPU
4-CPUSMP
Institute for Software Science – University of Vienna
P.Brezany14
Institute for Software Science – University of Vienna
P.Brezany15
Abstract Maschine Model
Institute for Software Science – University of Vienna
P.Brezany16
Cluster von PCs
Institute for Software Science – University of Vienna
P.Brezany17
Institute for Software Science – University of Vienna
P.Brezany18
Pipeline
No Pipeline
Institute for Software Science – University of Vienna
P.Brezany19
Towards Parallel Databases
Institute for Software Science – University of Vienna
P.Brezany20
Relational Data Model
Example
Cities Name Population LandMunich 1211617 BayernBremen 535058 Bremen. . . . . . . . .
Schema: (Cities: STRING, Population: INTEGER, Land: STRING
Relation (represented by a table): {(Munich, 1.211.617, Bayern), (Bremen, 535.058, Bremen), ...}
Key : {Name}
Institute for Software Science – University of Vienna
P.Brezany21
Relational Data Model - Queries
SELECT <Attribute List>
FROM <Relation Name>
[WHERE <Condition>] – option
.......... other options
Example: This is equivalent to:
SELECT * SELECT Name, Population, LandFROM Cities FROM Cities
SELECT Name, LandFROM CitiesWHERE Population > 600000
Institute for Software Science – University of Vienna
P.Brezany22
Institute for Software Science – University of Vienna
P.Brezany23
Institute for Software Science – University of Vienna
P.Brezany24
Grid Idea