introduction - puc-rio | homenoemi/pcp-10/aula1-introherlihy.pdf · • hardware – processors •...
TRANSCRIPT
![Page 1: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/1.jpg)
Introduction
Companion slides for The Art of Multiprocessor
Programming by Maurice Herlihy & Nir Shavit
![Page 2: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/2.jpg)
Art of Multiprocessor Programming
2
From the New York Times … SAN FRANCISCO, May 7. 2004 - Intel said on Friday that it was scrapping its development of two microprocessors, a move that is a shift in the company's business strategy….
![Page 3: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/3.jpg)
Art of Multiprocessor Programming
3
Moore’s Law
Clock speed flattening
sharply
Transistor count still
rising
![Page 4: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/4.jpg)
Art of Multiprocessor Programming
4
On Your Desktop: The Uniprocesor
memory
cpu
![Page 5: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/5.jpg)
Art of Multiprocessor Programming
5
In the Enterprise: The Shared Memory Multiprocessor
(SMP)
cache
Bus Bus
shared memory
cache cache
![Page 6: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/6.jpg)
Art of Multiprocessor Programming
6
Your New Desktop: The Multicore processor
(CMP)
cache Bus Bus
shared memory
cache cache All on the same chip
Sun T2000 Niagara
![Page 7: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/7.jpg)
Art of Multiprocessor Programming
7
Multicores Are Here
• “Intel's Intel ups ante with 4-core chip. New microprocessor, due this year, will be faster, use less electricity...” [San Fran Chronicle]
• “AMD will launch a dual-core version of its Opteron server processor at an event in New York on April 21.” [PC World]
• “Sun’s Niagara…will have eight cores, each core capable of running 4 threads in parallel, for 32 concurrently running threads. ….” [The Inquierer]
![Page 8: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/8.jpg)
Art of Multiprocessor Programming
8
Why do we care?
• Time no longer cures software bloat – The “free ride” is over
• When you double your program’s path length – You can’t just wait 6 months – Your software must somehow exploit
twice as much concurrency
![Page 9: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/9.jpg)
Art of Multiprocessor Programming
9
Traditional Scaling Process
User code
Traditional Uniprocessor
Speedup
Time: Moore’s law
![Page 10: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/10.jpg)
Art of Multiprocessor Programming
10
Multicore Scaling Process
User code
Multicore
Speedup
Unfortunately, not so simple…
![Page 11: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/11.jpg)
Art of Multiprocessor Programming
11
Real-World Scaling Process
User code
Multicore
Speedup
Parallelization and Synchronization require great care…
![Page 12: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/12.jpg)
Art of Multiprocessor Programming
12
Multiprocessor Programming: Course Overview
• Fundamentals – Models, algorithms, impossibility
• Real-World programming – Architectures – Techniques
![Page 13: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/13.jpg)
Art of Multiprocessor Programming
13
Multiprocessor Programming: Course Overview
• Fundamentals – Models, algorithms, impossibility
• Real-World programming – Architectures – Techniques
![Page 14: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/14.jpg)
Art of Multiprocessor Programming
14
Sequential Computation
memory
object object
thread
![Page 15: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/15.jpg)
Art of Multiprocessor Programming
15
Concurrent Computation
memory
object object
![Page 16: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/16.jpg)
Art of Multiprocessor Programming
16
Asynchrony
• Sudden unpredictable delays – Cache misses (short) – Page faults (long) – Scheduling quantum used up (really long)
![Page 17: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/17.jpg)
Art of Multiprocessor Programming
17
Model Summary
• Multiple threads – Sometimes called processes
• Single shared memory • Objects live in memory • Unpredictable asynchronous delays
![Page 18: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/18.jpg)
Art of Multiprocessor Programming
18
Road Map
• We are going to focus on principles first, then practice – Start with idealized models – Look at simplistic problems – Emphasize correctness over pragmatism – “Correctness may be theoretical, but
incorrectness has practical impact”
![Page 19: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/19.jpg)
Art of Multiprocessor Programming
19
Concurrency Jargon
• Hardware – Processors
• Software – Threads, processes
• Sometimes OK to confuse them, sometimes not.
![Page 20: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/20.jpg)
Art of Multiprocessor Programming
20
Parallel Primality Testing
• Challenge – Print primes from 1 to 1010
• Given – Ten-processor multiprocessor – One thread per processor
• Goal – Get ten-fold speedup (or close)
![Page 21: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/21.jpg)
Art of Multiprocessor Programming
21
Load Balancing
• Split the work evenly • Each thread tests range of 109
…
… 109 1010 2·109 1
P0 P1 P9
![Page 22: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/22.jpg)
Art of Multiprocessor Programming
22
Procedure for Thread i
void primePrint { int i = ThreadID.get(); // IDs in {0..9} for (j = i*109+1, j<(i+1)*109; j++) { if (isPrime(j)) print(j); } }
![Page 23: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/23.jpg)
Art of Multiprocessor Programming
23
Issues
• Higher ranges have fewer primes • Yet larger numbers harder to test • Thread workloads
– Uneven – Hard to predict
• Need dynamic load balancing
![Page 24: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/24.jpg)
Art of Multiprocessor Programming
24
Issues
• Higher ranges have fewer primes • Yet larger numbers harder to test • Thread workloads
– Uneven – Hard to predict
• Need dynamic load balancing
![Page 25: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/25.jpg)
Art of Multiprocessor Programming
25
17
18
19
Shared Counter
each thread takes a number
![Page 26: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/26.jpg)
Art of Multiprocessor Programming
26
Procedure for Thread i int counter = new Counter(1);
void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } }
![Page 27: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/27.jpg)
Art of Multiprocessor Programming
27
Counter counter = new Counter(1);
void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } }
Procedure for Thread i
Shared counter object
![Page 28: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/28.jpg)
Art of Multiprocessor Programming
28
Where Things Reside
cache
Bus Bus
cache cache
1
shared counter
shared memory
void primePrint { int i = ThreadID.get(); // IDs in {0..9} for (j = i*109+1, j<(i+1)*109; j++) { if (isPrime(j)) print(j); } }
code
Local variables
![Page 29: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/29.jpg)
Art of Multiprocessor Programming
29
Procedure for Thread i Counter counter = new Counter(1);
void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } }
Stop when every value taken
![Page 30: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/30.jpg)
Art of Multiprocessor Programming
30
Counter counter = new Counter(1);
void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } }
Procedure for Thread i
Increment & return each new value
![Page 31: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/31.jpg)
Art of Multiprocessor Programming
31
Counter Implementation
public class Counter { private long value;
public long getAndIncrement() { return value++; } }
![Page 32: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/32.jpg)
Art of Multiprocessor Programming
32
Counter Implementation
public class Counter { private long value;
public long getAndIncrement() { return value++; } } OK for single thread,
not for concurrent threads
![Page 33: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/33.jpg)
Art of Multiprocessor Programming
33
What It Means
public class Counter { private long value;
public long getAndIncrement() { return value++; } }
![Page 34: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/34.jpg)
Art of Multiprocessor Programming
34
What It Means
public class Counter { private long value;
public long getAndIncrement() { return value++; } }
temp = value; value = value + 1; return temp;
![Page 35: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/35.jpg)
Art of Multiprocessor Programming
35
time
Not so good…
Value… 1
read 1
read 1
write 2
read 2
write 3
write 2
2 3 2
![Page 36: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/36.jpg)
Art of Multiprocessor Programming
36
Is this problem inherent?
If we could only glue reads and writes…
read
write read
write
![Page 37: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/37.jpg)
Art of Multiprocessor Programming
37
Challenge
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; } }
![Page 38: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/38.jpg)
Art of Multiprocessor Programming
38
Challenge
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; } }
Make these steps atomic (indivisible)
![Page 39: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/39.jpg)
Art of Multiprocessor Programming
39
Hardware Solution
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; } } ReadModifyWrite()
instruction
![Page 40: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/40.jpg)
Art of Multiprocessor Programming
40
An Aside: Java™
public class Counter { private long value;
public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; } }
![Page 41: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/41.jpg)
Art of Multiprocessor Programming
41
An Aside: Java™
public class Counter { private long value;
public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; } }
Synchronized block
![Page 42: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/42.jpg)
Art of Multiprocessor Programming
42
An Aside: Java™
public class Counter { private long value;
public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; } }
Mutual Exclusion
![Page 43: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/43.jpg)
Art of Multiprocessor Programming
43
Mutual Exclusion or “Alice & Bob share a pond”
A B
![Page 44: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/44.jpg)
Art of Multiprocessor Programming
44
Alice has a pet
A B
![Page 45: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/45.jpg)
Art of Multiprocessor Programming
45
Bob has a pet
A B
![Page 46: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/46.jpg)
Art of Multiprocessor Programming
46
The Problem
A B
The pets don’t get along
![Page 47: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/47.jpg)
Art of Multiprocessor Programming
47
Formalizing the Problem
• Two types of formal properties in asynchronous computation:
• Safety Properties – Nothing bad happens ever
• Liveness Properties – Something good happens eventually
![Page 48: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/48.jpg)
Art of Multiprocessor Programming
48
Formalizing our Problem
• Mutual Exclusion – Both pets never in pond simultaneously – This is a safety property
• No Deadlock – if only one wants in, it gets in – if both want in, one gets in. – This is a liveness property
![Page 49: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/49.jpg)
Art of Multiprocessor Programming
49
Simple Protocol
• Idea – Just look at the pond
• Gotcha – Trees obscure the view
![Page 50: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/50.jpg)
Art of Multiprocessor Programming
50
Interpretation
• Threads can’t “see” what other threads are doing
• Explicit communication required for coordination
![Page 51: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/51.jpg)
Art of Multiprocessor Programming
51
Cell Phone Protocol
• Idea – Bob calls Alice (or vice-versa)
• Gotcha – Bob takes shower – Alice recharges battery – Bob out shopping for pet food …
![Page 52: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/52.jpg)
Art of Multiprocessor Programming
52
Interpretation
• Message-passing doesn’t work • Recipient might not be
– Listening – There at all
• Communication must be – Persistent (like writing) – Not transient (like speaking)
![Page 53: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/53.jpg)
Art of Multiprocessor Programming
53
Can Protocol
cola
cola
![Page 54: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/54.jpg)
Art of Multiprocessor Programming
54
Bob conveys a bit
A B
cola
![Page 55: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/55.jpg)
Art of Multiprocessor Programming
55
Bob conveys a bit
A B
cola
![Page 56: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/56.jpg)
Art of Multiprocessor Programming
56
Can Protocol
• Idea – Cans on Alice’s windowsill – Strings lead to Bob’s house – Bob pulls strings, knocks over cans
• Gotcha – Cans cannot be reused – Bob runs out of cans
![Page 57: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/57.jpg)
Art of Multiprocessor Programming
57
Interpretation
• Cannot solve mutual exclusion with interrupts – Sender sets fixed bit in receiver’s space – Receiver resets bit when ready – Requires unbounded number of inturrupt
bits
![Page 58: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/58.jpg)
Art of Multiprocessor Programming
58
Flag Protocol
A B
![Page 59: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/59.jpg)
Art of Multiprocessor Programming
59
Alice’s Protocol (sort of)
A B
![Page 60: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/60.jpg)
Art of Multiprocessor Programming
60
Bob’s Protocol (sort of)
A B
![Page 61: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/61.jpg)
Art of Multiprocessor Programming
61
Alice’s Protocol
• Raise flag • Wait until Bob’s flag is down • Unleash pet • Lower flag when pet returns
![Page 62: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/62.jpg)
Art of Multiprocessor Programming
62
Bob’s Protocol
• Raise flag • Wait until Alice’s flag is down • Unleash pet • Lower flag when pet returns
![Page 63: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/63.jpg)
Art of Multiprocessor Programming
63
Bob’s Protocol (2nd try)
• Raise flag • While Alice’s flag is up
– Lower flag – Wait for Alice’s flag to go down – Raise flag
• Unleash pet • Lower flag when pet returns
![Page 64: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/64.jpg)
Art of Multiprocessor Programming
64
Bob’s Protocol
• Raise flag • While Alice’s flag is up
– Lower flag – Wait for Alice’s flag to go down – Raise flag
• Unleash pet • Lower flag when pet returns
Bob defers to Alice
![Page 65: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/65.jpg)
Art of Multiprocessor Programming
65
The Flag Principle
• Raise the flag • Look at other’s flag • Flag Principle:
– If each raises and looks, then – Last to look must see both flags up
![Page 66: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/66.jpg)
Art of Multiprocessor Programming
66
Proof of Mutual Exclusion
• Assume both pets in pond – Derive a contradiction – By reasoning backwards
• Consider the last time Alice and Bob each looked before letting the pets in
• Without loss of generality assume Alice was the last to look…
![Page 67: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/67.jpg)
Art of Multiprocessor Programming
67
Proof
time
Alice’s last look
Alice last raised her flag
Bob’s last looked
Alice must have seen Bob’s Flag. A Contradiction
Bob last raised flag
![Page 68: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/68.jpg)
Art of Multiprocessor Programming
68
Proof of No Deadlock
• If only one pet wants in, it gets in.
![Page 69: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/69.jpg)
Art of Multiprocessor Programming
69
Proof of No Deadlock
• If only one pet wants in, it gets in. • Deadlock requires both continually
trying to get in.
![Page 70: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/70.jpg)
Art of Multiprocessor Programming
70
Proof of No Deadlock
• If only one pet wants in, it gets in. • Deadlock requires both continually
trying to get in. • If Bob sees Alice’s flag, he gives her
priority (a gentleman…)
![Page 71: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/71.jpg)
Art of Multiprocessor Programming
71
Remarks
• Protocol is unfair – Bob’s pet might never get in
• Protocol uses waiting – If Bob is eaten by his pet, Alice’s pet
might never get in
![Page 72: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/72.jpg)
Art of Multiprocessor Programming
72
Moral of Story • Mutual Exclusion cannot be solved by
– transient communication (cell phones) – interrupts (cans)
• It can be solved by – one-bit shared variables – that can be read or written
![Page 73: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/73.jpg)
Art of Multiprocessor Programming
73
The Arbiter Problem (an aside) Pick a point
Pick a point
![Page 74: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/74.jpg)
Art of Multiprocessor Programming
74
The Fable Continues
• Alice and Bob fall in love & marry
![Page 75: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/75.jpg)
Art of Multiprocessor Programming
75
The Fable Continues
• Alice and Bob fall in love & marry • Then they fall out of love & divorce
– She gets the pets – He has to feed them
![Page 76: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/76.jpg)
Art of Multiprocessor Programming
76
The Fable Continues
• Alice and Bob fall in love & marry • Then they fall out of love & divorce
– She gets the pets – He has to feed them
• Leading to a new coordination problem: Producer-Consumer
![Page 77: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/77.jpg)
Art of Multiprocessor Programming
77
Bob Puts Food in the Pond
A
![Page 78: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/78.jpg)
Art of Multiprocessor Programming
78
mmm…
Alice releases her pets to Feed
B mmm…
![Page 79: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/79.jpg)
Art of Multiprocessor Programming
79
Producer/Consumer
• Alice and Bob can’t meet – Each has restraining order on other – So he puts food in the pond – And later, she releases the pets
• Avoid – Releasing pets when there’s no food – Putting out food if uneaten food remains
![Page 80: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/80.jpg)
Art of Multiprocessor Programming
80
Producer/Consumer
• Need a mechanism so that – Bob lets Alice know when food has been
put out – Alice lets Bob know when to put out more
food
![Page 81: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/81.jpg)
Art of Multiprocessor Programming
81
Surprise Solution
A B
cola
![Page 82: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/82.jpg)
Art of Multiprocessor Programming
82
Bob puts food in Pond
A B
cola
![Page 83: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/83.jpg)
Art of Multiprocessor Programming
83
Bob knocks over Can
A B
cola
![Page 84: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/84.jpg)
Art of Multiprocessor Programming
84
Alice Releases Pets
A B
cola
yum… B yum…
![Page 85: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/85.jpg)
Art of Multiprocessor Programming
85
Alice Resets Can when Pets are Fed
A B
cola
![Page 86: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/86.jpg)
Art of Multiprocessor Programming
86
Pseudocode
while (true) { while (can.isUp()){}; pet.release(); pet.recapture(); can.reset(); }
Alice’s code
![Page 87: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/87.jpg)
Art of Multiprocessor Programming
87
Pseudocode
while (true) { while (can.isUp()){}; pet.release(); pet.recapture(); can.reset(); }
Alice’s code
while (true) { while (can.isDown()){}; pond.stockWithFood(); can.knockOver(); }
Bob’s code
![Page 88: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/88.jpg)
Art of Multiprocessor Programming
88
Correctness • Mutual Exclusion
– Pets and Bob never together in pond
![Page 89: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/89.jpg)
Art of Multiprocessor Programming
89
Correctness • Mutual Exclusion
– Pets and Bob never together in pond • No Starvation
if Bob always willing to feed, and pets always famished, then pets eat infinitely often.
![Page 90: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/90.jpg)
Art of Multiprocessor Programming
90
Correctness • Mutual Exclusion
– Pets and Bob never together in pond • No Starvation
if Bob always willing to feed, and pets always famished, then pets eat infinitely often.
• Producer/Consumer The pets never enter pond unless there is
food, and Bob never provides food if there is unconsumed food.
safety
liveness
safety
![Page 91: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/91.jpg)
Art of Multiprocessor Programming
91
Could Also Solve Using Flags
A B
![Page 92: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/92.jpg)
Art of Multiprocessor Programming
92
Waiting
• Both solutions use waiting – while(mumble){}
• Waiting is problematic – If one participant is delayed – So is everyone else – But delays are common & unpredictable
![Page 93: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/93.jpg)
Art of Multiprocessor Programming
93
The Fable drags on …
• Bob and Alice still have issues
![Page 94: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/94.jpg)
Art of Multiprocessor Programming
94
The Fable drags on …
• Bob and Alice still have issues • So they need to communicate
![Page 95: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/95.jpg)
Art of Multiprocessor Programming
95
The Fable drags on …
• Bob and Alice still have issues • So they need to communicate • So they agree to use billboards …
![Page 96: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/96.jpg)
Art of Multiprocessor Programming
96
E 1
D 2 C
3
Billboards are Large
B 3 A
1
Letter Tiles
From Scrabble™ box
![Page 97: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/97.jpg)
Art of Multiprocessor Programming
97
E 1
D 2 C
3
Write One Letter at a Time …
B 3 A
1
W 4 A
1 S
1
H 4
![Page 98: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/98.jpg)
Art of Multiprocessor Programming
98
To post a message
W 4 A
1 S
1 H
4 A
1 C
3 R
1 T
1 H
4 E
1
whew
![Page 99: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/99.jpg)
Art of Multiprocessor Programming
99
S 1
Let’s send another mesage
S 1 E
1 L
1 L
1 L
1 V
4
L 1 A
1
M 3
A 1
A 1
P 3
![Page 100: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/100.jpg)
Art of Multiprocessor Programming
100
Uh-Oh
A 1
C 3
R 1
T 1 H
4 E
1 S
1 E
1 L
1 L
1
L 1
OK
![Page 101: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/101.jpg)
Art of Multiprocessor Programming
101
Readers/Writers
• Devise a protocol so that – Writer writes one letter at a time – Reader reads one letter at a time – Reader sees
• Old message or new message • No mixed messages
![Page 102: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/102.jpg)
Art of Multiprocessor Programming
102
Readers/Writers (continued)
• Easy with mutual exclusion • But mutual exclusion requires waiting
– One waits for the other – Everyone executes sequentially
• Remarkably – We can solve R/W without mutual
exclusion
![Page 103: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/103.jpg)
Art of Multiprocessor Programming
103
Why do we care?
• We want as much of the code as possible to execute concurrently (in parallel)
• A larger sequential part implies reduced performance
• Amdahl’s law: this relation is not linear…
![Page 104: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/104.jpg)
Art of Multiprocessor Programming
104
Amdahl’s Law
Speedup=
…of computation given n CPUs instead of 1
![Page 105: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/105.jpg)
Art of Multiprocessor Programming
105
Amdahl’s Law
Speedup=
![Page 106: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/106.jpg)
Art of Multiprocessor Programming
106
Amdahl’s Law
Speedup=
Parallel fraction
![Page 107: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/107.jpg)
Art of Multiprocessor Programming
107
Amdahl’s Law
Speedup=
Parallel fraction
Sequential fraction
![Page 108: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/108.jpg)
Art of Multiprocessor Programming
108
Amdahl’s Law
Speedup=
Parallel fraction
Number of processors
Sequential fraction
![Page 109: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/109.jpg)
Art of Multiprocessor Programming
109
Example • Ten processors • 60% concurrent, 40% sequential • How close to 10-fold speedup?
![Page 110: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/110.jpg)
Art of Multiprocessor Programming
110
Example • Ten processors • 60% concurrent, 40% sequential • How close to 10-fold speedup?
Speedup=2.17=
![Page 111: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/111.jpg)
Art of Multiprocessor Programming
111
Example • Ten processors • 80% concurrent, 20% sequential • How close to 10-fold speedup?
![Page 112: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/112.jpg)
Art of Multiprocessor Programming
112
Example • Ten processors • 80% concurrent, 20% sequential • How close to 10-fold speedup?
Speedup=3.57=
![Page 113: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/113.jpg)
Art of Multiprocessor Programming
113
Example • Ten processors • 90% concurrent, 10% sequential • How close to 10-fold speedup?
![Page 114: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/114.jpg)
Art of Multiprocessor Programming
114
Example • Ten processors • 90% concurrent, 10% sequential • How close to 10-fold speedup?
Speedup=5.26=
![Page 115: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/115.jpg)
Art of Multiprocessor Programming
115
Example • Ten processors • 99% concurrent, 01% sequential • How close to 10-fold speedup?
![Page 116: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/116.jpg)
Art of Multiprocessor Programming
116
Example • Ten processors • 99% concurrent, 01% sequential • How close to 10-fold speedup?
Speedup=9.17=
![Page 117: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/117.jpg)
Art of Multiprocessor Programming
117
The Moral
• Making good use of our multiple processors (cores) means
• Finding ways to effectively parallelize our code – Minimize sequential parts – Reduce idle time in which threads wait without
![Page 118: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/118.jpg)
Art of Multiprocessor Programming
118
Multiprocessor Programming
• This is what this course is about… – The % that is not easy to make
concurrent yet may have a large impact on overall speedup
• Next week: – A more serious look at mutual exclusion
![Page 119: Introduction - PUC-Rio | Homenoemi/pcp-10/aula1-introherlihy.pdf · • Hardware – Processors • Software – Threads, processes • Sometimes OK to confuse them, sometimes not](https://reader035.vdocuments.site/reader035/viewer/2022071015/5fce36d4d3a0fa09486539ec/html5/thumbnails/119.jpg)
Art of Multiprocessor Programming
119
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License.
• You are free: – to Share — to copy, distribute and transmit the work – to Remix — to adapt the work
• Under the following conditions: – Attribution. You must attribute the work to “The Art of
Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work).
– Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license.
• For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to – http://creativecommons.org/licenses/by-sa/3.0/.
• Any of the above conditions can be waived if you get permission from the copyright holder.
• Nothing in this license impairs or restricts the author's moral rights.