concurrency: processes and events - …
TRANSCRIPT
![Page 1: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/1.jpg)
CSE333, Autumn 2019L26: Processes
Concurrency: Processes and EventsCSE 333 Autumn 2019
Instructor: Hannah C. Tang
Teaching Assistants:
Dao Yi Farrell Fileas Lukas Joswiak
Nathan Lipiarski Renshu Gu Travis McGaha
Yibo Cao Yifan Bai Yifan Xu
![Page 2: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/2.jpg)
CSE333, Autumn 2019L26: Processes
pollev.com/cse333
About how long did Exercise 17 take?
A. 0-1 HoursB. 1-2 HoursC. 2-3 HoursD. 3-4 HoursE. 4+ HoursF. I prefer not to say
2
![Page 3: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/3.jpg)
CSE333, Autumn 2019L26: Processes
Administrivia
❖ 🎊 No more exercises! 🎊
❖ HW4 due on Thursday (12/05)
▪ You can use at most ONE late day
❖ Guest lecture on Wednesday (12/04)
▪ Albert J. Wong, Google: threat modeling and system design
3
![Page 4: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/4.jpg)
CSE333, Autumn 2019L26: Processes
Administrivia
❖ Final exam on Wednesday (12/11)
▪ Final review sessions this weekend!
❖ Course evals
▪ Please fill them out! Your feedback is extremely valuable to us
▪ Comments are helpful!
▪ Your honesty is even more helpful!
4
![Page 5: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/5.jpg)
CSE333, Autumn 2019L26: Processes
Lecture Outline
❖ Processes
▪ fork() and wait()
▪ Concurrency using Processes
▪ Threads vs. Processes: A Story of Efficiency
❖ Event-based Concurrency
❖ Concurrency Wrapup
5
![Page 6: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/6.jpg)
CSE333, Autumn 2019L26: Processes
Review: Address Spaces
❖ A process executes within an address space
▪ Includes segments for different parts of memory
▪ Process tracks its current state using the stack pointer (SP) and program counter (PC)
6
0xFF…FF
0x00…00
OS kernel [protected]
Stack
Heap (malloc/free)
Read/Write Segments.data, .bss
Shared Libraries
Read-Only Segments.text, .rodata
SP
PC
![Page 7: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/7.jpg)
CSE333, Autumn 2019L26: Processes
Review: Multi-threaded Address Spaces
❖ After creating a thread
▪ Two threads of execution running in the address space• Original thread (parent) and new
thread (child)
• New stack created for child thread
• Child thread has its own values of the PC and SP
▪ Both threads share the other segments (code, heap, globals)• They can cooperatively modify
shared data
7
OS kernel [protected]
Stackparent
Heap (malloc/free)
Read/Write Segments.data, .bss
Shared Libraries
Read-Only Segments.text, .rodata
SPparent
PCparent
StackchildSPchild
PCchild
![Page 8: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/8.jpg)
CSE333, Autumn 2019L26: Processes
Creating New Processes
❖
▪ Creates a new process (the “child”) that is an exact clone* of the current process (the “parent”)
• Variables, file descriptors, open sockets, the virtual address space (code, globals, heap, stack), etc.
• *Everything is cloned except threads
❖ Primarily used in two patterns:
▪ Servers: fork a child to handle a connection
▪ Shells: fork a child that then exec’s a new program
8
pid_t fork(void);
![Page 9: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/9.jpg)
CSE333, Autumn 2019L26: Processes
fork() and Address Spaces
❖ fork() causes the OS to clone the address space▪ The copies of the
memory segments are (nearly) identical
▪ The new process has copies of the parent’s data, stack-allocated variables, open file descriptors, etc.
9
OS kernel [protected]
Stack
Heap (malloc/free)
Read/Write Segments.data, .bss
Shared Libraries
Read-Only Segments.text, .rodata
fork() CHILD
OS kernel [protected]
Stack
Heap (malloc/free)
Read/Write Segments.data, .bss
Shared Libraries
Read-Only Segments.text, .rodata
PARENT
SPparent
PCparent
SPchild
PCchild
![Page 10: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/10.jpg)
CSE333, Autumn 2019L26: Processes
fork()
❖ fork() has peculiar semantics
▪ The parent invokes fork()
▪ The OS clones the parent
▪ Both the parent and the child return from fork
• Parent receives child’s pid
• Child receives a 0
10
parent
OS
fork()
In-memoryresources
![Page 11: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/11.jpg)
CSE333, Autumn 2019L26: Processes
fork()
❖ fork() has peculiar semantics
▪ The parent invokes fork()
▪ The OS clones the parent
▪ Both the parent and the child return from fork
• Parent receives child’s pid
• Child receives a 0
11
parent child
OS
clone
In-memoryresources
In-memoryresources
![Page 12: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/12.jpg)
CSE333, Autumn 2019L26: Processes
fork()
❖ fork() has peculiar semantics
▪ The parent invokes fork()
▪ The OS clones the parent
▪ Both the parent and the child return from fork
• Parent receives child’s pid
• Child receives a 0
❖ Remember that processes become “zombies” after death
12
parent child
OS
child pid 0
In-memoryresources
In-memoryresources
![Page 13: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/13.jpg)
CSE333, Autumn 2019L26: Processes
waitpid()
❖
▪ Block until the passed-in process has changed state (usually terminated)
• Detailed process status available in status output parameter.
13
pid_t waitpid(pid_t pid, int *status,
int options);
![Page 14: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/14.jpg)
CSE333, Autumn 2019L26: Processes
I need a fork()ing demo!
❖ See fork_example.cc
14
![Page 15: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/15.jpg)
CSE333, Autumn 2019L26: Processes
Lecture Outline
❖ Processes
▪ fork() and waitpid()
▪ Concurrency using Processes
▪ Threads vs. Processes: A Story of Efficiency
❖ Event-based Concurrency
❖ Concurrency Wrapup
15
![Page 16: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/16.jpg)
CSE333, Autumn 2019L26: Processes
Multi-processes Search Engine: Architecture
❖ The parent process blocks on accept(), waiting for a new client to connect▪ When a new connection arrives, the parent calls fork() to
create a child process
▪ The child process handles that new connection and subsequent I/O, calls exit()’s when the connection terminates
16
![Page 17: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/17.jpg)
CSE333, Autumn 2019L26: Processes
Double-fork Trick
❖ There is no “process version” of pthread_detach()
▪ How do we tell the OS to clean up the process when it’s dead?
❖ Remember that processes become “zombies” after death▪ Option A: Parent calls waitpid() to “reap” children
▪ Option B: Parent terminates, causing children to be “adopted” by the root process (“init” or “systemd”)
17
![Page 18: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/18.jpg)
CSE333, Autumn 2019L26: Processes
Multi-process Search Engine: Request Flow
18
serverIn-memoryresources
![Page 19: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/19.jpg)
CSE333, Autumn 2019L26: Processes
Multi-process Search Engine: Request Flow
19
client
server accept()In-memoryresources
![Page 20: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/20.jpg)
CSE333, Autumn 2019L26: Processes
Multi-process Search Engine: Request Flow
20
client
server
serverfork() child
In-memoryresources
In-memoryresources
![Page 21: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/21.jpg)
CSE333, Autumn 2019L26: Processes
Multi-process Search Engine: Request Flow
21
client
server
serverfork() child
In-memoryresources
In-memoryresources
![Page 22: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/22.jpg)
CSE333, Autumn 2019L26: Processes
Multi-process Search Engine: Request Flow
22
client
server
server
fork() grandchild
In-memoryresources
In-memoryresources
serverIn-memoryresources
![Page 23: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/23.jpg)
CSE333, Autumn 2019L26: Processes
Multi-process Search Engine: Request Flow
23
client
serverIn-memoryresources
serverIn-memoryresources
child exit()’s / parent wait()’s
![Page 24: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/24.jpg)
CSE333, Autumn 2019L26: Processes
Multi-process Search Engine: Request Flow
24
client
serverIn-memoryresources
serverIn-memoryresources
parent closes itsclient connection
![Page 25: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/25.jpg)
CSE333, Autumn 2019L26: Processes
Multi-process Search Engine: Request Flow
25
serverIn-memoryresources
In-memoryresourcesclient server
![Page 26: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/26.jpg)
CSE333, Autumn 2019L26: Processes
Multi-process Search Engine: Request Flow
26
serverIn-memoryresources
In-memoryresourcesclient server
server
server
server
fork() grandchildexit()
fork() childIn-memoryresources
In-memoryresources
client
![Page 27: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/27.jpg)
CSE333, Autumn 2019L26: Processes
Multi-process Search Engine: Request Flow
27
client server
client server
serverIn-memoryresources
In-memoryresources
In-memoryresources
![Page 28: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/28.jpg)
CSE333, Autumn 2019L26: Processes
Multi-process Search Engine: Request Flow
28
client server
client server
client server
client server
client server
client server
client server
client server
client server
server
![Page 29: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/29.jpg)
CSE333, Autumn 2019L26: Processes
pollev.com/cse333
What happens when a grandchild process finishes?
A. Zombie until grandparent exits
B. Zombie until grandparent reaps
C. Zombie until systemd reaps
D. ZOMBIE FOREVER!!!
E. I’m not sure…
29
![Page 30: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/30.jpg)
CSE333, Autumn 2019L26: Processes
Lecture Outline
❖ Processes
▪ fork() and waitpid()
▪ Concurrency using Processes
▪ Threads vs. Processes: A Story of Efficiency
❖ Event-based Concurrency
❖ Concurrency Wrapup
31
![Page 31: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/31.jpg)
CSE333, Autumn 2019L26: Processes
How Fast is fork()?
❖ See forklatency.cc
❖ ~ 0.500 ms per fork*
▪ ∴ maximum of (1000/0.50) = 2,000 connections/sec/core
▪ ~175 million connections/day/core
• This is fine for most servers
• Too slow for super-high-traffic front-line web services
– Facebook served ~ 750 billion page views per day in 2013!Would need 3-6k cores just to handle fork(), i.e. without doing any work for each connection
❖ *Past measurements are not indicative of future performance – depends on hardware, OS, software versions, …
32
![Page 32: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/32.jpg)
CSE333, Autumn 2019L26: Processes
How Fast is pthread_create()?
❖ See threadlatency.cc
❖ ~0.070 ms per thread creation*▪ ~10x faster than fork()
▪ ∴ maximum of (1000/0.036) = 28,000 connections/sec
▪ ~2.4 billion connections/day/core
❖ Mush faster, but writing safe multithreaded code can be serious voodoo
❖ *Past measurements are not indicative of future performance – depends on hardware, OS, software versions, …, but will typically be an order of magnitude faster than fork()
33
![Page 33: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/33.jpg)
CSE333, Autumn 2019L26: Processes
Lecture Outline
❖ Processes
▪ fork() and waitpid()
▪ Concurrency using Processes
▪ Threads vs. Processes: A Story of Efficiency
❖ Event-based Concurrency
❖ Concurrency Wrapup
34
![Page 34: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/34.jpg)
CSE333, Autumn 2019L26: Processes
Review: Multi-“worker” Search Engine
35
client
client
client
client
client
client server
shareddata
structures
“The child process/thread handles that new connectionand subsequent I/O, then calls exit()/pthread_exit()
when the connection terminates”
client server
client server
serverIn-memoryresources
In-memoryresources
In-memoryresources
Processes Threads
![Page 35: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/35.jpg)
CSE333, Autumn 2019L26: Processes
Event-Driven Programming
❖ Your program is structured as an event-loop consisting of (mostly) independent, stateless tasks executing in any order
36
void ProcessOneTask(state) {
query_words = state.buffer;
for (idx : state.indices) {
...
}
...
}
while (1) {
event = OS.GetNextEvent();
state = GetState(event);
ProcessOneTask(state);
}
![Page 36: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/36.jpg)
CSE333, Autumn 2019L26: Processes
One Way to Think About It
❖ Threaded code:
▪ OS and thread scheduler switch between threads for you
▪ Each thread executes its task sequentially, and per-task state is naturally stored in the thread’s stack
❖ Event-driven code:
▪ You (or your framework) are the scheduler
• You (or your framework) also manages scheduling-related resources, such as the connection
▪ You have to bundle up task state into continuations (data structures describing what-to-do-next); tasks do not have their own stacks
37
![Page 37: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/37.jpg)
CSE333, Autumn 2019L26: Processes
Multi-Step Event-Driven Programming❖ Each step is a brand-new event
▪ Task state must include information about which step we’re on
38
void dispatch(task, event) {
switch (task.state) {
case READING_FROM_CONSOLE:
query_words = event.query;
async_read(index, query_words[0]);
task.state = READING_FROM_INDEX;
return;
case READING_FROM_INDEX:
results = event.results;
...
}
}
while (1) {
event = OS.GetNextEvent();
task = lookup(event);
dispatch(task, event);
}
![Page 38: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/38.jpg)
CSE333, Autumn 2019L26: Processes
Multi-Step, Event-Driven w/Async I/O
39
I/O 1.b
I/O 2.b
I/O 3.b
time
I/O 2.d
CPU 3.a
CPU 1.a
CPU 2.a
I/O 1.d
CPU 1.c
CPU 2.c
I/O 3.d
CPU 1.e
CPU 2.e
CPU 3.c
CPU 3.e
![Page 39: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/39.jpg)
CSE333, Autumn 2019L26: Processes
Lecture Outline
❖ Processes
▪ fork() and waitpid()
▪ Concurrency using Processes
▪ Threads vs. Processes: A Story of Efficiency
❖ Event-based Concurrency
❖ Concurrency Wrapup
41
![Page 40: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/40.jpg)
CSE333, Autumn 2019L26: Processes
Aside: Thread Pools
❖ In real servers, we’d like to avoid overhead needed to create a new thread or process for every request
❖ Idea: Thread Pools
▪ Create a fixed set of worker threads or processes on server startup and put them in a queue
▪ When a request arrives, remove the first worker thread from the queue and assign it to handle the request
▪ When a worker is done, it places itself back on the queue and then sleeps until dequeued and handed a new request
❖ Pairs naturally with event-based programming
42
![Page 41: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/41.jpg)
CSE333, Autumn 2019L26: Processes
Why Sequential?
❖ Advantages:
▪ Simple to write, maintain, debug
▪ The default. Supported everywhere!
❖ Disadvantages:
▪ Depending on application, poor performance
• One slow client will cause all others to block
• Poor utilization of resources (CPU, network, disk)
43
![Page 42: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/42.jpg)
CSE333, Autumn 2019L26: Processes
Why Concurrent Threads?
❖ Advantages:
▪ Almost as simple to code as sequential
▪ Concurrent execution with good CPU and network utilization
▪ Threads can run in parallel if you have multiple CPUs/cores
▪ Shared-memory communication is possible
❖ Disadvantages:
▪ Need language and OS support for threads
▪ If threads share data, you need locks or other synchronization
▪ Threads can introduce overhead (technical + cognitive)
▪ Threads have a “shared fate” (eg, “rogue” thread, shared limits)
44
![Page 43: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/43.jpg)
CSE333, Autumn 2019L26: Processes
Why Concurrent Processes?
❖ Advantages:
▪ Almost as simple to code as sequential
▪ Concurrent execution with good CPU and network utilization
▪ Processes almost certainly run in parallel thanks to OS time-sharing
▪ No need to synchronize access to in-memory structures
❖ Disadvantages:
▪ Processes are heavyweight
• Relatively slow to fork and context switching latency is high
▪ Communication between processes is complicated
▪ Fewer things to synchronize – but when you do need to synchronize, it’s hard!
45
![Page 44: Concurrency: Processes and Events - …](https://reader030.vdocuments.site/reader030/viewer/2022012703/61a54f6b3a0d493a587b8502/html5/thumbnails/44.jpg)
CSE333, Autumn 2019L26: Processes
Why Events?
❖ Advantages:
▪ For some kinds of programs – those with mostly-stateless, simple responses – leads to very simple and intuitive program
• Eg, GUIs: one event handler for each UI event
❖ Disadvantages:
▪ Can lead to very complex structure for some programs
• Sequential logic gets broken up into a jumble of small event handlers
• You have to package up all task state between handlers
46