1 capriccio: scalable threads for internet services matthew phillips
TRANSCRIPT
![Page 1: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/1.jpg)
1
Capriccio: Scalable Threads for Internet Services
Matthew Phillips
![Page 2: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/2.jpg)
2
Overview
• Motivation• Background
• History• Threads vs. event based systems• User-level threads vs. kernel threads
• Capriccio• Linked stacks• Resource-aware scheduler
• Evaluation
![Page 3: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/3.jpg)
3
Motivation
• High concurrency• Internet servers can have requests for
hundreds of thousands of connections.• Simplify programming model.
![Page 4: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/4.jpg)
4
Background
• History• Why threads are a bad idea (for most
purposes), John Ousterhout. Sun Labs. 1996.
• Race conditions, deadlocks, not scalable, etc.
• Popularity of Internet about the same time• Heavy demand on web servers
![Page 5: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/5.jpg)
5
Background
• History• Need for high concurrency servers
• Flash• SEDA
• Interest in systems other than thread based systems.
• Event based systems
• Problem: Event based systems are sometimes difficult to use.
• Programmers prefer linear program control
![Page 6: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/6.jpg)
6
Background
• History• Instead of making event based systems
more usable, “fix” thread based systems.• Investigate ways to make thread based
systems more scalable.• Authors presented Why events are a bad
idea (for high-concurrency servers). 2003• On the duality of operating system structures,
Lauer, Needham.1978.
![Page 7: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/7.jpg)
7
Background
• Why threads?• Can have equal or better performance
• More desirable programming model• Programmers prefer linear program control
• Existing systems use threads
![Page 8: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/8.jpg)
8
Background
• Kernel threads• Systems calls
• open, read, send, receive
• User-level threads• User-level libraries
![Page 9: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/9.jpg)
9
Background
• User-level threads• Advantages:
• Flexibility: A level away (abstraction) from kernel. Decouples applications and kernel.
• Performance: User-level threads are lightweight: Inexpensive synchronization, fast context switches
• Thus, user-level threading is used.
![Page 10: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/10.jpg)
10
Background
• Created user-level threading library for Linux
• All thread operations are O(1)• Now scalable
![Page 11: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/11.jpg)
11
Capriccio
• Linked stacks• Traditional call stacks• LinuxThreads allocates 2 megabytes per thread• Most threads only consume a few kilobytes• Lots of wasted virtual memory
• 32 bit system has 4 gigabyte limit
![Page 12: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/12.jpg)
12
Capriccio
• Uses weighted call graphs• Perform a whole-program analysis
• CIL used to read source
• Each function is a node• Each edge is a function call• Node weights are calculated from stack frames
![Page 13: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/13.jpg)
13
Capriccio
• Node weightsint sample()
{ int a, b;
int c[10];double d;
char e[8036];return 0;
}
![Page 14: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/14.jpg)
14
Capriccio
• Example
Path from Main->A->B is 2.3k
![Page 15: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/15.jpg)
15
Capriccio
• Example main(){char buf[512]; A(); C();}
void A(){char buf[820]; B(); D();}
void B(){char buf[1024];}
void C(){char buf[205]; D(); E();}
void D(){char buf[205];}
void E(){char buf[205]; C();}
![Page 16: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/16.jpg)
16
Capriccio
• Linked stacks• Recursion complicates things• Use checkpoints to deal with recursion
![Page 17: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/17.jpg)
17
Capriccio
• Placement of checkpoints• Break each cycle with checkpoint• Additional checkpoints
• Longest path from node to checkpoint, if predefined limit is exceeded, add checkpoint
![Page 18: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/18.jpg)
18
Capriccio
• Example
![Page 19: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/19.jpg)
19
Capriccio
• Resource aware scheduling• Uses blocking graph
• Node is location in program that is blocked• Node is composed of call chain used to reach
blocking point
![Page 20: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/20.jpg)
20
Capriccio
• Resource aware scheduling• Edges are annotated with average running
time - time it took the thread to pass between nodes
• Nodes are annotated with resources used on its outgoing edges.
![Page 21: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/21.jpg)
21
Capriccio
• Resource aware scheduling
![Page 22: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/22.jpg)
22
Capriccio
• Evaluation• Apache• Compared with Knot• Compared with Haboob
![Page 23: 1 Capriccio: Scalable Threads for Internet Services Matthew Phillips](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d145503460f949e8103/html5/thumbnails/23.jpg)
23
Capriccio
• Evaluation
15% speedup with Capriccio