Operating Systems 11/21/2013
CSC 256/456 1
11/21/2013 CSC 256/456 23
Multiprocessor Operating
Systems
CS 256/456
Dept. of Computer Science, University
of Rochester
11/21/2013 CSC 256/456 24
Multiprocessor Hardware • A computer system in which two or more CPUs share full
access to the main memory
• Each CPU might have its own cache and the coherence among multiple caches is maintained
– write operation by a CPU is visible to all other CPUs
– writes to the same location is seen in the same order by all CPUs (also called write serialization)
– bus snooping and cache invalidation
… … … Cache
CPU
Cache
CPU
Cache
CPU
Memory
Memory bus
11/21/2013 CSC 256/456 25
Multiprocessor Applications
• Multiprogramming
– Multiple regular applications running concurrently
• Concurrent servers
– Web servers, … …
• Parallel programs
– Utilizing multiple processors to complete one task (parallel matrix multiplication, Gaussian elimination)
– Strong synchronization
x = A B C
11/21/2013 CSC 256/456 26
Single-processor OS vs. Multiprocessor
OS • Single-processor OS
– easier to support kernel synchronization • coarse-grained locking vs. fine-grain locking
• disabling interrupts to prevent concurrent executions
– easier to perform scheduling • which to run, not where to run
• Multiprocessor OS
– evolution of OS structure
– synchronization
– scheduling
Operating Systems 11/21/2013
CSC 256/456 2
11/21/2013 CSC 256/456 27
Multiprocessor OS
• Each CPU has its own operating system
– quick to port from a single-processor OS
• Disadvantages
– difficult to share things (processing cycles, memory, buffer cache)
Bus
11/21/2013 CSC 256/456 28
Multiprocessor OS – Master/Slave
Bus
• All operating system functionality goes to one CPU
– no multiprocessor concurrency in the kernel
• Disadvantage
– OS CPU consumption may be large so the OS CPU becomes the bottleneck (especially in a machine with many CPUs)
11/21/2013 CSC 256/456 29
Multiprocessor OS – Shared OS
• A single OS instance may run on all CPUs
• The OS itself must handle multiprocessor synchronization
– multiple OS instances from multiple CPUs may access shared data structure
Bus
11/21/2013 CSC 256/456 30
Preemptive Scheduling
• Use timer interrupts or signals to trigger involuntary yields
• Protect scheduler data structures by locking ready list, disabling/reenabling prior to/after rescheduling
yield:
disable_signals
enqueue(ready_list, current)
reschedule
re-enable_signals
Operating Systems 11/21/2013
CSC 256/456 3
11/21/2013 CSC 256/456 31
Synchronization (Fine/Coarse-Grain
Locking)
• Fine-grain locking – lock only what is necessary for critical section
• Coarse-grain locking – locking large piece of code, much of which is unnecessary – simplicity, robustness – prevent simultaneous execution
Simultaneous execution is not possible on uniprocessor
anyway
11/21/2013 CSC 256/456 32
Anderson et al. 1989 (IEEE TOCS)
• Raises issues of
– Locality (per-processor data structures)
– Granularity of scheduling tasks
– Lock overhead
– Tradeoff between throughput and latency
• Large critical sections are good for best-case
latency (low locking overhead) but bad for
throughput (low parallelism)
11/21/2013 CSC 256/456 33
Performance Measures
• Latency
– Cost of thread management under the best
case assumption of no contention for locks
• Throughput
– Rate at which threads can be created, started,
and finished when there is contention
11/21/2013 CSC 256/456 34
Optimizations
• Allocate stacks lazily
• Store deallocated control blocks and stacks in
free lists
• Create per-processor ready lists
• Create local free lists for locality
• Queue of idle processors (in addition to queue of
waiting threads)
Operating Systems 11/21/2013
CSC 256/456 4
11/21/2013 CSC 256/456 35
Ready List Management
• Single lock for all data structures
• Multiple locks, one per data structure
• Local freelists for control blocks and stacks, single shared locked ready list
• Queue of idle processors with preallocated control block and stack waiting for work
• Local ready list per processor, each with its own lock
11/21/2013 CSC 256/456 36
Multiprocessor Scheduling • Timesharing
– similar to uni-processor scheduling – one queue of ready tasks (protected by synchronization), a task is dequeued and executed when a processor is available
• Space sharing • cache affinity
– affinity-based scheduling – try to run each process on the processor that it last ran on
• cache sharing and synchronization of parallel/concurrent applications – gang/cohort scheduling – utilize all CPUs for one
parallel/concurrent application at a time CPU 0
CPU 1
web server parallel Gaussian elimination
client/server game (civ)
11/21/2013 CSC 256/456 37
SMP-CMP-SMT Multiprocessor
Image from http://www.eecg.toronto.edu/~tamda/papers/threadclustering.pdf 11/21/2013 CSC 256/456 38
Resource Contention-Aware
Scheduling I • Hardware resource sharing/contention in multi-processors
– SMP processors share memory bus bandwidths
– Multi-core processors share L2 cache
– SMT processors share a lot more stuff
• An example: on an SMP machine
– a web server benchmark delivers around 6300 reqs/sec on one processor, but only around 9500 reqs/sec on an SMP with 4 processors
• Contention-reduction scheduling
– co-scheduling tasks with complementary resource needs (a computation-heavy task and a memory access-heavy task)
– In [Fedorova et al. USENIX2005], IPC is used to distinguish computation-heavy tasks from memory access-heavy tasks
Operating Systems 11/21/2013
CSC 256/456 5
11/21/2013 CSC 256/456 39
Resource Contention-Aware
Scheduling II • What if contention on a resource is unavoidable?
• Two evils of contention
– high contention ⇒ performance slowdown
– fluctuating contention ⇒ uneven application progress over the same amount of time ⇒ poor fairness
• [Zhang et al. HotOS2007] Scheduling so that:
– very high contention is avoided
– the resource contention is kept stable
CPU 0
CPU 1
high resource usage
low resource usage
high resource usage
low resource usage
medium resource usage
medium resource usage
Multi-Core Cache Challenges
• Cache performance is critical to computer
systems.
– memory access: several hundred cycles
– cache hit: L1 several cycles; L2 10~20 cycles
• Cache performance is poor.
– more than half of CPU’s area is dedicated to cache
– 30% of total execution time is due to
L2 cache miss [Basu, MICRO’06]
– low utilization:
more than 40% of lines are evicted
before reuse [Qureshi, ISCA’07]
source: http://www.intel.com
Multi-Core Cache Challenges
(cont.)
• Hardware manages cache at the grain of cache
lines.
– single program: data with different locality are mixed
together
– shared cache: uncontrolled sharing threads
– compete for space -> interference
• Using OS as an auxiliary to manage cache
– high-level knowledge of program
– running state of the entire system
– how? page coloring
Address Mapping in Cache
• physical memory address and cache
– cache size = line size * way size * # of set
– 512KB, 16-way, 64B line size L2 cache: 512
sets (9 bits to index)
32 bits addr. set index line offset
5 bits 9 bits
L2 Cache
cache set
Way-1 Way-16 …………
Operating Systems 11/21/2013
CSC 256/456 6
•color:
– cache: a group of cache sets
– memory: a group of physical pages
– (page N, N+4, N+8, … )
•page color:
– data belonging to a page color can only be cached by cache sets with same color
physical addr. page offset (12bits)
5 bits line off. 9 bits set index
physical page #
physical
mem pages
color index (2 bits-> 4 colors)
L2 Cache
What is a Page Color? Software partitioning by page
coloring
OS and Page Coloring
• What is the rule of OS?
– control the mapping between virtual memory
pages and physical pages via page table
physical addr. page offset physical page #
virtual addr. page offset virtual page #
page table
under the
control of OS
OS and Page Coloring (cont.)
• Color a page: map a virtual page to a physical
page with a particular color (lower bits of page #)
• Re-color a page: change the color in runtime
– flush TLB, copy page data, modify page table
– overhead
physical pages
virtual addr. page offset virtual page number
Operating Systems 11/21/2013
CSC 256/456 7
Shared Caches on Multicore Systems
[Tam et al. WIOSCA’97]
– Uncontrolled sharing threads
run mcf,art on Dual Core Power 5
Page Coloring Based Cache
Partitioning • Hardware partitioning mechanisms, not available
today
• Page coloring based cache partitioning
– Implemented on real system
– Guided phys pages allocation
-> controlled L2 cache usage
– Goal: optimize system overall
performance
Benefits of Cache Partitioning [Tam
et al. WIOSCA’07]
base:
multi-prg, no partitioning
Big Picture
A B D C
Select which applications run
together
X ………
…..
Control resource usage of co-running
applications
Resource-aware scheduling
Page coloring or Hardware throttling
Operating Systems 11/21/2013
CSC 256/456 8
Hardware Execution Throttling • Instead of directly controlling resource allocation,
throttle the execution speed of application that overuses resource
• Available throttling knobs
– Duty-cycle modulation
– Frequency/voltage scaling
– Cache prefetchers
Comparing Hardware Execution
Throttling to Page Coloring
• Kernel code modification complexity
– Code length: 40 lines in a single file, as a reference our
page coloring implementation takes 700+ lines of code
crossing 10+ files
• Runtime overhead of configuration
– Less than 1 microseconds, as a reference re-coloring a
page takes 3 microseconds
Existing Mechanism(II):
Scheduling Quantum Adjustment
• Shorten the time slice of app that overuses cache
• May let core idle if there is no other active thread
available
Thread B
Thread A idle
Thread B
Thread A idle
Thread B
Thread A idle Core 0
Core 1
time
Drawback of Scheduling Quantum
Adjustment Coarse-grained control at scheduling quantum granularity may
result in fluctuating service delays for individual transactions
Operating Systems 11/21/2013
CSC 256/456 9
New Mechanism:
Hardware Execution Throttling [Usenix’09]
• Throttle the execution speed of app that overuses cache
– Duty cycle modulation
• CPU works only in duty cycles and stalls in non-duty cycles
• Different from Dynamic Voltage Frequency Scaling
– Per-core vs. per-processor control
– Thermal vs. power management
– Enable/disable cache prefetchers
• L1 prefetchers
– IP: keeps track of instruction pointer for load history
– DCU: when detecting multiple loads from the same line within a time limit,
prefetches the next line
• L2 prefetchers
– Adjacent line: Prefetches the adjacent line of required data
– Stream: looks at streams of data for regular patterns
Comparison of Hardware Execution
Throttling to other two mechanisms
• Comparison to page coloring
– Little complexity to kernel • Code length: 40 lines in a single file, as a reference our page coloring implementation
takes 700+ lines of code crossing 10+ files
– Lightweight to configure • Read plus write register: duty-cycle 265 + 350 cycles, prefetcher 298 + 2065 cycles
• Less than 1 microseconds, as a reference re-coloring a page takes 3 microseconds
• Comparison to scheduling quantum adjustment
– More fine-grained controlling
Thread B
Core 0
Core 1
Thread A idle
Quantum adjustment Hardware execution throttling
time
57
Fairness Comparison
• On average all three mechanisms are effective in improving fairness
• Case {swim, SPECweb} illustrates limitation of page coloring
• Unfairness factor: coefficient of variation (deviation-to-mean ratio, σ / μ) of co-running apps’ normalized performances (normalization base is the execution-time/throughput when the application monopolizes the whole chip)
58
Performance Comparison
• System efficiency: geometric mean of co-running apps’ normalized performances
• On average all three mechanisms achieve system efficiency comparable to default sharing
• Case where severe inter-
thread cache conflicts exist
favors segregation, e.g.
{swim, mcf}
• Case where well-interleaved
cache accesses exist favors
sharing, e.g. {mcf, mcf}
Operating Systems 11/21/2013
CSC 256/456 10
Policies for Hardware Throttling-
Enabled Multicore Management
• User-defined service level agreements (SLAs) – Proportional progress among competing threads
• Unfairness metric: coefficient of variation of threads’ performance
– Quality of service guarantee for high-priority application(s)
• Key challenge
– Throttling configuration space grows exponentially as
the number of cores increases
– Quickly determining optimal or close to optimal
throttling configurations is challenging
TEMM: A Flexible Framework for
Throttling-Enabled Multicore
Management [ICPP’12] • Customizable performance estimation model
• Reference configuration set and linear approximation
• Currently incorporates duty cycle modulation and
frequency/voltage scaling
• Iterative refinement
• Prediction accuracy gets improved over time as more
configurations are added into reference set
