cache wrap‐up, system control flow · l19: cache wrap-up, system control flow cse351, autumn 2017...

CSE351, Autumn 2017L19: Cache Wrap-Up, System Control Flow

Cache Wrap‐Up, System Control FlowCSE 351 Autumn 2017

Instructor:Justin Hsia

Teaching Assistants:Lucas Wotton Michael Zhang Parker DeWilde Ryan WongSam Gehman Sam Wolfson Savanna Yee Vinny Palaniappan

http://xkcd.com/908/


Administrivia

Homework 4 due Friday (11/17) Lab 4 released Wednesday (11/15) Cache parameter puzzles and code optimizations

2


Anatomy of a Cache Question

Cache questions come in a few flavors:

1) TIO Address Breakdown

2) For fixed cache parameters, analyze the performance of the given code/sequence

3) For given code/sequence, how does changing your cache parameters affect performance?

4) Average Memory Access Time (AMAT)

3


Optimizations for the Memory Hierarchy

Write code that has locality! Spatial: access data contiguously Temporal: make sure access to the same data is not too far apart in time

How can you achieve locality? Adjust memory accesses in code (software) to improve miss rate (MR)• Requires knowledge of both how caches work as well as your system’s parameters

Proper choice of algorithm Loop transformations

4


Example: Matrix Multiplication

5

C

= ×

A B

ai* b*j

cij


Matrices in Memory

How do cache blocks fit into this scheme? Row major matrix in memory:

6

Cache blocks

COLUMN of matrix (blue) is spread among cache blocks shown in red


Naïve Matrix Multiply

# move along rows of Afor (i = 0; i < n; i++)

# move along columns of Bfor (j = 0; j < n; j++)# EACH k loop reads row of A, col of B# Also read & write c(i,j) n timesfor (k = 0; k < n; k++)c[i*n+j] += a[i*n+k] * b[k*n+j];

7

C(i,j) A(i,:)B(:,j)

C(i,j)


Cache Miss Analysis (Naïve)

Scenario Parameters: Square matrix ( ), elements are doubles Cache block size = 64 B = 8 doubles Cache size (much smaller than )

Each iteration:

misses

8

Ignoring matrix c




Each iteration:

misses

Afterwards in cache:(schematic)

98 doubles wide

Ignoring matrix c




Each iteration:

misses

Total misses: 2 3

10

Ignoring matrix c

once per element


Linear Algebra to the Rescue (1)

Can get the same result of a matrix multiplication by splitting the matrices into smaller submatrices (matrix “blocks”)

For example, multiply two 4×4 matrices:

11

This is extra (non‐testable)

material


Linear Algebra to the Rescue (2)

12

Matrices of size , split into 4 blocks of size ( = )

C22 = A21B12 + A22B22 + A23B32 + A24B42 = k A2k*Bk2

Multiplication operates on small “block” matrices Choose size so that they fit in the cache! This technique called “cache blocking”

C11 C12 C13 C14

C21 C22 C23 C24

C31 C32 C43 C34

C41 C42 C43 C44

A11 A12 A13 A14

A21 A22 A23 A24

A31 A32 A33 A34

A41 A42 A43 A144

B11 B12 B13 B14

B21 B22 B23 B24

B32 B32 B33 B34

B41 B42 B43 B44


material


Blocked Matrix Multiply

Blocked version of the naïve algorithm:

= block matrix size (assume divides evenly)

13

# move by rxr BLOCKS nowfor (i = 0; i < n; i += r)for (j = 0; j < n; j += r)for (k = 0; k < n; k += r)# block matrix multiplicationfor (ib = i; ib < i+r; ib++)for (jb = j; jb < j+r; jb++)for (kb = k; kb < k+r; kb++)c[ib*n+jb] += a[ib*n+kb]*b[kb*n+jb];


Cache Miss Analysis (Blocked)

Scenario Parameters: Cache block size = 64 B = 8 doubles Cache size (much smaller than ) Three blocks ( ) fit into cache: 2

Each block iteration: misses per block 2

14

/ blocks2elements per block, 8 per cache block

/ blocks in row and column

Ignoring matrix c





Afterwards in cache(schematic)

15



Ignoring matrix c





Total misses: 3

16



Ignoring matrix c


Matrix Multiply Visualization

Here = 100, = 32 KiB, = 30

17

Naïve:

Blocked:

≈ 1,020,000cache misses

≈ 90,000cache misses


Cache‐Friendly Code

Programmer can optimize for cache performance How data structures are organized How data are accessed

• Nested loop structure• Blocking is a general technique

All systems favor “cache‐friendly code” Getting absolute optimum performance is very platform specific• Cache size, cache block size, associativity, etc.

Can get most of the advantage with generic code• Keep working set reasonably small (temporal locality)• Use small strides (spatial locality)• Focus on inner loop code

18


The Memory Mountain

19

128m32m

8m2m

512k128k

32k0

2000

4000

6000

8000

10000

12000

14000

16000

s1s3

s5s7

s9s11

Size (bytes)

Read

throug

hput (M

B/s)

Stride (x8 bytes)

Core i7 Haswell2.1 GHz32 KB L1 d‐cache256 KB L2 cache8 MB L3 cache64 B block size

Slopes of spatial locality

Ridges of temporal locality

L1

Mem

L2

L3

Aggressive prefetching


Learning About Your Machine

Linux: lscpu ls /sys/devices/system/cpu/cpu0/cache/index0/

• Ex: cat /sys/devices/system/cpu/cpu0/cache/index*/size

Windows: wmic memcache get <query> (all values in KB) Ex: wmic memcache get MaxCacheSize

Modern processor specs: http://www.7‐cpu.com/

20


Roadmap

21

car *c = malloc(sizeof(car));c->miles = 100;c->gals = 17;float mpg = get_mpg(c);free(c);

Car c = new Car();c.setMiles(100);c.setGals(17);float mpg =

c.getMPG();

get_mpg:pushq %rbpmovq %rsp, %rbp...popq %rbpret

Java:C:

Assembly language:

Machine code:

01110100000110001000110100000100000000101000100111000010110000011111101000011111

Computer system:

OS:

Memory & dataIntegers & floatsx86 assemblyProcedures & stacksExecutablesArrays & structsMemory & cachesProcessesVirtual memoryMemory allocationJava vs. C


Leading Up to Processes

System Control Flow Control flow Exceptional control flow Asynchronous exceptions (interrupts) Synchronous exceptions (traps & faults)

22


Control Flow

So far: we’ve seen how the flow of control changes as a single program executes

Reality: multiple programs running concurrently How does control flow across the many components of the system? In particular: More programs running than CPUs

Exceptional control flow is basic mechanism used for: Transferring control between processes and OS Handling I/O and virtual memory within the OS Implementing multi‐process apps like shells and web servers Implementing concurrency

23


Control Flow

Processors do only one thing: From startup to shutdown, a CPU simply reads and executes (interprets) a sequence of instructions, one at a time This sequence is the CPU’s control flow (or flow of control)

24

<startup>instr1instr2instr3…instrn<shutdown>

Physical control flow

time


Altering the Control Flow

Up to now, two ways to change control flow: Jumps (conditional and unconditional) Call and return Both react to changes in program state

Processor also needs to react to changes in system state Unix/Linux user hits “Ctrl‐C” at the keyboard User clicks on a different application’s window on the screen Data arrives from a disk or a network adapter Instruction divides by zero System timer expires

Can jumps and procedure calls achieve this? No – the system needs mechanisms for “exceptional” control flow!

25


Java Digression

Java has exceptions, but they’re something different Examples: NullPointerException, MyBadThingHappenedException, … throw statements try/catch statements (“throw to youngest matching catch on the call‐

stack, or exit‐with‐stack‐trace if none”)

Java exceptions are for reacting to (unexpected) program state Can be implemented with stack operations and conditional jumps A mechanism for “many call‐stack returns at once” Requires additions to the calling convention, but we already have the

CPU features we need

System‐state changes on previous slide are mostly of a different sort (asynchronous/external except for divide‐by‐zero) and implemented very differently

26


material


Exceptional Control Flow

Exists at all levels of a computer system

Low level mechanisms Exceptions

• Change in processor’s control flow in response to a system event (i.e. change in system state, user‐generated interrupt)

• Implemented using a combination of hardware and OS software

Higher level mechanisms Process context switch

• Implemented by OS software and hardware timer

Signals• Implemented by OS software• We won’t cover these – see CSE451 and CSE/EE474

27


Exceptions

An exception is transfer of control to the operating system (OS) kernel in response to some event (i.e. change in processor state)

Kernel is the memory‐resident part of the OS Examples: division by 0, page fault, I/O request completes, Ctrl‐C

How does the system know where to jump to in the OS?

User Code OS Kernel Code

exceptionexception processing by exception handler, then:• return to current_instr,• return to next_instr, OR• abort

current_instrnext_instr

event

28


Exception Table

A jump table for exceptions (also called Interrupt Vector Table) Each type of event has a unique

exception number = index into exception table

(a.k.a interrupt vector) Handler is called each time

exception occurs

29

012

...n-1

ExceptionTable

code for exception handler 0

code for exception handler 1

code forexception handler 2

code for exception handler n‐1

...

Exception numbers


Exception Table (Excerpt)

30

Exception Number Description Exception Class

0 Divide error Fault

13 General protection fault Fault

14 Page fault Fault

18 Machine check Abort

32‐255 OS‐defined Interrupt or trap


Leading Up to Processes

System Control Flow Control flow Exceptional control flow Asynchronous exceptions (interrupts) Synchronous exceptions (traps & faults)

31


Asynchronous Exceptions (Interrupts)

Caused by events external to the processor Indicated by setting the processor’s interrupt pin(s) (wire into CPU) After interrupt handler runs, the handler returns to “next” instruction

Examples: I/O interrupts

• Hitting Ctrl‐C on the keyboard• Clicking a mouse button or tapping a touchscreen• Arrival of a packet from a network• Arrival of data from a disk

Timer interrupt• Every few ms, an external timer chip triggers an interrupt• Used by the OS kernel to take back control from user programs

32


Synchronous Exceptions

Caused by events that occur as a result of executing an instruction: Traps

• Intentional: transfer control to OS to perform some function• Examples: system calls, breakpoint traps, special instructions• Returns control to “next” instruction

Faults• Unintentional but possibly recoverable • Examples: page faults, segment protection faults, integer divide‐by‐zero

exceptions• Either re‐executes faulting (“current”) instruction or aborts

Aborts• Unintentional and unrecoverable• Examples: parity error, machine check (hardware failure detected)• Aborts current program

33


System Calls

Each system call has a unique ID number Examples for Linux on x86‐64:

34

Number Name Description

0 read Read file

1 write Write file

2 open Open file

3 close Close file

4 stat Get info about file

57 fork Create process

59 execve Execute a program

60 _exit Terminate process

62 kill Send signal to process


Traps Example: Opening File User calls open(filename, options) Calls __open function, which invokes system call instruction syscall

00000000000e5d70 <__open>:...e5d79: b8 02 00 00 00 mov $0x2,%eax # open is syscall 2e5d7e: 0f 05 syscall # return value in %raxe5d80: 48 3d 01 f0 ff ff cmp $0xfffffffffffff001,%rax ...e5dfa: c3 retq

User code OS Kernel code

Exception

Open fileReturns

syscallcmp

%rax contains syscall number Other arguments in %rdi,

%rsi, %rdx, %r10, %r8, %r9 Return value in %rax Negative value is an error

corresponding to negative errno

35


Fault Example: Page Fault User writes to memory location That portion (page) of user’s memory

is currently on disk

Page fault handler must load page into physical memory Returns to faulting instruction: mov is executed again! Successful on second try

int a[1000];int main (){

a[500] = 13;}

80483b7: c7 05 10 9d 04 08 0d movl $0xd,0x8049d10

User code OS Kernel code

exception: page faultCreate page and load into memoryreturns

movlhandle_page_fault:

36


Fault Example: Invalid Memory Reference

Page fault handler detects invalid address Sends SIGSEGV signal to user process User process exits with “segmentation fault”

37

int a[1000];int main(){

a[5000] = 13;}

80483b7: c7 05 60 e3 04 08 0d movl $0xd,0x804e360

User Process OS

exception: page fault

detect invalid addressmovl

signal process

handle_page_fault:


Summary

Exceptions Events that require non‐standard control flow Generated externally (interrupts) or internally (traps and faults) After an exception is handled, one of three things may happen:• Re‐execute the current instruction• Resume execution with the next instruction• Abort the process that caused the exception

38

cache wrap‐up, system control flow · l19: cache wrap-up, system control flow cse351, autumn 2017...

Documents