Cache OrganizationCache Organization

TopicsTopics Background Simple examples

– 2 –

Typical Cache OrganizationTypical Cache Organization

=?

cache line

address

tag index offset

tagarray

dataarray

Cache Organization Details (S, E, B)Cache Organization Details (S, E, B)E = 2e blocks (lines) per set

S=2s sets

setblock

0 1 2 B-1tagv

valid bitB = 2b bytes per cache block (the data)

Cache size:S x E x B data bytes

– 4 –

Example: Direct Mapped Cache (E = 1)Example: Direct Mapped Cache (E = 1)

S=2s sets

Direct mapped: One block per setAssume: cache block size 8 bytes

t bits 100Address of int:

0 1 2 7tagv 3 654

0 1 2 7tagv 3 654

0 1 2 7tagv 3 654

0 1 2 7tagv 3 654

find set

– 5 –

Example: Direct Mapped Cache (E = 1)Example: Direct Mapped Cache (E = 1)

t bits 100Address of int:

0 1 2 7tagv 3 654

match: assume yes = hit

block offset

tag

Direct mapped: One block per setAssume: cache block size 8 bytes

– 6 –

Example: Direct Mapped Cache (E = 1)Example: Direct Mapped Cache (E = 1)

t bitsAddress of int:

0 1 2 7tagv 3 654

match: assume yes = hit

block offset

tag

Direct mapped: One block per setAssume: cache block size 8 bytes

int (4 Bytes) is here

No match: old line is evicted and replaced

100

– 7 –

E-way Set Associative Cache (E = 2)E-way Set Associative Cache (E = 2)

E = 2: Two lines per setAssume: cache block size 8 bytes

t bits 100Address of short int:

0 1 2 7tgv 3 654 0 1 2 7tgv 3 654

0 1 2 7tgv 3 654 0 1 2 7tgv 3 654

0 1 2 7tgv 3 654 0 1 2 7tgv 3 654

0 1 2 7tgv 3 654 0 1 2 7tgv 3 654

find set

E-way Set Associative Cache (E = 2)E-way Set Associative Cache (E = 2)

t bits 100Address of short int:

0 1 2 7tgv 3 654 0 1 2 7v 3 654

compare both

match: yes = hit

block offset

tg

E = 2: Two lines per setcache block size 8 bytes

tg

E-way Set Associative Cache (E = 2)E-way Set Associative Cache (E = 2)

t bits 100Address of short int:

0 1 2 7v 3 654 0 1 2 7tgv 3 654

compare both

match: yes = hit

block offset

tg

E = 2: Two lines per setcache block size 8 bytes

short int (2 Bytes) is here

No match: •One line in set is selected for eviction and replacement

•Replacement policies: random, least recently used (LRU), …

tg

– 10 –

Assumed Simple CacheAssumed Simple Cache

2 ints per block2 ints per block

2-way set associative2-way set associative

2 blocks total2 blocks total

1 set1 set

i.e., same thing as fully associativei.e., same thing as fully associative

Replacement policy: Least Recently Used (LRU)Replacement policy: Least Recently Used (LRU)

Cache

Block 0

Block 1

– 11 –

Array Access: Key QuestionsArray Access: Key Questions

How many array elements are there per block?How many array elements are there per block?

Does the data structure fit in the cache?Does the data structure fit in the cache?

Do I re-use blocks over time?Do I re-use blocks over time?

In what order am I accessing blocks?In what order am I accessing blocks?

– 12 –

Simple ArraySimple Array

1 2 3 4A

Cachefor (i=0;i<N;i++){

… = A[i];}

Miss rate = #misses / #accesses

– 13 –

Simple ArraySimple Array

1 2 3 4A

Cachefor (i=0;i<N;i++){

… = A[i];}

Miss rate = #misses / #accesses =

(N//2) / N = ½ = 50%

– 14 –

Simple Array w outer loopSimple Array w outer loop

1 2 3 4A

Cache for (k=0;k<P;k++){ for (i=0;i<N;i++){

… = A[i]; }}

Assume A[] fits in the cache:

Miss rate = #misses / #accesses =

Lesson: for sequential accesses with re-use,If fits in the cache, first visit suffers all the misses

– 15 –

Simple Array w outer loopSimple Array w outer loop

1 2 3 4A

Cache for (k=0;k<P;k++){ for (i=0;i<N;i++){

… = A[i]; }}

Assume A[] fits in the cache:

Miss rate = #misses / #accesses =

Lesson: for sequential accesses with re-use,If fits in the cache, first visit suffers all the misses

(N//2) / N = ½ = 50%

– 16 –

Simple ArraySimple Array

1 2 3 4 5 6 7 8A

Cache for (i=0;i<N;i++){ … = A[i];

}

Assume A[] does not fit in the cache:

Miss rate = #misses / #accesses

– 17 –

Simple ArraySimple Array

5 6

7 81 2 3 4 5 6 7 8A

Cache for (i=0;i<N;i++){ … = A[i];

}

Assume A[] does not fit in the cache:

Miss rate = #misses / #accesses = (N/2) / N = ½ = 50%

Lesson: for sequential accesses, if no-reuse it doesn’tmatter whether data structure fits

– 18 –

Simple Array with outer loopSimple Array with outer loop

1 2 3 4 5 6 7 8A

Cache

Assume A[] does not fit in the cache:

Miss rate = #misses / #accesses =

for (k=0;k<P;k++){ for (i=0;i<N;i++){

… = A[i]; }}

(N/2) / N = ½ = 50%

Lesson: for sequential accesses with re-use,

If the data structure doesn’t fit,

same miss rate as no-reuse

– 19 –

2D array2D array

ACache

Assume A[] fits in the cache:

Miss rate = #misses / #accesses =

for (i=0;i<N;i++){ for (j=0;j<N;j++){ … = A[i][j];

}}

1 2

3 4

(N*N/2) / (N*N) = ½ = 50%

– 20 –

2D array2D array

ACache

for (i=0;i<N;i++){ for (j=0;j<N;j++){ … = A[i][j];

}}

Lesson: for 2D accesses, if row order and no-reuse,same hit rate as sequential, whether fits or not

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

Assume A[] does not fit in the cache:

Miss rate = #misses / #accesses = 50%

– 21 –

2D array2D array

ACache for (j=0;j<N;j++){

for (i=0;i<N;i++){ … = A[i][j];

}}

Lesson: for 2D accesses, if column order and no-reuse,same hit rate as sequential if entire column fits in the cache

1 2

3 4

Assume A[] fits in the cache:

Miss rate = #misses / #accesses = (N*N/2) / N*N = ½ = 50%

– 22 –

2D array2D array

ACache

Assume A[] does not fit in the cache:

Miss rate = #misses / #accesses

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

for (j=0;j<N;j++){ for (i=0;i<N;i++){ … = A[i][j];

}}

= 100%

Lesson: for 2D accesses, if column order, if entire column

doesn’t fit, then 100% miss rate (block (1,2) is gone after access to element 9).

– 23 –