x-ray: a non-invasive exclusive caching mechanism for raids lakshmi n. bairavasundaram muthian...
TRANSCRIPT
X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs
Lakshmi N. Bairavasundaram
Muthian Sivathanu
Andrea C. Arpaci-Dusseau
Remzi H. Arpaci-Dusseau
ADvanced Systems Laboratory
Computer Sciences Department
University of Wisconsin – Madison
Introduction Caching in modern systems
Multiple levels Storage: 2-level hierarchy
Level 1: File system (FS) cache Software-managed Main memory of host/client LRU-like cache replacement
Level 2: RAID cache Firmware-managed Memory inside RAID system Usually LRU replacement .......
File system cache
RAID cache
RAID
Application
Host
Introduction – contd. LRU
Replace LRU block Cache placement on read
Read Block no. 10
LRU MRU
Read Block no. 10
39 …….. 4523 10…….. 4523
Introduction – contd. LRU
Replace LRU block Cache placement on read
2 levels of LRU Redundant contents
……..
……..
Read Block no. 10
Read Block no. 10
Read Block no. 10
10
10
MRU
MRU
LRU
LRU10
LRU 10 MRU
MRULRU
11
11
12
12
….
….
FS Cache
RAID Cache
Introduction – contd. LRU
Cache placement on read Replace LRU block
2 levels of LRU Redundant contents
Goal: Exclusive caching 10
LRU 10 MRU
MRULRU
11
11
12
12
….
….
FS Cache
RAID Cache
Improved RAID Caching Multi-Queue (Zhou et al. 2001)
Add frequency component to cache policy Not strictly exclusive!
DEMOTE (Wong and Wilkes 2002) Change interface to disk File system issues “cache place” command Has perfect information and hence perfectly exclusive caches Interface changes – difficult to deploy
Ideal RAID Cache Exclusive caching
File system and RAID caches should have different contents
Global LRU Known to work well RAID cache should be a victim cache
No interface changes….
……
FS Cache
RAID Cache Block ReadVictim Block
LRU
MRU
X-RAY Observes disk traffic
Reads and writes to data and metadata
Builds a model of the FS cache Uses semantic knowledge Predicts size and contents of FS cache
Identifies set of exclusive blocks Recent victims of the FS cache
Reads blocks from disk into cache Result
A nearly exclusive cache without interface changes
File system cache
RAID cache
RAID
Host
Model of FS cacheX-RAY
Talk Outline Introduction File Systems Information and Inferences X-RAY Cache Design Results Conclusion
File System Operation Applications perform file reads and writes File system (Unix)
Translates file accesses to disk block requests Metadata
To maintain application data on disk and manage disk blocks Periodically written to disk Examples: inodes, bitmap blocks
File System Operation Inode
Pointers to data blocks File access information
Inode
Data Blocks
Latest access time
Pointers to data blocks
File
File System Operation File access
Use inode to obtain pointers to disk data blocks Read corresponding blocks from disk if they are not in FS cache Update the access time information in inode
Metadata updates Periodically check for “dirty” inodes and write to disk
The Problem To observe disk traffic and infer
the contents of FS cache Why difficult?
FS cache size changes over time Shares main memory with virtual
memory system
The Problem To observe disk traffic and infer
the contents of FS cache Why difficult?
FS cache size changes over time Disk cannot observe all FS-level
accesses
Read block:
10
10
Disk Read
11
11
101112
12
12
LRU
LRU
MRU
MRU
FS Cache
FS Cache Model
RAID
The Problem To observe disk traffic and infer
the contents of FS cache Why difficult?
FS cache size changes over time Disk cannot observe all FS-level
accesses
Read block:
10
10
Disk Read
11
11
10
12
12
LRU
LRU
MRU
MRU
13
FS Cache
FS Cache Model
RAID
The ProblemRead block:
10
11 12
12 13
13
LRU
LRU
MRU
MRU
FS Cache
FS Cache Model
RAID
To observe disk traffic and infer the contents of FS cache
Why difficult? FS cache size changes over time Disk cannot observe all FS-level
accesses
The ProblemRead block:
10
11 12
12 13
13
LRU
LRU
MRU
MRU
FS Cache
FS Cache Model
RAID
To observe disk traffic and infer the contents of FS cache
Why difficult? FS cache size changes over time Disk cannot observe all FS-level
accesses
Key observation We need information about
accesses that hit in FS cache File system maintains access
information in inodes
Talk Outline Introduction File Systems Information and Inferences X-RAY Cache Design Results Conclusion
Information Obtain information from observing disk traffic Knowledge of file system structures and operations
File system maintains time of last access in inodes Periodic inode writes Assuming whole file access, all blocks are in FS cache
Assume file system cache policy is LRU
Inferences Read for data block
Block will be placed in file system cache (MRU block)
Read for previously read data block Block became victim in file system cache Blocks with an earlier access time should also be victims
Inode write: new access time , no disk read observed All blocks belonging to file are in FS cache Other blocks with later access time should also be present
Talk Outline Introduction File Systems Information and Inferences X-RAY Cache Design Results Conclusion
Design
Recency list (R-list) List of data blocks ordered
by access time Cache Begin (CB) pointer
Divides R-list into inclusive and exclusive regions
RAID Cache contents Subset of blocks in exclusive
region
LRU MRUA, 1 B, 1 D, 3C, 2 F, 5E, 3
CBInclusive regionExclusive region
Block number Access time
Blocks the RAID should cache
Blocks expected to be in FS cache
Disk Read
LRU MRUA , 1 B , 1 C , 2 D , 3 E , 3 F , 4
CBInclusive regionExclusive region
Read Block ‘D’ ; time = 6
Disk Read
LRU MRUA , 1 B , 1 C , 2 D , 3 E , 3 F , 4
CBInclusive regionExclusive region
Read Block ‘D’ ; time = 6
Disk Read
LRU MRUA , 1 B , 1 C , 2 D , 6E , 3 F , 4
CBInclusive regionExclusive region
Read Block ‘D’ ; time = 6
Inode Write – Access time change
LRU MRUA , 1 B , 1 C , 2 D , 3 E , 4 F , 5
CBInclusive regionExclusive region
G , 7
Inode “23” : access time = 6
Semantic knowledge Inode “23” == blocks D & E
Blocks D, E : access time = 6
Inode Write – Access time change
LRU MRUA , 1 B , 1 C , 2 D , 3 E , 4 F , 5
CBInclusive regionExclusive region
G , 7
Blocks D, E : access time = 6Inode “23” : access time = 6
Inode Write – Access time change
LRU MRUA , 1 B , 1 C , 2 F , 5
D , 6 E , 6
CBInclusive regionExclusive region
G , 7
Blocks D, E : access time = 6Inode “23” : access time = 6
X-RAY Cache
LRU MRUA , 1 B , 1 C , 2 F , 5 D , 6 E , 6
CBInclusive regionExclusive region
G , 7
RAID Cache (size = 2 blocks)
Keep track of additions to window in exclusive region
X-RAY Cache
Read newly-added blocks from disk Replace blocks no longer in the window Additional disk bandwidth
Idle time, extra internal bandwidth, freeblock scheduling
LRU MRUA , 1 B , 1 C , 2 F , 5 D , 6 E , 6
CBInclusive regionExclusive region
G , 7
RAID Cache (size = 2 blocks)
Talk Outline Introduction File Systems Information and Inferences X-RAY Cache Design Results
Tracking FS Cache Contents RAID Cache Performance
Conclusion
Results – Tracking Accurate size and content prediction Highly responsive to FS cache size changes Tolerates changes in inode write interval Partial file reads
X-RAY performs well if percentage of partially accessed files is < 40% (typical traces have less than 30%)
Results – Cache Performance
Performs better than LRU and Multi-Queue
Close to DEMOTE, in spite of imperfect information
Hit rate advantage translates to lower read latency
Additional Results File system cache policy is not LRU
Clock, 2Q X-RAY performs nearly as well as before It performs better than both LRU and Multi-Queue
Idle time requirements X-RAY reads blocks into cache only during idle time It performs well if idle time is greater than one-third of actual idle time
observed in the trace
More in the paper …
Conclusion Easy deployment is an important goal in developing technology
Avoid interface changes – use non-invasive mechanisms
Higher-level systems maintain various pieces of information about data they manage Provide low-level systems with basic semantic knowledge
Semantic intelligence for managing RAID caches Use access information in metadata to track file system cache contents
and cache exclusive blocks In spite of imperfect information, X-RAY performs nearly as well as
changing the interface
Semantically-smart Disk Systems Availability, security and performance improvements
Questions ?
ADvanced Systems Laboratory (ADSL)
Computer Sciences, University of Wisconsin-Madison
http://www.cs.wisc.edu/adsl