file systems
DESCRIPTION
File Systems. Thomas Plagemann University of Oslo (includes slides from: Carsten Griwodz , P ål Halvorsen, Kai Li, Andrew Tanenbaum , Maarten van Steen). Todays Plan. Long-term Information Storage. Must store large amounts of data - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/1.jpg)
File Systems
Thomas PlagemannUniversity of Oslo
(includes slides from: Carsten Griwodz, Pål Halvorsen, Kai Li, Andrew Tanenbaum, Maarten van Steen)
![Page 2: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/2.jpg)
2
Todays Plan
![Page 3: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/3.jpg)
3
Long-term Information Storage
1. Must store large amounts of data
2. Information stored must survive the termination of the process using it
3. Multiple processes must be able to access the information concurrently
![Page 4: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/4.jpg)
Why Files?• Physical reality
– Block oriented– Physical sector
numbers– No protection among
users of the system– Data might be
corrupted if machine crashes
• File system abstraction– Byte oriented– Named files– Users protected from
each other– Robust to machine
failures
![Page 5: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/5.jpg)
5
File Naming
Typical file extensions.
![Page 6: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/6.jpg)
6
File Structure
• Three kinds of files– byte sequence– record sequence– tree
![Page 7: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/7.jpg)
7
File Types
(a) An executable file (b) An archive
![Page 8: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/8.jpg)
8
File Access• Sequential access
– read all bytes/records from the beginning– cannot jump around, could rewind or back up– convenient when medium was mag tape
• Random access– bytes/records read in any order– essential for data base systems– read can be …
• move file marker (seek), then read or …• read and then move file marker
![Page 9: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/9.jpg)
9
File Attributes
Possible file attributes
![Page 10: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/10.jpg)
10
File Operations
1. Create2. Delete3. Open4. Close5. Read6. Write
7. Append8. Seek9. Get attributes10.Set Attributes11.Rename
![Page 11: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/11.jpg)
11
An Example Program Using File System Calls (1/2)
![Page 12: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/12.jpg)
12
An Example Program Using File System Calls (2/2)
![Page 13: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/13.jpg)
13
DirectoriesSingle-Level Directory Systems
• A single level directory system– contains 4 files– owned by 3 different people, A, B, and C
![Page 14: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/14.jpg)
14
Two-level Directory Systems
Letters indicate owners of the directories and files
![Page 15: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/15.jpg)
15
Hierarchical Directory Systems
A hierarchical directory system
![Page 16: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/16.jpg)
16A UNIX directory tree
Path Names
![Page 17: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/17.jpg)
17
Directory Operations
1. Create2. Delete3. Opendir4. Closedir
5. Readdir6. Rename7. Link8. Unlink
![Page 18: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/18.jpg)
18
Now we are opening the box
Inodes
Bit map
FS layout
![Page 19: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/19.jpg)
File System Components• Disk management
– Arrange collection of disk blocks into files
• Naming– User gives file name, not track or
sector number, to locate data• Security
– Keep information secure• Reliability/durability
– When system crashes, lose stuff in memory, but want files to be durable
User
FileNaming
Fileaccess
Diskmanagement
Diskdrivers
![Page 20: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/20.jpg)
20
File System Implementation
A possible file system layout
![Page 21: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/21.jpg)
21
Implementing Files (1)
(a) Contiguous allocation of disk space for 7 files(b) State of the disk after files D and E have been removed
![Page 22: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/22.jpg)
22
Implementing Files (2)
Storing a file as a linked list of disk blocks
![Page 23: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/23.jpg)
23
Implementing Files (3)
Linked list allocation using a file allocation table in RAM
![Page 24: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/24.jpg)
File Allocation Table (FAT)• Approach
– A section of disk for each partition is reserved
– One entry for each block– A file is a linked list of blocks– A directory entry points to the
1st block of the file• Pros
– Simple• Cons
– Always go to FAT– Wasting space
619
399
foo 217
EOF
FAT
0
217
399
619
![Page 25: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/25.jpg)
25
Implementing Files (4)
An example i-node
![Page 26: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/26.jpg)
26
Implementing Directories (1)
(a) A simple directoryfixed size entriesdisk addresses and attributes in directory entry
(b) Directory in which each entry just refers to an i-node
![Page 27: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/27.jpg)
27
Implementing Directories (2)
• Two ways of handling long file names in directory– (a) In-line– (b) In a heap
![Page 28: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/28.jpg)
28
The UNIX V7 File System (1)
A UNIX V7 directory entry
![Page 29: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/29.jpg)
29
The UNIX V7 File System (2)
A UNIX i-node
![Page 30: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/30.jpg)
30
The UNIX V7 File System (3)
The steps in looking up /usr/ast/mbox
![Page 31: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/31.jpg)
31
Shared Files (1)
File system containing a shared file
![Page 32: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/32.jpg)
32
Shared Files (2)
(a) Situation prior to linking(b) After the link is created(c)After the original owner removes the file
![Page 33: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/33.jpg)
33
Block Size
![Page 34: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/34.jpg)
34
Disk Space Management (1)
• Dark line (left hand scale) gives data rate of a disk• Dotted line (right hand scale) gives disk space efficiency• All files 2KB
Block size
![Page 35: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/35.jpg)
35
Disk Space Management (2)
(a) Storing the free list on a linked list(b) A bit map
![Page 36: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/36.jpg)
36
Disk Space Management (3)
(a) Almost-full block of pointers to free disk blocks in RAM- three blocks of pointers on disk
(b) Result of freeing a 3-block file(c) Alternative strategy for handling 3 free blocks
- shaded entries are pointers to free disk blocks
![Page 37: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/37.jpg)
37
Disk Space Management (4)
Quotas for keeping track of each user’s disk use
![Page 38: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/38.jpg)
38
How many FSs do we have on one PC?
![Page 39: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/39.jpg)
39
Virtual File System
![Page 40: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/40.jpg)
40
Creating a VFS
• Boot time of OS: register root FS with VFS• Mounting of others FSs (boot time or later):
register with VFS• Registering: provide function pointers to the
FS specific calls
![Page 41: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/41.jpg)
41
Using a FS via VFS
• Open(“/usr/include/unistd.h”, O_RDONLY)
![Page 42: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/42.jpg)
42
VFS Structure
![Page 43: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/43.jpg)
43
Network File System (NFS)
![Page 44: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/44.jpg)
44
NFS Architecture
![Page 45: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/45.jpg)
45
A Look into Linux-like “Mechanics”
![Page 46: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/46.jpg)
File System Data Structures
Processcontrolblock
...
Openfilepointerarray
Open filetable(systemwide)
File descriptors(Metadata)
Filedescriptors
File systeminfo
Directories
File data
![Page 47: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/47.jpg)
Open File
• File name lookup and authenticate
• Copy the file descriptors into the in memory data structure, if it is not in yet
• Create an entry in the open file table (system wide) if there isn’t one
• Create an entry in PCB• Link up the data structures• Return a pointer to user
PCB
fd = open( FileName, access)
Openfiletable
Metadata
Allocate & link updata structures
File name lookup& authenticate
File system on disk
![Page 48: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/48.jpg)
OpenO
pera
ting
Sys
tem
open(name, mode)
sys_open() vn_open():1. Check if valid call 2. Allocate file descriptor3. If file exists, open for read. Otherwise, create a new file.
Must get directory i-node. May require disk I/O.4. Set access rights, flags and pointer to v-node5. Return index to file descriptor table
fd
![Page 49: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/49.jpg)
From User to System View• What happens if user wants to read 10 bytes from
a file starting at byte 2?– seek byte 2– fetch the block– read 10 bytes
• What happens if user wants to write 10 bytes to a file starting at byte 2?– seek byte 2– fetch the block– write 10 bytes– write out the block
![Page 50: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/50.jpg)
Read BlockPCB
Openfiletable
Metadata
read( fd, userBuf, size )
Find open filedescriptor
read( fileDesc, userBuf, size )
Logical phyiscal
read( device, phyBlock, size )Get physical block to sysBufcopy to userBuf
Disk device driver
Buffercache
![Page 51: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/51.jpg)
ReadO
pera
ting
Sys
tem
bufferread(fd, buf*, len)
sys_read() dofileread() (*fp_read==vn_read)():1. Check if valid call and mark file as used 2. Use file descriptor as index in file table
to find corresponding file pointer3. Use data pointer in file structure to find v-node4. Find current offset in file 5. Call local file system
VOP_READ(vp,len,offset,..)
![Page 52: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/52.jpg)
ReadO
pera
ting
Sys
tem
VOP_READ(...) is a pointer to a read function in the corresponding file system, e.g., Fast File System (FFS)
READ():1. Find corresponding inode2. Check if valid call - file size vs. len + offset 3. Loop and find corresponding blocks
• find logical blocks from inode, offset, length• do block I/O, fill buffer structure
e.g., bread(...) bio_doread(...) getblk()
• return and copy block to user
VOP_READ(vp,len,offset,..)
getblk(vp,blkno,size,...)
![Page 53: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/53.jpg)
ReadO
pera
ting
Sys
tem
A B C D E F G H I J K L
Mgetblk(vp,blkno,size,...)
1. Search for block in buffer cache, return if found(hash vp and blkno and follow linked hash list)
2. Get a new buffer (LRU, age)3. Call disk driver - sleep or do something else
4. Return bufferVOP_STRATEGY(bp)
![Page 54: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/54.jpg)
ReadO
pera
ting
Sys
tem
VOP_STRATEGY(bp)
VOP_STRATEGY(...) is a pointer to the corresponding driver depending on the hardware, e.g., SCSI - sdstrategy(...) sdstart(...)
1. Check buffer parameters, size, blocks, etc.2. Convert to raw block numbers3. Sort requests according to SCAN - disksort_blkno(...)4. Start device and send request
![Page 55: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/55.jpg)
Readfile attributes
...
data pointer
data pointer
data pointer
data pointer
data pointer
...
...Ope
rati
ng S
yste
m
M
![Page 56: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/56.jpg)
ReadO
pera
ting
Sys
tem
A B C D E F G H I J K L
1. Search for block in buffer cache, return if found(hash vp and blkno and follow linked hash list)
2. Get a new buffer (LRU, age)3. Call disk driver - sleep or do something else
4. Return buffer M
M
Interrupt to notify end of disk IOKernel may awaken sleeping process
![Page 57: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/57.jpg)
ReadO
pera
ting
Sys
tem
READ():1. Find corresponding inode2. Check if valid call - file size vs. len + offset 3. Loop and find corresponding blocks
• find logical blocks from inode, offset, length
• do block I/O, e.g., bread(...) bio_doread(...) getblk()
• return and copy block to user
buffer
M
![Page 58: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/58.jpg)
58
Outlook to Research Issues
![Page 59: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/59.jpg)
59
Outlook to Research Issues (cont.)
• Energy aware OS• Impact of memory speed on performance• Video streaming on small devices• Sensor data processing on very small
devices
![Page 60: File Systems](https://reader035.vdocuments.site/reader035/viewer/2022070501/56816923550346895de05a6f/html5/thumbnails/60.jpg)
60
Concluding Questions
• What is the basic layout of FS on disk?• What is the relationship between a directory
and an i-node?• How can we manage free blocks?• What is the difference between i-node and
v-node?