silberschatz, galvin and gagne 2002 12.1 operating system concepts chapter 12: file system...
Post on 20-Dec-2015
235 views
TRANSCRIPT
Silberschatz, Galvin and Gagne 200212.1Operating System Concepts
Chapter 12: File System Implementation
File System Structure File System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency and Performance Recovery Log-Structured File Systems - omit for now NFS - omit for now
Annotated for use with Silberschatz’s book used during CS350, Fall 03
Instructor's annotation in blue Last updated 12/4/03+
Silberschatz, Galvin and Gagne 200212.2Operating System Concepts
File-System Structure
File structure Logical storage unit Collection of related information
File system resides on secondary storage (disks). Design problem:
How should the file system look to the user … attributes, operations, directory structure
Creating algorithms and data structures to map the logical file system to physical secondary storage device.
File system organized into layers. File control block (FCB) – storage structure consisting of information
about a file – stored on disk, copied to memory for use.
Silberschatz, Galvin and Gagne 200212.3Operating System Concepts
Manages file attribute/control infoManages directory structure: symbolic file names to logical block#,Uses File Control Block (FCB)
Logical block# in physical block# out, (knows location of file and allocation scheme used)
Generic cmds to dev drivers using disk addresses:cylinder, track, sector
Device drivers: interfaces to hardware/controllers, gets generic commands.
Hardware controllers
Layered File System
Silberschatz, Galvin and Gagne 200212.4Operating System Concepts
A Typical File Control Block
Location on disk
Silberschatz, Galvin and Gagne 200212.5Operating System Concepts
Overview On-disk structures
Boot control block – 1st block of partition - used for booting if this is a bootable partition. May also have information on another partition which may be where booting may start (the Master Boot Record, MBR).
Partition control block- attributes of partition, FCB pointers, number blocks in partition, free block count, free block pointers, etc.
Directory structure The FCB’s - (inode in UNIX)
In-Memory structures In memory copy of partition table Cache of recently accesses directories System-wide open file table copy of FCBs for all open files +more Per-process open file table (pointers to system-wide open file table) –
accessed via file descriptor returned from open call – reduces search time
Silberschatz, Galvin and Gagne 200212.6Operating System Concepts
In-Memory File System StructuresThe following example (fig. 12.3) illustrates the necessary file system structures provided by the operating systemsSee Sect. 12.2.1 for scenario description
opening a file
reading a file
Silberschatz, Galvin and Gagne 200212.7Operating System Concepts
Scenario for file operationssee p. 415 of text
Creating a file Allocate a new FCB Read appropriate directory into memory, and add the new file
name and FCB to it Write it back to disk
Opening a file Directory structure searched – parts cached un memory FCB copied to System-wide-open file table File descriptor returned which indexes into the per-process-
open file table. Reading a file
Using file descriptor, access the per processes open file table Which in turn gets the FCB from system wide open file table FCB has information needed to access the data in the file
Silberschatz, Galvin and Gagne 200212.8Operating System Concepts
Directory Implementation
Must search director for a required file
Linear list of file names with pointer to the data blocks. simple to program time-consuming to execute
Hash Table – linear list with hash data structure. decreases directory search time collisions – situations where two file names hash to the
same location fixed size
Silberschatz, Galvin and Gagne 200212.9Operating System Concepts
File system allocation(new slide)
Many files of variable size stored on the same disk
Main problem is how to allocate disk blocks for these files so that disk space is utilized effectively and files can be accessed quickly.
Three major allocations methods: contiguous Linked indexed
Silberschatz, Galvin and Gagne 200212.10Operating System Concepts
Contiguous Allocation
Each file occupies a set of contiguous blocks on the disk.
Minimal head movement – high performance(minimal head movement) - used by IBM VM system
Simple – only starting location (block #) and length (number of blocks) are required.
Random access – for file stating at block b, a block in that file at i blocks into the file would be at block b+i
Good for sequential access also Wasteful of space– external fragmentation – same problem as in dynamic
contiguous memory management (ch 9) – with same solutions.
Files cannot grow – if we over-allocate then internal fragmentation. Difficult to estimate how much space is needed for file
Modified contiguous allocation – give “extents” to make file grow – just postpones the original problem
Silberschatz, Galvin and Gagne 200212.11Operating System Concepts
Contiguous Allocation of Disk Space
Start pointes to 1st block In file
Length is the number of blocks in file
Silberschatz, Galvin and Gagne 200212.12Operating System Concepts
Extent-Based Systems
Many newer file systems (I.e. Veritas File System) use a modified contiguous allocation scheme.
Extent-based file systems allocate disk blocks in extents.
An extent is a contiguous block of disks. Extents are allocated for file allocation. A file consists of one or more extents.
–solves growth problem, but just postpones the original problem- what should be the size of the extent: too large – internal fragmentation, too small: external fragmentation.
Silberschatz, Galvin and Gagne 200212.13Operating System Concepts
Linked Allocation Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk.
Pointer is to next block – file is treated like a linked list
Sequentially follow pointers to access a given block
pointerblock =
Silberschatz, Galvin and Gagne 200212.14Operating System Concepts
Linked Allocation (Cont.)
Simple – need only starting address Free-space management system – no waste of space No random access
File-allocation table (FAT) – disk-space allocation used by MS-DOS and OS/2. – Combines linked with indexed (see later)
Incorrect block location calculations removed
Silberschatz, Galvin and Gagne 200212.15Operating System Concepts
Linked Allocation (Cont.)
Solves external, internal (minimal) fragmentation problem, and growth problem – only limit is free space pool(as paging did in memory management, but no random access)
Only sequential access –direct access support poor (must be “simulated” )
Access is slow and variable because of having to read all pointers in blocks before the one desired.
Each block has wasted space for pointer – minimized by clustering blocks – but this may result in increased internal fragmentation
Reliability a problem: lost or damaged pointers
Silberschatz, Galvin and Gagne 200212.17Operating System Concepts
Indexed Allocation Brings all pointers together into the index block. One “logical” index block per file … in general may have multiple linked index blocks – see
later. Logical view.
index table
Solves internal, external fragmentation, and growth problem In addition allows direct/random accessIn simplest form, size of index block determines number of blocks in fileIf index block too large, then wasted space for small files.If index block not large enough then need to link index blocks together – see later.
Silberschatz, Galvin and Gagne 200212.19Operating System Concepts
Indexed Allocation (Cont.)
Need index table (ie., block) Random access Dynamic access without external fragmentation, but have
overhead of index block. Index block permanently stored on disk, but may be
cached in memory – to further improve disk performce. If cached, we have the old problem of keeping the disk updated – especially if we have a crash.
Incorrect block location calculations removed
Silberschatz, Galvin and Gagne 200212.20Operating System Concepts
Indexed Allocation – Mapping (Cont.)(new slide)
Problem is when the file so large that there are not enough entries in a single index block.
Want to map from logical to physical in a file of unbounded length Use a scheme in which the index blocks are themselves linked:
Link blocks of index table (no limit on size).
The last entry of a linked block of an index table is a pointer to the next index table
Now no logical limit on the size of a file Multilevel index block is another approach to this problem – see
below: Can combine linked index block idea with multi-level: UNIX
I-node scheme –see below
Silberschatz, Galvin and Gagne 200212.21Operating System Concepts
Indexed Allocation – Mapping (Cont.) Multi-level Index blocks - See text
outer-index
index table file
Silberschatz, Galvin and Gagne 200212.22Operating System Concepts
File-Allocation Table Used for windows/OS/2 PC systems Combines linked and indexed concepts
===> See Instructor’s notes: “Comments on File System Implementation” For
detailed description – available on CS350 website
Silberschatz, Galvin and Gagne 200212.23Operating System Concepts
Combined Scheme: UNIX (4K bytes per block)See text
The UNIX Inode
Performance tradeoff favors small files
Silberschatz, Galvin and Gagne 200212.24Operating System Concepts
Free-Space Management See instructors notes: “Comments on File System
Implementation” from the web site. Bit vector (n blocks)
…
0 1 2 n-1
bit[i] = 0 block[i] free
1 block[i] occupied
A bit for every block on disk – offset into map is block number
Block number calculation (moving from left to right):
(number of bits per word) *(number of 0-value words) +offset of first 1 bit in 1st nonzero wordNote:“number of 0-value words” = # of leading all zero words
Silberschatz, Galvin and Gagne 200212.25Operating System Concepts
Free-Space Management (Cont.) Bit map requires extra space. Example:
block size = 212 bytes - 4K page (common)disk size = 230 bytes (1 gigabyte) - on the small siden = 230/212 = 218 bits (or 32K bytes)a 1.3 GB with 512byte page has bit map of 332KB - stresses memory if cached But easy to get contiguous files
Linked list (free list) Link together all free disk blocks. Keep pointer to 1st free block in a special location & cache it Cannot get contiguous space easily No waste of space
Grouping – modification of free-list approach – an indexed approach 1st block in partition (an index block) contains pointers to n free data blocks Last block in the above n free blocks is another index block pointing to another n free blocks
with the last of these being another pointer block, etc. Similar the the linked index block approach for file allocation. Advantage: the addresses of a large number of free blocks can be found quickly
Counting Keep track of groups of contiguous blocks. Just need address of first block in group & number of free contiguous blocks following – similar
to extents Shorter index list
Silberschatz, Galvin and Gagne 200212.26Operating System Concepts
Free-Space Management (Cont.)
Need to protect: Pointer to free list Bit map
Must be kept on disk Copy in memory and disk may differ. Cannot allow for block[i] to have a situation where bit[i] =
1 in memory and bit[i] = 0 on disk. Solution - keep memory copy & disk copy in synch:
Set bit[i] = 1 in disk. Allocate block[i] Set bit[i] = 1 in memory
Silberschatz, Galvin and Gagne 200212.27Operating System Concepts
Linked Free Space List on DiskFree (unused) blocks physicallyLinked together.
Pointer to 1st block may be cachedin Memory
The FAT method has free spaceManagement built into the access scheme.
The text unjustifiably criticizes thismethod as inefficient due to the need for transverse the list. We should never have to transverse thelist. As long as we have a pointer tothe first block in the list, we need only to access the first block: Takethe first block if you need a new block, or add a free block the head of the list.
Silberschatz, Galvin and Gagne 200212.28Operating System Concepts
Efficiency and Performance Efficiency dependent on:
disk allocation and directory algorithms types of data kept in file’s directory entry - maintaining “statistics” in
the directory - see p. 390? UNIX: vary cluster size as file grows (minimizes internal
fragmentation) UNIX: Pre-allocates inodes on a disk to distribute this structure,
wastes some space but improves performance - see below
Performance disk cache – separate section of main memory for frequently used blocks free-behind and read-ahead – techniques to optimize sequential access improve PC performance by dedicating section of memory as virtual disk,
or RAM disk. History: was frequently done on PCs when all you had was a floppy.
UNIX: distribute inodes across partition - keep data blocks near files inode - minimize seeks
Silberschatz, Galvin and Gagne 200212.29Operating System Concepts
Various Disk-Caching Locations
Track buffer: page 434
• Note the difference between a RAM disk and disk cache:• Contents of RAM disk under user control - same as a real disk• Contents of disk cache under control of OS
• cache disk referenced blocks on assumption that they will be needed again (temporal locality)• Use I/O protocol for disk cache
Disk Controller
Silberschatz, Galvin and Gagne 200212.30Operating System Concepts
Page Cache A page cache caches pages rather than disk blocks using virtual
memory techniques. More efficient that caching disk blocks (disk cache) - higher
performance Use memory management protocol for page cache – uses
memory calls or instructions.
Memory-mapped I/O uses a page cache.
Routine I/O (I/O reads and writes) through the file system uses the buffer (disk) cache.
Problem: what if you want to do memory mapped “paged” caching if the underlying architecture uses ordinary I/O?
This leads to the following figures.
Silberschatz, Galvin and Gagne 200212.31Operating System Concepts
I/O Without a Unified Buffer Cache
Fig. 12.11p. 435
File I/O interfaceUse read/write calls
Because the virtualMemory cannot Interface with theBuffer cache, the Data in buffer cacheIs copied to pageCache – double Caching - inefficient
Virtual MemoryInterface: use memory references
Page caching mappedonto buffer caching
pagesDiskblocks
Silberschatz, Galvin and Gagne 200212.32Operating System Concepts
I/O Using a Unified Buffer CacheA unified buffer cache uses the same page cache to cache both memory-mapped pages and ordinary file system I/O:
page cache?
Only a single Buffer needed – More efficient.
Silberschatz, Galvin and Gagne 200212.33Operating System Concepts
Recovery
“lazy” disk update may be used - can leave the disk in an inconsistent state if a crash ocurrs before a disk update is made.
Consistency checking – compares data in directory structure with data blocks on disk, and tries to fix inconsistencies - a scandisk function?
Use system programs to back up data from disk to another storage device (floppy disk, magnetic tape).
Recover lost file or disk by restoring data from backup Ex. The Master Boot Record is backed up - better to
resore an obsolete version than none at all.
Silberschatz, Galvin and Gagne 200212.34Operating System Concepts
Log Structured File Systems <<omit>>
Log structured (or journaling) file systems record each update to the file system as a transaction.
All transactions are written to a log. A transaction is considered committed once it is written to the log. However, the file system may not yet be updated.
The transactions in the log are asynchronously written to the file system. When the file system is modified, the transaction is removed from the log.
If the file system crashes, all remaining transactions in the log must still be performed.
Silberschatz, Galvin and Gagne 200212.35Operating System Concepts
Network File System - NFS: Overview NFS views a set of interconnected machines as
having independent file system. Goal: allow sharing of file systems omong machines
on the network in a transparent manner. Sharing is not mandatory – it is upon request of a machine.
Sharing is based on a client/server relationship. Sharing is allowed between any pair of machines
raher than only with a dedicated server. The file mounting protocol is used to make a remote
directory accessible in a transparent manner.
Silberschatz, Galvin and Gagne 200212.36Operating System Concepts
The Sun Network File System (NFS)Omit NFS slides for now
An implementation and a specification of a software system for accessing remote files across LANs (or WANs).
The implementation is part of the Solaris and SunOS operating systems running on Sun workstations using an unreliable datagram protocol (UDP/IP protocol and Ethernet.
Silberschatz, Galvin and Gagne 200212.37Operating System Concepts
NFS (Cont.) Omit NFS slides for now
Interconnected workstations viewed as a set of independent machines with independent file systems, which allows sharing among these file systems in a transparent manner. A remote directory is mounted over a local file system
directory. The mounted directory looks like an integral subtree of the local file system, replacing the subtree descending from the local directory.
Specification of the remote directory for the mount operation is nontransparent; the host name of the remote directory has to be provided. Files in the remote directory can then be accessed in a transparent manner.
Subject to access-rights accreditation, potentially any file system (or directory within a file system), can be mounted remotely on top of any local directory.
Silberschatz, Galvin and Gagne 200212.38Operating System Concepts
NFS (Cont.) Omit NFS slides for now
NFS is designed to operate in a heterogeneous environment of different machines, operating systems, and network architectures; the NFS specifications independent of these media.
This independence is achieved through the use of RPC primitives built on top of an External Data Representation (XDR) protocol used between two implementation-independent interfaces.
The NFS specification distinguishes between the services provided by a mount mechanism and the actual remote-file-access services.
Silberschatz, Galvin and Gagne 200212.39Operating System Concepts
Three Independent File Systems Omit NFS slides for now
Silberschatz, Galvin and Gagne 200212.40Operating System Concepts
Mounting in NFS Omit NFS slides for now
Mounts Cascading mounts
Silberschatz, Galvin and Gagne 200212.41Operating System Concepts
NFS Mount Protocol Omit NFS slides for now
Establishes initial logical connection between server and client.
Mount operation includes name of remote directory to be mounted and name of server machine storing it. Mount request is mapped to corresponding RPC and forwarded
to mount server running on server machine. Export list – specifies local file systems that server exports for
mounting, along with names of machines that are permitted to mount them.
Following a mount request that conforms to its export list, the server returns a file handle—a key for further accesses.
File handle – a file-system identifier, and an inode number to identify the mounted directory within the exported file system.
The mount operation changes only the user’s view and does not affect the server side.
Silberschatz, Galvin and Gagne 200212.42Operating System Concepts
NFS Protocol Omit NFS slides for now
Provides a set of remote procedure calls for remote file operations. The procedures support the following operations: searching for a file within a directory reading a set of directory entries manipulating links and directories accessing file attributes reading and writing files
NFS servers are stateless; each request has to provide a full set of arguments.
Modified data must be committed to the server’s disk before results are returned to the client (lose advantages of caching).
The NFS protocol does not provide concurrency-control mechanisms.
Silberschatz, Galvin and Gagne 200212.43Operating System Concepts
Three Major Layers of NFS Architecture Omit NFS slides for now
UNIX file-system interface (based on the open, read, write, and close calls, and file descriptors).
Virtual File System (VFS) layer – distinguishes local files from remote ones, and local files are further distinguished according to their file-system types. The VFS activates file-system-specific operations to handle
local requests according to their file-system types. Calls the NFS protocol procedures for remote requests.
NFS service layer – bottom layer of the architecture; implements the NFS protocol.
Silberschatz, Galvin and Gagne 200212.44Operating System Concepts
Schematic View of NFS Architecture Omit NFS slides for now
Silberschatz, Galvin and Gagne 200212.45Operating System Concepts
NFS Path-Name Translation Omit NFS slides for now
Performed by breaking the path into component names and performing a separate NFS lookup call for every pair of component name and directory vnode.
To make lookup faster, a directory name lookup cache on the client’s side holds the vnodes for remote directory names.
Silberschatz, Galvin and Gagne 200212.46Operating System Concepts
NFS Remote Operations Omit NFS slides for now
Nearly one-to-one correspondence between regular UNIX system calls and the NFS protocol RPCs (except opening and closing files).
NFS adheres to the remote-service paradigm, but employs buffering and caching techniques for the sake of performance.
File-blocks cache – when a file is opened, the kernel checks with the remote server whether to fetch or revalidate the cached attributes. Cached file blocks are used only if the corresponding cached attributes are up to date.
File-attribute cache – the attribute cache is updated whenever new attributes arrive from the server.
Clients do not free delayed-write blocks until the server confirms that the data have been written to disk.