Advanced Operating Systems - Spring 2009Lecture 19 – Monday March 30, 2009Dan C. Marinescu

Email: dcm@cs.ucf.eduOffice: HEC 439 B. Office hours: M, Wd 3 – 4:30 PM.

TA: Chen YuEmail: yuchen@cs.ucf.eduOffice: HEC 354. Office hours: M, Wd 1.00 – 3:00 PM.

1

Last, Current, Next Lecture Last time:

Caching Introduction to I/O Subsystem

Today I/O subsystem File System Implementation

Next time: File System Interface and Mass Storage Structure

2

Communication among asynchronous sub-systems: polling versus interruptsPolling periodically checking the status of an I/O device Interrupt deliver data or status information when status

information immediately . Intel Pentium Vector Table

3

Interrupts: used for I/O and for exceptionsCPU Interrupt-request line triggered by I/O device Interrupt handler receives interruptsTo mask an interrupt ignore or delay some interrupts Interrupt vector to dispatch interrupt to correct handler

Based on prioritySome non-maskable

4

5

DMA Bypasses CPU to transfer data directly between I/O device and memory; it allows subsystems within the computer to access system memory for reading and/or writing independently of CPU: disk controller, graphics cards, network cards, sound cards, GPUs (graphics processors),   also used for intra-chip data transfer in multi-core processors,. 

Avoids programmed I/O for large data movement Requires DMA controller

6

Device drivers and I/O system calls Multitude of I/O devices

Character-stream or block Sequential or random-

access Sharable or dedicated Speed of operation Read-write, read only, or

write onlyDevice-driver layer hides

differences among I/O controllers from kernel:

I/O system calls encapsulate device behaviors in generic classes

7

Block and Character Devices

Block devices (e.g., disk drives, tapes) Commands e.g., read, write, seek Raw I/O or file-system access Memory-mapped file access possible

Character devices (e.g., keyboards, mice, serial ports) Commands e.g., get, put Libraries allow line editing

8

Network Devices and Timers

Network devices Own interface different from bloc or character devices Unix and Windows NT/9x/2000 include socket interface

Separates network protocol from network operation Includes select functionality

Approaches vary widely (pipes, FIFOs, streams, queues, mailboxes)

Timers Provide current time, elapsed time, timer Programmable interval timer for timings, periodic interrupts ioctl (on UNIX) covers odd aspects of I/O such as clocks and

timers

9

Blocking and Nonblocking I/OBlocking process suspended until I/O completed

Easy to use and understand Insufficient for some needs

Nonblocking I/O call returns control to the process immediately User interface, data copy (buffered I/O) Implemented via multi-threading Returns quickly with count of bytes read or written

Asynchronous process runs while I/O executes I/O subsystem signals process when I/O completed

10

Synchronous Asynchronous

11

Kernel I/O Subsystem

SchedulingSome I/O request ordering using per-device queueSome OSs try fairness

Buffering – store data in memory while transferring to I/O device.To cope with device speed mismatch or transfer size

mismatchTo maintain “copy semantics”

12

13

Kernel I/O Subsystem and Error HandlingCaching fast memory holding copy of data

Always just a copyKey to performance

Spooling holds output for a device that can serve only one request at a time (e.g., printer).

Device reservation provides exclusive access to a deviceSystem calls for allocation and de-allocationPossibility of deadlock

Error handling: OS can recover from disk read, device unavailable, transient

write failures When I/O request fails error code. System error logs hold problem reports

14

I/O Protection I/O instructions are priviledgedUsers make system calls

15

Kernel Data StructuresKernel keeps state info for I/O components, including open file

tables, network connections, device control blocs

Complex data structures to track buffers, memory allocation, “dirty” blocks

Some use object-oriented methods and message passing to implement I/O

16

17

Hardware OperationsOperation for reading a file:

Determine device holding file Translate name to device representation Physically read data from disk into buffer Make data available to the process Return control to process

18

STREAMS in UnixSTREAM a full-duplex communication channel between a

user-level process and a device in Unix System V and beyond

A STREAM consists of:- STREAM head interfaces with the user process- driver end interfaces with the device- zero or more STREAM modules between them.

Each module contains a read queue and a write queueMessage passing is used to communicate between queues

19

20

I/O major factor in system performance:Execute

device driver, kernel I/O codeContext switches

Data copyingNetwork traffic stressful

21

Improving Performance

Reduce number of context switchesReduce data copying Reduce interrupts by using large transfers, smart

controllers, polling Use DMABalance CPU, memory, bus, and I/O performance for

highest throughput

22

File System

Basic ConceptsAllocation MethodsFree-Space Management Efficiency and PerformanceFault tolerance and recoveryNFS

23

Basic conceptsFile Collection of related information residing on

secondary storage.File control block data structure summarizing

information about a file

24

25

Virtual File Systems (VFS)Support an object-oriented implementation of a file

systems.The VFS API used for different types of file systems.

26

Directory ImplementationDirectory provides information about files on a system

or on a physical device.

Linear list of file names with pointer to the data blocks.simple to programtime-consuming to execute

Hash Table – linear list with hash data structure.decreases directory search timecollisions – situations where two file names hash to the

same locationfixed size

27

Physical layout of a fileAn allocation method refers to how disk blocks are allocated for files:Contiguous allocation Each file occupies a set of contiguous blocks on the

disk Simple – only starting location (block #) and length (number of blocks)

required Random access Wasteful of space (dynamic storage-allocation problem) Files cannot grow

Linked allocation Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk. Simple – need only starting address Free-space management system – no waste of space No random access Mapping

Indexed allocation Brings all pointers together into the index block.

pointerblock =

28

29

Mapping from logical to physical

Block to be accessed = ! + starting addressDisplacement into block = R

LA/512

Q

R

Extent-Based Systems

Extent a contiguous block of disks. Extent-based file systems allocate disk blocks in extents.

A file consists of one or more extents.Many newer file systems (I.e. Veritas File System) use a

modified contiguous allocation scheme

30

Linked Allocation

Block to be accessed is the Qth block in the linked chain of blocks representing the file.Displacement into block = R + 1

File-allocation table (FAT) – disk-space allocation used by MS-DOS and OS/2.

LA/511Q

R

Maping

31

index table

32

Indexed AllocationNeed index tableRandom accessDynamic access without external fragmentation, but

have overhead of index block.Mapping from logical to physical in a file of maximum

size of 256K words and block size of 512 words. We need only 1 block for index table.

LA/512Q

R

Q = displacement into index tableR = displacement into block

33

Indexed Allocation – Mapping Mapping from logical to physical in a file of

unbounded length (block size of 512 words).Linked scheme – Link blocks of index table (no

limit on size).

LA / (512 x 511)Q1

R1

Q1 = block of index tableR1 is used as follows:

R1 / 512Q2

R2

Q2 = displacement into block of index tableR2 displacement into block of file:

34

Indexed Allocation – Mapping (Cont.)Two-level index (maximum file size is 5123)

LA / (512 x 512)Q1

R1

Q1 = displacement into outer-indexR1 is used as follows:

R1 / 512Q2

R2

Q2 = displacement into block of index tableR2 displacement into block of file:

35

Indexed Allocation – Mapping (Cont.)

outer-index

index table file

36

37

Free-Space ManagementBit vector (n blocks)

…

0 1 2 n-1

bit[i] = 0 block[i] free

1 block[i] occupied

Block number calculation

(number of bits per word) *(number of 0-value words) +offset of first 1 bit

38

Free-Space Management (Cont.)Bit map requires extra space

Example:block size = 212 bytesdisk size = 230 bytes (1 gigabyte)n = 230/212 = 218 bits (or 32K bytes)

Easy to get contiguous files Linked list (free list)

Cannot get contiguous space easilyNo waste of space

Grouping Counting

39

Free-Space Management (Cont.)Need to protect:

Pointer to free listBit map

Must be kept on disk Copy in memory and disk may differ Cannot allow for block[i] to have a

situation where bit[i] = 1 in memory and bit[i] = 0 on disk

Solution: Set bit[i] = 1 in disk Allocate block[i] Set bit[i] = 1 in memory

40

Efficiency and Performance

Efficiency dependent on:disk allocation and directory algorithmstypes of data kept in file’s directory entry

Performancedisk cache – separate section of main memory for frequently

used blocks free-behind and read-ahead – techniques to optimize

sequential access improve PC performance by dedicating section of memory as

virtual disk, or RAM disk

41

Caching for regular I/O operations and memory mapped I/O

A unified buffer cache uses the same page cache to cache both memory-mapped pages and ordinary file system I/O

42

A page cache caches pages rather than disk blocks using virtual memory techniques: used by memory-mapped I/O Routine I/O through the file system uses the buffer (disk) cache

I/O without a unified buffer cacheI/O with a unified buffer cache

Fault -toleranceRecovery

Consistency checking – compares data in directory structure with data blocks on disk, and tries to fix inconsistencies

Backup/Restore Use system programs to back up data from disk to another storage device

Log Structured File Systems All transactions are written to a log A transaction is considered committed once it is written to

the logHowever, the file system may not yet be updated The transactions in the log are asynchronously written to the file

system When the file system is modified, the transaction is removed from

the log If the file system crashes, all remaining transactions in the log must

still be performed43