Files

CS 470 - Spring 2002

Overview

• Example: FAT File System

• File Organization

• File System Organization– File Directories and File Sharing– Record Blocking– Disk space allocation

• Example: Unix File Systems

Fat Disk StructureBoot Sector

Other Reserved

File AllocationTable (FAT)

File AllocationTable Copy

Root Directory

File and DirectoryArea

Reserved Sectors

Allocation chainsand Free List

Redundant Copy

Fixed size Arrayof root directory entriesAll other directories andfiles are located here

FAT Boot Sector

• Location: First sector of the drive

• Contains:– Jump to first level boot code (byte offset 0)– Disk Parameter Block (byte offset 11) which

describes the drive– Other identifying information like the volume

label (byte offset 43) – First level boot code (typically at offset 0x40)

Disk Parameter Block

• Geometry– BytesPerSector[2] – SectorsPerTrack[2] – Heads[2] – Sectors[2] or

LargeSectors[4]– Media– HiddenSectors[4]

• Structure Sizes– ReservedSectors[2]– NumberOfFats – SectorsPerFat[2] – NbrOfRootEntries[2]– SectorsPerCluster

Field sizes in bytes are indicated in square brackets

FAT Directory Entries

Typedef struct _DIRENT {

UCHAR FileName[11];

UCHAR Attributes;

UCHAR NtByte;

UCHAR

CreationCSec;

FAT_TIME_STAMP

CreationTime;

FAT_DATE

LastAccessDate;

USHORT

ExtendedAttributes;

FAT_TIME_STAMP

LastWriteTime;

USHORT

FirstClusterOfFile;

ULONG FileSize;

} DIRENT;

File Allocation Table

18

0

0xff7

1a

0

0xfff

14

8dot3 Name

. . .

First Cluster (15)

File Size (5a0)

0xfffDirectory Entry 14

16

17

18

19

1aFree Cluster

Bad Cluster

FAT12

FOO BAR. . .1b32f

Dir Entry

• How to translate FAT tables entries to sector offsets in the disk partition

• RootDirSize = NbrRootEntries * sizeof(DIRENT) / SectorSize

• FileAreaOffset = NbrReservedSectors + NbrFat*SectorsPerFat + RootDirSize

• SectorOffset = FileAreaOffset + (FAT_Offset - 2) * SectorsPerCluster

Fat_Offset to SectorOffset

Long Filename Directory Entrytypedef struct _LFN_DIRENT {

UCHAR Ordinal;

UCHAR Name1[10]; // 5 Unicode chars

UCHAR Attributes; // Should be 0x0f

UCHAR Type;

UCHAR CheckSum; // of 8dot3 name

WCHAR Name2[6];

USHORT MustBeZero;

WCHAR Name3[2]; } LFN_DIRENT;

DIRENT with LFN

0x44

Characters 26 to 383

Characters 13 to 252

First 13 Unicode Chars1

Up to 13 Unicode Chars

8dot3 Name

Ordinal

All have sameChecksumField

Calculate rotated checksum from this

File Structure

CS 470 - Spring 2002

Segment II

File Organization• Vocabulary

– Field - basic element of data– Record - collection of related fields– File - collection of similar records

• File Organization Choice Criteria– Rapid access– Ease of updates– Economy of storage– Reliability

Some File Types

• Pile

• Sequential

• Indexed Sequential

• Indexed

• Direct or Hashed

Pile

• Variable length records with variable fields

• Fields must be self describing

• No order to records - exhaustive search

F1 F2 F5

F1 F3

Record 1

Record 2

F4 DataF4 Hdr

F3

Sequential File• Fixed length records of fixed format fixed

length fields. Distinguished key field.

• Records sorted in key field order

• Search is typically sequential

• Insertions are not easy - so usually accumulated in separate overflow file with periodic update of the master file

17 F2 F3 F4 F524 F2 F3 F4 F526 F2 F3 F4 F5

14 F2 F3 F4 F5 Record 1Record 2

Record 3Record 4

Key

Indexed Sequential Files

• Sequential file with index file and overflow file for updates

• Records still in order of key index

• Overflow file also accessible through index - Records have Next_Record field which can point into overflow file.

• Multiple level so indexing can speed lookups

Indexed Sequential File

5

42

35

18

9

11

18

35

42

13

38

15

5Index

OverflowFile

MasterFile

Key

Indexed Files

• Index for each searchable field, but each record must have at least one searchable field

• Records not sorted, no overflow needed

• Allows for variable length fields and records with missing fields

• Exhaustive index - one entry per record, versus Partial index - on entry per record containing the indexed field.

Indexed File

PartialIndex Exhaustive

Index

Indexed File

Hashed Files

• Hashed access with overflow tables to handle collisions

• Usually used with fixed length records

• Very fast access

• No way to sequentially access all records

Hashed File

8

37

91

231

19

54

14

83

24

87

23

43

Overflow Table37

87

23

91

24

14

Hashed File

Pointers calculated via hashing function

File Systems

CS 470 - Spring 2002

Segment III

Directories

• Contents– Name– File type and organization– Address: Size, location– Access control information– Usage Information - times, current usage

• Structure - e.g. directory tree

• Operations: Search, Create file, Delete file, and List Directory

File Sharing

• Access Rights - Allow various users or groups to share files

• Simultaneous Access control - Allow multiple threads to share the same file at the same time

Record Blocking

• Three approaches– Fixed Blocking– Variable length records with records

restricted to a single block - no spanning– Variable length records which can overlap

(span) two blocks

• First two may have internal fragmentation

• Third is hard to update

Record Blocking Methods

Rec 1 Rec 2 Rec 3

Rec 4 Rec 5 Rec 6

Rec 1 Rec 2 Rec 3 Rec 4

Rec 5 Rec 6 Rec 6 R7

Rec 3 Rec 4 Rec 5

Rec 1 Rec 2

Rec 4

Block 1

Block 2

Block 1

Block 2

Block 1

Block 2

SpanningRecords

Fragment

Disk Space Allocation (1 of 2)• Recorded in File Allocation Table

• Pre-allocation (at file creation) versus dynamic allocation of space

• Allocation size - variable or fixed, large or small, contiguous or not

• Allocation Methods– Contiguous - just need address and total

size– Chained - pointer to next block– Indexed - index refers to file portions

Allocation Methods

0 UU213

0 21

3 543

21

0 12

5

Indexed

Chained

Contiguous

Disk Space Allocation (2 of 2)

• Free Space Management (Disk Allocation Table)– Bit maps of Free Blocks– Chained Free Portions– Index of Free Portions

• Reliability Problems– Caused by improper shutdown when File

and Disk allocation tables kept in memory– Locking is often too expensive

Free List Management

0

14 42205

432 4344333

321

51 522

1

Indexed

Chained

Bit Map 1 1 0 0 1 0 1 1 1 1

Free Allocated

Free

Unix File System

CS 470 - Spring 2002

Segment IV

Classic Unix File System

Superblock

InodeTable

Files andDirectories

Superblock

Free Inode List

Free Block List

InodeTable

BufferCache

Main MemoryDisk Drive

FileTable

V6 Unix Inode Entries

• i_mode - type (dir, char or block special, or ordinary) and file permissions

• i_nlink - number of names (hard links)

• i_uid, i_gid - user and group owners

• i_size0, i_size1 - size of file in bytes

• i_ addr[8] - location of file on disk

• i_atime, i_mtime - access and modification times

Classic V7 Unix Inode

Rest of inode

256 More Block Nbrs

10 Direct Block Nbrs

256 More Indirect Block Nbrs

256 Block Nbrs 255 More of these

Block number ofTriple Indirect Block

Inode

File location part of inode

Double IndirectBlock

Single Indirect Block

Unix File Allocation Limits

Level Nbr of Blks Nbr of Bytes

Direct 10 10 KB

Single Indirect 256 256 KB

Double Indirect 256 x 256 64 MB

= 64 K

Triple Indirect 256 x 64 K 16 GB

= 16 M

Classic Unix Free Lists

Superblock Free Block List

255 Free Block Nbrs

255 Free Block Nbrs

Superblock Free Disk Inode List

One per diskPartition

One per diskPartition

Summary

• File Organization

• File System Organization– File Directories– File Sharing– Record Blocking– Disk space allocation

• Examples: Unix and FAT File Systems