1 file systems guide to operating systems second edition

1

File Systems

Guide to Operating Systems Second Edition

2

Understanding File System Functions• A file system is a design for storing and managing files on

storage media

• The file systems used by personal computer operating systems perform the following general functions:

– Enable files to be organized through directories and folders

– Partition and format disks to store and retrieve information

– Establish file-naming conventions

– Provide utilities for functions such as file compression and disk defragmentation

– Provide for file and data integrity

– Provide storage media management functions

– Enable error recovery or prevention

3

Understanding File System Functions

• The overall purpose of a file system is to create a structure for filing data

• A file is a set of data that is grouped in some logical manner, assigned a name, and stored on the disk

• The data contained in files can be text, images, music and sounds, video, or Web pages for the Internet

• But no matter what kind of data is stored in the file system, it must be converted into digital format—a series of 1s and 0s, or electrical signal “ons” and “offs” —that the computer understands

4


• There must be a way to write digital information onto disk, track it, update it when necessary, and call it back when the user, or a program under the user’s control, wants it

• To achieve all this, the operating system typically groups disk sectors in some logical way, creates a record of this structure, and builds a directory to track the type of data stored in each file

5

A File System

6


• In addition to the names of files and where to find them on the disk, directories also store the following information:

– Date and time the directory or file was created (a timestamp for that directory or file)

– Date and time the directory or file was last modified (another form of timestamp)

– Directory or file size

– Directory of file attributes, such as security information, or if the directory or file was backed up

7

Designing a Directory Structure

• For users, one of the most important features of a file system is the ability to store information according to a pattern of organization that is enabled by the use of directories

• In the early Windows systems and UNIX systems, files are organized by directories, while in later Windows versions and the Mac OS, these are called folders

8


• Directories and folders can be organized in a

hierarchy that is similar to a tree structure

9


• When you design a directory structure consider directories for the following:

– Operating system files

– Software applications

– Work files, such as word processing, graphics, spreadsheets, and database files

– Public files that you share over the network

– Utilities files

– Temporary files

10

Disk Storage

• When a hard disk is delivered from the manufacturer, it is low-level formatted

• A low-level format is a software process that marks the location of disk tracks and sectors

• Every disk is divided into tracks, which are like several circles around a disk

• Each track is divided into sections of equal size called sectors

• Figure 3-4 illustrates a hard disk divided into tracks and sectors

11

Disk Tracks and Sectors

12

Block Allocation

• The operating systems discussed in this book use a method called block allocation to keep track of where specific files are stored on the disk

• With block allocation the disk is divided into logical blocks (in MS-DOS and Windows these are called clusters), which are in turn mapped to sectors, heads, and tracks on the disk

• The data regarding block allocation is stored on the disk itself, using two techniques

13

Block Allocation

• One technique uses a fixed portion of the disk to store this data, such as the FAT (file allocation table) file system used by MS-DOS and supported by all versions of Windows

• The other technique uses various locations on the disk to store a special type of file that is used for directory and file allocation information, such as the New Technology File System (NTFS) and the UNIX file systems

14

Partitions

• Before a file system can be placed on a hard disk, the disk must be partitioned and formatted

• Partitioning is the process of blocking a group of tracks and sectors to be used by a particular file system, such as FAT or NTFS

• After a disk is partitioned, it must be high-level formatted (usually referred to as just formatted) so that the partition contains the disk divisions and patterns needed by a particular operating system to store files

15

Partitions

• Figure 3-5 illustrates a Windows 2000 system that has multiple partitions (FAT, NTFS, and compact disk file system (CDFS)) used for segmenting data on two hard disks and a CD-ROM drive

16

Partitions

• Partitioning can typically be done only on hard disks or large removable disks

• Floppy disks and other low-capacity media do not support partitioning because there simply isn’t enough room on these low-capacity disks to make partitioning practical

• On systems in the IBM/Intel PC hardware architecture, typically there is room to store information for up to four partitions on each disk

17

Partitions

• This area is known as the partition table in MS-DOS, Mac OS, and Windows, and the disk label in UNIX

• In addition to the disk label and partition table, there is another piece of disk reserved, known as the boot block in UNIX, or the Master Boot Record (MBR) in MS-DOS and Windows

• This area hold a tiny program used to begin booting an operating system from a disk

18

The Basic MS-DOS File System

• The first Microsoft file system, called the file allocation table (FAT) file system started with MS-DOS in the early eighties, and later versions of FAT are in many ways similar to the earliest version

• FAT uses a file allocation table to store directory information about files, such as filenames, file attributes, and file location

• MS-DOS versions prior to 4.0 use the FAT12 file system in which the maximum size of a file system is 32 MB

19

The Basic MS-DOS File System

• Beginning with MS-DOS 4.0, the FAT16 file system is used

• Another important characteristic of FAT is its use of “8.3” filenames, which can be up to eight characters long, followed by a period and an extension of three characters, such as Filename.ext

20

Partitioning

• The FAT file system supports two partitions per hard drive, a primary partition and a secondary partition

• The secondary partition may then be divided further into a maximum of three logical drives

• Each of these four possible logical drives can hold an individual MS-DOS file system

• Although 26 drive definitions are technically possible, MS-DOS reserves drive A: and B: for floppy drives, practically limiting the number of hard drives (including logical drives) to 24

21

Partitioning

• Each MS-DOS file system is assigned a letter followed by a colon: A:, B:, C:, and so on through Z:

• Typically, C: is reserved for the first hard disk or removable disk file system (and is normally the system that contains the boot partition)

22

Partitioning

• With a new system called large block allocation (LBA), it is now possible to make file systems much larger than 512 MB under MS-DOS

• LBA translates larger logical blocks to smaller ones for support of larger physical disks

• However, the disk controller and disk must support LBA, and they must be configured for LBA

• The trick to LBA is that MS-DOS is told that the sector size of the hard disk is greater than 512 bytes per sector, which results in the ability to have much larger file systems

23

Formatting

• In MS-DOS (and in many Windows versions), the file system is placed on the partition using the format command

• This command writes all of the file system structure to the disk

• In the case of a floppy disk, it uses the first sector of the disk as the boot block

• As with many system-level commands, format includes several additional switches that modify precise program operation

24

format Command Switches

25

Formatting

• Command lines frequently use switches (extra code) to change the way a particular command operates

• The boot block is placed in the first sector on the disk

• Next comes the root directory, where the system stores file information, such as name, start cluster, file size, file modification date and time, and file attributes (file characteristics such a Hidden, Read-only, Archive, and so on)

26

Formatting

• The root directory on every partition is a fixed size that can contain a maximum of 512 entries in FAT16 (and unlimited entries in FAT32)

• When a file is stored to disk, its data is written in the clusters on the disk

• The filename is stored in the directory, along with the number of the first cluster in which the data is stored

27

Formatting

• When the operating system fills the first cluster, data is written to the next free cluster on the disk

• When a file is completely written to the disk, the FAT entry for the final cluster is filled with all 1s, which means end of file

• At this time, the directory entry for the file is updated with the total file size

• This is commonly referred to as the “linked-list” method

28

Formatting

• Clusters are of a fixed length, and if a file does not exactly match the space available in the clusters it uses, you can end up with some unused space at the end of a cluster

• Unusable or troubled spots on the disk are marked in the FAT as bad clusters, and these areas are never used for file storage

• Formatting a disk removes all data that was on the disk because you lost the directory and FAT data needed to get to the data

29

Formatting

• On disks that have never been formatted, the format command writes new sector and track markers on the disk

• The format process on a hard disk is the same as on a floppy disk, with two exceptions

• The first is related to the size of each entry in the FAT table, which is 16 bits long on any disk larger than 16 MB

• The second difference is in the cluster size, larger partitions use larger cluster sizes

30

Formatting

• The FAT tables and root directory are found at the beginning of each partition, and they are always at the same location

• Each item in a directory consists of 32 bytes

31

Formatting

• In each entry, information about the file is stored, including the filename, the file change date and the file size and the file attributes

• Apart from the filename, each directory entry also contains some status bits that identify the type of filename contained in each entry

• The status bits in use are Volume, Directory, System, Hidden, Read-only, and Archive

• The Volume bit indicates a file system volume label, or a nickname for the file system

32

Formatting

• The attrib command can be used to look at or set these attributes

• Typing attrib in a directory shows all of the attribute settings for all the files, whereas typing attrib followed by a filename shows only attributes specific to that file

33

The Windows 95/98/Me FAT File Systems

• There are two versions of FAT that can be used with Windows 95, 98, and Me, depending on the version of Windows you are running, and the size of the disk you are using:

– FAT16, similar to the system used in MS-DOS/Win 3.x

– FAT32, a new system introduced in Windows 95, release 2 (OSR2)

34

Windows 95/98/Me FAT16

• In Windows 95, there is support for long filenames (LFNs)

• A filename in the Windows 95 file system:

– Can contain as many as 255 characters

– Is not case sensitive

• Since LFN characters are stored in Unicode, a coding system that allows for representation of any character in any language, it is possible to use any character known to Windows in a filename

35


• The advantage of LFNs and Unicode is that LFNs can be read by Mac OS and UNIX systems

• Formerly, letters and digits were represented by ASCII (American Standard Code for Information Interchange) values

• The problem with this standard is that it uses an entire byte to represent each character, which limits the number of characters that can be represented to 255

36


• This is not enough to handle all the characters needed to represent world languages, including several different alphabets (Greek, Russian, Japanese, and Hindi, for example)

• ASCII deals with this problem by employing different character sets, depending on the characters you’re trying to represent

• Unicode is a 16-bit code that allots two bytes for each character, which allows 65,536 characters to be defined

37

FAT32

• To accommodate the increasing capacities of hard disks, and avoid the problem of cluster size, the second release of Windows 95 (OSR2) introduced an improved FAT file system called FAT32

• FAT32 shares characteristics of FAT16 for Windows 95

• FAT32 allows partitions of up to 2 TB (theoretically); blocks can be allocated with clusters as small as 8 KB, and the maximum file size is raised to 4 GB

38

FAT32

• FAT32 works by using many bits per FAT allocation unit, and is therefore incompatible with other operating systems such as MS-DOS and Windows NT

• However, it is compatible with Windows 2000 and XP

• You can choose the FAT32 file system when the Windows 95/98 fdisk utility is run, but unfortunately, the utility does not simply ask if you wish to use FAT32

39

FAT32

• DO NOT use FAT32 if you plan to access your hard disk from MS-DOS or Windows NT because it is not compatible with these systems

40

Windows 95/98/Me File System Utilities

• The fdisk and format utilities in Windows 95, 98, and Me work identically to those in MS-DOS

• There are now also graphical equivalents of these tools that you can launch from the Windows user environment, such as the Windows 95 format tool shown in Figure 3-11

41

The Windows NT/2000/XP/.NET File Systems

• Windows NT supports two types of file

systems: the extended FAT16 system

used by Windows 95, and the NTFS up

through version 4 (for Windows NT 4.0)

42

Extended FAT16

• The extended FAT16 file system under Windows NT/2000/XP/.NET has the same functionality as the FAT16 system under Windows 95, and it is possible to have multiple operating systems on one disk (called dual-boot or multiboot systems) with the FAT16 file system

43

FAT32

• Windows 2000, XP, and .NET Server all support FAT32, which is the same file system that is used in Windows 95 OSR2, Windows 98, and Windows Me

• Sometimes users choose to employ FAT16 or FAT32 because they are familiar with these file systems, or they have dual-boot systems, such as a system with Windows XP and Windows 98

44

NTFS

• Four important advantages of NTFS compared to FAT16 and FAT32 are:

– Ability to compress file and directory contents on the fly

– Better recoverability and stability

– Less disk fragmentation

– Local file and folder-level security

45

NTFS

• NTFS enables the use of LFNs that are compatible with LFN FAT filenames

• If an LFN is used in NTFS, that file can be copied to a FAT16 or FAT32 volume, and the filename remains intact

• As a full-featured network file system, NTFS is equipped with security features that meet the U.S. government’s C2 security specifications

• C2 security refers to high-level, “top secret” standards for data protection, system auditing, and system access, which are required by some government agencies

46

NTFS

• NTFS can be scaled to accommodate very large files, particularly for database applications

• A Microsoft SQL Server database file might be 20 GB or larger

• Another NTFS feature is its ability to keep a log of file system activity

47

NTFS

• An important volume-handling feature of NTFS is the ability to create extensions on an existing volume, such as when new disk storage is added

• Last, NTFS is less prone to file corruption than FAT, in part because it has a “hot fix” capability, which means that if a bad disk area is detected, NTFS automatically copies the information from the bad area to another disk area that is not damaged

48

NTFS

• The way NTFS keeps track of files and clusters is a little different from the FAT file system

• Rather than using a structure of FAT tables and directories, NTFS uses a Master File Table (MFT)

• Like the FAT tables and directories, this table is located at the beginning of the partition

• When a file is made in NTFS, a record for that file is added to the MFT

49

FAT16, FAT32, and NTFS Compared

50

NTFS Utilities

• Care should be taken to use only utilities designed to work with NTFS; serious damage can occur if other utilities are used

• When you upgrade Windows NT Server 4.0 to Windows 2000 or Windows .NET, the current NTFS disk partitions are automatically converted to basic disks, which use traditional NTFS disk management techniques, with a limited number of volumes on one disk

51

CDFS and UDF

• The Universal Disk Format (UDF) file

system is used on CD-ROM and large

capacity Digital Video Disk-Read Only

Memory (DVD-ROM) media, which are

used for huge file storage to

accommodate movies and games

52

The UNIX File System

• The UNIX file system works a little differently from anything discussed up to this point

• “UNIX file system” is really a misnomer

• In Linux, the extended file system (ext or ext fs) is native and installed by default

• Ext provides an advantage over all other file systems that can be used with Linux because it enables the use of the full range of built-in Linux commands, file manipulation, and security

53

File Systems Supported by UNIX

54

File Systems Supported by UNIX

55


• The main difference between native UNIX file systems, such as ext3 and ufs, and those covered earlier lies in the way information is physically stored on the disk

• The ufs UNIX file system (and also ext/ext2/ext3) uses the concept of information nodes, or inodes

56


• Each inode can store some information about a file on the file system

• The information stored on an inode identifies: the inode number, the owner of the file, the group in which the file is placed, the size of the file, the date the file was created, the date the file was last modified and read, the number of links to this inode, and information regarding the location of the blocks in the file system in which the file is stored

57


• The superblock contains information about the layout of blocks, sectors, and cylinder groups on the file system

• This information is the key to finding anything on the file system, and it should never change

• A UNIX system can have many file systems

• Unlike the MS-DOS/Windows environment, where each file system must have a letter of the alphabet assigned to it to enable access, UNIX mounts file systems as a sub-file system of the root file system

58


• There are several other special files in the UNIX file system

• Disks themselves, are, for example, referenced by a special inode called a device

• There are two types of devices, raw devices and block devices

• A raw device has no logical division in blocks, whereas a block device does

59

The Macintosh File System

• The original Macintosh Filing System (MFS) of 1984 was limited to keeping track of 128 documents, applications, or folders

• As larger disks became available, the need for directories and subdirectories became obvious, and Apple responded with the Hierarchical Filing System (HFS) in 1986

• Like FAT16, HFS divides a volume (the Mac term for a disk or disk partition) into, at most, 216

(65,536) units

60


• On PC systems, these units are called clusters or allocation units

• On the Mac, they are called allocation blocks, but the principle is the same

61


• The first two sectors of a Mac-formatted disk are the boot sectors, or boot blocks in Macintosh terminology

• The boot blocks identify the filing system, the names of important system files, and other important information

• The boot blocks are followed by the volume information block, which points to other important areas of information, such as the location of the system files, and the catalog and extents trees

62


• The catalog b-tree is the list of all files on the volume

• The extents b-tree keeps track of the location of the file fragments, or extents

• In terms of filename length, the Mac OS has always supported what might be called “medium” filenames of up to 31 characters in length

• UNIX and Windows operating systems use filename extensions such as .txt and .gif to identify file types

63


• The Mac uses invisible type codes and creator codes

• As an example, files created with Apple’s SimpleText text editor have a type code of TEXT, and a creator code of ttxt

• The type and creator codes facilitate the Mac’s use of icons

• One way in which Macintosh files are unique is that Mac files can contain two parts, or forks: the data fork and the resource fork

64


• The data fork contains frequently changing information (such as word processing data), while the resource fork contains information that is fixed (such as a program’s icons, menu resources, and splash screens)

• Apple’s free ResEdit utility can edit file resources

• Using ResEdit, a programmer can modify a program’s version number, splash screens, default memory allocation, icons, menu items, window resources, dialog text, and many other properties

65


• The fact that Mac files have invisible type and creator codes and two forks can create problems when storing files on non-Macintosh servers, or transferring files over the Internet

66


• The need to store Mac files on non-Mac computers has led to several Mac file formats for bulletin board services, online services, and the Internet

• One of these formats is MacBinary, which joins the two forks into one, and safely stores the type and creator codes and finder flags

67


• Apple’s equivalent of the UNIX link and Windows shortcut is the alias, introduced in System 7.0 in 1991

• Files, folders, applications, and disks can be aliased

68


• The Mac OS ships with two basic disk utilities: Disk First Aid and Drive Setup

• Drive Setup, a replacement for the older Apple HD Setup, formats and partitions Apple IDE and SCSI hard drives

• Drive Setup checks the hard drive for a ROM that is present in drives shipped with Apple computers

• Most versions of the Mac OS include a basic find file utility

1 file systems guide to operating systems second edition

Documents

file systemchapter

file compression

file systemsguide

filing dataa file

storage mediathe file

hard disk

directory structurewhen

directory structuredirectories