file systems and mass storage sorin manolache [email protected]
Post on 21-Dec-2015
222 views
TRANSCRIPT
2S. Manolache, Process programming and operating systems, File systems and mass storage
Last on TTIT61
BindingCompile time, load time, execution time
SwappingContiguous memory allocation
External fragmentationPaging
Internal fragmentation, sharing, protectionSegmentation
External fragmentation, sharing, protectionVirtual memory
Page replacementThrashing
3S. Manolache, Process programming and operating systems, File systems and mass storage
Lecture Plan
1. What is an operating system? What are its functions? Basics of computer architectures. (Part I of the textbook)
2. Processes, threads, schedulers (Part II , chap. IV-VI)
3. Synchronisation (Part II, chap. VII)
4. Primary memory management. (Part III, chap. IX, X)
5. File systems and secondary memory management (Part III, chap. XI, XII, Part IV)
6. Security (Part VI)
4S. Manolache, Process programming and operating systems, File systems and mass storage
Outline
The concept of fileOperations on filesAccess methods
DirectoriesOperations on directoriesDirectory hierarchies
File sharingProtectionDisk scheduling
5S. Manolache, Process programming and operating systems, File systems and mass storage
Files
Named collection of related information that is stored on secondary storage
Smallest allotment of logical secondary storage (when we want to store something on the secondary storage, we store it in files)
Format of files is typically defined by the creator
6S. Manolache, Process programming and operating systems, File systems and mass storage
File Attributes
Name (identifier for human use) Identifier (typically a numeric identifier for the internal use of
the OS)Type (for OS that support file types)SizeTime, date, user identification (last access, last modification,
creation)Location (on the device)Protection (permissions to read/write/execute/etc.)
7S. Manolache, Process programming and operating systems, File systems and mass storage
Operations on Files
CreationNameProtection information
DeletionName
WritingReadingTruncatingRepositioning within a file
8S. Manolache, Process programming and operating systems, File systems and mass storage
Opening a File If the file to be read from (written to) was specified by its
name to the read (write) system callsThe OS would have to lookup the disk blocks
corresponding to the named file for each system call invocation performance penalty
Most OS require the user to perform an open system call thatMaps the file name to an identifier Initialises a memory structure with the disk location of the
opened file and other data (see slides)
Implicit opening: automatically open at first access, close at process exit
9S. Manolache, Process programming and operating systems, File systems and mass storage
Disks
Cylinder
Heads
10S. Manolache, Process programming and operating systems, File systems and mass storage
Disk Organisation
Tracks
Gap
Sector
11S. Manolache, Process programming and operating systems, File systems and mass storage
Disk Organisation
The geometry of disks is given in C/H/S (Cylinders/Heads/Sectors)
Initially used to corresponded to the true physical geometry
The access granularity is the physical block (sector)The operating system maps logical records on physical
blocks
Disk space is always allocated in blocksFiles may not have a size equal to an integer multiple of
the block size last block not fully used Internal fragmentation
12S. Manolache, Process programming and operating systems, File systems and mass storage
Access Methods
Direct accessSequential access Indexed access
13S. Manolache, Process programming and operating systems, File systems and mass storage
Direct Access
Direct accessRead (write) system call do specify the relative block
number from where to read (where to write to)E.g.
write(fd, buf, sizeof(buf), 30); -- writes sizeof(buf) bytes from the buffer buf to the 30th block of the file identified by fd.
14S. Manolache, Process programming and operating systems, File systems and mass storage
Sequential Access
Sequential accessThe block from where to read (where to write) is not
specified. The OS keeps a file pointer that it modifies accordingly.
A read (write) operation reads (writes) data from the current file offset, stored in the file pointer
After the read or write, the value of the file pointer is incremented with the amount of transferred records.
E.g.: write(fd, buf, sizeof(buf)) – write sizeof(buf) bytes
from the buffer buf to the file identified by fd at the current file offset. Increment the file pointer with sizeof(buf)
15S. Manolache, Process programming and operating systems, File systems and mass storage
Indexed Access
143520, 1311607896, 1312853661, 1400
143520, $10245679, $30509877, $5510978, $15
Block 1311
607896, $20610942, $10790134, $15829842, $8
Block 1312
853661, $10898541, $20934625, $6973147, $7
Block 1400
Index file
File
16S. Manolache, Process programming and operating systems, File systems and mass storage
Writing to Files in Unix
WritingWhich file to write toWhat to write
The write system call does not specify a file offset (a write pointer pointing at the position in the file where the writing should begin)
Sequential access
17S. Manolache, Process programming and operating systems, File systems and mass storage
Reading from Files in Unix
ReadingWhich file to read fromHow much to readWhere to put what we read
The read system call does not specify a file offset (a read pointer pointing at the position in the file where the reading should begin)
18S. Manolache, Process programming and operating systems, File systems and mass storage
Read/Write Pointers
Typically, a file is used either for reading or for writing by a process
The OS keeps just one single file pointer for both reading and writing
19S. Manolache, Process programming and operating systems, File systems and mass storage
File Operations in Unix
int creat(const char *name, int permissions) int open(const char *name, int flags) int read(int fd, char *buffer, int requested_size); int write(int fd, const char *buffer, int size); int close(int fd); long lseek(int fd, long offset, int whence); int stat(const char *name, struct stat *status);
20S. Manolache, Process programming and operating systems, File systems and mass storage
Usage Example
int fd, n;
char wbuf[] = “Hello!\n”, rbuf[MAX_BUF];
fd = creat(“source.txt”, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);
write(fd, wbuf, sizeof(wbuf));
n = read(fd, rbuf, MAX_BUF);
close(fd);
From which file offset will read read? What's the value of n? What will rbuf contain?
21S. Manolache, Process programming and operating systems, File systems and mass storage
Usage Example
int fd, n;
char wbuf[] = “Hello!\n”, rbuf[MAX_BUF];
fd = creat(“source.txt”, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);
write(fd, wbuf, sizeof(wbuf));
close(fd);
fd = open(“source.txt”, O_RDONLY);
n = read(fd, rbuf, MAX_BUF);
close(fd);
22S. Manolache, Process programming and operating systems, File systems and mass storage
Directories
Special files that contain directory entriesA directory entry is a data structure containing the file
attributes
/
bin/
ls
lib/
libc.solibgcc.so
vmlinux-2.6.11
bin, dir, root:root, rwxr-xr-x, 10050lib, dir, root:root, rwxr-xr-x, 52175vmlinux-2.6.11, reg, root:root, r-x------, 120311
ls, reg, root:root, r-xr-xr-x, 10052
libc.so, reg, root:root, r-xr-xr-x, 52177libgcc.so, reg, root:root, r-xr-xr-x, 60621
root dir
bin dir
lib dir
directory entry
23S. Manolache, Process programming and operating systems, File systems and mass storage
Operations on Directories
List the contents of the directoryDelete (unlink) a fileRename a fileOpen and close the directory
Search for a fileTraverse the file system
24S. Manolache, Process programming and operating systems, File systems and mass storage
Directory Hierarchies
Tree-structured directoriesLeaf nodes are files, all other nodes are directories
Acyclic-graph directoriesSame with the exception that the structure is an acyclic
graphGeneral graph directories
May contain cycles
25S. Manolache, Process programming and operating systems, File systems and mass storage
Acyclic Graph Directories
We need reference counters. A file is removed and its blocks marked as free when the reference counter reaches 0.
Removing = unlinking
/
bin/
ls
lib/
libc.solibgcc.sokernel
vmlinux-2.6.11
bin, dir, root:root, rwxr-xr-x, 10050lib, dir, root:root, rwxr-xr-x, 52175vmlinux-2.6.11, reg, root:root, r-x------, 120311
ls, reg, root:root, r-xr-xr-x, 10052
libc.so, reg, root:root, r-xr-xr-x, 52177libgcc.so, reg, root:root, r-xr-xr-x, 60621kernel, reg, sys:root, r-xr-x---, 120311
root dir
bin dir
lib dir
26S. Manolache, Process programming and operating systems, File systems and mass storage
Symbolic Links
A hard link is a directory entry pointing to a different file. It contains no blocks of its own
A symbolic link is a special file, very short one, that contains the name of the file that it points to
/
bin/
ls
lib/
libc.solibgcc.sokernel
vmlinux-2.6.11
bin, dir, root:root, rwxr-xr-x, 10050lib, dir, root:root, rwxr-xr-x, 52175vmlinux-2.6.11, reg, root:root, r-x------, 120311
ls, reg, root:root, r-xr-xr-x, 10052
libc.so, reg, root:root, r-xr-xr-x, 52177libgcc.so, reg, root:root, r-xr-xr-x, 60621kernel, link, guest:guest, rwxrwxrwx, 71220
root dir
bin dir
lib dir
27S. Manolache, Process programming and operating systems, File systems and mass storage
File Sharing
When files can be sharedShould all writes be allowed to occur or should the OS
protect the user actions from each other?Should a write be immediately visible to all the other
users who share the file?
28S. Manolache, Process programming and operating systems, File systems and mass storage
yyyyyyyyyyxxxxxxxxxx
File Sharing
fd = open(“file.txt”, O_RDWR);n = read(fd, buf, 10);
n = read(fd, buf, 5);printf(“%s\n”, buf);
fd = open(“file.txt”, O_RDWR);n = read(fd, buf, 10);
write(fd, “xxxxxxxxxx”, 10);printf(“%s\n”, buf);
n = read(fd, buf, 5);printf(“%s\n”, buf);
printf(“%s\n”, buf);write(fd, “yyyyyyyyyy”, 10);
printf(“%s\n”, buf);n = read(fd, buf, 10);
000000000088888888885555555555
0
0file.txt, ftp:users, rw-rw-rw-, ,10228
01
18
…
183
186
10
10
15
20
2030
30
close(fd);
121
close(fd);
0
OS-wide open file table
Proc A open file table Proc B open file table
29S. Manolache, Process programming and operating systems, File systems and mass storage
Protection
Keep safe from improper accessWe introduce the notion of file owner
A user ID kept on the disk in the directory entryUsers have IDsProcesses, besides their process IDs (pid), have user
IDs, typically the user ID of the user that executes them (user ID (uid) or effective user ID (euid))
Files typically are owned by the user who creates themThe effective user ID of the file creator is written in the
directory entryBesides owner, a file is characterised by its group
30S. Manolache, Process programming and operating systems, File systems and mass storage
Protection
Controlled access is introduced by specifying which users (or user groups) are allowed to perform operations on the file
Examples of controlled operations:ReadWriteExecuteAppendDeleteList
31S. Manolache, Process programming and operating systems, File systems and mass storage
Unix File Protection
Read:Read for files, list for directories
Write:Write/modify for files, create/delete new entries for
directoriesExecute:
Execute for files, change directory rights for directories
32S. Manolache, Process programming and operating systems, File systems and mass storage
Access Control Lists (ACL)
Each file (directory) has an access control list attachedThe access control list specifies for each controlled
operation the users that are allowed to perform this operation
Advantage:Very general and flexible
Disadvantages:Difficult to construct if we do not know all the users
beforehandDirectory entry of variable size, more difficult to manage
33S. Manolache, Process programming and operating systems, File systems and mass storage
Condensed ACL Use of condensed ACLs instead Use per-owner, per-group, per-others permissions
E.g.: Sara writes a book, Jim, Dawn, and Jill help her. Sara has all the rights, Jim, Dawn, and Jill may read or write but not delete, all the others may only read
Sara is the owner, has rw- permissions A group book is created, the file is owned by user Sara and group book,
Jim, Dawn, and Jill are added to group book, the group has rw- permissions
Others have r-- permissions The directory in which the book resides has rwxr-xr-x permissions If Sara wants Joe to have read/write access to chapter 1, she cannot add
him to group book Instead, user Joe is added to the ACL
34S. Manolache, Process programming and operating systems, File systems and mass storage
Condensed ACL
What if:kim:staff rw-r-xr-- script.shUser kim belongs to group staffShould kim be allowed to execute script.sh? If we consider that the permissions of the owner apply,
then no If we consider that the permissions of the group apply,
then yesPrecedence given to most specific
35S. Manolache, Process programming and operating systems, File systems and mass storage
Disk Access Scheduling
OS has to ensure that the resources are used efficientlyBandwidth = transferred bytes / length of interval between
first request and completion of last request
Seek timeRotational latency
36S. Manolache, Process programming and operating systems, File systems and mass storage
FCFS Disk Scheduling
Following requests: cylinders 98, 183, 37, 122, 14, 124, 65, 67
Head initially at cylinder 53
Total head movement of 640 cylinders
0 98 1831223714 6553
37S. Manolache, Process programming and operating systems, File systems and mass storage
Shortest Seek Time First
Movement of only 236 cylindersMay cause starvationNot optimal!! If we moved from 53 to 37 and then 14,
before 65, 67, etc. 208 cylinders
0 98 1831223714 6553
38S. Manolache, Process programming and operating systems, File systems and mass storage
SCAN Scheduling
When we reach one end, it is more likely that requests are closer to the other end than close to the read/write head
Those also waited the longest C-SCAN algorithm
0 98 1831223714 6553
39S. Manolache, Process programming and operating systems, File systems and mass storage
Circular-SCAN Algorithm
0 98 1831223714 6553
40S. Manolache, Process programming and operating systems, File systems and mass storage
LOOK and C-LOOK Algorithms
0 98 1831223714 6553
0 98 1831223714 6553
41S. Manolache, Process programming and operating systems, File systems and mass storage
Which One?
Depends on the loadDepends on file allocationSSTF and LOOK seem reasonable alternatives
Real disk geometry is hidden to the OSDisk manufacturers include a scheduling in the hard disk
controllerThen why not let the hard disk do all the scheduling?Because some requests have different semantics and have
to be treated differently (accesses of a higher priority process, or paging, for example)
42S. Manolache, Process programming and operating systems, File systems and mass storage
Summary
FilesOperationsSharingProtection
Directory hierarchiesDisk scheduling algorithms