unix file system
TRANSCRIPT
![Page 1: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/1.jpg)
Lecture On“Unix File System and Other Issues”
M M Gore CSED MNNIT
![Page 2: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/2.jpg)
Introduction• “Technically, Unix is a simple, coherent system which
pushes a few good ideas to the limit.”—Sunil Das• “The greatest virtues of Unix, in my opinion, are those that
emerged as a result of the way that it developed.”—Peter Salus
• “The growth and development of Unix is an exciting sociological tale. . . . The nature of the individuals and their interactions is what made Unix vital.”—Peter Salus
• “UNIX is simple and coherent, but it takes a genius (or at any rate, a programmer) to understand and appreciate its simplicity.”—Dennis Ritchie
• The history of Unix is a story of intrigue, adaptability, desire, cunning, intellectual honesty, and the pursuit of freedom and happiness.—Mark Shacklette
![Page 3: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/3.jpg)
Unix Core Philosophy Summary I
• Each program should do one thing and do it well. If you need something new, add a new simple program rather than bloat existing programs with new features.
• Allow the output of every program to be the input to another independent program. Don't clutter output with extraneous information. Avoid columnar and binary formats for I/O. Don't insist on interactive input.
• Design and build software, even operating systems, to be tested early on in the development cycle. Throw away the clumsy parts and rewrite them.
![Page 4: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/4.jpg)
Unix Core Philosophy Summary II• Favor general-purpose software tools, even if you have to
take the time to stop and write the tools yourself.• Honesty: Admit your bugs, even advertise them so that
others won’t trip over them• Do not assume, a priori, that your users are idiots• When forced to decide between portability and speed,
always opt for portability (speed will come, Grasshopper)• Give users choice (as in shell, editor, etc.)• Provide online help (from Multics)• Provide source code for freedom and empowerment(cf. McIlroy, Pinson, Tague, "Unix Time-Sharing System Forward", The Bell System Technical Jounal, July -Aug 1978;
57.6.2, p. 1902)
![Page 5: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/5.jpg)
The Unix File System
• Unix is Pro-Family (Hierarchical)– Parent, child, current directories
• Absolute pathnames (start at root)• Relative pathnames (start where you are)• Important standard directories:
– / /home /usr /usr/bin– /sbin /usr/sbin /etc /var– /dev /tmp /usr/ucb
![Page 6: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/6.jpg)
The Home Directory
• Your home directory is defined in /etc/passwd
• You can use any of the following commands to get home:
– cd
– cd ~
– cd $HOME
– cd /full/path/to/your/home/dir
• Change to the previous (last) directory:
– cd -
![Page 7: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/7.jpg)
Man Pages
• Man man
• Sections:
– user
– system calls
– C library functions/X Windows functions
– devices and network interfaces
– file formats
– games and demos
– environments/miscellaneous
– system administration/maintenance
• whatis keyword
• man [1,2] write (1 is user command, 2 is system call)
• man -k keyword (or apropos keyword)
![Page 8: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/8.jpg)
“Everything in Unix is a File”• A file has a name and an associated inode, which contains
detailed information about the file • What is a directory?
– A file containing a list of other files, some of which may be other directories
– A file whose contents is a list of name + inode pairs• The directory structure is an abstraction that presents
files on a file system in a hierarchical manner—but the files’ contents remain littered across the physical file system.
• A file is a stream of bytes• A file is referenced by a user via a filename. A filename
can be up to 255 bytes.• A file is referenced by the system via the inode
![Page 9: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/9.jpg)
inode details• Every file is associated with a potentially unique inode. In fact, early
Unix systems refered to filenames as “links”, that is, names “linked” to an inode.
• The inode contains information about the file and the inode itself, like:– File type (regular, link, directory, etc.)– Number of Hard Links to the inode– Associated file byte stream length in bytes– Device ID where the file is located (/dev/hda1)– Inode number of this file– File owner’s userid and groupid– mtime, atime, and ctime– permissions (rwx)
• ls -i (ls -1iF)• The stat command
![Page 10: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/10.jpg)
File Permissions
• Files have three categories of permissions:– user (owner)– group– other (everyone else NOT in one of the above)
• r (4): Read permission (can open the file)• w(2): Write permission (can modify it)• x (1): Execute permission (can run it)
![Page 11: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/11.jpg)
Directory Permissions
• Directories have three categories of permissions:– user (owner)
– group
– other (everyone else NOT in one of the above)
• r (4): Read permission (can ls the filenames)• w(2): Write permission (can modify the dir)• x (1): Execute permission (can cd into dir)• t (sticky bit): individual ownership only
![Page 12: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/12.jpg)
Links• Hard Links
– ln origfile linkfile– a directory entry with a unique name referencing a particular inode– ls -i will list out inodes for files (ls -1iF)– Only superuser can hard link to a directory– Hard links are only meaningful within a single filesystem, not across
mount points– A hard link’s inode is the same number as the linked file’s inode
• Soft (Symbolic) Links:– ln -s origfile linkfile– Anyone can create a soft link to a directory– A softlink can refer to another file on another filesystem– ls -[l]F will reveal softlinks (noted by -> pointer and @ notation)– A softlink’s contents is the name of the file pointed to.
![Page 13: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/13.jpg)
Redirection
• Unix has three default file handles (defined in /usr/include/unistd.h):– Standard Output (stdio, 1)– Standard Error (stderr, 2)– Standard Input (stdin, 0)
• By default, standard output is sent to the current process owner’s terminal
• Redirection causes the standard output of the current process to go to some other designated file:– ls -la >/tmp/some.other.file– cat /tmp/some.other.file
![Page 14: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/14.jpg)
Job Control
• & puts current process in background
• jobs prints out current jobs in shell
• kill %n terminates a given job
• fg [%n] moves a job to the foreground
• bg [%n] moves a job to the background
![Page 15: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/15.jpg)
Filters• Filters are programs that are written to accept input from STDIN in
addition to any other forms of input.• Filters send output to STDOUT.• The list of Unix Filters includes:
– cat– cut– less– grep– sort– tr– uniq– wc– tail, head– lpr
![Page 16: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/16.jpg)
The Four Stages of Compilation
• preprocessing• compilation• assembly• linking
![Page 17: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/17.jpg)
gcc driver program (toplev.c)
• cpp: C PreProcessor• cc1: RTL (Register Transfer Language) processor• as: assembler• ld: loader (linker)
![Page 18: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/18.jpg)
The GNU CC Compilation Process
• GCC is portable:– multiplatform (intel, MIPS, RISC, Sparc,
Motorola, etc.)– multiOS (BSD,AIX, Linux, HPUX, mach,
IRIX, minix, msdos, Solaris, Windoze, etc.)– Multilingual (C, Objective C, C++, Fortran,
etc.)• Single first parsing pass that generates a parsing
tree
![Page 19: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/19.jpg)
The GNU CC Compilation Process• Register Transfer Language generation
– close to 30 additional passes operate on RTL Expressions (RTXs), constructed from partial syntax trees
– gcc –c –dr filename.c– RTL is Lisp-like
• cond(if_then_else cond then else)• (eq: m x y)• (set lval x)• (call function numargs)• (parallel [x0 x1 x2 xn])
• Final output is assembly language, obtained by mapping RTX to a machine dependency dictionary– ~/mark/pub/51081/compiler/i386.md
![Page 20: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/20.jpg)
Assembler Tasks
• converts assembly source code into machine instructions, producing an “object” file (called “.o”)
![Page 21: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/21.jpg)
Loader (Linker) tasks
• The Loader (linker) creates an executable process image within a file, and makes sure that any functions or subprocesses needed are available or known. Library functions that are used by the code are linked in, either statically or dynamically.
![Page 22: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/22.jpg)
Preprocessor Options
• -E preprocess only: send preprocessed output to standard out--no compile
– output file: file.c -> file.i file.cpp -> file.ii
• -M produce dependencies for make to stdout (voluble)
• -C keep comments in output (used with -E above):– -E -C
• -H printer Header dependency tree
• -dM Tell preprocessor to output only a list of macro defs in effect at end of preprocessing. (used with -E above)
– gcc -E -dM funcs.c |grep MAX
![Page 23: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/23.jpg)
Compiler Options
• -c compile only• -S send assembler output source to *.s
– output file: file.c -> file.s • -w Suppress All Warnings
– gcc warnings.c– gcc -w warnings.c
• -W Produce warnings about side-effects (falling out of a function)– gcc -W warnings.c
![Page 24: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/24.jpg)
Compiler Options (cont)
• -I Specify additional include file paths• -Wall Produce many warnings about questionable
practices; implicit declarations, newlines in comments, questionable lack of parentheses, uninitialized variable usage, unused variables, etc.– gcc -Wall warnings.c
• -pedantic Warn on violations from ANSI compatibility (only reports violations required by ANSI spec).– gcc -pedantic warnings.c
![Page 25: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/25.jpg)
Compiler Options (cont)
• -O optimize (1,2,3,0)– -O,-O1 base optimizations, no auto inlines, no loops– -O2 performs additional optimizations except inline-
functions optimization and loop optimization– -O3 also turns on inline-functions and loop
optimization– -O1 default
• -g include debug info (can tell it what debugger):– -gcoff COFF format for sdb (System V < Release 4)– -gstabs for dbx on BSD– -gxcoff for dbx on IBM RS/6000 systems– -gdwarf for sdb on System V Release 4
![Page 26: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/26.jpg)
Compiler Options (cont)
• -save-temps save temp files (foo.i, foo.s, foo.o)• -print-search-dirs print the install, program, and
libraries paths• -gprof create profiling output for gprof• -v verbose output (useful at times)• -nostartfiles skip linking of standard start files,
like /usr/lib/crt[0,1].o, /usr/lib/crti.o, etc.• -static link only to static (.a=archive) libraries• -shared if possible, prefer shared libraries over
static
![Page 27: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/27.jpg)
Assembler Options (use gcc -Wa,-options to pass options to assembler)
• -ahl generate high level assembly language source – gcc -Wa,-ahl warnings.c
• -as generate a listing of the symbol table– gcc -Wa,-as warnings.c
![Page 28: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/28.jpg)
Linker Options (use gcc -Wl,-options to pass options to the loader)
• gcc passes any unknown options to the linker• -l lib (default naming convention liblib.a)• -L lib path (in addition to default /usr/lib and /lib)• -s strip final executable code of symbol and
relocation tables– gcc -w –g warnings.c ; ls -l a.out ; gcc -w -
Wl,-s warnings.c ; ls -l a.out• -M create load Map to stdout
![Page 29: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/29.jpg)
How do Shared Libraries Work?
• When a program runs that depends on a shared library (discover with ldd progname), the dynamic linker will attempt to find the shared library referenced by the soname
• Once all libraries are found, the dependent code is dynamically linked to your program, which is then executed
• Reference: The Linux Program-Library HOWTO
![Page 30: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/30.jpg)
Unix File I/O
![Page 31: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/31.jpg)
Unix System Calls
• System calls are low level functions the operating system makes available to applications via a defined API (Application Programming Interface)
• System calls represent the interface the kernel presents to user applications
![Page 32: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/32.jpg)
A File is a File is a File--Gertrude Stein
• Remember, “Everything in Unix is a File”• This means that all low-level I/O is done by
reading and writing file handles, regardless of what particular peripheral device is being accessed—a tape, a socket, even your terminal, they are all files.
• Low level I/O is performed by making system calls
![Page 33: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/33.jpg)
User and Kernel Space
• System memory is divided into two parts:– user space
• a process executing in user space is executing in user mode
• each user process is protected (isolated) from another (except for shared memory segments and mmapings in IPC)
– kernel space• a process executing in kernel space is executing in kernel
mode• Kernel space is the area wherein the kernel executes• User space is the area where a user program normally executes,
except when it performs a system call.
![Page 34: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/34.jpg)
Anatomy of a System Call
• A System Call is an explicit request to the kernel made via a software interrupt
• The standard C Library (libc) provides wrapper routines, which basically provide a user space API for all system calls, thus facilitating the context switch from user to kernel mode
• The wrapper routine (in Linux) makes an interrupt call 0x80 (vector 128 in the Interrupt Descriptor Table)
• The wrapper routine makes a call to a system call handler (sometimes called the “call gate”), which executes in kernel mode
• The system call handler in turns calls the system call interrupt service routine (ISR), which also executes in kernel mode.
![Page 35: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/35.jpg)
Regardless…
• Regardless of the type of file you are reading or writing, the general strategy remains the same:– creat() a file– open() a file– read() a file– write() a file– close() a file
• These functions constitute Unix Unbuffered I/O• ALL files are referenced by an integer file
descriptor (0 == STDIN, 1 == STDOUT, 2 == STDERR)
![Page 36: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/36.jpg)
read() and write()
• Low level system calls return a count of the number of bytes processed (read or written)
• This count may be less than the amount requested• A value of 0 indicates EOF• A value of –1 indicates ERROR• The BUFSIZ #define (8192, 512)
![Page 37: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/37.jpg)
read()
#include <unistd.h>
ssize_t read(int fd, void * buf, size_t count);
• If read() is successful, it returns the number of bytes read
• If it returns 0, it indicates EOF• If unsuccessful, it returns –1 and sets errno
![Page 38: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/38.jpg)
write()
#include <unistd.h>
ssize_t write(int fd, void * buf, size_t count);
• If write() is successful, it returns the number of bytes written to the file descriptor, this will usually equal count
• If it returns 0, it indicates 0 bytes were written• If unsuccessful, it returns –1 and sets errno
![Page 39: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/39.jpg)
open()
#include <fcntl.h>int open(const char * path, int flags[, mode_t
mode]);
• flags may be OR’d together:– O_RDONLY open for reading only– O_WRONLY open for writing only– O_RDRW open for both reading and writing– O_APPEND open for appending to the end of file– O_TRUNC truncate to 0 length if file exists– O_CREAT create the file if it doesn’t exist
• path is the pathname of the file to open/create• file descriptor is returned on success, -1 on error
![Page 40: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/40.jpg)
creat()
• Dennis Ritchie was once asked what was the single biggest thing he regretted about the C language. He said “leaving off the ‘e’ on creat()”.
• The creat() system call creates a file with certain permissions:int creat(const char * filename, mode_t mode);
• The mode lets you specifiy the permissions assigned to the file after creation
• The file is opened for writing only
![Page 41: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/41.jpg)
open() (create file)
• When we use the O_CREAT flag with open(), we need to define the mode (rights mask from sys/stat.h):– S_IRUSR read permission granted to OWNER– S_IWUSR write permission granted to OWNER– S_IXUSR execute permission granted to OWNER– S_IRGRP read permission granted to GROUP
• etc.– S_IROTH read permission granted to OTHERS
• etc.• Example:
int fd = open(“/path/to/file”, O_CREAT, S_IRUSR | S_IWUSR | S_IXUSR | S_IRGRP | S_IROTH);
![Page 42: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/42.jpg)
close()
#include <unistd.h>
int close( int fd );
• close() closes a file descriptor (fd) that has been opened.
![Page 43: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/43.jpg)
lseek()
#include <sys/types.h>#include <unistd.h>long lseek(int fd, long offset, int startingpoint)• lseek moves the current file pointer of the file associated
with file descriptor fd to a new position for the next read/write call
• offset is given in number of bytes, either positive or negative from startingpoint
• startingpoint may be one of:– SEEK_SET move from beginning of the file– SEEK_CUR move from current position– SEEK_END move from the end of the file
![Page 44: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/44.jpg)
Error Handling
• System calls set a global integer called errno on error:– extern int errno; /* defined in /usr/include/errno.h */
• The constants that errno may be set to are defined in </usr/include/asm/errno.h>. For example:– EPERM operation not permitted– ENOENT no such file or directory (not there)– EIO I/O error– EEXIST file already exists– ENODEV no such device exists– EINVAL invalid argument passed
#include <stdio.h>void perror(const char * s);
![Page 45: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/45.jpg)
stat():int stat(const char * pathname; struct stat *buf);
• The stat() system call returns a structure (into a buffer you pass in) representing all the stat values for a given filename. This information includes:– the file’s mode (permissions)– inode number– number of hard links– user id of owner of file– group id of owner of file– file size– last access, modification, change times– less /usr/include/sys/stat.h => /usr/include/bits/stat.h– less /usr/include/sys/types.h (S_IFMT, S_IFCHR, etc.)
• Example: ~/UofC/51081/pub/51081/stat/mystat.c
![Page 46: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/46.jpg)
Introduction to make
Debugging with gdb and ddd
Introduction to Systems Programming:
Processes and Signals
![Page 47: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/47.jpg)
make
![Page 48: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/48.jpg)
What is make?
• make is used to:
– save time by not recompiling files that haven't changed
– make sure all files that have changed do get recompiled
![Page 49: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/49.jpg)
The Concept
• make is a program that will update targets on the basis of changes in dependencies.
• Although it is mostly used to build software by compiling and linking, it can be used to manage any construction project that involves creating something based on something else (e.g., using nroff over a series of book chapters).
• A makefile is nothing more than dependencies and rules. A rule describes HOW to create the target from the dependencies.
![Page 50: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/50.jpg)
Calling Convention and Options
• -n don't make, but print out what would be done
• -k keep going, don't stop on errors, which is the default
• -f run makefile specified by filename• Default makefile naming convention
– makefile– Makefile
![Page 51: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/51.jpg)
Dependencies and Rules
• Dependencies and Syntax– target: dep1 dep2 depn– make will build the first target it finds– this target is commonly called "all"
• all: bigapp• Rules
– It is a rule that every rule must begin with a single TAB character!
• [TAB] gcc -c 1.c• make has several built-in rules
– make -p will show them to you
![Page 52: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/52.jpg)
Macros and Multiple Targets• a MACRO is a substitutable syntax to give flexibility and
genericity to rules• Forms:
– MACRONAME=value– access with either:
• $(MACRONAME) or• ${MACRONAME} or (sometimes)
$MACRONAME– undefine a MACRO with:
• MACRONAME=• A macro can be redefined at the command line:
– make CC=aCC #for HP Ansi compiler• Examples: (make2, make3)
![Page 53: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/53.jpg)
Suffix Rules• a Suffix Rule is a directive that applies rules and macros to generic
suffixes• tell make about a new suffix: SUFFIXES: .cpp• tell make how to compile it: .cpp.o:• then the rule: $(CC) -xc++ $(CFLAGS) -I$(INCLUDE) -c $<• Built in suffix macros:
– $@ The full name of the current target– $? A list of modified dependencies (a list of files newer than the
target on which the target depends)– $< The single file that is newer than the target on which the target
is dependent– $* The name of the target file, WITHOUT its suffix (i.e., without
the .c or .cpp, etc.)
![Page 54: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/54.jpg)
Debugging with gdb and ddd
![Page 55: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/55.jpg)
What is a bug?
• a bug exists when executable code returns or causes results that are unintended or undesirable.– You can only have a bug in code that's
compiled or a shell script that's executed by the shell (ie. the compiler or shell do not give errors about compilation).
• Don't confuse design errors with code bugs (don't confuse design with implementation)
![Page 56: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/56.jpg)
Finding bugs
• Problem statement: Code runs fast and furious--we must find out "where" in the code the problem originates.
• Solution statement:– attempt to make bug repeatable--this is empirical
analysis, pure and simple.– printf's can help, displaying variables, but they're
limited.• gcc -o cinfo -DDEBUG cinfo.c• cinfo
– __DATE__, __TIME__, __LINE__
• Examples: (in ~mark/pub/51081/debug) cinfo.c
![Page 57: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/57.jpg)
Interactive Debuggers• But interactive debuggers are MUCH better, because they
offer:– run time code stepping– variable analysis and modification– breakpoints (multiple forms)
• Compile for debugging: -g – Try to void optimizing when debugging
• remaining problems:– loop tracing (problem doesn't arise until loop has
executed 1M times)– Optimization problems– Intermittency
• Examples: debug3 (gdb); debug4 (ddd)
![Page 58: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/58.jpg)
Introduction to Systems Programming
Processes
Signals
![Page 59: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/59.jpg)
Introduction to Processes• Multiuser OS
– Ability of an OS to have multiple users using the system at the same time
• Multitasking OS– Ability of an OS to run multiple programs at the same
time– “Pay No Attention To The Man Behind the Screen”
• Concurrency versus Parallelism– timesharing quantums done by the system scheduler
(called swapper), which is a kernel thread and has process ID of 0
![Page 60: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/60.jpg)
What is a Process?
• A process is an executable “cradle” in which a program may run
• This “cradle” provides an environment in which the program can run, offering memory resources, terminal IO, via access to kernel services.
• When a new process is created, a copy of the parent process’ environment variables is provided as a default to the new process
• A process is an address space married to a single default thread of control that executes on code within that address space
• ps -yal
![Page 61: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/61.jpg)
Introduction to Processes• Other kernel threads are created to run the following
services (various Unix kernels vary, YMMV):
– initd (1): parent initializer of all processes
– keventd (2): kernel event handler
– kswapd (3): kernel memory manager
– kreclaimd (4): reclaims pages in vm when unused
– bdflush (5): cleans memory by flushing dirty buffers from disk cache
– kupdated (6): maintains sanity of filesystem buffers
![Page 62: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/62.jpg)
User and Kernel Space• System memory is divided into two parts:
– user space• a process executing in user space is executing in
user mode• each user process is protected (isolated) from
another (except for shared memory segments and mmapings in IPC)
– kernel space• a process executing in kernel space is executing in
kernel mode• Kernel space is the area wherein the kernel executes• User space is the area where a user program normally
executes, except when it performs a system call.
![Page 63: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/63.jpg)
Anatomy of a System Call• A System Call is an explicit request to the kernel made via a
software interrupt• The standard C Library (libc) provides wrapper routines,
which basically provide a user space API for all system calls, thus facilitating the context switch from user to kernel mode
• The wrapper routine (in Linux) makes an interrupt call 0x80 (vector 128 in the Interrupt Descriptor Table)
• The wrapper routine makes a call to a system call handler (sometimes called the “call gate), which executes in kernel mode
• The system call handler in turns calls the system call interrupt service routine (ISR), which also executes in kernel mode.
![Page 64: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/64.jpg)
ELF (Executable and Linking Format)
• Heap is for dynamic memory demand (malloc())
• Stack is for function call storage and automatic variables
• BSS (Block Started by Symbol) stores uninitialized static data
int array[100];
• Data Segment stores initialized static data
char name[] = “bob”;
• Multiple processes can share the same code segment
dynamiclibraries
unitializeddata area
(BSS)NULLed out
initialized datasegment
(loaded from objectfile on disk)
DA
TA
SE
GM
EN
T
Heap
Stackgrows
grows
Text Segment(YCGH)
![Page 65: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/65.jpg)
C Language Allocation
dynamiclibraries
unitializeddata area
(BSS)NULLed out
initialized datasegment
(loaded from objectfile on disk)
DA
TA
SE
GM
EN
T
Heap
Stackgrows
grows
Text Segment(YCGH)
char * p = malloc(1024);
int iarray[20];
int iarray2[] = { 1,2,3 };
int main() { ... }
int myfunc(int x, float y) { int z; }
![Page 66: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/66.jpg)
The Linux Process Descriptor• Each Linux process is described by a task_struct structure
defined in include/linux/sched.h• This structure holds information on most aspects of a
process in memory, including, among other items:– process state– next and previous task pointers– next and previous runnable task pointers– Parent, Child, and Sibling pointers– tty information– current directory information– open file descriptors table– memory pointers– signals received
![Page 67: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/67.jpg)
Task State• TASK_RUNNING: running or waiting to be executed
• TASK_INTERRUPTIBLE: a sleeping or suspended process, can be awakened by signal
• TASK_STOPPED: process is stopped (as by a debugger or SIGTSTP, Ctrl-Z)
• TASK_ZOMBIE: process is in “walking dead” state waiting for parent process to issue wait() call
• TASK_UNINTERRUPTIBLE: task is performing critical operation and should not be interrupted by a signal (usually used with device drivers)
![Page 68: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/68.jpg)
Signal Processing
“Introduction to Interprocess Communication”
![Page 69: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/69.jpg)
What is a Signal?• A signal is a software interrupt delivered to a process by
the OS because:– it did something (oops)– the user did something (pressed ^C)– another process wants to tell it something (SIGUSR?)
• A signal is asynchronous, it may be raised at any time (almost)
• Some signals are directly related to hardware (illegal instruction, arithmetic exception, such as attempt to divide by 0)
• Others are purely software signals (interrupt, bad system call, segmentation fault)
![Page 70: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/70.jpg)
Common Signals
• SIGHUP (1): sent to a process when its controlling terminal has disconnected
• SIGINT (2): Ctrl-C (or DELETE key)• SIGQUIT (3): Ctrl-\ (default produces core)• SIGSEGV (11): Segmentation fault• SIGILL (4): Illegal instruction (default core)• SIGUSR[1,2]: User-defined signals (10,12)• kill –l will list all signals• SIGFPE (8):Floating Point Exception (divide by
0; integer overflow; floating-point underflow)
![Page 71: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/71.jpg)
Chris Brown’s Top 6 List of Things to Do with a Signal Once You Trap It
1. Ignore a signal
2. Clean up and terminate
3. Handle Dynamic Configuration (SIGHUP)
4. Report status, dump internal tables
5. Toggle debugging on/off
6. Implement a timeout condition
(cf. Chris Brown, Unix Distributed Programming, Prentice Hall, 1994)
![Page 72: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/72.jpg)
Reliable and Unreliable Signal APIs
• Signal model provided by AT&T Version 7 was “not reliable”, meaning that signals could get “lost” on the one hand, and programs could not turn signal delivery “off” during critical sections, on the other hand.
• BSD 4.3 and System V Release 3 delivered reliable signals, which solved many of the problems with signals present in Version 7.
• And if that weren’t enough, SVR4 introduced POSIX signals.
![Page 73: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/73.jpg)
Signal Disposition
• Ignore the signal (most signals can simply be ignored, except SIGKILL and SIGSTOP)
• Handle the signal disposition via a signal handler routine. This allows us to gracefully shutdown a program when the user presses Ctrl-C (SIGINT).
• Block the signal. In this case, the OS queues signals for possible later delivery
• Let the default apply (usually process termination)
![Page 74: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/74.jpg)
Names & Contents of Important UNIX Directories
Directory
Description
/ root - kernel
/sbin files required to start the system and scripts to control the boot process
![Page 75: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/75.jpg)
Names & Contents of Important UNIX Directories Cont…
/etc
Files required to boot the system and communicate, and scripts to control the boot process
/etc/config System configuration option
files
/etc/cron.d cron access files and FIFO
![Page 76: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/76.jpg)
Names & Contents of Important UNIX Directories Cont…
/etc/default Default system configuration
/etc/dfs Distributed file sharing configuration
/etc/fs Static file system specific mount commands
![Page 77: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/77.jpg)
Names & Contents of Important UNIX Directories Cont…
/etc/fdmnsFile domain names and devices, symbolic links to the file volumes
/etc/inet Internet services configuration
/etc/init.d Internet service scripts run by init
![Page 78: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/78.jpg)
Names & Contents of Important UNIX Directories Cont…
/etc/lib Shared libraries required for boot
/etc/lp Line printer system configuration
/etc/mail Mail configuration
![Page 79: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/79.jpg)
Names & Contents of Important UNIX Directories Cont…
/etc/net Configuration for transport-independent network services
/etc/opt Optional software package configuration files
/etc/rc#.d Operations performed when entering run level # (S, 0, 1, 2, 3)
![Page 80: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/80.jpg)
Names & Contents of Important UNIX Directories Cont…
/etc/saf Service access facility configuration
/etc/security Security audit configuration
/etc/sec Security audit configuration
![Page 81: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/81.jpg)
Names & Contents of Important UNIX Directories Cont…
/usr Directories of system files
/usr/bin System binary files
/usr/etc Further system communication and administration programs
![Page 82: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/82.jpg)
Names & Contents of Important UNIX Directories Cont…
/usr/sbin Further system communication and administration programs
/usr/lib Libraries of object files, sendmail
/usr/4lib SunOS 4.1 libraries required for binary compatibility
![Page 83: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/83.jpg)
Names & Contents of Important UNIX Directories Cont…
/usr/5bin System V binaries
/usr/5include System V include files
/usr/5lib System V libraries
![Page 84: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/84.jpg)
Names & Contents of Important UNIX Directories Cont…
/usr/aset Automated security enhancement tool files and programs
/usr/ucb BSD binaries
/usr/bsd BSD binaries
![Page 85: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/85.jpg)
Names & Contents of Important UNIX Directories Cont…
/usr/ccs Compiler support system
/usr/dt CDE desktop hierarchy
/usr/lib/fs File system dependent modules
![Page 86: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/86.jpg)
Names & Contents of Important UNIX Directories Cont…
/use/lib/lp Line printer database and programs
/usr/lib/netsvc Network service utilities
/usr/lib/nfs NFS daemons and programs
![Page 87: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/87.jpg)
Names & Contents of Important UNIX Directories Cont…
/usr/lib/nis NIS+ programs and setup scripts
/usr/lib/saf SAF daemons and programs
/var Directories for administrative programs and logs
![Page 88: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/88.jpg)
Names & Contents of Important UNIX Directories Cont…
/var/adm System log and account files
/var/log System log files
/var/spool/mail Mail spool directory
![Page 89: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/89.jpg)
Names & Contents of Important UNIX Directories Cont…
/var/mail Mail spool directory
/var/yp NIS tables and Makefile for updating NIS
/var/nis NIS+ tables
![Page 90: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/90.jpg)
Names & Contents of Important UNIX Directories Cont…
/var/spool Directories for corn, logs, etc.
/var/sadm Databases maintained by package administration utilities
/var/inst Database maintained by inst utility
/var/saf Service access facility log and account files
![Page 91: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/91.jpg)
Names & Contents of Important UNIX Directories Cont…
/dev devices directory
/dev/dskBlock disk devices directory
![Page 92: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/92.jpg)
Names & Contents of Important UNIX Directories Cont…
/dev/rdsk Raw disk devices directory
/dev/ptsPseudo terminal (pty) devices directory
/dev/rmt Raw tape device directory
/dev/term Terminal devices directory
/dev/sadEntry points for STREAMS administrative drivers
![Page 93: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/93.jpg)
Names & Contents of Important UNIX Directories Cont…
/devices Physical devices directory
/home/usr/people/usr/users User directories
/tftpboot/usr/local/boot Client boot programs
/tmp Temporary files
![Page 94: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/94.jpg)
Names & Contents of Important UNIX Directories Cont…
/usr/local Locally installed files
/opt Locally installed packages and files
/kernel Contains the kernel and drivers for the kernel
/platform Hardware specific files for kernel support
![Page 95: Unix File System](https://reader033.vdocuments.site/reader033/viewer/2022061201/5478b9d0b4af9fbb4f8b45a4/html5/thumbnails/95.jpg)
Names & Contents of Important UNIX Directories Cont…
/standStandalone environment programs, can be accessed from the PROM
/procFor process access file system, it provides access to all current processes
/sys Object files to reconfigure the
kernel
/vol Vold mount points