1

UNIX Foundations

The Process and the Kernel

Course Description

The Goals for this course Understand UNIX Understand Operating Systems

Prerequisites: COSC 2P13 : Introduction to Operating Systems COSC 2P91 : Procedural Programming

Course Description An intensive study of computer operating

system design Multiprogramming Time-sharing Real-time processing Job and task control Synchronization of concurrent processes and

processors Resource scheduling Protection Management of hierarchical storage.

How to study this course Read and remember

Read the book, remember the concepts and commands

Think Think operating systems as natural

administrative agents Practice

Coding with UNIX, use and understand of UNIX commands and get the results

The Textbook Used Uresh Vahalia, UNIX Internals: The

New Frontiers, Prentice Hall, 1996 Why we use this book?

UNIX is one of the most popular operating systems of the world.

If you understand UNIX, you can understand other operating systems.

References William Stallings, Operating Systems,5th

Ed., Prentice Hall,2005 A. Tanenbaum, Modern Operating

Systems, 2nd ed. Prentice Hall 2001 Kay A. Robbins and Steven Robbins, UNIX

Systems Programming, Prentice Hall,2003 W. R. Stevens, Advanced Programming in

the UNIX Environment, Addison Wesley, 1992

7UNIX

3BSD

SVR2

SVR34.2BSD

4.3BSD

4.4BSD

SVR4

SunOS

Solaris

4BSD

SCO UNIXAIX

HP-UX

ULTRIX

Digital UNIX

XENIX

Summary of UNIX History

SVID POSIX

XPG

8

Summary of UNIX History

V1...V6

V7

32V

V8

V10

Plan9

PWB

PWB2

SIII

SYSV

V.2

V.3

V.3.2

SVR4

SCO

Xenix2

Xenix BSD

2BSD...2.9BSD

2.10BSD

2.11BSD

3BSD

4BSD

4.2BSD

4.3BSD

4.4BSDUltrix

Mach

OSF1

LINUX

Solaris

AIX SUNOS

9

Flexibility Traditional Kernel: file, scheduling,

executable file formats

10

Modern UNIX Kernel

11

Assessment of UNIX

Advantages: - open software process, - well designed, small and simple kernel, - text files in system databases, - simple, uniform interface to I/O devices, - portability (written in C)

12

Assessment of UNIX

Disadvantages: - expansion of the more and more complex I/O library, - unfriendly user interface, - building block approach to tools, but not to the kernel, - too many versions and standards, - monolithic, unmodular and more and more complex kernel.

13

UNIX Architecture Hardware is surrounded by the

operating-system Operating system is called the kernel Comes with a number of user services

and interfaces shell C compiler

14

UNIX Architecture

Hardware

Kernel

Shell Editors, andPrivate User Programs

Com

pile

r C

ompo

nent

s

Com

pile

rApplication Programs

15

The layers of a UNIX system.

UserInterface

16

UNIX Utility Programs

A few of the more common UNIX utility programs required by POSIX

17

UNIX shell programming

cp src dest head –20 file ls *.c sort <in >out sort <in >temp;head –30<temp;rm temp sort <in | head –30 grep ter *.t | sort | head –20 | tail –5 >foo sort <x | head &

18

Process In UNIX

Process is an instance of a running program.

Lifetime: fork/vfork->exec->exit Well-defined hierarchy: parent,child,init, init process: the top process swapper & pagedeamon Orphans: the parent process is terminated.

19

20

Process Creation Submission of a batch job User logs on Created to provide a service such as

printing Spawned by an existing process

21

Process Termination Batch job issues Halt instruction User logs off Process executes a service request to

terminate Error and fault conditions

22

Reasons for Process Termination

Normal completion Time limit exceeded Memory unavailable Bounds violation Protection error

example write to read-only file Arithmetic error Time overrun

process waited longer than a specified maximum for an event

23

Reasons for Process Termination

I/O failure Invalid instruction

happens when try to execute data Privileged instruction Data misuse Operating system intervention

such as when deadlock occurs Parent terminates so child processes terminate Parent request

24

Process States The Running state

The process that gets executed (single CPU)

The Ready state any process that is ready to be executed

The Blocked state when a process cannot execute until some

event occurs (ex: the completion of an I/O)

25

Process States The New state

OS has performed the necessary actions to create the process

has created a process identifier has created tables needed to manage the process

but has not yet committed to execute the process (not yet admitted)

because resources are limited

The Exit state Termination moves the process to this state It is no longer eligible for execution Tables and other info are temporarily preserved for

auxiliary program

26

Five-State Process Model

New Ready Running Exit

Blocked

Admit

EventOccurs

Dispatch Release

Time-out

EventWait

27

Single Blocked Queue

Admit

Ready Queue

Dispatch

Time-out

Event Wait

ReleaseProcessor

Blocked Queue

EventOccurs

28

Multiple Blocked Queues

Admit

Ready Queue

Dispatch

Time-out

ReleaseProcessor

Event 1 Wait

Event 1 Queue

Event 1Occurs

Event 2 Wait

Event 2 Queue

Event 2Occurs

29

Suspended Processes Processor is faster than I/O so all

processes could be waiting for I/O Swap these processes to disk to free

up more memory Blocked state becomes suspend state

when swapped to disk Two new states

Blocked, suspend Ready, suspend

30

Process State Transition Diagram with Two Suspend States

Ready

AdmitAdmit Suspend

Dispatch

Time out

Ready,suspend

Ready

BlockedBlocked,suspend

EventOccurs

Activate

EventOccurs

Activate

Suspend

Running Exit

EventWait

31

Processor State Information

Contents of processor registers User-visible registers Control and status registers Stack pointers

Program status word (PSW) contains status information

32

Process Control Information

Additional information needed by the operating system to control and coordinate the various active processes scheduling and state information data structuring interprocess communication process privileges memory management resource ownership and utilization

33

Process Creation Assign a unique process identifier Allocate space for the process Initialize process control block Set up appropriate linkages

Ex: add new process to linked list used for scheduling queue

Other maintain an accounting file

34

When to Switch a Process Interrupts

Clock process has executed for the maximum allowable time slice

I/O Memory fault

memory address is in virtual memory so it must be brought into main memory

Trap error occurred may cause process to be moved to Exit state

Supervisor call such as file open

35

Change of Process State Save context of processor including program

counter and other registers Update the process control block with the new

state and any accounting information Move process control block to appropriate

queue - ready, blocked Select another process for execution Update the process control block of the process

selected Update memory-management data structures Restore context of the selected process

36

UNIX Process State Initial (idle) Ready to run Kernel/User running Zombie Asleep for 4BSD: stopped/suspend

37

Process states and state transitions

38

Process Context User address space:

code, data, stack, shared memory regions,

Control information: u area, proc, kernel stack, ATM

Credentials: UID & GID Environment variables:

inherited from the parent

Hardware context(in PCB of u area): PC, SP, PSW, MMR, FPU

39

User Credentials Superuser: UID=0, GID=1 Real IDs: log in, send signals Effective IDs: file creation and access exec:

suid mode: that of the owner; sgid mode: that of the calling process

setuid / setgid: SV & BSD are different with these

saved UID, saved GID in SV setgroup in BSD

40

The u Area part of the process space, needed only when running.

PCB proc pointer UID s Arguments, results, error status from system calls Signal handlers and related information Info from the program header, text, data, stack size, MM info Open file descriptor Vnodes & controlling terminal pointers CPU usage statistics, profiling info, disk quotas, & resource

limits Per-process kernel stack

41

The proc Process table

Id Location of the kernel address map for the u area Current process state Forward and backward pointers in scheduled queue Sleep channel (7.2.3) Scheduling priority and related (Chapter 5) Signal handling info(Chapter 4). MM info Pointers to link active, free, zombie processes in lists Flags Pointers to keep structures on a hash queue by PID Hierarchy info

42

A typical process hierarchy in 4.3BSD UNIX

Process ID

Parent Process IDpointer to parent’s proc

Pointer to the oldest childPointer to the younger sibling

43

Typical Functions of an Operating-System Kernel

Process Management Process creation and termination Process scheduling and dispatching Process switching Process synchronization and support for inter-

process communication Management of process control blocks

Memory Management Allocation of address space to processes Swapping Page and segment management

44

Typical Functions of an Operating-System Kernel

I/O Management Buffer management Allocation of I/O channels and devices to

processes Support Functions

Interrupt handling Accounting Monitoring

45

Operating System Control Structures

An OS maintains the following tables for managing processes and resources: Memory tables I/O tables File tables Process tables

46

Memory Tables Allocation of main memory to

processes Allocation of secondary memory to

processes Protection attributes for access to

shared memory regions Information needed to manage virtual

memory

47

I/O Tables I/O device is available or assigned Status of I/O operation Location in main memory being used

as the source or destination of the I/O transfer

48

File Tables Existence of files Location on secondary memory Current Status Attributes Sometimes this information is

maintained by a file-management system

49

Process Table Process image consists of program,

data, stack, and attributes Attributes

process control block

50

Process Control BlockProcess Identification

Unique numeric identifier may be an index into the primary process

table User identifier

who is responsible for the job

51

Execution of the Operating System

Nonprocess Kernel execute kernel outside of any process operating system code is executed as a

separate entity that operates in privileged mode

Execution Within User Processes operating system software within context

of a user process process executes in privileged mode when

executing operating system code

52

Execution of the Operating System

Process-Based Operating System major kernel functions are separate

processes a process is invoked by the operating

system

53

The UNIX kernel A special program that runs directly on the

hardware. Implements the process model and services. Resides on disk, in a file /vmunix or /unix. Bootstrapping: loads the kernel. Initializes the system and sets up the

environment, remains in memory before shut down

54

UNIX Services

System call interface Hardware exceptions

Divide by 0, overflowing user stack Interrupts

Devices Swapper, pagedaemon

55

The Kernel interacts with processes and devices

BACK

56

Mode,Space & Context By modes: some critical resources can

be protected. Kernel Mode: More privileged, kernel

functions User Mode: Less privileged, user functions

Virtual Memory VM space Address Translation Maps Memory Management Unit

57

Kernel data Current process & Context switch One instance of the kernel Global data structure Pre-process objects System call, Mode Switch User area: info. about a process Kernel stack:

58

Context UNIX kernel is re-entrant: several

processes may be involved in kernel activities concurrently.

Execution context Process context: System context (Interrupt context):

59

Execution mode and Context

60

Executing in Kernel Mode 3 types of events:

Device interrupts Exceptions Traps or software interrupts

Dispatch table System context: interrupts Process context: traps, exceptions &

software interrupts

61

The System Call Interface

syscall(): the starting point in kernel mode, but in process context. Copy arguments , save hardware

context on the kernel stack. Use system call number to index

dispatch vector Return results in registers, restore

hardware context, to user mode, control back to the library routine.

62

UNIX Interrupt Handling Interrupt handler(interrupt service routine):

runs in kernel mode and system context, not permitted to block.

the time used to service an interrupt charged to the interrupted process

The clock interrupt handler charges the clock tick to the current process

ipl(interrupt priority level)- specified for each interrupt and saved in interrupt register of the processor status word

Interrupts are preemptive

63

Setting the interrupt priority in 4.3BSD and SVR4

64

Interrupt handling

65

Synchronization

UNIX is re-entrant

UNIX is non-preemptive, keeping the kernel states consistent.

Relinquish CPU voluntarily.

66

Blocking Operations Blocks the process (make it sleep). Lock: a single bit flag wanted(flag): sleep(): wake(): wake all the waiting

processes. upon waking up: check once again to

make sure.

67

Algorithm for resource locking

68

Blocking Interrupts Blocking interrupts while accessing critical

sections. int x = splbio();/* raise ipl*/ modify disk buffer cache; splx(x); /*restore ipl*/

Critical region:few & brief. Blocked interrupts may access the critical

region. Different interrupts may have the same ipl Blocking an interrupt may block others.

69

UNIX Process Implementation

70

New Processes & Programs fork:

creates a new process. returns 0 to the child, PID to the

parent exec:

begins to execute a new program

71

Using fork & exec if ((result = fork()==0){ /* child code*/ … … if (execve(“new program”),…)<0) perror(“execve failed!”); } else if (result<0) { perror(“fork”);/*fork failed*/ } /*parent continures here*/

72

Shell creating a process

A highly simplified shell

73

The ls Command

Steps in executing the command ls typed to the shell

74

Process Creation Almost an exact clone of the parent.

Reserve swap space for the child Allocate a new PID and proc structure for the child Initialize proc structure Allocate address translation map (ATM) Allocate u area and copy Update the u area to refer to the new ATM & Swap space Add the child to the set of processes sharing the text region of

the program Duplicate the parent’s data and stack regions update ATM to

refer to these new pages. Acquire references to shared resources inherited by the child Initialize the hardware context Make the child runnable and put it on a scheduler queue Arrange to return with 0 Return the PID to the parent

75

fork Optimization It is wasteful to make an actual copy of

the address space of the parent Copy-on-write (SV) : only the pages that

are modified must be copied vfork (BSD): The parent loans the

address space and blocks until the child returns to it.

dangerous!

csh: exploits it.

76

Invoking a New Program Process address space

Text: code Initialized data: Uninitialized data(bss): Shared memory(SV): Shared libraries: Heap: dynamic space User stack: space allocated by the kernel

77

exec System Call Parse & access Verify the permission Read the header and check if valid executable If the file has SUID or SGID bits set in its mode, change the

caller’s effective UID or GID to that of the owner Copy the arguments to exec and the env. variables into kernel. Allocate swap space for the data and stack region Set up the new address space. Copy the arguments and env. variables back onto the new user

stack. Reset all signal handlers to default actions. Initialize the hardware context.

78

Process Termination (exit())

Turns off all signals. Closes all open files. Releases the text file and other resources such as the current

directory. Writes to the accounting log. Saves resource usage statistics and exit status in the proc

structure. Changes state to SZOMB, and puts the proc on the zombie

process list. Makes the init process inherit any live children of the exiting

process. Releases the address space, u area, ATM, and swap space. Notifies the parent by sending it a SIGCHLD signal. Wakes up the parent if it is asleep. Calls swtch() to schedule a new process to run.

79

Awaiting Process Termination

wait(stat_loc);/* SV, BSD & POSIX*/ wait3(statusp, options, rusagep); /*BSD*/ waitpid(pid, stat_loc, options);/*POSIX*/ waitid(idtype, id, infop, options);/*SVR4*/

80

Zombie Processes after exit(), process holds only proc structure

and is in zombie state until cleaned completely.

wait() (issued by the parent or the init process)frees the proc structure and completes process exit,

If the child dies before the parent which does not wait for it, the proc is not released until the system is rebooted.

Parent can specify that it will not wait for its children

81

LINUX

Modular structure – collection of modules, some of them loadable and unloadable on demand,

Module – object file whose code be linked and unlinked from the kernel at runtime, executed in kernel mode on behalf of the current process

Loadable module characteristic – Dynamic linking, -- Stackable modules – may serve as libraries when they are referenced by client modules higher up in the hierarchy, and as clients when they reference modules further down the hierarchy.

82

83

Linux components

At user level – tasks (specific to Linux – combine features of processes and threads);

At kernel level – interacting collection of components;

Hardware components – in this case IA-64 Intel Itanium architecture

84

85

Linux Task Data Structure

State Scheduling information Identifiers Interprocess communication Links Times and timers File system Address space Processor-specific context

86

Linux States of a Process/Thread

Running Interruptable Uninterruptable Stopped Zombie

87