unix programming environment part 3-1 basic commands and an introduction to b-shell

Unix Programming Unix Programming EnvironmentEnvironment

Part 3-1 Basic Commands and An Introduction to B-Shell

2002/09/25 Draft Version

2002/10/01 Rev1.0

Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA

AgendaAgenda 1. The Basics – getting started ( 1.1 )1. The Basics – getting started ( 1.1 )

2. Overview of the system architecture - concepts2. Overview of the system architecture - concepts

3. using TTY – Unix terminal3. using TTY – Unix terminal

4. An introduction to B-Shell ( 1.4 )4. An introduction to B-Shell ( 1.4 )

OverviewOverview

Filename ExpansionFilename Expansion

Standard I/O – redirect I/OStandard I/O – redirect I/O

PipePipe

Customizing our personal operating environment Customizing our personal operating environment

5. X-Windows5. X-Windows

demodemo


1. The Basics – getting started( 1 )1. The Basics – getting started( 1 )

Reading the book( 1.1 )Reading the book( 1.1 )

1. logging in 1. logging in

2. some basic commands2. some basic commands

3. interrupt the running programs3. interrupt the running programs

4. logout4. logout

5. mail & write5. mail & write

6. online manual – man6. online manual – man

WinhelpWinhelp

man manman man

man –k “command”man –k “command”

xman, gnome-help-broswer, web-based manualxman, gnome-help-broswer, web-based manual


1. The Basics – getting started( 2 )1. The Basics – getting started( 2 ) Sections: Sections:

1. user commands; (1M. administrator commands )1. user commands; (1M. administrator commands )

2. system calls2. system calls

3. library functions( standard C, networking, threads, etc )3. library functions( standard C, networking, threads, etc )

4. File formats4. File formats

5. Standards, Environments and Macros5. Standards, Environments and Macros

6. Demos6. Demos

7. Device and Network Interfaces7. Device and Network Interfaces

9. DDI and DKI 9. DDI and DKI


2. Overview of the system architecture2. Overview of the system architecture


3. Using Unix terminals – TTY ( 1 )3. Using Unix terminals – TTY ( 1 ) 1. Scenarios1. Scenarios

H/W: host + serial ports(Hyerterm, minicom), host + NIC( telnet, rsh ), PC( H/W: host + serial ports(Hyerterm, minicom), host + NIC( telnet, rsh ), PC( CRT+ Adaptor + KB )CRT+ Adaptor + KB )

X-based App, Screen-based App, line-based AppX-based App, Screen-based App, line-based App

Job control & signalsJob control & signals

2. the internal structure: hierarchical model2. the internal structure: hierarchical model

General TTY data structure,General TTY data structure,

Line Disciplines, Line Disciplines,

TTY Drivers( pty, serial )TTY Drivers( pty, serial )

3. TTY’s Line Discipline3. TTY’s Line Discipline

““echo”; echo”;

Mode: canonical, raw, cbreakMode: canonical, raw, cbreak

Special Control CharactersSpecial Control Characters

The “stty” command: Demo Linux & SolarisThe “stty” command: Demo Linux & Solaris


3. Using Unix terminals – TTY ( 2 )3. Using Unix terminals – TTY ( 2 )

4. POSIX Programming Interfaces4. POSIX Programming Interfaces

See AUPE – Chapter 11, 19See AUPE – Chapter 11, 19

5. other applications using terminal I/O5. other applications using terminal I/O

Printer, vi( ncurse ), PPP, PPPOEPrinter, vi( ncurse ), PPP, PPPOE

6. Further Readings: 6. Further Readings:

<AUPE> - C11( POSIX API ), C19( PTY )<AUPE> - C11( POSIX API ), C19( PTY )

< Linux Source > - tty.c/h, n_tty.c, serial.c< Linux Source > - tty.c/h, n_tty.c, serial.c

< Design of UNIX System >, Bach< Design of UNIX System >, Bach

< ncurse lib >< ncurse lib >


4.1 An Introduction4.1 An Introduction of Unix Shells (1)of Unix Shells (1) 1. What is a shell?1. What is a shell?

A shell is simply A shell is simply a macro processora macro processor that executes commands. A Unix shell is both a that executes commands. A Unix shell is both a command interpreter and a programming languagecommand interpreter and a programming language..

a command interpretera command interpreter : : providing the user interface to the rich set of Unix utilities.providing the user interface to the rich set of Unix utilities.

a programming languagea programming language: allowing these utilitites to be combined. Files containing : allowing these utilitites to be combined. Files containing commands can be created, and become commands themselves. Like any high-level commands can be created, and become commands themselves. Like any high-level language, the shell provides variables, flow control constructs, quoting, and language, the shell provides variables, flow control constructs, quoting, and functions.functions.

Shells may be used Shells may be used interactively or non-interactivelyinteractively or non-interactively: they accept input typed from the : they accept input typed from the keyboard or from a file.keyboard or from a file.

A shell allows execution of Unix commands, both A shell allows execution of Unix commands, both synchronously and asynchronouslysynchronously and asynchronously..

The shell waits for synchronous commands to complete before accepting more The shell waits for synchronous commands to complete before accepting more input; asynchronous commands continue to execute in parallel with the shell while it input; asynchronous commands continue to execute in parallel with the shell while it reads and executes additional commands. reads and executes additional commands.

The The redirection constructsredirection constructs permit fine-grained control of the input and output of those permit fine-grained control of the input and output of those commands, and the shell allows control over the contents of their environment. commands, and the shell allows control over the contents of their environment.

Unix shells also provide Unix shells also provide a small set of built-in commandsa small set of built-in commands (builtins) implementing (builtins) implementing functionality impossible (e.g., cd, break, continue, and exec), or inconvenient (history, functionality impossible (e.g., cd, break, continue, and exec), or inconvenient (history, getopts, kill, or pwd, for example) to obtain via separate utilities. getopts, kill, or pwd, for example) to obtain via separate utilities.


4.1 Overview of Unix Shells (2)4.1 Overview of Unix Shells (2) 2. Shell Operation2. Shell Operation

Reads its inputReads its input from a file, from a string supplied as an argument to from a file, from a string supplied as an argument to the `-c' invocation option, or from the user's terminal. the `-c' invocation option, or from the user's terminal.

Breaks the input into words and operatorsBreaks the input into words and operators, obeying the , obeying the quoting quoting rulesrules. These tokens are separated by . These tokens are separated by metacharactersmetacharacters. . AliasAlias expansion is performed by this step. expansion is performed by this step.

Parses the tokens into simple and compound commands. Parses the tokens into simple and compound commands.

Performs the Performs the various shell expansionsvarious shell expansions, breaking the expanded , breaking the expanded tokens into lists of filenames (“Filename Expansion”) and commands tokens into lists of filenames (“Filename Expansion”) and commands and arguments. and arguments.

Performs any necessary Performs any necessary redirectionsredirections and removes the redirection and removes the redirection operators and their operands from the argument list. operators and their operands from the argument list.

ExecutesExecutes the command. the command.

Optionally Optionally waitswaits for the command to complete and collects its exit for the command to complete and collects its exit status.status.


4.1 Overview of Unix Shells (3)4.1 Overview of Unix Shells (3) 3. Variants of Unix Shells3. Variants of Unix Shells

The Bourne Shell( /bin/sh ): The Bourne Shell( /bin/sh ):

Developed by Steve Bourne at Bell Labs. Its control flow constructs Developed by Steve Bourne at Bell Labs. Its control flow constructs are reminiscent of Algol 68. are reminiscent of Algol 68.

C-Shell: C-Shell:

Done by Bill Joy at Berkeley. It was build on the 5th Edition shell( not Done by Bill Joy at Berkeley. It was build on the 5th Edition shell( not the Bourne shell). Its control flow looks more like the C language, and the Bourne shell). Its control flow looks more like the C language, and supports additional features – job control, a history mechanism, and supports additional features – job control, a history mechanism, and command-line editing. command-line editing.

The Korn Shell( /bin/ksh ):The Korn Shell( /bin/ksh ):

A successor to the Bourne shell and being provided in SVR4. A successor to the Bourne shell and being provided in SVR4.

Developed by David Kern at Bell Labs. It is upward-compatible from Developed by David Kern at Bell Labs. It is upward-compatible from the Bourne shell and included those features that mode the C shell the Bourne shell and included those features that mode the C shell popular.popular.

Bash: Bash:

the GNU Shell, used in GNU O.S. like Linux, FreeBSD and Cygwin.the GNU Shell, used in GNU O.S. like Linux, FreeBSD and Cygwin.

To get details about Bash, please read the supplementary reading.To get details about Bash, please read the supplementary reading.


4.2 B-Shell: Filename Expansion (1)4.2 B-Shell: Filename Expansion (1) 1. Pattern Matching1. Pattern Matching

The special pattern characters have the following meanings: The special pattern characters have the following meanings:

* * : Matches any string, including the null string. : Matches any string, including the null string.

?? : Matches any single character. : Matches any single character.

[...][...] : Matches any one of the enclosed characters. : Matches any one of the enclosed characters.

The special pattern characters The special pattern characters MUST be quotedMUST be quoted if they are to be matched if they are to be matched literallyliterally. .

2. Filename Expansion2. Filename Expansion After word splitting, sh scans each word for the characters `*', `?', `(', and After word splitting, sh scans each word for the characters `*', `?', `(', and

`['. `['.

If one of these characters appears, then the word is regarded as a If one of these characters appears, then the word is regarded as a patternpattern, and replaced with an , and replaced with an alphabetically sortedalphabetically sorted list of file names list of file names matching the pattern. matching the pattern.

If no matching file names are found, If no matching file names are found, the word is left unchangedthe word is left unchanged. .

Options in Bash: left unchanged, or removedOptions in Bash: left unchanged, or removed

The character The character `.' at the start of a filename`.' at the start of a filename or immediately following or immediately following a a slashslash must be matched explicitly. When matching a file name, the slash must be matched explicitly. When matching a file name, the slash character must always be matched explicitly. character must always be matched explicitly.


4.2 B-Shell: Filename Expansion (2)4.2 B-Shell: Filename Expansion (2) Examples from the textbook( p20, 21 )Examples from the textbook( p20, 21 )

PathnamePathname

‘‘*’, ‘?’ can not be used to create files and directories*’, ‘?’ can not be used to create files and directories

Quoting: Quoting:

Escape Character Escape Character

Example: $ ls \?Example: $ ls \?

Single Quotes Single Quotes

Example: $ ls ‘?’Example: $ ls ‘?’


4.5 B-Shell: Process ( 1 )4.5 B-Shell: Process ( 1 )

1. Introduction1. Introduction

A “process” is A “process” is a virtual machinea virtual machine which provides the runtime which provides the runtime environment for executable programs. environment for executable programs.

Process Context: ( proc struct, u-area ), PIDProcess Context: ( proc struct, u-area ), PID

The process-local file descriptor table: FSThe process-local file descriptor table: FS

Memory Mapping table: memoryMemory Mapping table: memory

The hierarchical structure: the parent process( ppid ) and its The hierarchical structure: the parent process( ppid ) and its context context

We will further understand “Process” using O.S. designers’ eyes We will further understand “Process” using O.S. designers’ eyes in Part-6 for these basic concepts are critical for our in Part-6 for these basic concepts are critical for our programming. programming.


4.5 B-Shell: Process ( 2 )4.5 B-Shell: Process ( 2 ) 2. examples from the textbook: 2. examples from the textbook:

date; whodate; who

wc ch* > wc.out &wc ch* > wc.out &

pr ch* | lpr & pr ch* | lpr &

wait, kill pid, nicewait, kill pid, nice

psps

nohupnohup

at time, batchat time, batch

3. exit status3. exit status

Every command returns an exit status (sometimes referred to as a return status ). A successful command returns a 0, while an unsuccessful one returns a non-zero value that usually may be interpreted as an error code.The exit status must be a decimal number in the 0 - 255 range.

$? gives the exit status of the last command executed in the function.

See waitpid() to get more information on how to encode the return status.


4.5 B-Shell: Process ( 3 )4.5 B-Shell: Process ( 3 ) 4. Job Control4. Job Control

Job control refers to the ability to selectively stop (suspend) the execution of Job control refers to the ability to selectively stop (suspend) the execution of processes and continue (resume) their execution at a later point. processes and continue (resume) their execution at a later point.

A user typically employs this facility via an interactive interface supplied jointly A user typically employs this facility via an interactive interface supplied jointly by the system's terminal driver and a shell supporting job control. by the system's terminal driver and a shell supporting job control.

The system maintains the notion of a current terminal The system maintains the notion of a current terminal process group IDprocess group ID. . Members of this process group (processes whose process group ID is equal to Members of this process group (processes whose process group ID is equal to the current terminal process group ID) receive keyboard-generated signals the current terminal process group ID) receive keyboard-generated signals such as SIGINT. These processes are said to be in such as SIGINT. These processes are said to be in the foregroundthe foreground. . Background processesBackground processes are those whose process group ID differs from the are those whose process group ID differs from the terminal's; such processes are immune to keyboard-generated signals. Only terminal's; such processes are immune to keyboard-generated signals. Only foreground processes are allowed to read from or write to the terminal. foreground processes are allowed to read from or write to the terminal. Background processes which attempt to read from (write to) the terminal are Background processes which attempt to read from (write to) the terminal are sent a sent a SIGTTIN (SIGTTOU) signalSIGTTIN (SIGTTOU) signal by the terminal driver, which, unless caught, by the terminal driver, which, unless caught, suspends the process.suspends the process.

““^Z”, jobs, fg, bg^Z”, jobs, fg, bg

We will model these concepts in Part-6, and discuss programming interfaces.We will model these concepts in Part-6, and discuss programming interfaces.


4.5 B-Shell: Process ( 4 )4.5 B-Shell: Process ( 4 )

Session

Background Process GroupSession Leader = Controlling

Process Background Process Group

Foreground Process Group

ControllingTerminal

Han

gup

Sig

nal

Terminal In

put and terminal-generated signals


4.3 B-Shell: I/O Redirection (1)4.3 B-Shell: I/O Redirection (1)

1. Introduction

For “processes”, there are always three default "files" open, stdin (the keyboard), stdout (the screen), and stderr (error messages output to the screen), which are inherited from their parent processes.

Before a command is executed, its process’s input and output may be redirected using a special notation interpreted by the shell.

Redirection may also be used to open and close files for the current shell execution environment.


4.3 B-Shell: I/O Redirection (2)4.3 B-Shell: I/O Redirection (2)

2. Redirecting Input: [n]<2. Redirecting Input: [n]<wordword

3. Redirecting Output: [n]>[|]3. Redirecting Output: [n]>[|]wordword

4. Appending Redirected Output: [n]>>4. Appending Redirected Output: [n]>>wordword

5. Redirecting Standard Output and Standard Error5. Redirecting Standard Output and Standard Error

&>&>wordword, >&, >&wordword, semantically equivalent to “>, semantically equivalent to “>wordword 2>&1” 2>&1”

6. Here Documents6. Here Documents

7. Duplicating File Descriptors 7. Duplicating File Descriptors

8. Opening File Descriptors for Reading and Writing 8. Opening File Descriptors for Reading and Writing

We discuss the features(6, 7, 8) in the Chapter 3.We discuss the features(6, 7, 8) in the Chapter 3.


4.4 B-Shell: Pipelines4.4 B-Shell: Pipelines

1. Introduction1. Introduction

““Pipelines” should be the ancestor of the middleware tech. Pipelines” should be the ancestor of the middleware tech.

The format: [time [-p]] [!] The format: [time [-p]] [!] command1command1 [| [| command2command2 ...] ...]

If the pipeline is If the pipeline is not executed asynchronouslynot executed asynchronously, the shell waits for , the shell waits for all commands in the pipeline to complete. all commands in the pipeline to complete.

Each command in a pipeline is executed in its own subshell. The Each command in a pipeline is executed in its own subshell. The exit status of a pipeline is exit status of a pipeline is the exit status of the last commandthe exit status of the last command in in the pipeline. If the reserved word the pipeline. If the reserved word `!'`!' precedes the pipeline, the exit precedes the pipeline, the exit status is the logical negation of the exit status of the last status is the logical negation of the exit status of the last command. command.

How does the UNIX kernel support “pipelines”? How does the UNIX kernel support “pipelines”?

How does the shell implement “pipelines”?How does the shell implement “pipelines”?

who | grep mary | wc -lwho | grep mary | wc -l


4.6 B-Shell: Customizing Environment(1)4.6 B-Shell: Customizing Environment(1) 1. Shell Startup files 1. Shell Startup files

the POSIX standard for startup files: the POSIX standard for startup files:

`/etc/profile‘: `/etc/profile‘: systemwide defaults

`~/.profile‘: `~/.profile‘: user-specific default settings

For Bash: For Bash:

/etc/profile: systemwide defaults, mostly setting the environment (all Bourne-type shells, not just Bash)

/etc/bashrc: systemwide functions and aliases for Bash

$HOME/.bash_profile: user-specific Bash environmental default settings, found in each user's home directory (the local counterpart to /etc/profile)

$HOME/.bashrc: user-specific Bash init file, found in each user's home directory (the local counterpart to /etc/bashrc). Only interactive shells and user scripts read this file. For example, Bash is being run by the remote shell daemon, usually rshd.


4.6 B-Shell: Customizing Environment(2)4.6 B-Shell: Customizing Environment(2) 2. An Introduction to Variables

carefully distinguish between the name of a variable and its value.

If variable1 is the name of a variable, then $variable1 is a reference to its value, the data item it contains.

$variable actually a simplified alternate form of ${variable}.

tmp=/tmp/ps; ps a > ${tmp}a # ps a > /tmp/psa

Unlike many other programming languages, b-shell does not segregate its variables by "type". Essentially, b-shell variables are only strings-valued.

But, depending on context, Bash permits integer operations and comparisons on variables.

An uninitialized variable has a "null" value - no assigned value at all (not zero!).

the assignment operator (no space before & after)

Note that = can be either an assignment or a test operator, depending on context.


4.6 B-Shell: Customizing Environment(3)4.6 B-Shell: Customizing Environment(3) local variables: variables visible only within a code block or function

environmental variables:

variables that affect the behavior of the shell, the user interface and the child processes

“export”

Special variables: Special variables:

The command “set”: The command “set”: display all defined variablesdisplay all defined variables

HOME, PATH, PS1, PS2, IFS, TERMHOME, PATH, PS1, PS2, IFS, TERM

unix programming environment part 3-1 basic commands and an introduction to b-shell

Documents

programming language

execution of unix commands

unix terminals tty

tty unix terminal4

introduction of unix

user commands

c design of unix system

administrator commands