spsl unit1
Post on 23-Jan-2017
344 Views
Preview:
TRANSCRIPT
Shell Programming & Scripting Languages
Dr.K.Sasidhar
Shell Programming & Scripting LanguagesCourse Objectives
This course explains UNIX Operating System in thorough with
its commands, editor, file systems along with applications.
Implementation of Shell scripts with Bash shell programming
using functions, Process communication, Administration.
Scripting languages extends the functionality of existing
software. This was addressed by the Python language with its
features and programming with object oriented concepts.
This course also explains about the designing of integrated web
Applications and Data base interfaces.
Course Outcomes
Identify the basic commands as well as different utilities of the UNIX & analyze why to use unix.
Understand the commands, functions and Practice with real time applications
Learn about shell and how to write, debug & execute shell script along with different commands.
Explain file handling and directory handling using different system calls.
Discuss the fundamentals of the Process as well as Signal functions & controlling of process using signals.
Learning the Python language programming with its features Design the web applications through python Implementation of object oriented programming through python.
Prerequisites
Dos commands C programming
UNIT – I Contents Introduction to Unix:- Architecture of Unix,
Features of Unix , Unix utilities – process utilities, disk utilities, networking commands, text processing utilities and backup utilities.
Introduction to UNIX file system, vi editor, file handling utilities, security by file permissions.
Software Collection of Programs 2 types
SystemSoftware
ApplicationSoftware
Software
Application software Easy-to-use programs designed to perform specific
tasks System software
Programs that support the execution and development of other programs
Two major types Operating systems Translation systems (compilers & linkers)
Software
Computer Software Relationships
UserInterface
Basic Input and Output Services (BIOS)• needed for a computer to boot up
User Interface Operating System
User Interface Application Programs
Computer Hardware
Controls and manages the computing resources Examples
Windows, Unix, MSDOS, Important services that an operating system provides:
Security: prevent unauthorized users from accessing the system
Commands to manipulate the file system Input and output on a variety of devices Window management
Operating System
Operating SystemAn Operating System is a system software that provides interface between the user and computer hardware. Operating System controls (manages) hardware and software.provides support for peripherals such as keyboard, mouse, screen, disk drives, … software applications use the OS to communicate with
peripherals. The OS typically manages (starts, stops, pauses, etc)
applications.
What is UNIX
• UNIX is a Multi-user and Multi-tasking operating system.
• That is it is designed to be used by many people at the same time (multi-user) and for multiple tasks at a time.
• Runs on a variety of processors• It provides a number of facilities:
– management of hardware resources– directory and file system– loading / execution / suspension of programs
12
UNIX versionsM
ultic
s
Unic
s
UNIX
V1
UNIX
V3
UNIX
V4
UNIX
V7
1965 1969 1970 1973 1973 1979BSD U
NIX
AT&T UNIX
BSD
4.1
BSD
4.2
BSD
4.4
1981 1983 1993
System
IIISys
tem V
1982 1984
1998 – X/OPEN & IEEE started work to merge two standards2001 – Single UNIX Specification Version 3 (SUSV3) Also known as IEEE1003.1:20012002 – ISO standardized SUSV3 and IEEE1003.1:2001
Multiple StandardsAT&T – System V Interface DefinitionX/OPEN – X/OPEN Portability Guide (XPG)IEEE – Portable Operating System Interface for Computing Environments (POSIX)
UNIX History
The UNIX operating system was developed at AT & T Bell Labs in late 1960s.
It originally began as a one man project led by Ken Thompson of Bell Labs, and has since grown to become the most widely used operating system.
In the time since UNIX was first developed, it has gone through many different generations and even mutations.
Some differ substantially from the original version, like Berkeley Software Distribution (BSD) or Linux.
Why to Use UNIX?
• Multi-tasking / multi-user
• Networking capability
• Distributed Processing
• graphical (with command line)
• Easy to program
• portable (PCs, mainframes, super-computers)
15
Features of UNIX Operating System Multi-user & Multi-tasking – UNIX allows multiple users to log onto the
system, and have each run multiple tasks. This is standard for most modern OS.
Large Number of Applications – There are an enormous amount of applications available for UNIX. They range from commercial applications such as CAD, Maya, WordPerfect, to many free applications.
Free Applications and Open source Operating System - of all of the applications available under UNIX, many of them are free. Most of the development that we do in programming courses is done under the Linux OS.
Less Resource Intensive - UNIX installations tend to be much less demanding on system resources. In many cases, the old family computer that can barely run Windows is more than sufficient to run the latest version of Linux.
Internet Development - Much of the backbone of the Internet is run by UNIX servers. Many of the more general web servers run UNIX with the Apache web server - another free application.
Structure of the UNIX system
Applications
Shell
\Kernel (OS)
Hardware
There are many standard applications:
• file system commands
• text editors
• compilers
• text processing
UNIX System Structure ….. The Kernel - handles memory management, input and output
requests, and program scheduling. The kernel is the heart of OS. It provides the basic software connection to the hardware.
The Shell and Graphical User Interfaces (GUIs) - UNIX shell provides a “command line” interface which allows the user to type in commands. These commands are translated by the shell into something the kernel can comprehend, and then executed by the kernel.
Primary command-line shells are: Bourne shell, Bash Shell, korn Shell, C Shell
The Built-in System Utilities - Programs that allow a user to perform tasks which involve complex actions. Utilities provide user interface functions that are basic to an operating system.
Examples: Commands to see the contents of a directory, move & copy files, remove files, etc...
Application Software & Utilities – The programs that are bundled with the OS distribution, or available separately. These can range from additional or different versions of basic utilities, to full scale commercial applications.
UNIX System Structure …..
Flavors of Unix
There are many versions of Unix: SCO UNIX
SysV (from AT&T) BSD (from Berkeley) Solaris (Sun) IRIX (SGI) AIX (IBM) LINUX (free software)
File System is a group of files The relevant information arranged in a logical manner. Different Operating systems have different file systems. In Unix, everything will be treated as a file. Whether it is application or
device or utility or data or directory It is similar to tree data structure. The files will be denoted with absolute or relative path
Ex: /home/usr2/file1 absolute path Relative path is nothing but pwd (present working directory) cd ram relative path To check the block size the command is $cmchk
The UNIX File System
The UNIX File System
• A simplified UNIX directory/file system:
/ Root
bindev lib
User
home/usr tmp. . .
. .
. User2
User1 User3
22
cat date
etc
• /bin: contains executable files for most of the unix commands.
• /dev: contain files that control various input & output devices.
• /etc: contains system administration files, such as password file.
• /lib: contains all the library functions in binary form.• /usr: contains several directories each associated with a
particular user.• /tmp: contain the temporary files created by unix or by
any user.• /etc: contains configuration files of the system.
23
• File: It is a container for storing information.
• A file is of the following types.
• Text file or Ordinary file : contains printable characters.
Ex: program source code, documents, scripts
• Binary file: contains both printable & non printable characters. Ex: gif, jpeg, executable etc..
• Directory file: It maintains some details of the files & subdirectories that it contains.
24
Files
• Every directory entry contains 2 components.• 1.file name.• 2. Unique identification number for the file or
directory.
25
Files The file systems in UNIX maintains with four blocks
Boot Block Bootstrap loader program Super Block state of the file system– how large it
is, etc.. Inode Table Data Block contains actual file contents. An
allocated block can belong to only one file in the file system.
Inode Table The information related to all the files is stored in an Inode Table on the
disk. For each file there is an inode entry in the table. Each entry is made up of 64 bytes and contains relevant details for that
file. The details are:
Owner of the file Group to which the owner belongs Type of the file File access permissions Date and time of last access Date and time of last modification Number of links to the file Size of the file Addresses of the blocks where the file is physically present
How Does Unix Access Files? A file is identified by Unix by a unique “ Inode number”
associated with it. Can display the inode number with a file by using the
command ls-i Ex: $ls – i reports
reports 12324 Here 12324 is inode number. The inode no is nothing but the index into the inode table
where the information about the file is stored. Among all the slots the inode table slot number 12324 contains
information about the file reports
Storage of Files Each inode entry in the inode table specifies
completely where the contents of the files are stored on the disk.
These addresses may be numbered 0 through 12. The addresses 0 to 9 point to 1kb blocks on disk. Ex: a file of 3kb may have its entries as shown in the
following diagram.
owner GroupFile Type
PermissionsAccess Time
Modification Time Inode Modi. Time File size 0123456
12
497052313401
4970
5231
3401
Creating files cat
Used to create a fileTo append the filesTo display the file content.
$cat > test Type the content of the file and press Ctrl d to
stop / save the file. $cat test
Displays the content of file test. $cat sample1 sample2 >> newsample
newsample file will be created with the contents of sample and sample2
Creating files …
If new sample already contains something it would be overwritten.
If you want it should remain intact and content of sample1 and sample2 should get append to it then use
“ append output redirection operator” >> Ex: $ cat sample1 sample2 >> newsample
Cp (Copying Files)• To create an exact copy of a file you can use the “cp” command.
Syntax: cp [-option] source destinationEg: cp file1 file2Here the contents of file1 is copied to file2. if file2 is not existed
then it will be created.
Eg: cp file1 file2 dirfile1 file2 contents are copied to the directory named as dir.
• To copy files from or to different directories is by specifying their names with the path.
• $cp /usr/aa/chapter1 /usr/aa/newbook/chap1
33
File handling utilities
Cp turns to interactive when –i option is used &
destination file also exists.
$cp -i file1 file2overwrite file2 (yes/no)?
Y at this prompt overwrites the file.
34
Copying Files
• mv (Moving and Renaming Files)
Used to rename the files / directories.
$ mv test sample
Here test is renamed as sample, provided sample directory name is not already existing.
• Moving a file implies removing it from its current location and copying it at a new location.
35
mv command
Listing Files and Directories
$ls lists files and directoriesls – a hidden file. Displays the hidden files also.$ ls p* lists the files starts with p$ls ?ain lists the files whose names match with ain
from second character onwardsls [a-m][c-z][4-9]?? this will list all 5 character
filenames in the current directory whose first character is in the range a to m,
second character in the range c to z, the third character is in the range 4 to 9, where as the fourth and fifth character are any valid characters.
ls - l ls –l
long listing Out put: total 22 no. of disk blocks occupied -rwxr-x—x 1 user1 group 02 jan 16 10:15 ecm32 drwxr-xr-x 1 user1 group 04 jan 16 12:23 ecm31
File types ( meaning of first character)
File Type Meaning
- Ordinary file
d Directory file
c Character special file
b Block special file
l Symbolic link file
s Semaphore
p Named pipe
M Shared memory file
file command Recognizes several types of files. Works with wild cards also $ file *
• Used to create links (both soft & hard links).• It creates the alias & increase the link count by one.• $ln file1 file2• Hard link means the link will be established with the
data on the disk• Softlink is nothing but symbolic link that refers to
existed filename. ( it refers to fine name itself.)
40
ln command
Hard Link hard links : Refer to the specific location of physical data.
Soft Link symbolic links: Refer to a symbolic path indicating the abstract location of another file
wc command Counts the number of lines, words and
characters in the specified file or files.
-l counts and display no. of lines in the file -w counts and display no. of words in the file -c counts and display no. of bytes in the file. -m displays the no. of characters in the file. -L displays the length of the longest line in
the file.
$ wc –lc file1 file2
• Rm (Deleting Files)– To delete or remove a file, you use the “rm”
command. – Ex 1: $rm mylisting will delete “mylisting”. -f deletes the file forcibly –i option removes the files interactively. Ex 2: $rm –i file1 With –r option recursively removes directories. $rm –r dir1
44
• rmdir (remove directories )• Removes empty directories.• $rmdir book removes directory named book if it is empty.• $rmdir dbs doc dmc Removes the directories dbs doc dmc.
45
• find: Searches the files in a directory hierarchy • It recursively examines a directory tree to look for
matching file and then takes some action on the selected files.
Syntax:• find path_list selection_criteria action• To locate all files named a. out use$find / -name a. out –print• ‘/’ indicates search should start from root directory.
46
• To locate all c files in current directory $find . -name “*.c” –print• To find all files begin with an uppercase letter use $find . –name ‘[A-Z]*’ –print Find operators:Find uses 3 operators!,-a ,-o
47
Security by file permissionsUnix follows a 3-tiered file protection system.There are 3 categories- owner ,group ,others.
owner’s permission group permission other’s permission r w x r - x r - -Each category contains read ,write ,execute permissions .
rwx->presence of all permissions.
r-x->absence of write permission
r-- -> absence of write ,execute permission
48
Security by file permissions• Chmod: change file permission mode
chmod sets a file’s permission (read, write and execute) mode for all three categories of users (owner, group and others)
49
Syntax
chmod category operation permission file(s)
The command contains three components:
• category of user (owner, group or others)
• operation to be performed (assign or remove a permission)
• Permission type (read, write or execute)
50
• Abbreviations used by chmodCategory
u-user g-group o-others
operation +-assign permission -remove permission=-assigns absolute permission a-all
permissionsr-read permission w-write permission
x-execute permission
51
octal methodPermission weight
read(r) 4 write (w) 2 execute (e) 1 Ex: $chmod 700 myfile
chown This command changes file owner and / or
group ownership for each given file.
Syntax: chown [OPTION]... [OWNER][:[GROUP]] FILE Ex1: chown ramu file1 Ex2: chown ram /path/to/file1 /path/to/file2
/path/to/file3 or $ chown ram /path/to/{file1, file2, file3}
Disk related Commands
$df: disk filesystem Displays free as well as used disk space for all the file systems on the
computer. i.e. The information of device name, total blocks, total disk space, used
disk space, available disk space and mount points on a file system.
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/cciss/c0d0p2 78361192 23185840 51130588 32% / /dev/cciss/c0d0p5 24797380 22273432 1243972 95%
/home Ex: $df
/ (/dev/root): 12970 blocks 27857 i-nodes The root file systems is /dev/root.df reports the number of free disk
blocks and free inodes for this file system. If we want a more detailed information about disk usage we have to
use: $df -ivt
disk utilitiesulimit
user limit and it contains a value which signifies the largest file that can be created by the user in the file system.
Ex: $ulimit 2097152 the user cannot create a file whose size
is bigger than 2097152 bytes, or 2048 KB. To reduce this value:$ulimit 1 Allocates 512 bytes of space. After the user logout the
value will change
General Commands
cal calendar $cal 2 2009
banner prints a message in large letters which
looks like a banner. $ banner ECM III Year
file commands …rm used to remove a file. If you have write
permission to the file, then only you can remove the file.
mkdir creates a directory.$mkdir ecm$mkdir –p works/bpb/unix/book$mkdir dbs doc dmcCreates three directories
pwd: present working directory
sort used to sort the contents of a file. Can merge multiple sorted files and store the
result in the specified output file. $ sort myfile To sort the contents of several files at one shot: $sort –o result file1 file2 file3 To merge already sorted files sort –m file1 file2
Text Processing Utilities
Sort options-b Ignores leading spaces and tabs-c Checks if files are already sorted. If already
sorted then it do nothing-d Sorts in directory order-f Ignore cases-m Merges files that have already been sorted.-n Sorts in numeric order-ofile Sorts output file.-r Reverses sort-tc Separates fields with character-u Unique output
cut command It cuts or picks up a given number of
character or fields from the specified file. In a database, to see only some fields( say 2nd,
7th fields) then use cut command. $cut –f 2,7 emp To see from 2nd field upto 7th field. $cut –f 2-7 emp
head, tail – Other text processing Utilities To view the beginning or end of the lines of a
file use head and tail commands.
$head -5 myfile$pg +10 -15 -p “page no. %d” –s myfile
Starts displaying of myfile contents, 15 lines at a time from 10th line onwards. At the end of each page the page no will be displayed.
diff command Compares two files, directories and reports all
the differences between two files.
diff file1 file2
63
more• more displays files on a page-by-page basis• Format: more [options] filename
• Options (selected):• -n : number of lines to show at a time
• Commands to use while in more:• space : advance a page at a time• return : advance a line at a time• q : quit more
more examplemore +10 -15 -s –d myfile yourfile -s option squeeze multiple blank lines in a file
to a single blank line. -d option changes the normal -- more –
prompt displayed at the end of each page to explanatory ‘[ hit space to continue, delete to abort]’
65
uniq
• uniq removes duplicate adjacent lines from a file• To ensure that all of the data is unique you should sort the file first
(recall that sort also can produce unique values with the –u option)
• Format: uniq [options] filename
• Options (selected):
-c : count the instances
-d : print only the duplicates (once)
-u : print only the unique lines
66
uniq Examples
uniq –u abc only prints out the unique lines in file abc
uniq –c abc only prints out the count of unique lines assuming
duplicates are next to each other
Calculations in Unix
$bc 10/2*2 quit$bc for(i=1;i<=5;i++) I 1
2345
Process utilities
• Ps (process status)• Display some process attributes.• Ex: $ps
PID TTY TIME CMD
1078 pts/2 0:00 bash
• Ps presents a snapshot of the process table.
68
Process utilities
• –f option displays a full listing that includes the
PPID.
• –u option followed by user-id displays the processes
owned by the user-id.
• –e option displays the system processes.
• -a option displays all processes for all the users.
• -t option displays the process launched at a particular
terminal 69
nohup command The processes we have executed should not
die even after we log out, for this purpose we have to use nohup command.
nohup stands for no hangups. Ex: $ nohup sort emp.dat > output.emp If we don’t mention the output file then the
sorted output will be stored in the file nohup.out
Killing a process
In case of system hanging, we have to kill the running process.
To kill the process, we must know the process-id using ps command.
Ex: kill 9876
Changing Process Priorities The priority of a process is decided by a
number associated with a number. This number is called nice value of the process.
The nice value of a process can range between 0 to 39.
20 is the default value of the process. nice cat employee.dat nice value increases to 30 $nice -15 cat employee.dat
at command, batch command Used to execute the unix commands at a
future date and time. $at 16:30 echo “ it’s 4:30 PM! Backup your files and
logout” ctrl + dbatch command:The system decides when to execute a
sequence of commands. i.e. whenever it is free, the system will execute these commands.
crontab command More powerful than at command. Carry out a
submitted job everyday for years together.
• Who: know the users
• Displays the users currently logged in the system.
• $who
• Whoami: Show you the owner of this account
• $whoami
• W: Tell you who is logging in and doing what!
• $w
75
• Finger: Displays the information about the users. Who are connected and who can receive the messages
• $finger user
Find out the personal information of a user
• $finger name
Try to find the person’s info. by his/her name
• finger email-address
Try to find the person’s info across the network76
finger options -i displays the idle time of the system
Ex: $ finger –iLogin tty when idleRamu *tty01 Wen Jan27 8 Minutes 14
seconds
* means message is set to -n
Back Up commands ufsdump
1. Used for complete file system backup .2. It copies every thing from regular files in a file system to special character and block device files.2. It can work on mounted or unmounted file systems.
tar:1. Used for single or multiple files backup .2. Can’t backup special character & block device files ( 0 byte files ).3. Works only on mounted file system.
cpio:1. Used for single or multiple files backup .2. Can backup special character & block device files .3. Works only on mounted file system.4. Need a list of files to be backed up .5. Preserve hard links and time stamps of the files .
Compress and Un-Compress files compress -v file_name
gzip filename uncompress file_name.Z
orgunzip filename
Disk utilities
• Du: disk usage
• Du command estimate the file space usage on the disk.
• It produces a list containing the usage of each subdirectory of its argument and finally produces a summary.
• $du /home/usr1
80
Disk utilities
• Df: displays the amount of free space available on the disk. The output displays for each file system separately.
• $df
• Mount:
• Used to mount the file systems.
• Takes 2 arguments-device name ,mount point.
81
Disk Utilities
• Mount uses an option to specify the type of file system.
• To mount a file system on the /oracle directory on Linux system use
$mount –t ext2 /dev/hda3 /oracle
$mount –t iso9660 /dev/cdrom /mnt /cdrom
$mount –t vfat /dev/hda1 /msdos
$mount –t msdos /dev/fd0 /floppy82
Disk utilities
• Umount: unmounting file systems• Unmounting is achieved with the umount command.
which requires either file system name or the mount point as argument.
• $umount /oracle• $umount /dev/hda3• Unmounting a file system is not possible if the file is
opened.
83
• ulimit: user limit
• It contains a value which signifies the largest file that can be created by the user in the file system.
• When used by itself it displays the current setting.
• $ulimit
unlimited
User can also set the ulimit value by using
$ulimit 10
84
unmask:
When u create files and directories, the default permissions that are assigned to them depend on the system’s default setting. Actually this default is transformed
By subtracting the user mask from it to remove one or more permissions. This value is evaluated by umask without arguments.
$umask
02285
Networking commands
• ftp: file transfer protocol
• ftp is used to transfer files. It can be used
with host name.
$ftp Saturn
Connected to Saturn
220 Saturn ftp server
Name (Saturn: summit ): Henry
Password: ******86
• To quit ftp use close and then bye or quit.
• ftp>close
221 good bye
• ftp>bye
• Transferring files:
can be of 2 types.
• Uploading( put & mput):
• To upload the web pages & graphic files to website.• The put command sends a single file to the remote machine.
87
• ftp>binary 200 type set to I• ftp>put penguin. gif To copy multiple files use mput.• ftp>mput t*.sql Downloading files: get & mget• To download the files from remote machine use get &
mget.• ftp>get ls-lR.gz• ftp>_
88
Networking commands
telnet: Remote login
If u have an account on the host in a local network (or on internet ),u can use this with the host name or the ip address as argument.
$telnet SaturnTrying to 192.168.0.1…Connected to Saturn
89
• Login:----
• Password:-----
• quit telnet by using exit command.
• telnet prompt:
When telnet used without Ip address the system displays a telnet> prompt . U can invoke a login session from here with open.
telnet> open 192.168.0.8
Trying to 192.168.0.8…
Connected to 192.168.0.8
90
Networking commands
• rlogin: remote login without password• rlogin is the Berkley's implementation of the remote
login facility.• U can log on to ur own identical remote account
without using either the user name or password.• $rlogin Jupiter• Last login :….• rlogin is terminated with ctrl+d or exit or logout.
91
vi Editor
• vi is a full screen text editor
• It was created by Bill Joy.
• Bram Moolenaor improved it and called it vim (vi improved).
• Invoking vi:
• $Vi file name
92
vi Editor
We notice a tilde (~) on each line following the cursor. A tilde represents an unused line. If a line does not begin with a tilde and appears to be blank, there is a space, tab, newline, or some other nonviewable character present.
vi editor Command Description
vi filename Creates a new file if it already does not exist, otherwise opens existing file. vi -R filename Opens an existing file in read only mode.view filename Opens an existing file in read only mode.
Modes of operation
• Vi has 3 mode of operation.
1.Command mode: In this mode all the keys pressed by the user are interpreted as commands. It may perform some actions like move cursor, save, delete text, quit vi, etc.
2. Input/Insert mode: used for inserting text.
– start by typing i; finish with Esc
95
Modes of operation
• Ex mode or last line mode:• Used for giving commands at command line. • The bottom line of vi is called the command
line.
96
Basic Cursor Movements
j down onek up oneh move cursor one place to leftl right onew move forward one wordb back one word
97
Finishing a vi Session• Get to command mode (press ESc)
ZZ save changes to the file and quit(no RETURN)
:q! quit without saving(press RETURN)
:wq! Saves the file & quit.
98
Inserting Text
• Move to insertion point
• Switch to input mode: i
• Start typing; BACKSPACE or DELETE for deletion
• ESC finish; back in command mode
No RURN
99
Deletion
• Must be in command mode.
x Delete character that cursor is on. dd Delete current line. D Delete from cursor position to
end of line u Undo last command
100
Screen Commands
• Ctrl-f To move screen forward• Ctrl-b To move one screen backwards.• Ctrl-u Moves the cursor half a screen forward.• Ctrl- d Moves the cursor half screen
backward.• Ctrl-r or Ctrl-l To redraw the original screen.• Ctrl-g Displays the status on the status line. name of file, current line number
top related