unix 101: introduction to unix (i.e. unix for windows users)mkegel/coursenotes.pdf · unix 101:...

36
Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction to UNIX (i.e. Unix for Windows Users) The cold hard truth · this course is NOT sponsored by the CS dept. · you will not receive any credit at all introduce ourselves to the students · Should introduce myself and Marshall are seniors in CS and CS/Math And what do we think we’re doing... · here to teach you fundamentals of Unix · to teach you the student to teach yourself about unix · to give you the necessary basics to go out and teach yourself whatever you might need to know to get work done on unix · catch a fish for a man vs teach a man to fish · trying to follow Mudd philosophy - teach you to teach yourself who this course is targeted at · those students who have never worked with unix before and would to get some experi- ence doing · those students who have had some experi- ence but would like to learn more · someone who is lost, confused, and looking for spiritual enlightenment mashall has a great religion for you to join - we sin so jesus didn’t die in vain ask around, see who has what experience Some reasons to learn about Unix · linux and OS X · solid and basic understanding of unix not enough time in the regular school year to teach you this and teach you everything else you need to know and also because the CS department doesn’t have the resources, or want to of- fer this course on its own - maybe they will in the future we needed something to do with our sum- mer vacation the goodness of our hearts, etc. To make this perfectly clear... · will not teach you to program · will not teach you emacs tricks; will provide a reference sheet not about learning to use a single applica- tion will not solve world hunger, peace, or to be a better person 2 Computers on Campus Available computing resources · graphics lab · terminal room - printer · AC and LAC labs - printer and scanner · Engineering lab - to be avoided Servers · odin, thuban, banshee · knuth, knuth, wilkes, muddcs, arc, cortana knuth is the one you care about main server everyone will be working on - give specs. People to know · Tim Buchheim and Roger Wiechman Bring them milk, cookies and be nice to them, they make things work 1

Upload: phungnhu

Post on 15-Mar-2018

272 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)

Mark Kegel

September 7, 2005

1 Introduction to UNIX (i.e.Unix for Windows Users)

• The cold hard truth

· this course is NOT sponsored by the CSdept.

· you will not receive any credit at all

• introduce ourselves to the students

· Should introduce myself and Marshall

Ù are seniors in CS and CS/Math

• And what do we think we’re doing...

· here to teach you fundamentals of Unix

· to teach you the student to teach yourselfabout unix

· to give you the necessary basics to go outand teach yourself whatever you might needto know to get work done on unix

· catch a fish for a man vs teach a man tofish

· trying to follow Mudd philosophy - teachyou to teach yourself

• who this course is targeted at

· those students who have never worked withunix before and would to get some experi-ence doing

· those students who have had some experi-ence but would like to learn more

· someone who is lost, confused, and lookingfor spiritual enlightenment

Ù mashall has a great religion for you to join- we sin so jesus didn’t die in vain

Ù ask around, see who has what experience

• Some reasons to learn about Unix

· linux and OS X

· solid and basic understanding of unixÙ not enough time in the regular school year

to teach you this and teach you everythingelse you need to know

Ù and also because the CS departmentdoesn’t have the resources, or want to of-fer this course on its own - maybe they willin the future

Ù we needed something to do with our sum-mer vacation

Ù the goodness of our hearts, etc.

• To make this perfectly clear...

· will not teach you to program· will not teach you emacs tricks; will provide

a reference sheetÙ not about learning to use a single applica-

tionÙ will not solve world hunger, peace, or to be

a better person

2 Computers on Campus

• Available computing resources

· graphics lab· terminal room - printer· AC and LAC labs - printer and scanner· Engineering lab - to be avoided

• Servers

· odin, thuban, banshee· knuth, knuth, wilkes, muddcs, arc, cortana

Ù knuth is the one you care aboutÙ main server everyone will be working on -

give specs.

• People to know

· Tim Buchheim and Roger WiechmanÙ Bring them milk, cookies and be nice to

them, they make things work

1

Page 2: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

3 Philosophy and History ofUNIX

• Origin of Unix

Ù computers were mainframes that peopleworked at, much like knuth

Ù teletypes - basically a typewriter pluggedinto a computer

· Ken Thomson implemented the first UNIXenvironment

Ù had been working at Bell Labs on a projectcalled Multics (Multiplexed ...)

Ù Unix was kind of a castrated Multics - muchsmaller because invented on an underpow-ered system - PDP-7

Ù became popular mainly because it was dis-tributed to Universities

Ù will want to display Unix timeline

• Unix Today

Ù programmers environment - because of thisis extremely powerful

· still very much a tinkerer’s environment

· unix encourages and support ”hacking” thesystem

· Fundamentally multi-user and multitasking

Ù designed in the days of mainframe environ-ments when many people would all work onone computer system

Ù we still use the same model in many wayswith knuth

Ù clean, simple interface

Ù Chalkboard exercise

Ù unix is also multi-layered in design, mean-ing that functionality separated into levelsgoing from the hardware to the user, stuffis also abstracted aways at each layer

Ù user | (shell, other programs) | OS(kernel)| hardware

Ù at top user at bottom hardware, what’s in-between?

Ù scales very well - from supercomputers tomicrowaves (embedded systems)

Ù two different flavors of unix - BSD (FreeBSD,OSX) and System V (Solaris, Linux)

Ù about as different as mango, peach, kiwi, straw-berry ice-cream and peach, strawberry, kiwi,mango ice-cream with bits of vanilla

Ù especially that way in linux, its all just annoying

Ù it doesn’t matter what you run, what we aregoing to teach you here will apply across mostunixes though not all

• What is Unix?

Ù unix in some sense is not an operating sys-tem but rather a programming language(C)/philosophy/interface (POSIX)

Ù the development of C (and the internet) areall connected

Ù unix is (kind of) C, the Standard C Li-brary, and POSIX (Portable Operating Sys-tem Standard) which is the API that pro-grammers use to interact with the systemkernel

Ù the design and implementation of unix alsofollows a certain philosophy about howan OS should be designed, how programsshould be written, etc.

Ù everything else is just historical baggage

Ù makes unix extremely flexible - can startfrom scratch with a new implementationbut still same ”design”

Ù what would be some good philosophical ideas tofollow when building an OS?

Ù what are some good rules of thumb? what trade-offs do you want to make?

• Unix philosophy

· programs should be designed to operate to-gether

· output of one could be used as the in-put to another

Ù text streams -> universal input

· Programs are viewed as simple tools thatcombined will do complex tasks

· each program should do one thing ex-tremely well

Ù LEGO blocks - simple tools but can createvery complex systems

Ù another analogy - hammer, wrench, saw

Ù Avoid the ginormous programs that do ev-erything

2

Page 3: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù Windows has this problem (think of Of-fice)

Ù Windows programs also don’t integratevery nicely - no pipeline

· want to keep everything as simple as possi-ble

· KISS - keep it simple stupidÙ this applies everywhere - both in OS

interaction (system calls and librarycalls) and in program creation

· separate mechanism from policy

Ù Unix (and programs that run on Unix)provide a mechanism to do somethingbut do not lock you down to one policy

Ù policy - how a program should look andbehave

Ù users have to decide on policyÙ games and GUIs are generally a good

examples - mods and skins

Ù walk up to some one and try to scare them

· principle of least surprise

· applications will generally produce nooutput if successful

· applications that fail will make it evi-dent (sirens if necessary)

· programs should do the least surprisingthing

Ù do not try to feed emacs a stray cat

· there is no ”one true way” - unix is notperfect!

Ù unix has things wrong with it, andthings that unless you understand willcause you a great deal of pain

Ù The Unix Hater’s Handbook - writtento expose the flaws in unix

· from a stanford site:http://pangea.stanford.edu/computerinfo/unix/overview/advantages.html

· ”The traditional command line shell in-terface is user hostile

· designed for the programmer, not thecasual user.

· Commands often have cryptic namesand give very little response to tell theuser what they are doing. Much use ofspecial keyboard characters - little ty-pos have unexpected results.

· To use Unix well, you need to under-stand some of the main design features.Its power comes from knowing how to

make commands and programs interactwith each other, not just from treatingeach as a fixed black box.

· Richness of utilities (over 400 standardones) often overwhelms novices. Doc-umentation is short on examples andtutorials to help you figure out how touse the many tools provided to accom-plish various kinds of tasks.”

Ù ”Unix *is* user-friendly. It’s just pickyabout its friends.”

Ù very true - have to be able to understandthe system at some intuitive level to be ableto work with it

Ù also if you don’t like something you can fixit, maybe not easily, but you can fix it

Ù What makes a system secure? What needs to beprotected?

Ù What is a good way of protecting things, likeproperty? Ownership

• Unix security

· concept of ownership - everything on thesystem is owned by someone

Ù ownership implies responsibility - so who-ever owns these files is in the end responsi-ble for them

· protect users from one another and fromdoing dangerous things

· protect privacy -> data· protect work -> programs

· protect programs and files from users - orrather allow only certain users to do certainthings

Ù security: not perfect, can break, and willcause you to scream a few months down theline

4 Types of users

Ù what powers is each user going to have?

Ù do you need to have a class of users that managethe system?

Ù what kinds of users are there going to be?

• administrative users

· root - is the best known account of this type- aka The SuperUser

3

Page 4: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

· complete control over everything on the sys-tem

· can do anything that they want (mostly) -has to respect file perms.

· can become any user on the system

Ù loaded gun with the safety off, easy to shootyourself in the foot

· is a standard account on everyUNIX/Linux/BSD machine

· used for system administra-tion/maintenance

· what this is will cover later

Ù any other user however can be granted ad-min privileges, so root isn’t strictly neces-sary to run a system

Ù why would you want? what if you tried todelete all the files on the system? shouldn’tbe able to do

Ù should there be different types of normal users?is there a better way to organize responsibility?

• ”normal” users

· have a home directory - directory to keepall your files to yourself

Ù cannot generally see others home directo-ries, Odin had this problem and Charlie stilldoes

· can use most of the programs on the system

· any special configuration files are generallystored in your home directory

· on knuth you have about 120 Mb for allyour files

Ù normal users have to obey the security poli-cies of the system, superusers can ignoremost of them

Ù on your own system can have as much asyou want ;-) or if you’re CS staff

Ù the CS department does daily backups so ifyou lose a file you REALLY want then youcan ask them to restore it

Ù for what jobs do you need a third type of user?

Ù what jobs do you not want root doing, or a nor-mal user? webserver

• other users: nobody, ssh, ftp, etc.

· used to run daemons in the background

Ù run ps -aux | grep daemon

Ù for security reasons do not want to have aweb server running as root

Ù what if someone owned your web daemon -would own your system...is bad

Ù unless you’re setting up your own systemdon’t have to worry about this

Ù and even then still don’t really have toworry about this since most of these userswill have been created by default - rarelyhave to create a new system user

Ù now that we have users, how do we managethem?

Ù should all users be lumped in to one category?

• groups

Ù are the unix solution to separating respon-sibilty amoung users

· every user on the system belongs to one ormore groups

· these are sets of users which all share somecharacteristic

Ù on the board draw a Venn diagram

Ù one group, on knuth at least, is students

Ù every student that has an account belongsto this group

Ù consultants are additionally members of theoperator group

Ù however since consultants are also students,we draw another distinction

· every user has a primary group - this is agroup to which file which you create will beassigned

Ù can generally be a member to as manygroups as the admin wants you to be

Ù some groups serve special purposes alongwith the ”special” users

· can have groups ftp, www, etc.

· exists to allow easier management of a sys-tem by administrative users

5 Logging In

• Loggin In

· two parts: username and password

Ù username can be anything, though someare reserved for special purposes (root, bin,etc.)

4

Page 5: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù on knuth it will be the first character ofyour first name, followed by your last name(hopefully); on knuth the username limit is16 characters

Ù so on knuth I am: mkegel; Marshall ismpierce

• password should be strong

· what makes a strong password?

· min 8 characters in length (turing only paysattention to the first 8 characters of a pass-word)

· special characters (Numbers and

@ ! # $ % ^ & . , ( ), even spaces)

· uppercase characters

· should not look like a dictionary word

· pass-phrases are good things

· I took CS60# could become ->1 t00k CS 6o# (thats a one withtwo zeros in took and a lower case O)

Ù Demonstrate logging in to knuth

Ù notice that no password characters show upwhen typing

6 Introducing the shell

Ù log into knuth with the default knuth setup

Ù what you see when starting

Ù the default knuth setup, easiest and mostfamiliar setup

Ù components of the prompt: username, time,command number, and directory

Ù change the prompt to something much simpler,like %

• Sessions of the Heart

· unix is at its heart a cmd line system - insome respects this is much more powerfulthan having just a graphical interface

Ù it was created in the days of the teletype,like a telegraph with a keyboard only up-percase letters though - so very limited

Ù so the teletype would be hooked up to thegiant room filling computer

Ù the teletype would send the input, and theoutput would be printed out on a sheet ofpaper, this meant that the teletype and thecomputer could be located either next toeach other or across the country, the com-puter couldn’t tell

Ù later the sheet of paper changed to a CRTmonitor - would just print the text to thescreen

Ù over time the teletype changed to a fullkeyboard but the fundamental relation be-tween the teletype and computer (inputand output) for most things hasn’t reallychanged

Ù obviously today you have both upper andlower case characters

Ù most of you probably come from either a win-dows or mac world where most things can bedone with a graphical interface, for certain thingsa GUI (graphical user interface) can be verygood (ex. web browsing), however, certain otherthings are made much simpler by the use of thecmd line.

Ù for you, however, the best reason to learn thecommand line is that you have to

Ù the CS department has chosen to have studentsdo most of their work on unix systems, thereforeto make your life easier you should learn how touse them

Ù Why should you learn to use the command line?What does it offer?

• The Blinking Cursor of Doom

· simple, quick, and powerful

Ù what does that mean?

Ù can do an enormous amount of work, withvery few commands

Ù for certain tasks are much superior to agraphic interface

· becoming a unix power user involves be-coming familiar with the cmd line

Ù we will focus this course on using the commandline

Ù change prompt to something simple - probablyuse a special account on farmboy

Ù for now we want a simple prompt, don’t want todistract you

5

Page 6: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

• Home sweet Home

· when you log in you will automatically beat your home directory

· home directories on knuth, and many othersystems (OS X is different) are located at/home/username

· on OSX home directories are at/Users/username

Ù for me: /home/mkegel (knuth, farmboy)/Users/mkegel (shadow)

· directories are denoted with /, windowsuses

Ù there are other directories that we will getto later, but on knuth you should never haveto worry about them

• Unix directory tree

Ù what do i mean the home is /home/mkegel?no c:?

· file systems can generally be seen as a tree

· a cdrom or hard-drive can generally be lo-cated anywhere in the tree

· no preset structure

· working with devices we’ll get to much laterso just sit tight for now

Ù there is no c:, d:, a: like on windows, every-thing starts at / (root)

· is filesystem independent

• Introducing the Shell

Ù what is the shell? talked about multi-layered design but what is it?

· The shell is the highest level interactionlayer between you and the OS

· it’s what sits between the teletype and thecomputer - makes the computer easier touse

· it runs whatever commands you give it,with whatever arguments you give it

· the most important thing in learning to useunix in a productive manner is to becomecomfortable with using a command line

Ù that’s not a computer fan, that’s the shellpurring...

• syntax of a command

· will use

· things in brackets optional, angle bracketsnot optional

Ù cmd [switches] [arg1] ... [argn]

Ù that is how commands will appear and beused on the command line

• switches

Ù What is an argument and what is a switch?

· switches are arguments that you pass into aprogram that modify the programs behav-ior - switch the behavior of the program

· sometimes called flags, we’ll use the termswitch though

· different than what is generally referred toas an ’argument’

· a switch will generally have the form ”-?”where ? is some character

Ù most programs take multiple switches andalso accept multiple forms for what is thesame action

· other style, GNU style, is "--switch"

· most programs accept many switches

Ù gpg (encryption key generator) accepts 360!

Ù demo the tar stuff from the command line

Ù can have multiple switches, example tar

Ù tar (tape archive) is an archiving programfrom the days of tape drives

Ù file archives are given the extension ”.tar”(gee big surprise!)

· to uncompress will most often type

Ù tar -xvf [tar file]

Ù ”-xvf” could have been typed ”-x -v -f” buteasier to write the first way

Ù on old archaeic systems this may not actu-ally be the case but on new systems it mostoften is

• Arguments

· data that you want to program to operateon

Ù what is an argument? what would you passin?

· filenames, text strings, man pages

Ù all of this stuff is sensitive to spaces, so ”ab”is not ”a b”

Ù not every command takes arguments; some re-quire them

6

Page 7: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

· some basic commands to get started with - thesecommands are more complicated than what weare actually showing here, you’ll be able to findout more stuff about them after the end of thislecture

Ù these are also the commands that you’ll be usingthe most often

Ù students are going to have to write all of thisdown, will probably want to put on the chalk-board

Ù cd [directory] :: changes the current di-rectory

Ù pwd :: prints the current directory you arein

Ù mv <source> <target> :: moves the’source’ file to ’target’ file/directory

Ù cp <source> <target> :: moves the’source’ file to the ’target’ directory

Ù rm <file1> [file2] ... [filen] :: removesfiles

Ù ls :: gives you a listing of the files in thedirectory

Ù mkdir <name> :: creates a new directory

Ù rmdir <name> :: removes an empty direc-tory

Ù man [command name] :: prints the ”manpage” for various system commands

Ù some commands are interactive

Ù all these programs just run, do their thing, andthen quit

Ù what we generally refer to as an application isinteractive; commands are generally not interac-tive

Ù emacs and the shell are both applications

• some things to know

· note: unix is case sensitive - AAAA is dif-ferent than aaaa - this shouldn’t be too sur-prising

· also most unix commands are done in lower-case as you’ve just seen

· most switches to programs will also be inlower case, although some need to use both- pay careful attention to both the switchyou’re using and the case of the letter(s)you may be passing in

· there are also special characters will in dif-ferent contexts will be interpreted differ-ently

• Ctrl+World = Mwahahaa!

· discuss C-? and M-? notation

· you see this everywhere, particularly in doc-umentation, so is very useful to know

· is incredibly simple: C == Ctrl; M = Alt(but stands for meta)

· therefore C-x is Ctrl and lowercase x heldtogether

· are what might be called shortcut keys un-der windows

· one other notation is caret notation; ^hwould mean C-h

• quitting programs

· most programs can be quit through eitherCtrl-c or Ctrl-d

· C-d sends an EOF character

· C-c sends an interrupt signal that causesthe program to clean up and then to exit

· if C-c doesn’t work and you have to quit theprogram, then C- will cause the program toexit without cleaning up first (as well asdumping core)

• tab completion

· wouldn’t it be nice if say you have somelong directory name, ./thisdirectorynameis-toolong/ but there was some way thatyou didn’t have to type it out every time,hmmm...

· most shells (and any that you wouldnormally use) provide whats called tab-completion, allowing you to complete par-tially written statements into their full form

Ù demo this feature

Ù so given partial data the shell will deter-mine what sequences of characters will ap-propriately complete what you’ve writtenon the cmd line

Ù this is accomplished by hitting the tab key

Ù hit tab key to show different completions

Ù once you’ve typed enough to make the restof the string unique (that is there is onlyone possible thing that you could be typing)the shell will automatically complete whatyou’re typing when you hit tab

7

Page 8: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù demonstrate on the long directory name

Ù since directory names have to be unique onthe same level, tab-completion work really,really well

Ù but what if you want to tab complete otherthings?

Ù like say the arguments to a functions, orswitches...

Ù luckily this is setup by default with the shellyou’re using, so you can tab complete anynumber of things, man pages, arguments,etc.

• files and directories

Ù up arrow gives old commands, continuepressing to work back through your en-tire command history, useful for when youhave to type the same commands again andagain - like

Ù make program

Ù the down arrow with of course cycle forwardin your history

• command line editing

· the left and right arrow keys move the cur-sor allowing you to edit the cmd you’vetyped on the screen - that way you can goback and correct something without havingto retype the entire command

· other useful key commands are (assumingthey aren’t already used):

· C-? will delete back· C-d will delete forward· C-u will delete an entire line· C-k will delete all of the line forward

from the cursor· C-l clears the terminal· C-y will paste the characters you have

deleted· C-a will move the cursor to the begin-

ning of the line· C-e will move to the end of a line· M-b moves back a word· M-f forward a word· C-_ will undo the last thing typed· C-r will do a history search· there are many more...and most impor-

tantly you can create your own!

• special characters

· the shell interprets a number of charactersas being special

· that is some characters have meanings tothe shell other than what you may havetyped

· so which characters are special?

· # $ * ? [ ] ( ) = |^ ; < > ‘ $ " ’ \

Ù what do you need special characters for?

Ù for example the semi-colon ’;’ separatescommands on the cmd line

Ù cmd1; cmd2; cmd3;...

Ù but what if you want to pass a special char-acter to a program?

• Escape From ”The Shell”

Ù what is you want to pass these charactersinto a program?

· have to escape them; \?, where ? is thespecial character you want to use

· demo echo semi; versus echo semi\;

· so when you type \; the special meaning of; is disabled

· you can also disable the special meaning bysurrounding the character in quotes, either” or ’ (double or single quotes)

· demo echo ";"

· how quoting is more powerful since all thespecial characters inside of the quotes getdisabled - well mostly, there are some rules

· pg. 527 UPT

· ’xxx’ - everything get disabled

· "xxx" - everything but $ ’ and \

· the character has another meaning thatyou have to worry about

· if you have a at the end of a line then thenew line will be removed and the shell willtreat the two lines as one long continuousline

Ù echo Who \stole \the cookie, \from the \cookie jar \?

8

Page 9: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

7 Finding help

• Finding Help

Ù our goal is not to teach you unix, but togive you the basics, then have you be ableto teach yourselves what you need to know

Ù but sometimes you have to ask for help,and often times this is the best way tolearn something new, don’t beat your headagainst the wall if someone you know al-ready has the answer and can help you fig-ure it out!

· so part of teaching yourself is knowing whento ask questions

· there is one resource that is the best and itis? ...Google!

· whenever you have a question about some-thing the first thing that you should alwaysdo is google for the answer - most likelysomeone else has already taken the time tosolve and write up whatever solution youare looking for

· lots of very good resources

· websites (particularly forums), books,documentation, and people

· some good websites to check out are:linuxdoc.org, freebsd.org

· some good books are: UNIX PowerTools, FreeBSD Handbook...

· people: Marshall, Mark, Mac, andmany others!!

· Computer Science Department Staff andConsultants

· they have people hired to help you out!!You should use them as a resource!

· Staff take care of the various boxen thedepartment runsÙ give a listing of the boxes, their pur-

pose, and their addresses· knuth.cs.hmc.edu – shell server

Ù find new mail server· muddcs.hmc.edu – web

· Consultants sit in the terminal roomand answer Unix and CS related ques-tions

· give a list of who are the consultants,emails, and schedule

· The CS department website

· QREF’s - documentation writtenspecifically for YOU

· is a great resource for general informa-tion, system policy, etc.

Ù give the URL for the QREF’s· http://www.cs.hmc.edu/qref/

· the professors will also help you out as muchas they can

· some good mailing addresses for questions

· [email protected] - linux mailing listfor hmc students and others

Ù give out others (?), maybe fo-rums.muddstudents.org

• Built In Docs

· man pages

· purpose is to document how each of thevarious programs/commands on thesystem work, and how to get them todo what you want

· generally short on examples, but ingeneral well worth reading

· can document other thingsÙ get in the habit of using man pages -

saves time and effort!· what man pages are not

· general reference, programmingguides

Ù what man pages are· very specific references on a limited

set of system commands· essentially they document how to

use the programs on the systemÙ some man pages break these rules but

in general they seem to be followedÙ demo man

· Other systems

· info· is mainly about emacs

· the ’help’ command for bash

· Most programs come with more documen-tation than just a man page

· check out: /usr/doc and/usr/share/doc

· slackware uses /usr/doc and is reallygood

8 How to use man

• how man is organized - sections - explain whateach section covers

9

Page 10: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

· 1 - Commands available to users

· 2 - Unix and C system calls

· 3 - C library routines for C programs

· 4 - Special file names (Devices and DeviceDrivers)

· 5 - File formats, protocols, and conventionsfor files used by Unix

· 6 - Games

· 7 - Conventions, Macro packages, Wordprocessing packages and Misc.

· 8 - System administration commands andprocedures

Ù always need to keep in mind that one com-mand could be in two plus different sectionsat the same time

Ù allows for a better organization of documen-tation

• some switches

Ù man -k perl

Ù so then ”man -k” allows you to search forman pages

Ù ”-s” - specify the section to search

Ù ”-a” - show all man pages that match thesearch, not just the first

Ù man -a sync (demonstrate this)

• how to read man pages - the parts of a man page

Ù at the very top is the is the name and sec-tion number; for man it is man(1)

Ù NAME - just a quick one line description

Ù SYNOPSIS - all the arguments that theprogram accepts

Ù how to tell if something is optional orrequired

Ù optional things are surrounding inbrackets ’[xxx]’

Ù required things are notÙ show example - man requires at least

one argumentÙ man man

Ù DESCRIPTION - a paragraph ”summary”of the program

Ù OPTIONS - what each of the switches tothe program do

Ù ENVIRONMENT - what environment vari-ables the program expects

Ù SEE ALSO - other related man pages to see

Ù BUGS - bugs the program may have

Ù FILES - files that the program may use(usually configuration files)

Ù AUTHOR - who wrote the program

• navigating a man page

Ù man by default uses more (more is a pagerthat sucks)

Ù on knuth man is setup to use less, but beaware!!

Ù so learning to use man is really like learningto use less

· arrows move man page up/down left/right

· space bar moves a screen at a time

· to search would type /pattern, then pressenter

· demo this feature

· typing / or n again moves to the next in-stance of pattern

· N moves to the previous instance

· b : back - like page up

· f : forward - page down

Ù man almost always uses the less program asits pager, so learn to use less!

Ù you’ll also use less for lots of other textviewing

Ù exercise - now for a man page hunt

9 Making programs work to-gether on the cmd line

• Working Together

Ù ask question before showing slide

Ù unix is about more than just running oneprogram at a time (that’s windows job); inunix programs work together, so we’ll showyou how to do that

Ù How do you get programs to talk? Files?Pass in a program as an argument? Whatshould handle this mechanism? Whatabout basic program I/O?

Ù Already know that can accept argumentsand print stuff to the terminal.

Ù Text streams are a universal I/O format

10

Page 11: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

• Working Together

· unix runs more than one program simulta-neously

· programs need to work together

· programs use stdin, stdout, stderr for gen-eral I/O purposes

· by default I/O comes from cmd line

• What stdout, etc do

Ù you should already know about stdin, std-out and stderr from CS70 or CS60

Ù they allow a single program to get inputfrom the cmd line or print output to thecmd line

· stdin - represents the characters you are in-puting

· stdout - the standard place for the programto print output

· general goes to the terminal you ran theprogram from

· stderr - like stdout but for error output

Ù need to show distinction between stderr andstdout

Ù in C++ the three are accessed throughcout, cin, and cerr

Ù can accessed in other ways through otherlanguages

Ù so we have some idea of what stdin andstdout are, they represent the input we getfrom the terminal and the output we sendto the screen, but how do you think thatthey actually work?

• Everything in Unix is a file

· devices (keyboard, mice, monitor)

· directories (yep even directories)

· links (we’ll cover what these are)

· and what you normally think of as files

Ù what does it mean to be a file? anythingused with read() and write()?

Ù put quotes around file -> ”file”, aren’t re-ally files, just treated like files

Ù a file is just a stream of data, in some caseswith limitations as to how that stream ishandled

• Are they file?

· so stdin, stdout and stderr are files? no, butare treated as files

· are actually file descriptors, so beingtreated like files

· why so surprising? you read() from stdin,write() to stdout and stderr

· each program has its own set stdout, stdinand stderr that is managed by the Operat-ing system

· how do you combine simple programs? howdo you get two programs to talk?

Ù good time to use the chalk board

Ù ask group for ideas

Ù we know that two programs can share thesame file - can have the same file open twodifferent times

Ù so to communicate one program could writeto a file, the other could read

Ù [program one] --> [file descriptor/pipe] -->[program two]

Ù can use the stdin, stdout, and stderr files tohave programs talk to each other

Ù [program one] --> [stdout/stdin] --> [pro-gram two]

Ù the stdout of one program IS the stdin ofanother

Ù very flexible and powerful solution

• Using pipes

· stdin, stdout, and stderr are actually pipes

· instead of the program printing its outputto the screen it is piped to the second pro-gram

· keyword: pipe

· you can do this from the shell

· the syntax is

Ù cmd1 | cmd2

Ù the ’|’ character is known as the pipe andis located on the key, so Shft- is |

Ù example of this in action - first run who andtell what wc does, then run this

Ù who | wc -l

Ù (gives the number of terminals open on thesystem)

Ù get a longer piping example

· programs used in this way are called ’filters’

11

Page 12: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù filters are the tools that you use to maketasks easier, and you need them to be good

Ù if you can’t use filters you’re never going tobe able to exploit the power of the system

Ù cause you won’t be able to leverage thework of other people

Ù how many people think that they could im-plement ls? how about a good implementa-tion?

Ù to make them good you have to follow acertain set of standards and conventions

Ù if you don’t the filter will be difficult to use

Ù so what are these conventions? we dis-cussed many of them at the beginning ofthe class

Ù KISS, principle of least surprise, you’ll learnmost of these just by using a UNIX system

Ù demo piping on terminal

• Redirecting Output

Ù can do more than just piping between pro-grams

· you can redirect the output of a program toa file

· you can redirect a program to read stdinfrom a file

Ù keyword: redirection

· you can have whatever file you want to bestdin, or stdout/stderr, using the shell

· redirect file to stdin

Ù cmd1 < file1

· You use the < to have file be stdin for theprogram

· redirect stdout to file

Ù cmd2 > file2

· So you use > to redirect the output

Ù Some notes on redirecting output

Ù the file cannot already exist (if you haveNOCLOBBER set)

Ù won’t execute if it doesÙ since this is set by default on knuth you

need to be aware of it

· You can append to the end of a file (addlines of standard output at the end withoutwriting over what is already there, usefulfor log files) by doing the following

Ù cmd4 >> file

Ù have a demo ready to appendÙ echo "Hello there!" > newfile

Ù echo "Good to see you." >> newfile

Ù cat newfile

• Working with stderr

Ù word of warning: this is very shell depen-dent, but since you are working with zsh, oryou ought to be the syntax provided hereshould work

· send both stdin and stdout withÙ cmd1 |& cmd2

Ù cmd >& file

Ù spacing matters so pay attention to itÙ can also append stderr to a file - better for

log files since you don’t generally want tosend error messages along a pipeline, onlywant real output

· to append stderr very much like redirectingstderr

Ù cmd5 >>& file

· to redirect just stderr to a fileÙ cmd 2> file

Ù redirecting just stderr along a pipe is a bitmore difficult

Ù demo this concept

• Complex Redirection

· can modify stdin, etc, file descriptors fromthe cmd line

· stdin=0, stdout=1, stderr=2 : are file de-scriptors (integers)

· file descriptors 3 to 9 are free for you to use· examples:

· 2>&1

· 1>/dev/null

· 1>file is same as >file

· 2>&1 1>&2 (this won’t actually swapstderr and stdout)

· 3>&2 2>&1 1>&3 (this will swap thetwo)

Ù demo this concept

Ù not every program takes things from stdin orsends output to stdout

Ù for example can’t do

Ù emacs < myfile

Ù instead have to send myfile in as an argument toemacs

12

Page 13: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

10 Another way to combineprograms

• The Back-Tick

· you may remember the ‘ as being a specialcharacter

· ‘ is located on the key above tab

· is called a back-tick or back-quote

· anytime you surround a string with back-ticks the shell will treat it as a commandand then replace the string with the outputof the program

Ù kill -HUP ‘cat /var/run/sshd.pid‘

Ù this will restart your ssh server, since ‘cat..‘returns the PID of sshd

· another example

Ù emacs ‘grep -l error *.c‘

Ù this will let you edit all of the files that havethe word error in them

Ù lots of other uses, mainly in shell scripting

11 You Ought to Know this bynow

• Review

· Philsophy and History of Unix

· Piping, Redirection, Back-Ticks

· Finding Help

· Users and Groups

· Some Basic Commands

· What the Shell is and why its cool

· Few Other tidbits

Ù if you don’t know what we’re talking abouthere, then you’ll want to review before go-ing on

12 Processes and the Kernel -Actually running programs

Ù do most of this on the board

Ù some of the commands we’ve been runninghaven’t actually been ”programs” in the sensenormally described

Ù example: cd is a builtin, its a part of theshell and not a stand alone program likesay emacs is

Ù need to have some sense of the basic, underlyingabstractions of how the computer works - not ona bits and byte levels, but higher up

Ù exercise: back to the layer thing, how the com-puter is organized

Ù again is a layer thing - bottom up

Ù hardware - the machine itself - just electri-cal signals

Ù develop on the board a better layered model

Ù user | shell - applications - libraries | kernel- modules | hardware - remote systems

Ù kernel - a mediator between the hardware(and remote systems) and the user - pro-vides a set (library) of functions for con-trolling the hardware that every programyou would write would need (ex. drawingtext to the screen, writing to disk)

Ù applications - do the things that you wantthe computer to do

Ù libraries contain useful code that you’drather not reimplement

Ù the user - running applications and makingsense of the electrical signals that the com-puter processes - only here do things haveany ”meaning”

Ù in understanding unix you don’t so much aboutthe hardware - leave that to the engineers

Ù really only care about the applications layer -how to make programs do what you want

Ù but in order to fully appreciate and use unix youhave to understand at some level how applica-tions interact with the kernel, and what a kernelis

• What is a process, program

Ù what is a program and what is a process?

Ù ask what a program/executable is - just thefile containing instructions

Ù is very broad: does it matter what kind ofinstructions?

Ù the instruction could be machine codeÙ could be byte code like in java or com-

piled python

13

Page 14: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù or could be an interpreted scripting lan-guage

Ù what is process?

Ù a program in actionÙ an abstraction provided by the kernelÙ every process thinks its the only one

running - runs in a self-contained envi-ronment with the kernel being respon-sible for enforcing this abstraction

Ù how is a program different from a pro-cess?

Ù how many copies of each do you normallyhave?

Ù 1 for executable (one file containing thebinary code)

Ù can have different versions but thosedon’t count

Ù n for processes

Ù why would you have more than one processof a single program?

Ù need them to be doing different things- two xterm windows for example

• The Kernel?

Ù how do the processes get run?

· the job of the kernel is to manage themany processes on the machine and runthem in some sane order so that thesystem is both responsive and produc-tive

· the kernel also provides an interface toall the hardware on the system

· this interface is uniform across archi-tectures and hardware

· so you always call read() though thedisk might actually be just a networkshare somewhere in asia

Ù does the kernel just run processes? basi-cally, and do what processes need done

Ù what might be the other jobs of the ker-nel?

Ù manage devicesÙ reading/writing filesÙ handling system callsÙ enforcing system policies - security for

exampleÙ allows for multiple users on a single sys-

temÙ abstracts a common interface for differ-

ent hardware - creates an abstration

Ù kernel does more than this

Ù take Operating Systems if you want to write akernel

Ù take CS105 to learn more about how ComputerSystems work

Ù as users all we care about is running programs,leave the other stuff to the CS geeks

Ù and unix is very good at running programs

Ù so what does this have to do with anything?

Ù not a whole lot other than making you un-derstand that you can run multiple pro-grams with fear of them crashing the sys-tem like on some other systems we couldmention

• Making Forrest Run

· unix is a multitasking environment

Ù can run multiple processes at the sametime

Ù no fear of them interactingÙ each process runs in its own little envi-

ronment isolated from othersÙ unless you WANT them to interact!

very key!

· unix is a multiuser environment

Ù can handle multiple users using the sys-tem at the same time

Ù so if you are going to run multiple programsfrom the shell(s) it would be helpful if youknew how to manage them

Ù with GUIs most of these things are notreally an issue, but its still very use-ful to have some idea in case you messthings up

Ù can open as many xterms as you want,but at some level you need to knowthis...

Ù from a single shell session can runabout as many programs as you wouldever want to - constrained by memoryand processing speed

Ù normally though you are only ever run-ning one program at once in a shell -this is referred to as the foreground pro-cess

· we’ll assume one shell and that you need torun multiple programs

14

Page 15: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

· is called ”job control”

Ù why would you need this even though wehave GUIs?

Ù because you won’t always have a GUI,may end up working at just a terminal

• Making Forrest Run

· three types of processes: foreground, back-ground, suspended

· foreground process is the one that receiveskeyboard input (if interactive) and prints tothe terminal

Ù mv ..

Ù ls ..

Ù rm ..

Ù sometimes you want to run other programs

Ù say stop reading a man page to copy a file,or read email

Ù could open another terminal - do youreally want to do this for every pro-gram? what if you only have a limitednumber of terminals?

· or could ”suspend” the process inwhich case it stops executing

· or put it in the ”background”, and con-tinue to let it run

· background process don’t recieve input, butcan still print to the terminal

· suspended processes just aren’t doing any-thing - but are not terminating

• Sending the Right Signals

· recall that C-c (SIGINT) kills the programyou are currently running

· so does C-\ (SIGQUIT)

Ù sometimes its C-d (sends an EOF – usuallyfor interactive programs)

· to suspend a running process use C-z(SIGTSTP)

Ù common mistake to think that the programhas just suddenly died

Ù people unfamiliar with the system will juststart over, being angry and frustrated atthe work they ”lost”

Ù do a demo of suspending a process

Ù demo reading ”man page”, suspending, andusing pine

· will want to demonstrate that pine cannot be suspended by default

• Background Processes

· when we type

Ù cmd

· this program is the one we’ll be running inthe foreground

· to run a program in the background we type

Ù cmd &

· this causes the program to run, but not inthe foreground - so it won’t be receivingany input that you type unless you switchto that program

Ù just because a process is backgrounded doesnot mean it isn’t running, and that sus-pended process aren’t running

Ù you could have a copy operation running inthe background

Ù music program in the background

Ù also note that the system has to run allthese, so no free lunch!

Ù remember: limited by RAM, hard drivespeed (swapping), and cpu power

· to switch back to the process you just sus-pended you use the fg command

· can of course suspend multiple processes

· demo two other man pages, also suspend

· will have three jobs in the background withcmd line open

· open emacs in background and switch to it

Ù emacs &

Ù do some thing on the cmd line

Ù fg

Ù this is one of the most useful things to know,so remember it!

Ù why this is useful - give this example sce-nario

Ù are logged in remotely with no X ses-sion

Ù have an emacs window open andchange a bit of code

Ù too slow to exit emacs and restartÙ so press C-z to suspendÙ then can rebuild from the cmd lineÙ then can switch back to emacs to con-

tinue working

15

Page 16: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù really only need to remember C-z to sus-pend and fg to bring the process back

Ù draw on chalk board what is happening whenyou suspend or background a process

Ù shell -> process -> shellÙ shell -> (splits) -> shell (-> process)Ù and more

• Mid-level management

· use the jobs command to list all the jobsthat you have running

Ù jobs

[1] + Suspended man page

[2] Suspended man perl[3] - Suspended man printf

Ù the number identifies each job

Ù we use the fg command to switch back towhich ever job we want using the number

Ù by default fg will bring back the jobwith the plus sign

Ù this is the current jobÙ the minus sign was the job that used to

be the current job

Ù the rest of the output of jobs is obvious

Ù Stopped means it doesn’t receive anycpu time

Ù running means just that

Ù example to go back to man perl we wouldtype

Ù fg %2

Ù nifty trick: if we know the name of the pro-cess, say we have a single emacs processalong with other can quickly switch back toemacs by using a pattern matching schemefor fg

Ù instead of %n, you have &?xxxx where xxxxis the first part of the name of the runningprocess

Ù obviously you need enough of the nameto disambiguate the process

Ù so returning to emacs could be as simple astyping

Ù fg %em

Ù don’t have to use jobs and look there, justdo a single fg command

Ù can also do some of the same tricks with bg

Ù bg %n

Ù continues job n in the background

Ù also cover %+ and %-

Ù what happens if we background emacs?what happens if we suspend emacs and letit continue in the background with bg?

Ù now that we can manage a few processeswhat about the processes on the rest of themachine?

13 Understanding process own-ership

• Process ownership

· every process on the machine has a par-ent/child status

· so when you log in and get a shell that pro-cess will act as the parent to all the otherprocesses you create on the machine

· there is one master process on the machine,init

· every process is a descendent of init

· every orphaned process is a child of init

· a zombie is a process which has exited buthasn’t been reaped by its parent

Ù draw out a parent/child diagram from initdown to the shell

· init -> getty -> yourshell (zsh) -> your programs

· init -> sshd -> yourshell (zsh) -> your programs

Ù parents can query for the exit state of aprocess

• Process Ownership

· each user on the system can own processes,so when you log in the shell that you see isowned by you

· to identify all the processes on the machineeach process is given a PID or process idthat uniquely identifies the process

· you use the ps command to see the pro-cesses that are running

Ù run just by itself (no switches) will justprint out your process that are being runfrom the terminal you logged in on

Ù to see all processes

Ù ps -aux

16

Page 17: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

· can see what everyone is running with this

Ù point out which column is the process id

Ù pids are not always 5 digitsÙ init for example will always have its

PID equal to 1Ù on OS X pids are generally 3 or 4 digitsÙ linux tends to be 5 digitsÙ will most always between 2 and 32000

Ù so why is this useful?

• How to be like Charlie Manson

Ù what if we want to kill a runaway processor any other process

Ù imagine you have a java application in aninfinite loop

· this would be a good time for CS60 stu-dents to pay attention

· don’t want it running forever, so want tokill that process

· we use the kill cmd to do this

Ù kill [PID | %n]

Ù two ways to kill

Ù (1) if the process is running in the back-ground, can use background number

Ù (2) can use the process id (PID)

Ù also demo

Ù killall <proc name>

Ù we can use ps to find the process id

Ù demonstrate using ps to find the id and killthe process

Ù but how did we know that the process wasout-of-control in the first place?

· nifty utility on unix called top

Ù will list out the processes in order of cputime

Ù demonstrate top - point out the PID column

Ù can use this to see if any of your processesare out of control

Ù should be able to point out several aspects oftop

Ù memory usage, cpu time, state and pri-ority

Ù top is made more useful by seeing only theprocesses that you own

Ù first start top

Ù top

Ù then press the ’u’ key; then type your user-name, or another username if you want tosee their processes

Ù then enter

Ù demo this feature also

Ù to get help on the other commands top of-fers use the ’h’ key when running top

Ù to quit top press the ’q’ key or C-c

Ù top is really nifty, so spend a few minutestrying to figure it out

Ù digression: what would happen if say, wewere root and we typed

Ù kill 1

Ù what would happen? is that even referringto a process?

Ù in general killing a parent does not killchild, child is orphaned and becomes a childof init

Ù init is the exception

Ù yes it is - init (or launchd on Tiger) is givenPID of 1

Ù so typing kill 1 while root would kill youentire system - basically all processes stopand the system has to be rebooted

Ù this is a bad thing and should be avoided

• Process priority

Ù what is process priority?

· priority defines how much time the partic-ular process should get from the machine

· processes with a low priority gets run lessoften than a process with a high priority

· as a user you can’t actually change the pri-ority, only root and the scheduler can dothat

· what you can do is set the *niceness* of theprocess

· a higher nice value means that the processis nicer to other processes on the system,giving them more time to get stuff done

Ù so a high nice value leads to a low priorityand a low nice value leads to a ”normal”priority

Ù to set the niceness of a process you use thenice cmd

Ù nice cmd

17

Page 18: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù if you are going to be running a long andfairly intensive job on knuth please use, andbe, nice!

Ù nothing worse than locking the system upfor fifty people while you try to computeprime numbers ;-)

Ù show how mprime has been niced onodin

Ù to use nice run it like such

Ù nice [-n NUM] cmd args... NUM=0 to19 (lowest) (root for 0 to -20 (highest))

Ù find a demo for nice

Ù check out the man page for more info

Ù does nice have an effect?

Ù nice will have an effect if the process is *cpubound* meaning that it requires the proces-sor more than anything, doesn’t really needany input

Ù on *IO bound* processes not so much sincethey wait for input anyway, and while wait-ing the system will do other things, won’thave much effect

Ù now know all about managing processes andhow the system handles the

14 Some programs to remem-ber

Ù this is a list of programs that we consider to bethe most useful or that are basic to being ableto use a unix system

Ù locate, echo, cat, clear, less, head, tail

Ù locate

Ù allows you to find files on the system

Ù has to search a database so not always up-to-date

Ù usage

Ù locate <filename>

Ù accepts wildcards for files which we’ll get tolater but which you may already be familiarwith...

Ù echo

Ù prints whatever arguments that you give itto standard output

Ù useful for when you want to write/appendsome string to a file

Ù or just when you want to write to stdout

Ù usage

Ù echo "text string"

· or

Ù echo text string and more text

Ù before using you should really read the manpage (it has some very interesting options)

Ù cat

Ù like echo, instead takes file names as argu-ments

Ù prints the contexts of each file in order tostandard out

Ù one of the more useful commands, espe-cially if you want to look at a small textfile

Ù cat is short for concatenate - you may thinkthey should have used ”cons”

Ù usage

Ù cat <file1> <file2>

Ù if no file is given will take its input fromstdin

Ù end the text stream with C-d

Ù will then print back to stdout

Ù most useful when combined with redirec-tion and piping

Ù clear

Ù clears the screen (also C-l) - useful for whenyour terminal has filled up with junk andyou’d like to make it go away

Ù clear

Ù more

Ù is a pager

Ù sometimes you have more text than can befit on a single window, so need to way tosee the entire text of a document

Ù more lets you view the entire text of a docu-ment, but is limited in that you can’t scrollup

Ù use the space bar to advance to the nextscreen

Ù less

Ù less is also a pager

Ù less is a better pager than more however

18

Page 19: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù everyone knows the old adage ”less ismore”, there is also another pager calledmost

Ù most > less > more

Ù have already had exposure to this programthrough the man command

Ù man opens up its man pages in less (not bydefault but generally)

Ù but of course can use less to view more thanjust man pages

Ù usage

Ù less <file>

Ù and some of the key commands to make lessdo what you want - interactive

Ù arrows move man page up/down left/right

Ù space bar moves a screen at a time

Ù to search would type /pattern, then pressenter

Ù demo this feature

Ù typing / or n again moves to the next in-stance of pattern

Ù N moves to the previous instance

Ù b : back - like page up

Ù f : forward - page down

Ù head

Ù prints out the first n lines of a file to stdout

Ù useful for when you have a sorted list andwant only the first n

Ù by default (if n is not specified) will printout the first 10

Ù usage

Ù head [-n XX] <file>

Ù tail

Ù like head, in that it will print out the last nlines instead of the first

Ù more useful in that files, especially log files,are often appended to

Ù can then follow these files for as long as theyare open and see what is being added tothem

Ù usage

Ù tail [-n XX | -f | +NUM] <file>

Ù ”-n” : gives last XX lines of file

Ù ”-f” : keeps file open

Ù ”+NUM” : shows all but the first NUMlines of a file

Ù now on to some other commands

Ù the next two are both searching commands -one is for searching through stdin, the other forsearching through a directory tree

Ù grep

Ù allows you to search for a regular expressionwithin a file

Ù often though you just want to find a stringwithin a file, not everything matching a cer-tain pattern

Ù this is really just a copout since we don’twant to show you regex’s until later

Ù so to search through a single file

Ù grep [switches] "string" file1 ... fileN

Ù grep has some useful switches that make itnicer to work with

Ù ”-c” : instead of printing lines that matchwill instead count the number of lines thatmatch and print that instead

Ù ”-n” : print the line number on which thepattern was found

Ù ”-H” : prints which file the pattern wasfound in

Ù ”-r” : if a directory name is given will recur-sively descend through the hierarchy look-ing for matches in all the files in that tree

Ù there are also different versions of grep, soyou’ll want to check out the grep man page,as usual

Ù find

Ù at its most basic level find just descendsthrough a directory tree and prints out thenames of whatever files it finds there

Ù so you can replicate (in functionality butnot speed) the locate command with

Ù find ./ | grep pattern

Ù this is generally much slower than locate,but has to go through the entire file sys-tem starting at ./ (which is where ever youcurrently are)

Ù but find is much more complex than thisand can do locate all on its own

Ù find ./ -name glob -print

19

Page 20: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù the ”-name” switch will search for the glob,and ”-print” will print it to standard out(note that glob != regex)

Ù find can of course do much more interestingthings

Ù try to find out what this does, and why youwould need it

Ù find ./ -name ".foo" -perm +444

-exec chmod -R a+r {} \;

Ù these are other commands which are useful, butnot exactly necessary to know

Ù w

Ù prints out who is on the system

Ù w

Ù which

Ù prints out which command exactly will beexecuted when you just type cmd

Ù useful for when you want to figure out eitherwhich program you are using or where theprogram you are using is

Ù which cmd

Ù du

Ù displays the amount of space that a file (ordirectory) is taking up on disk

Ù du file1 file2 file3 ...

Ù to get the output to be a bit more useful,will want the ”-h” switch

Ù demonstrate this

Ù if file is actually a directory, du will recursethrough the directory printing out the sizeof each file as its gets them finally printingout how large the directory is in total at theend

Ù this is not very useful, so pass in the ”-s”switch to suppress the extra output

Ù this creates a useful ”sizeof” command

Ù du -sh file1 file2 file3 ...

Ù df

Ù shows a detailed listing of the amount ofspace being used on each file system that ismounted

Ù show the output of and explain

Ù made nicer by the ”-h” option

Ù show this output

Ù at the end of this would be a good time to poll tofigure out what other programs people would liketo know about;

Ù questions like ”what is the program that does....?” sorting, calculator, etc.

15 Files, programs and file, andthe filesystem

Ù what is a file? some review from before

Ù a file is a stream of bytes; a file isn’t always nec-essarily on disk

Ù traditionally think of a file as something like atext file or mp3

Ù in unix though we know that everything is a file:directories, devices, etc.

Ù review the unix security model

Ù users have a home directory

Ù users not allowed to do random (dangerous)things - only allowed to do those things thatthey have permission to do

Ù there is a superuser that can do anything

Ù users belong to groups, and have a primarygroup

• Files

Ù how should an OS handle security for files?how do you tell who owns a file? what’s agood way to manage sensitive files?

· files have their own security model

· files have an owner and a group that theybelong to

· user is you

· group is the group the file belongs to

· so when you create a file, the file is createdwith you as the owner, your primary groupas the group the file belongs to, and all thepermissions on

· so we’ve defined two categories that a filebelongs to, what else is left?

· what about the other people on the system?

· so in all a file is concerned with three setsof people: user, group, others

20

Page 21: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

· others are by definition everyone else on thesystem

· besides owner, group, etc., files have per-missions associated with what people insideof these classifications can do with the file

• File Permissions

Ù based on the security model above whatkinds of permissions should we have? whatkind do we have?

Ù so just because you own the file doesn’t nec-essarily mean that you can write to it, orexecute it if its a shell script

· so what things can you do to a file?

· well unix defines only three (there are morethat could be useful, but unix is old and thespec. hasn’t changed with the times)

· these three are: read, write and execute

· can set these three permissions for any cat-egories: user, group, others

· so the permissions look like: rwx|rwx|rwx

· the first rwx is the user, the second groupand the third others

Ù have an exercise to figure out what the per-missions of several files mean

Ù so when you see rw-rw-rw what does thismean? etc.

Ù now on to octal permissions

Ù so rather than writing out a bitmap eachtime you want to say what the permissionsof a file are, what would be a more compactrepresentation?

Ù use permissions base 8

Ù they are permissions base 8, because eachpermission group (user, group, others) canbe represented by a number from 0 to 7

Ù if we write out 0 to 7 in binary we see thatwe require 3 bits (hmmm!)

Ù _ _ _ so these bits have value 4 2 1 and aremapped to r w x

Ù 4 2 1

Ù so 7 looks like 1 1 1

Ù and 4 looks like 1 0 0

Ù combining this we can write permissions outas a group of three numbers 0 to 7

Ù permissions look like XXX

Ù the first X is user, the second is group, andthe third is others

Ù examples of common permissions settings

Ù 666: everyone read, writeÙ 777: do anything to fileÙ 022: user can’t do anything, group w,

other wÙ 744: user do everything, group read,

other read

Ù as an exercise write out the octal permis-sions in the rwxrwxrwx form

• Changing Ownership

Ù do an ls -l on some files

Ù demonstrate the two following programs

· chgrp - group· chown - owner

• Changing permissions

· chmod - will also change ALL aspects of afile if wanted

Ù demonstrate some standard examples -make file world readable

Ù switches u, g, o and the +-= syntax, withperms r,w,x,X

Ù also cover -R for recursive

Ù since may want to change all files in adirectory

• special files - directories

· directories are files too, though can’tread/write to it as you would a regular file

· but because are files, they have to have thesame permissions that other files on the sys-tem has

· unfortunately the permissions on a direc-tory act a bit differently (which you mightexpect)

Ù list how permissions affect a directory file

· r can list with cd

· x can access the file within

· w means can write a file to that directory

• dot-files

Ù so when you type the ls cmd, do you seeevery file?

Ù obviously not since I’m asking this question

Ù do an example of a hidden file - .hidden or”.you don’t see me”

Ù is that all the files in the directory?

21

Page 22: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù not necessarily...

Ù ls -a

· ”-a” switch will show ALL files in the direc-tory

· this is useful because dot-files are generallyused to hide configuration information forprograms away from the user

· also keeps the configuration information inyour home directory so that each user canhave their own settings as they like

Ù some examples of .files

Ù one common one you may care about is your.zshrc, this controls some of the settings youhave in your shell as you login

Ù another is .emacs - controls the settings foremacs

Ù you can also have .directories

Ù these are directories that are hidden

Ù do the same thing as .files (hide config info),but for programs that may have many con-figuration files that need to be hidden

Ù so is it a nice thing that just ls doesn’t showall the files in a directory by default? yes,but is still historical crap

Ù a better solution might be possible

Ù show a gftp listing of my home on farmboy

Ù so when you do a ls -a, do you recall seeingtwo files one named ”.” and the other ”..”?

· probably not but these are two special files

· the file named . is this directory

Ù do a literal translation for whatever di-rectory in on board

Ù so if you typeÙ cd .

Ù you stay in the same placeÙ what if you see a dot in the full path

name of a file?Ù ex. /foo/bar/./baz/box/Ù where is this referring?Ù to /foo/bar/baz/box/Ù so it means nothing, should just ignore

itÙ ./ is the same thing as .

· the file named .. is the parent directory

Ù you should already know that if youwant to go up in the directory hierarchythat you type

Ù therefore .. is the parent directory ofthe current directory

Ù ../ is the same as ..Ù cd ..

Ù so can probably guess that is you seeÙ /foo/bar/../baz/box/Ù you know where this refers to? where

does this go?Ù answer: /foo/baz/box/Ù so you can’t ignore these, but can be

useful for when you need to craft a pathto a file

• other special files

· tilde (~) denotes your home directory

Ù cd ~

· will take you to your home directory, a plaincd will do the same

· username denotes the home directory ofthat user

• relative and absolute paths

Ù can refer to files in two different ways

Ù what if you have written a program andwant to access a certain file - config.cfg

Ù how do you refer to this file?

· two ways: relative and absolute path

· absolute paths never change and describe in”absolute” terms where a file is in the tree

Ù emacs /home/mkegel/school/fall05/graphics.txt

· absolute paths will always start with /

· you are starting at the root and workingdown the tree

· relative paths start with a ./ - the directorytree before the ./ is not known

Ù don’t necessarily know where the programwill be stored, but always know where theconfig.cfg file will be in relation to the pro-gram

Ù or should at least know where it will be

Ù can refer to it then by saying

Ù cmd ./one/two/config.cfg

Ù the ./ is filled in by the OS or the shell -depending on context

22

Page 23: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

16 Links - symbolic andhardlinks

Ù so now that we’ve confused you this much whatif you wanted to have two files have the samename?

Ù why would you want this?

Ù how about when you upgrade software - couldhave the program named emacs, but the /real/name might be emacs-21.2, an older versioncould be emacs-20.7

Ù you want to continue to access this programthrough the same name, but have it always referto the new version

Ù how might you do this?

• File Links

· unix uses a thing called links

· links are a special type of file which trans-parently redirect you to a different file

· so you can type emacs, but the OS willknow that you really mean emacs-21.2

· there are two types of links: soft and hardlinks

• Types of links

· a hard link, which was what we createdabove, is literally a different name for thesame file

· the new name is stored in the directory itssupposed to be in so takes up virtually nomemory

· the hard link literally points to the samedata on disk

· a hard link cannot cross file-systems (sayyou have two different hard drives and wantto have one file refer to another)

· you cannot hard link to a directory

· a soft-link is different and more flexible

· it is a separate file (not just an entry in adirectory file) that the OS interprets as thereal file, basically just a type of pointer

· soft-links can go across file systems and canpoint to directories

· in practice you should probably almost al-ways create soft-links

· can link both directories and regular files

• Creating Links

Ù ln [OPTIONS] target link_name

· the basic syntax is backwards, so to ”link”emacs to emacs-21.2 you would type

Ù ln emacs-21.2 emacs

· to create one pass the ”-s” switch to ln

Ù ln -s emacs-21.2 emacs

· this will create a soft-link

Ù show output of ls -l

· symlinks don’t always work, for examplewhen trying to get to things through ftp- have to mount instead

17 File globbing

Ù most people are probably familiar with file globs,but for now, what is a file glob?

Ù why might you want file globbing?

· many times will want to work with multiple files

· say you want to mv or cp or rm all the filesin a directory

· how do you do that?

· you use a thing called file globbing

· is very similar to UNIX regular expressions, butshould not confuse them

· globbing is provided by the shell

Ù this is key - is one of the many complaints thepeople have about unix

· the shell is responsible for expanding globs

Ù programs generally only take file names anddon’t generally do pattern matching

Ù why should is be the responsiblity for the shellto expand globs? is this a good thing or a badone?

Ù some people see this as a bad thing - for you youdon’t have to care

· some standard globs - see page 658 U.P.T.

· * - matches all files in a directory (not dot-files though) - match anything

· will not match dot-files, however

· ? - match one character

23

Page 24: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

· [a-z] - any character a to z - characterclasses

· (x|y|...) - match either x or y - this isnot pipe!

· {a,b,...} - expands to ”a” ”b”

Ù give some examples and ask what they wouldmatch

· a[a-z]*

· ?[c]*c

Ù can put these anywhere in the expression

Ù by far the most common is to do

Ù ex1*ex2

Ù *ex3

Ù ex4*

Ù to match dot-files have to put a dot explicitly

Ù to match all dot-files in a directory woulddo .*

Ù show listing of ls .*

Ù note that this also lists ”.” and ”..”

Ù so what would happen say if we did this...

Ù chmod -R a+r ./

Ù as root from /home?

Ù much badness is what...since more than justthe down directory will be changed

Ù .. follows to its parent and so you’ll alsochange all the files in all the directories atthe same level and below in the tree

Ù draw out to illustrate

18 The tree of life - How theUnix directory tree is orga-nized

• Unix Directories

Ù now that you understand how to work withfiles, how to work with programs shouldknow a little bit more about the environ-ment you are working in

Ù unix is in some way a very structured envi-ronment

Ù 35 years of tradition have specified how youdo certain things and why they are the rightthings to do

Ù and then we come to the way the directorystructure is organized

Ù what do I mean by this?

Ù so with windows you were probably com-fortable having all your programs be in-stalled to C:\Program Files\Program; soeverything you installed was in a commonplace

Ù what if some one decided one dayto instead install everything toC:\Your Programs\Program andC:\My Programs\Program; but all theold ones you had would (by default) installto the old directory (Program Files)

Ù unix has a ”unified hierarchy”

Ù this would be quite a pain

Ù in unix the same thing exists, but across thedifferent unixes that exist

Ù of course unix is supposed to be indepen-dent of the directory structure on which itsworking, just makes it a pain for people inreal life

Ù this is an historical artifact

Ù in order to gain market share one thing theyteach you in business school is to differen-tiate your product

Ù so when unix began to come out in variouscommercial flavors the companies changedthings a bit to reflect how their product wasdifferent

Ù the most obvious and easy way to changean OS is to change where you put things

Ù so you end up with apache (a web daemon)being in four different places in four differ-ent flavors of unix

Ù luckily the madness has continued!!!!!

Ù linux suffers from the same problem thoughperhaps not to the same extent

Ù where things are put in debian is differentfrom gentoo, is different from...

Ù FreeBSD has some resemblance to the linuxdirectory structure similar

Ù just to limit the confusion we’ll just go overthe basics which apply to most unix sys-tems, except for OS X which is very differ-ent

· *now the real material*

Ù draw a /

24

Page 25: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù so we know that everything in unix startsat /

Ù so how do we decide what goes under this?

Ù we don’t get to so much, though if you wantcreate your own distro...

· so lets start with something you alreadyknow /home

• General Directories

Ù cd into each directory as they are being dis-cussed

Ù /home is where the heart (and your home-work) is

Ù in some sense this is true since this is wherethe home directories of all the users on thesystem are stored

Ù the home directories generally have theform /home/username, though this isn’tstrictly necessary, just useful

Ù rest of this is taken from

Ù man hier (on OSX, also present onFreeBSD, haven’t seen it on Linux)

Ù see also pg 822 UPT

Ù /bin/ user utilities fundamental to bothsingle-user and multi-user environments

Ù /dev/ block and character device files fd/file descriptor files; see fd(4)

Ù /etc/ system configuration files and scripts

Ù X11/ configuration information for X· NFS: exports· system: group, passwd, and shadow· shell: profile· sudo: sudoers· network and inet: hosts, hosts.allow,

hosts.deny, hosts.equiv· lots of other stuff

Ù /sbin/ system programs and administra-tion utilities fundamental to both single-user and multi-user environments

Ù /tmp/ temporary files

Ù /usr/ contains the majority of user utilitiesand applications

Ù bin/ common utilities, programmingtools, and applications

Ù games/ Unix games, also fortuneÙ include/ standard C include filesÙ lib/ archive libraries

Ù libexec/ system daemons and systemutilities (executed by other programs)

Ù local/ executables, libraries, etc. notincluded by the basic operating system

Ù sbin/ system daemons and system util-ities (executed by users)

Ù share/ architecture-independent datafiles

Ù share/man/ manual pagesÙ src/ storage for kernel/other source

code under linux

Ù /var/ multi-purpose log, temporary, tran-sient, and spool files

Ù log/ misc. system log files

Ù /lib - shared libraries and kernel modules

Ù /opt - larger static packages - where youwould install KDE for example

Ù /boot - kernel image and other boot files

Ù this is where the linux kernel gets putÙ is sometimes a separate partition - Red

Hat does thisÙ probably a good idea since if your main

one gets fried can still boot

Ù /mnt - temporary/permanent mount pointfor devices

Ù devices can be mounted anywhere, soone solution, if you have lots of harddrives, optical drives, cameras, scan-ners, etc. is to make lots of mountpoints under /mnt, so /mnt/dvd/,/mnt/ipod, etc.

Ù is this a good thing?Ù all up to your own preferencesÙ /mnt is meant for (at least in FreeBSD)

a single device that won’t be mountedfor any length of time

Ù would mount to a different point, say/ipod, so mounted right from root

Ù are multiple solutions, could also havea /vol (/Volumes)

19 Some special devices

• Special files

· /dev/null

· this is a sink for input, you can sendanything here and it will just go intothe void

25

Page 26: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

· useful for when you need a place to getrid of output, but don’t want to wastespace on the disk

· if read from will give you an EOF

· /dev/zero

· produces an infinite stream of zeros ifread from, good for when you need tonull something out

· if written to will just give a permissionerror (?? Is this always true ??)

· /dev/random and /dev/urandom

· when read produces a random streamof bytes

· permission error when try to write to

20 Environment Variables andShell Variables

• Variables

· these are special variables that all programsknow about

· they exist in the background and let pro-grams know about certain things

· where to find programs to run, whicheditor to use, etc

· serves the same purpose as the registry(sort of) in windows

Ù cover the distinction between the two

· environment variables are inheritedacross all shells you may have and allprograms that you run

· shell variables are local to each instanceof a shell that you may have

· each shell gets a clean slate of shell vari-ables when it starts

· analogy - programming: global versuslocal scope

· variable naming convention

· CAPS are environment

· lower-case are shell

· please follow convention!!!

Ù you generally only ever care about environ-ment variables

Ù show listings of the variables

· how do you see what environment variablesyou have?

· simple use the printenv command

Ù printenv

· what about shell variables?

Ù set (lists everything: functions, shell, andenv. variables)

Ù pg 705 has a good listing of pre-defined en-vironment variables

• Common Variables

· good ones to know about - will be all caps

· PATH - when you type in a commandthese are the directories that the shellwill search for the cmd you have typed

· HOME - where your home directory islocated

· HOSTNAME - name of the system youare on

· SHELL - full path name of shell you areusing

· USER - who you are logged in as· UID - your user id· TERM - type of terminal you are using

21 The Different Shells

Ù now is a good time to talk about what shell youare using

Ù there are in fact many different shells!

Ù you’ve probably been using zsh, the knuth de-fault, all this time with ever knowing it

Ù there are a multitude of shells, each with differ-ent strengths and features

Ù most fall into two different categories

Ù those based on sh, which was the first shell

Ù those based on csh

Ù give proper pronunciation lesson

Ù the default on knuth is zsh, which is a hybrid ofsh and csh

Ù other shells include bash (probably the mostwidely used shell), tcsh, ksh (Korn-Shell)

Ù other random shells: ash, fish, pdksh (free ver-sion of Korn-shell)

Ù probably the best (certainly the most fully fea-tured) shell is zsh

Ù we recommend that you use either zsh or, if youhave to, bash

26

Page 27: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù why use zsh?

Ù its just plain nifty

Ù will tab complete many very cool things,and even not so cool ones

Ù e.g can tab complete kill and man for ex-ample

Ù also we are only going to demonstrate forthis shell, you have the tools to figure outhow to do the same things for yourself onyour own shell

Ù so how are they different? what really distin-guishes a shell?

Ù in terms of user interface, shells are almost iden-tical

Ù shells provide the ability to run more one com-mand at a time

Ù if you string commands together in a file, withsome control structures like if, while, and casestatements: you get a language

Ù called shell scripting

Ù fundamental different between csh and sh - cshis more based on C syntax

Ù sh is just funky and has a weird syntax, but forshell scripting is better (better control over out-put and input)

Ù so when we say a shell is based on another, whatwe really mean is that zsh or bash can run scriptswritten for sh

Ù and that zsh and bash offer scripting options thatweren’t available with sh

Ù the same for tcsh

Ù we won’t show you shell scripting yet, but youwill learn about it

22 Setting Environment Vari-ables

• Setting Env. Variables

Ù to set environment variables in zsh

Ù export NAME=value

Ù can also do this

Ù NAME=value; export NAME

Ù you can see the output of a variable by usingthe echo cmd

Ù echo $ENV

Ù have to remember the $

Ù demonstrate this for the students

Ù the most common thing to do is to addpaths to PATH

Ù may have to do this for a classÙ may want to do this depending on

which box you are logged into

Ù from the cmd line for zsh we would type

Ù echo $PATH (to see path)

Ù export PATH=$PATH:/newpath

· or -

Ù export PATH=/newpath:\$PATH

Ù the two are different!

Ù first puts the new path at the end

Ù the second puts the new path at the front

Ù is important because when searching for thecmd to run, the shell will take the first onethat matches, so you want to make sure itsmatching the right cmd

Ù unfortunately you would have to do thisevery-time you logged in

Ù would get boring

• Setting Env. Variables

Ù maybe there is an easy way fix this prob-lem?

Ù ask crowd for suggestion

Ù real answer: this is why you have dot-files

Ù we know dot-files hold configuration infor-mation

Ù zsh is no exception to this rule

Ù zsh has several files that hold login infor-mation

Ù /.zprofileÙ /.zloginÙ /.zshrcÙ and more

Ù depending on default setup will want toshow how to do this add path

Ù the one that you care about is .zshrc

Ù once you change your .zshrc it won’t imme-diately take effect

27

Page 28: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù either you have to log out and restart yourshell or you can apply the change from thecmd line

Ù to apply from the cmd line use the sourcecmd

Ù source .file

Ù will cause the shell to read the configurationinformation, and apply the changes

Ù but this doesn’t cause the shell to reset itsold configuration, it just overwrites what itchanges

Ù so your PATH will be two copies of yourold path followed by the added directoryassuming you put it at the end

Ù not always what you want, and shouldn’tcause any problems unless you’re doingsome funky things in your .zshrc or .zlogin

Ù *changing the prompt*

Ù well if you can change your PATH whatother things can you change?

Ù lots of things, but the most fun to changeis the prompt

Ù demonstrate some nifty prompts in zsh andhow to change them

Ù other shells options

Ù shells are complex beasts and offer loads ofoptions, one of the most annoying is theshell beep

Ù to alert you to an error when you mistypesomething, the shell will cause the com-puter to emit a beep

Ù this is the most annoying thing ever

Ù to turn it off we would use the setopt com-mand that zsh has

Ù setopt nobeep

Ù we would put this line in our zshrc; by de-fault it should already be there

Ù there are lots of options so check them out!

Ù may want to read through the default zshrcto see what’s set

23 Using alias in the shell

• ALIAS

Ù it is a builtin cmd that allows you to ”alias”one set of commands to another

· so you can refer to a complicated, or repet-itive, command with a smaller, easier totype alias

Ù you always want to use these when you aretyping some long string over and over againthat you cannot just tab complete

Ù is very much like a macro

Ù what is one common thing that you mightbe typing?

Ù how about ”ssh [email protected]

Ù so to create an alias we need a name: howabout sshtoknuth

Ù from the cmd line would type: (spacing isimportant)

Ù alias sshtoknuth=’ssh [email protected]

Ù can also use ” when creating alias’ (good ifyou want to reference env. variables)

Ù this alias is, of course, only good for so longas the shell is open

Ù once you quit the alias is gone

Ù the basic syntax is

Ù alias name=’cmd arg1 arg2’

Ù but what would be a good name? some-thing better than sshtoknuth

Ù the whole point of the alias (for users) is tomake your life easier

Ù so want to choose a simple name that is easyto remember

Ù how about: knuth

Ù says what you want, is simple and short

Ù what would have happened though if we’daliased this to ssh?

Ù question of precedence...and the alias wins

Ù in zsh and bash when you type somethingon the cmd line that you want to be takenas a cmd, then if that text string has beenaliased to something else it will be replaced

Ù is this a good thing or bad thing?

Ù used in the appropriate places is OK

Ù in general, however, is a bad thing becausethen you get used to a potentially brokenor misleading command action.

Ù almost always want to use a new name

Ù one example is ls

Ù by default what you get on knuth is not/usr/local/bin/ls

28

Page 29: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù instead you get ”/usr/local/bin/ls -F”

Ù do an example of what each outputs

Ù discuss why this might be a bad thing, tellabout -G on GNU ls (hide groups)

Ù should watch which options you pass by de-fault

Ù so how did I know that all of you had this?

Ù demonstrate the defaults alias’ that peoplehave in zshrc

Ù for certain programs you’ll want the samebehavior every-time you login

Ù so to make an alias more or less permanentyou would put it in your zshrc

Ù so if you wanted the alias we talked aboutbefore you would add this line

Ù alias knuth=’ssh [email protected]

Ù spacing is important so follow closely

24 More about zshrc

Ù so the zshrc can set options and aliases for you,what else can it do at login?

Ù the at login is important since you can do thisall from the cmd line; just gets old if you wantit every time

• creating files and the umask

· by default every permission is ON, so itsyour job to turn them OFF

· the umask is the thing that does this

Ù show in zshrc

Ù written out in octal permissions (rememberthose?)

Ù write out the permissions of the umask

Ù ask what the permissions of a newly createdfile would look like

Ù ask if the results are good or not

Ù what might be a good umask; what would bea bad one?

25 UNIX programs for Win-dows geeks

· there are many different tools that unix offers

· you should now know how to make them worktogether and some of why they work as they do,but don’t exactly know what tools are out there

• Useful Applications

Ù these are almost all graphical tools and areessentially replacements for programs thatstudents would be using on either their Macor Win box

· web browser - firefox and mozilla, kon-queror

· text editing - jedit, emacs, vi

· word processing - abiword, open office

· document creation - latex

· spreadsheet - gnumeric, open office

· music - xmms, amarok (my favorite)

· movies - totem, xine, mplayer, vlc

· graphical terminals - eterm, aterm, xterm,rxvt, gnome-terminal, konsole

· mail - thunderbird, evolution, pine/muttthrough xterm

· pdf viewers - xpdf, kpdf, gpdf

· ftp - ncftp or lftp (cmd line), gftp

· cd burning - k3b

· aim - gaim, kopete

· bit torrent - azureus

· file manager - konqueror, nautilus

· spell checkers - ispell and aspell

· Windows like desktop - GNOME, KDE

· window managers - fluxbox, xfce, windowmaker, enlightenment

26 su and sudo

Ù up to this point have been fairly general, infor-mation applies to almost all users of a *nix sys-tem

Ù this information is really only useful to thosewho will using Linux on their own system, notso much for those people who are just going tolog into knuth

• sudo and su

Ù discussed before how use

· su allows you to assume the role of the supe-ruser (aka root; is default behavior) withouthaving to directly log in as root

Ù su <username>

· allows you to become that user (changesyour UID and GID), but have to know theirpassword

29

Page 30: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

· if you have first become root, you can dosu <username> without a password

· sudo allows you superuser powers withouthaving to become root - more fine grainedcontrol than with su

· sudo is flexible, you can have limited pow-ers, or have the whole shebang

· sudo privileges are controlled in/etc/sudoers

· is a regular text file, like almost all config-uration files on unix

· can add powers for users, or groups

· generally only modifiable by root (kind of asecurity risk otherwise)

Ù can also get a shell out of sudo by using the-s switch

• editing /etc/sudoers

Ù demonstrate adding

Ù mkegel ALL=(ALL) ALL

Ù %wheel ALL=(ALL) ALL

Ù %special ALL=(root)NOPASSWD:/usr/bin/myprog

Ù joe user local host=/sbin/prog

· the basic format is:USER HOST=[(RUN_AS)] CMD

Ù see page 992 of unix power tools

Ù would you ever want to let a person run emacs(or vi) with sudo? no: they would have accessto a root shell...bad thing need to watch out for?reason why you would want: let them edit sys-tem files

Ù why might you need to use sudo and su? whatkinds of things should be be an administrator todo?

27 Mounting partitions

Ù this information is really only useful to thosewho will using Linux on their own system, notso much for those people who are just going tolog into knuth

• mounting partitions*

Ù when managing a system you will almostalways be modifying ”files” that ”normal”users should never touch

Ù these files are generally configuration files

Ù one of the most basic things to configureon a newly installed linux system are thepartitions and how the hard disk will bemanaged

Ù in windows separate partitions would justmagically show up (not always, but mosttimes)

Ù more too it than this, but the managementof the disks was done mostly by the oper-ating system

Ù also in windows the most the you would re-ally ever do is put the disk/partition at G:,H:, or so on

Ù other devices were located at D: (generallya cdrom or dvd drive)

Ù A: - floppy drive

Ù C: - your main disk - the disk you bootedoff of

· these ”locations” are known as mountpoints

· in unix you don’t have a:, c:

· instead disks and other devices can be”mounted” most anywhere

Ù that is if you have a newly installed harddrive you have to decide where in thefilesystem tree that drive will be accessed

Ù this is what makes the file system(s) trans-parent to both users and programs

Ù draw out a diagram of how multipledrives/cdroms/devices can be accessed on afilesystem tree

Ù generally mount stuff in /mnt, but can beanywhere

Ù can also mount what you might considerimportant directories from other drives

Ù for example: /home on my system is an-other partition that I created for the ex-press purpose of holding everything that Iwould want in my home directory

Ù / holds everything else I need for my sys-tem, but is separate from what I have in myhome directory

Ù this is a nice because it is very flexible andcompletely transparent to you and to anyprogram that you might run

Ù one example of this being done is witheach of your, and mine, home directorieson knuth

30

Page 31: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù your home directories are mounted locallyon knuth when you log in, but where theywere before that doesn’t really matter

Ù obviously knuth can’t have enough physi-cal space to hold everything that we mightwant - home directories, staff accounts,projects, random data, backups

Ù all this disk space is provided transparentlyacross a network through something calledNFS or network file system (or No FuckingSecurity, nfs assumes you are on a trustednetwork)

Ù so your home directories are located on onemachine, while you are working on knuthand doing things to your files, editing, com-piling, etc.

Ù however that was just a setup for whatyou’ll likely do on your own machine

• mounting devices

· in linux, we won’t worry about bsd or other*nix, devices are located in a /dev like wetalked about before

· a device is of course just a file - is actuallya special file called a device node - tells thesystem where in the system the device canbe found (for PCI devices major and minornumber of device)

Ù device node != regular file or directory

Ù one thing to note: the linux 2.4 kernel isdifferent from the 2.6 kernel in one majorway -> the 2.6 kernel does use the /devfilesystem approach to devices; 2.6 is justmore intelligent and doesn’t enumerate ev-ery possible device

Ù in the 2.4 kernel every device that youcould possibly have attached to the sys-tem was enumerated and given a devicenode in /dev

Ù in 2.6 the kernel is a little bit smarterabout devices now and can handlethem with some intelligence

Ù for you this won’t matter too much un-less you’re trying to hook in some ex-otic device

Ù or if you have sound/video/usb prob-lems (or want to use these devices) thenyou’ll get to know /dev pretty well

Ù generally a device will be owned by root andin one of several groups: disk, video, audio,root, etc.

Ù hard drives are generally given the designa-tion hdXN, where X is [a-f] and is the driveand N is [0-9]

Ù so hda1 would be the first partition onthe master hard drive

Ù hda2 would be the second partitionÙ hdb would be the second hard driveÙ drives have up to 4 primary partitions,

and can have more logical partitionsÙ generally if you just say hda, or hdb you

mean the entire device and not any onepartition

· scsi drives (like what you’d have underfirewire) are denoted the same as harddrives except with sd instead of hd nota-tion

· another would be /dev/input/mouse0 (onmy system at least) for the mouse

Ù most often though will deal with hard drivesand partitions or other comparable (NFS)things

Ù so then how do you get a partitionmounted?

Ù easy with the mount cmd

Ù mount is fairly smart, can usually figure outwhich file system is on the device

Ù note to windows users: Most any computerwith XP will use NTFS which is a filesystemthat linux can only read (trouble writing)

Ù FAT32 will work just fine

Ù to use mount you need a directory on whichto mount, so if we were to mount /dev/hda4on /mnt/windows, then windows wouldneed to be a directory under /mnt

• mount command

Ù also to use mount you have to be root orhave root powers (sudo)!

Ù mount /dev/device /mnt/mntpoint

Ù to specify the file system use the -t switchlike such

Ù mount -t vfat /dev/hda4 /mnt/windows

Ù to un-mount the device use the umountcommand like such

Ù umount /dev/windows

Ù also cover --bind (GNU, different on BSD)- can bind one filesystem to two points -useful for when symlinks won’t work (net-working stuff)

31

Page 32: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù when you are setting up a system its quite helpfulto know which tools are available

Ù most systems you get will either come with win-dows installed, in which case the entire harddrive is one windows partition

Ù or you got a new hard drive

Ù (Windows Case)

Ù You’ll want to get your hands on a programcalled Partition Magic

Ù another good program is GNU Parted

Ù if you know where to look on the network(ffs.muddstudents.com *ahem*) you mightbe able to find the program

Ù it can do most anything that you’ll want,including converting, moving and resizingWindows partitions

Ù you probably shouldn’t let Partition Magictouch any linux partitions

Ù (New HD Case)

Ù you’ll want to use cfdisk or fdisk to createnew partitions

Ù use mkfs to then format the partition

Ù (General Case)

Ù the cmd df will show how much space isavailable on each mounted filesystem

Ù df

Ù the cmd du will show you how much spacesomething is taking up

Ù du <filename>

Ù useful alias

Ù alias sizeof=’du -sh’

• /etc/fstab

Ù So I mentioned that I have /home mountedfrom another partition

Ù this happens on boot, but how?

· /etc/fstab is a configuration file that letsthe system know which devices are to bemounted, where, who can mount, and withwhat other options (executable flag beingone)

Ù show my /etc/fstab - point out what eachcolumn is

· there are other things in things in fstab thanjust hard drives, but you can ignore them

· to modify fstab you have to be root(wouldn’t want random users messing withthis)

· fstab tells the system which filesystems areto be mounted at boot time

· also lets the system know which mountpoints the system has

· so say I wanted to mount windows then alli would have to type would be

Ù mount /mnt/windows

· and the system would auto-magically knowwhat to do

· you also use fstab to mount NFS exports,but we won’t cover that here

· do not fubar this file, will make you un-happy

28 Using cron

· cron is a daemon that runs in the backgroundand schedules tasks

Ù such tasks might be backing up a hard drive,updating the locate database

Ù this configuration information is stored in acrontab file

Ù generally you just give crontab a file containingwhat you want done and when - this can be doneas a regular user

Ù crontab mysched

· also, edit the crontab with crontab -e

· mysched will then contain single line entries foreach task that you want cron to do

· each line is formatted like this

Ù mins hrs day-of-month month weekday [username] cmd

Ù the username is only there in BSD systems, butcheck your local docs first

Ù you can give a single number (7), range (3-8),list (5,7,9), or * to indicate all valid numbers fora given entry

Ù mins: 0 to 59; hrs: 0 to 23 (military time); day:0 to 31; month: 1 to 12 weekday: 0 (Sunday) to6 (on BSD it goes from 1-7 with 1=Sunday)

Ù show a random cron example

32

Page 33: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù something scary for Friday the 13th, and some-thing cheerful for Christmas, and school startingevery Monday

Ù cron can also make your life easier by just run-ning things on a consistent schedule - say youneed to run things on a hourly, daily, weekly, ormonthly basis

Ù cron will run scripts that you put into/etc/cron.xxxx/

Ù where xxxx is daily, hourly, etc

Ù on those intervals

Ù you’ll have to be root to put stuff in these direc-tories though

29 Wine - Windows ”emula-tion” for linux

Ù use wine!!

Ù there isn’t much else to tell you

Ù install wine

Ù do this from the cmd line

Ù wine WinApp.exe

Ù hope for the best; does not work for every appli-cation

Ù if you want to run games (or itunes, or quick-time) then you’ll need a thing called cedega

Ù there is good documentation, you just need tofind it

30 Cygwin

Ù can get lots of GNU programs

Ù can get X11

Ù runs on Windows

Ù get a poorly implemented version of a terminal

Ù but can do unix-y type stuff on windows if youabsolutely have to

31 Sharing files under linux

Ù use a package of programs called samba

Ù allows you to share and access windows networkshares

Ù really easy to use but has issues with some of thenewer versions of the windows sharing protocol- may not be able to access all files

Ù better yet why don’t you just use ftp?

Ù ftp is much easier to setup, and you don’t haveto dirty yourself with Windows slime

32 Emacs and vi

Ù please use emacs, or if you have a graphical en-vironment something like jedit

Ù vi should only ever be used in the most desperateof circumstances

Ù emacs and vi

Ù hand out reference cards

Ù demonstrate some nifty tricks (M-q for one)

Ù how to exit and save

Ù F10 is menu in emacs

Ù vi is a modal editor - some people ”just getthis” way of editing

Ù not our goal to teach students to use these

Ù and besides by this point in the semester youshould already know how to use emacs

33 More topics to talk about

Ù shell scripting

Ù go read Unix Power Tools

Ù demonstrate a for loop on the cmd line how-ever

Ù perl and python

Ù both powerful interpreted languages

Ù easy to learn and use; generally only needto know one but should learn at least one

Ù checkout these sites:

Ù diveintopython.org and python.org

Ù perl.org and perl.com

33

Page 34: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù cannot stress how useful knowing a lan-guage like one of these is

Ù learn it!

Ù logging in through ssh

Ù how to X forward with ssh (-X | -Y)

Ù how to use ftp/sftp and scp

Ù why ftp is a bad idea - any password send in theclear

34 Things I left out

Ù speaking of other frustrating things also mentionthe C-q and C-s control flow ”bug”

Ù C-q resume console I/O

Ù C-s stop console I/O

35 Regex Info

Ù 3 slides for this topic

Ù metacharacters:

. * + ? [] ^ $ { } \ () |

Ù POSIX classes: [:alnum:] [:digit:] [:blank:] etc

Ù two main commands that we’ll cover that useregular expressions: sed and grep

Ù grep just searches for the given pattern

Ù sed can do that too, but often sed is used toreplace things

Ù grep ’foo’ file1

Ù returns every line that contains the string’foo’

Ù grep ’^foo’ file1

Ù returns every line that starts with foo

Ù grep ’foo$’ file1

Ù returns every line that ends with foo

Ù grep ’^foo$’ file1

Ù only lines that are exactly ’foo’

Ù grep ’fo*’

Ù would match f, fo, fooooooooooooooooo,etc

Ù grep ’fo?’

Ù would match f, fo

Ù grep ’f[oe]*’ file1

Ù would match f, fo, foe, fe, fee, etc, but notfie

Ù grep ’^f[oe]{1,2}$’ file1

Ù would match fo, fe, foe, feo, fee, foo, butnot f or fooo

Ù grep ’f[^aieou]*’

Ù would match f followed by 0 or more non-vowels

Ù grep ’(fa|e)o*’

Ù would match fa, faoooo, e, eoooo, but notfae

Ù Note that grep ’fo+’ is equivalent to grep ’foo*’

Ù sed is generally used for substitutions. There aretwo really handy commands for sed: s(ubstitute)and d(elete). Coupled with addresses, you cando some really cool stuff.

Ù By default, sed applies the commands to everyline. You can change this by specifying an ad-dress pattern to use. Patterns can be line num-bers, regexes, or things like $.

Ù No address -> apply commands to every line

Ù 1 address -> only on lines that match the addressstring

Ù 2 addresses -> only on lines between the first linethat matches the first address and the first linethat matches the second address

Ù You can negate addresses by appending a !.

Ù So, just ’d’, without an address, will delete everyline.

Ù 1d will delete just the first line.

Ù 1,5d will delete lines 1-5.

Ù $d will delete the last line. This is not the sameas $ = end of line.

Ù /^$/d will delete every blank line.

Ù 1,/^$/d will delete from the first line until thefirst blank line.

34

Page 35: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù All the addressing stuff also applies to the ’s’command, but to clarify exactly what ’s’ does,we won’t mix addressing and s.

Ù s/tree/cat/ replaces the first instance of ’tree’with ’cat’.

Ù s/tree/cat/g replaces every instance of treewith cat (g for global).

Ù s/tree/(&)/g replaces every instance of treewith (tree). & references the entire match.

Ù s/tr\(ee\)/\1/ replaces ’tree’ with ’ee’.

Ù s/\(tr\)\(ee\)/\2\1/ replaces ’tree’ with’eetr’.

Ù s/\(tr\)\(ee\)/\2\2\2\1/ replaces ’tree’ with’eeeeeetr’.

Ù s/<\([^>]\)>\(.*\)<\/\1>/[\1]\2\[\/\1]/replaces ’<b>foo</b>’ with ’[b]foo[/b]’

36 tr info

Ù 1 slide for this topic

Ù tr is very simple. It takes stuff it finds in the firstargument and replaces it with the stuff it finds inthe second argument. It does take position intoaccount: something found in the 17th position inthe first argument will be replaced by the thingin the 17th position of the second argument.

Ù echo "abc" | tr ’a’ ’z’ = zbc

Ù echo "abc" | tr ’[a-z]’ ’[A-Z]’ = ABC

Ù echo "abc" | tr ’[abc]’ ’[cba]’ = cba

Ù It’s also handy for deleting newlines or someother character you may have a particular ha-tred of: tr -d ’\n’

Ù This is what you would do if you really hatedyour grader

Ù strip out all comments

Ù remove all newlines

Ù get a failing grade on that assignment

37 screen info

Ù 2 slides for this topic

Ù screen is basically a way to have multiple ter-minals inside one standard terminal window.(Think job control, but instead of pausing onejob to go back to the shell, you could simply havea bunch of shells running, only one of which wasin the foreground at any given time.) In the daysof large resolutions and good terminal emulatorsand window manager, screen isn’t that handyfor doing stuff on a local session. It is, however,extremely useful for managing shells on remotesessions.

Ù Make a new (default) session:

Ù screen

Ù Resume the default screen:

Ù screen -r

Ù or

Ù screen -x (like -r, but better – allows multi-heading)

Ù Make a new screen session with a title:

Ù screen -S foo

Ù Resume foo:

Ù screen -x foo

Ù You can also use just -x f, if you don’t have anyother screens starting with ’f’.

Ù If you’ve got a un-resumable screen session, tidyit up with:

Ù screen -wipe

Ù Show available sessions:

Ù screen -ls

Ù The screen commands I use a lot:

Ù ^a c make a new shell

Ù ^a <space> switch to the next shell

Ù ^a k kill the current shell

Ù ^a d detach; go back to the original shell leavingthe current session intact

Ù The screen commands I use not so much:

Ù ^a l redraw the screen in case things got funky

35

Page 36: Unix 101: Introduction to UNIX (i.e. Unix for Windows Users)mkegel/CourseNotes.pdf · Unix 101: Introduction to UNIX (i.e. Unix for Windows Users) Mark Kegel September 7, 2005 1 Introduction

Ù ^a a send a literal ^a

Ù ^a ? (shift /) show help

Ù ^a " (shfit ’) show current shells in a list

36