unix basics - university of minnesota · unix basics presented by: patton fast user support group...
TRANSCRIPT
Unix Basics
Presented by: Patton FastUser Support Grouphelp-line: [email protected] 612-626-0802personal: [email protected] 612-625-6573
Research.We make it happen.
I. Warnings!
II. Basic Concepts
III. Basic Commands
IV. The Fun Stuff
V. Resources
Please Ask Questions!
Outline
• If you delete a file, it is GONE! The file can only be retrieved from a backup.
• If you overwrite a file it has been changed forever. The original file can only be restored from a backup.
• Home directories are backed up nightly.
• The scratch spaces are NOT backed up!
Bottom line? Be Careful With Your Files!
Warnings!
Scratch Space Time Limits!
• Regatta (/regscratch1 and /regscratch2) – 7 days, 50 Gb
• SP (/scratch1 and /scratch2) – 7 days, 50 Gb
• Altix (/scratch1 and /scratch2) – 14 days
• Netfinity (/scratch1 and /scratch2) – 14 days
• SDVL (/scratch) – 14 days
• BSCL (/wrk and /wrk2) – 14 days
Bottom line? Be Careful With Your Files!
Warnings!– scratch space
Netfinity Linux Cluster160 Xeon processors224 Gb of memory2.0 Tb of storage
www.msi.umn.edu/user_support
Basic Concepts– resources
IBM SP370 Power3 processors663 Gb of memory
4 Tb of storage
http://www.msi.umn.edu/sp http://www.msi.umn.edu/netfinity
SGI Altix176 Itanium2 processors352 Gb of memory
5 Tb of storage
http://www.msi.umn.edu/altixhttp://www.msi.umn.edu/pseries
IBM pSeries312 Power4 processors664 Gb of memory
7 Tb of storage
Computational Genetics Lab16 CPU SunFire 68004 Dell PCs1 Mac G51 SGI Fuel1 SGI Tezro1 IBM IntelliStation
Basic Sciences Computing Lab48 CPU SGI Altix16 CPU SGI Onyx49 SGI Octanes8 CPU Sun UltraSparcIII1 SunBlade 20002.5 Tb of storage
www.msi.umn.edu/user_support
Scientific Dev. & Vis. Lab9 SunBlade 2000s 7 SGI Octanes1 SGI Onyx
Basic Concepts– resources
Med. Chem./Supercomputing Inst. Vis. Workstation Lab
3 SGI Octanes3 SGI O2’s 2 SunBlade 2000s
http://www.msi.umn.edu/bscl http://www.msi.umn.edu/vwl
http://www.msi.umn.edu/sdvlhttp://www.msi.umn.edu/cgl
Basic Concepts– console
Console Login:
• Enter username
• Enter password
Consoles normally have a graphical-windows type interface, and always have priority over remote logins.
Console Logout:
Logout procedures vary between consoles. Make sure you logout! Otherwise your files and account are open to the next person. You could be logged out by an administrator.
After your first login you should change your initial password to something that is easy for you to remember, but not easy for someone to guess.
%> yppasswd (Institute machines)
%> passwd (general)
You should NEVER give your password to anyone!
If you forget your password, send a message to [email protected] and your password will be reset.
Note: Never share accounts or passwords!
Basic Concepts– changing passwords
ssh [options] machine secure shell; a program for logging into a remote machine providing encrypted communications between machines
options:-l login name-X sets environment variables for porting X-display
example:
type: ssh -l pfast sp open a secure connection for the user pfast on the machine sp.msi.umn.edu
type: ssh bi2 open a secure connection to bi2.msi.umn.edu with current username
SSH clients:
http://www.msi.umn.edu/user_support/ssh
Basic Concepts– remote login
Why?
• So that you get all of your email sent to one account.
• Makes it easier for sys admins to communicate with you.
How?
1) Create the file .forward in your home directory
2) On the first line:
3) Create this same file in all of your accounts that you want to forward.
Basic Concepts– forwarding email
man [options] command displays manual pages about command-k keyword search for a command
info command displays Info document of command
When in doubt, check the man/info pages!
Need extra details? Check the man/info pages!
Basic Concepts– command line help
Basic Concepts– file structure overview
/
bin@ home/ lib@ scratch/ usr/
user1/ user2/
file1
mail/ source/ work/
proj1/ proj2/
.cshrc
input output
bin/ lib/ local/
user1/ user2/
dir1/ dir2/
file1/ file2/directory ( / )executable ( * )link ( @ )
more*
Basic Concepts– metacharacters
? single character wild card
* wild card, arbitrary number of characters
~ home directory of current user
~name home directory of user name
. current directory
.. parent directory (back one)
command [options] argument
example:
ls -al /scratch
command name
[options] (usually preceded by a dash)
argument (quite often a filename or directory)
Note: UNIX commands, options, and arguments are all CasE seNsiTive!
Basic Concepts– command anatomy
Command Description
ls [options] dirname list the contents of dirname
pwd display full pathname
mkdir dirname create the directory dirname
cd dirname change to the directory dirname
mv [options] source destination move a file or directory
cp [options] source destination copy a file or directory
rm [options] source remove a file or directory
ssh [options] hostname remote login
scp [options] user@host1:file user@host2:file remote copy; file transfer
Basic Commands
skip basic commands
ls [options] dirname list the contents of dirname
options:
-a list all files including hidden files[hidden files are preceded by a ‘.’; eg .cshrc]
-l long listing showing ownership, permissions, and links
examples:
type: ls /home/user/temp view the contents of a directory with absolute pathname /home/user/temp
type: ls ../../temp list the contents of a directory using a relative path.
Listing Contents
mkdir dirname create the directory dirname
examples:
type: mkdir work create the directory work/ in the current working directory
type: mkdir work/proj1 create the directory proj1/ in the work/ directory
type: mkdir /wrk/user2 create the directory user2/ in the /wrk/ directory
Make Directory
cd change to your home directory
cd dirname change to the directory dirname
examples:
type: cd ~tom change to tom’s home directory
type: cd /wrk/user2 change to the directory /wrk/user2
Change Directory
mv [options] presname newname rename a file
mv [options] srcfile destdir move a file to another directory
options:
-i confirm overwrites
example:
type: mv ~user1/file ./outputfile moves file from user1’s home directory to the current working directory and renames it output file
Note: Be careful when overwriting files!
Moving or Renaming
cp [options] srcfile destfile copy a file to another filename
cp [options] srcfile destdir copy a file to another directory
options:
-i confirm overwrites-R recursively copy a directory and its
contents, copies symbolic links
example:
type: cp -R proj1 proj2 copy the directory proj1/ and name it proj2/
Note: Be careful when overwriting files!
Copying
cp [options] srcfile destfile copy a file to another filename
cp [options] srcfile destdir copy a file to another directory
options:
-i confirm overwrites-R recursively copy a directory and its
contents, copies symbolic links
example:
type: cp -R proj1 proj2 copy the directory proj1/ and name it proj2/
Note: Be careful when overwriting files!
Copying
Tip:If you have large files to move from the scratch directories,copy (cp) them rather than moving (mv) them.
rm [options] filename remove a file
options:
-f remove without prompting-i confirm removal-r recursively remove a directory and its contents
example:
type: rm -rf /scratch/user2/dir1 remove the directory /scratch2/user2/dir1and its contents
Note: Be careful when removing files!Especially with wild cards (*,?)!
Removing
scp [options] host:file1 host:file2 secure copy; copy files between hosts on a network using the ssh protocol
options:-r recursive (used for directories)
example:type: scp pfast@wind:”output/*.out” . copy the output files from host wind
with username pfast to current directory
type: scp -r mail origin:mail_copy copy the mail directory to the origin and rename the directory mail_copy
Note: You can use sftp if scp is not available.
Remote Copy
scp [options] host:file1 host:file2 secure copy; copy files between hosts on a network using the ssh protocol
options:-r recursive (used for directories)
example:type: scp pfast@wind:”output/*.out” . copy the output files from host wind
with username pfast to current directory
type: scp -r mail origin:mail_copy copy the mail directory to the origin and rename the directory mail_copy
Note: You can use sftp if scp is not available.
Remote Copy
Tip:If you have large files to transfer, use sftp instead ofscp.
chmod [options] who ops permission filename change file permissionsuse ls -l to view permissions
options
–R recursively change permissions
who can be any combination of:u (user) g (group)o (other/world) a (all or ugo)
ops adds or takes away permission, and can be:+ (add permission) – (remove permission)
permission can be any combination of:r (read) w (write) x (execute) X (check user first)
Changing Permissions
history display list of most recent commands
!string repeat command beginning with string
up/down arrows scroll up and down through most recent commands; csh and tcsh
left/right arrows use for editing current line
Control-E end of lineControl-A beginning of line
examples:
type: !ls repeat last ls command
Command Repetition
TAB finish the current command, filename, or or directory if possible, or show theControl-D possible completers
(works with c and tc shells)
Command Completion
tar [options] files collect or extract files or directories into or from an archive
options:c create a new archivex extract files from archivef store files in archivev print function letter and name of files– store to standard output
example:type: tar xvf file.tar extract the files and or directories from the
archive, file.tar
Tarring
compress file compress file and rename it file.Z(the program automatically renames the file)
uncompress file.Z uncompress file.Z and rename it file
gzip file compress file and rename it file.gz(the program automatically renames the file)
gunzip file.gz uncompress file.gz and rename it file
Compression
Process Control
Control-C cancel a foreground job
Control-Z stop (interrupt) a foreground job
jobs list of background jobs
bg run stopped job in the background
fg run stopped job in the foreground
& appended to the end of a command will place that job in the background
example:type: blastp swissprot test.seq > test.out & run WU-BLAST job in the
background
ps [options] display the status of the current processes and the process id-number
options:-a all processes-u display processes owned by a particular user
jobs shows any jobs that are currently running in the background or suspended
Process Control
kill –9 id-number terminate a process owned by you; id-number (process id) can be found with the ps command
or
kill %job-number terminate a process owned by you; job-number can be found with the jobscommand
Process Control
du -sk file display total disk usage of file or directory in kilobytes
df display the current disk usage for all connected file systems
stat file view attributes of a file or directory
quota -v list your disk quota
Other Commands
cat [options] files dumps the contents of files to the screen
more [options] files a pager, that lets you view one page at a time
less file similar to more, but better
Viewers
Graphical:
nedit filename if not available, ask to have it installed
xemacs filename (jot no longer works on SGI)
Command-line:
pico filename full-screen text editor available on most UNIX systems
vi filename full-screen text editor available on virtually every UNIX system
Editors
The Fun Stuff– redirect
> command output redirection (create new, overwrites existing file)
>> command output redirection (append)
< command input redirection (from file)
examples:
how to: combine individual sequence files into one file
type: cat seq1 seq2 > seq
how to: email a file to a collaborator
type: mail -s “results” [email protected] < new_blast_results
The Fun Stuff– piping
command1 | command2 | … directs standard output of one command into standard input for the next command
examples:
type: ls -al | more look at the ls output one page at a time
type: qstat -a | grep user1 check the queue on the origin for any processes being run by user1
grep [options] string file find and print out lines containing the string in a file or multiple files
options:
-i ignore case
examples:
how to: extract sequence information from protein structure files
type: grep SEQRES pdbfile | cut -c20-70 > seqs
The Fun Stuff– pattern searching
.cshrc shell startup file
Some things to add to your .cshrc file:
alias alias-string com-string abbreviate a command string with an alias string
examples:
type: alias la ls -a create a shortcut for ls -a
type: alias q qstat -u pfast check if you have any jobs running in the queue (origin)
module load(unload) software load or unload the environment settings for a particular software
source /usr/local/gcg/gcgstartup runs the gcgstartup script to set the necessary env. variables
The Fun Stuff– .cshrc file
UNIX in a Nutshell by O’Reilly
Developing Bioinformatics Computer Skills by O’Reilly
http://www.iats.missouri.edu/servlets/knowledgebase/article/422
http://www.geek-girl.com/unix.html
http://riceinfo.rice.edu/Computer/Documents/Unix/unix1.01.pdf
Resources