------------------------- Week 06 Notes for NET2003 ------------------------- -Ian! D. Allen - idallen@idallen.ca - www.idallen.com Remember - knowing how to find out an answer is more important than memorizing the answer. Learn to fish! RTFM! (Read The Fine Manual) ---------------------------------------------------------------------------- Review: - you can compare binary and text files - you can "data mine" for lines and fields in text files - you can quote command lines to hide them from the shell - you can use the "argv" program to debug command line argument parsing - you understand basic shell variables, both local and exported (environment) ---------------------------------------------------------------------------- * Abbreviation "ALN" = your "Advanced Linux Networking" text by R.W.Smith * Subscribe to a network security mailing list, e.g. BUGTRAQ and/or SANS ---------------------------------------------------------------------------- Command substitutions: treating commands as variables ----------------------------------------------------- see Notes: command_substitution.txt - Command Substitution - $(unix command) See /etc/init.d/apache2 for a sample network start-up that uses both back-quotes ("`") and $(cmd) syntax in the same file: PIDFILE=`grep -i ^PidFile $i | tail -n 1 | awk '{print $2}'` CNT=$(expr $CNT + 1) Q: How do you execute a command and save its output in a variable? ---------------------------------------------------------------------------- Where does the shell look for command names? $PATH --------------------------------------------------- see Notes: search_path.txt - Shell search PATH New commands: which - which $PATH entry contains this executable file name? whereis - where is this executable file in the system (including the man page) "which" looks in $PATH; "whereis" does not. Q: How do I change the directories where the shell looks for commands? Q: Where does the shell look for the executable "date" program? Q: What command shows which program will execute when I type "date"? ---------------------------------------------------------------------------- Shell order of expansion - what happens first? ------------------------ see Notes: order_of_processing.txt - Order of Shell Command Line processing CRITICAL POINT: Double-quote all your $variable and $(command) expansions! see Notes: data_mining.txt - Using commands and pipes to "mine" and extract ---------------------------------------------------------------------------- Process listing and control --------------------------- The "ps" command on Linux is a mix of the BSD "ps" command and the incompatible SystemV UNIX "ps" command. From "man ps": This version of ps accepts several kinds of options: 1 UNIX options, which may be grouped and must be preceded by a dash. 2 BSD options, which may be grouped and must not be used with a dash. 3 GNU long options, which are preceded by two dashes. - ps # BSD: some of your processes - ps x # BSD: all of your processes - ps xl # BSD: all your processes, long format - ps xlww # BSD: all your processes, long format, full wide listing - ps ax # BSD: all processes - ps -e # UNIX: all processes - ps -elww # UNIX: all processes, long format, full wide - ps f # BSD: ascii art hierarchical display (forest) - pstree - kill - killall - jobs (interactive shells only) Q: how do you see all your processes? Q: how do you see all processes for all users? ---------------------------------------------------------------------------- Symbolic Links -------------- Symbolic Links - similar to Windows/Mac aliases, but they work even for directories - can symlink to directories (cannot hard link to directories) - can point to nonexistent files or directories - a "dangling symlink" A leading "l" in ls output: $ ls -l /bin/*sh -rwxr-xr-x 1 root root 664084 Apr 21 2006 /bin/bash lrwxrwxrwx 1 root root 21 Aug 22 22:30 /bin/csh -> /etc/alternatives/csh lrwxrwxrwx 1 root root 4 Aug 22 18:42 /bin/rbash -> bash lrwxrwxrwx 1 root root 4 Aug 22 18:42 /bin/sh -> bash -rwxr-xr-x 1 root root 303344 Dec 1 2005 /bin/tcsh $ ln -s nosuchfile bar $ ls -l bar lrwxrwxrwx 1 idallen idallen 10 Feb 9 10:42 bar -> nosuchfile $ cat bar cat: bar: No such file or directory The system start-up script directories /etc/rc?.d/ contain symlinks. Q: How do you create a symlink in /tmp/foo that contains the path "/bin"? ---------------------------------------------------------------------------- Shell Control Structures ------------------------ see Notes: exit_status.txt - Return Code, Exit Status, test, if, and while Shells find and run commands; their control structures do the same! - if statement sometimes mistakenly called "if loop" - while command ; do ... ; done - while test 2 -eq 2 ; do echo same ; done - use "let" to do arithmetic (no $ needed in "let") e.g. let x=x+1 let y="x*(3+y)" # protect GLOB chars and () - loop to count up to 10 x=0 while [ $x -lt 10 ] ; do let x=x+1 ; echo "x = $x" ; done Testing file attributes: - test -e pathname # see if pathname exists - test -s pathname # see if pathname exists and is non-empty - test -r pathname # see if pathname exists and is readable all permission tests: -r -w -x - test -d pathname # see if pathname is a directory other tests: -d -f -L (-h) Testing strings - test "$foo" = "bar" - test -z "$foo" # test to see if $foo is an empty string - test -n "$foo" # test to see if $foo is NOT an empty string Don't confuse test -n "$foo" with test -s "$foo" - one tests the string $foo the other tests the size of the file named by $foo. Scripts that violate Structured Programming (one exit per program). - see Notes file deep_nesting.txt * Useful tests for IF statements: test -z -n -r -w -x -s -e -L -f -d Combining tests using -a or -o: test -r foo -a -w foo Q: How do you test to see if a string is empty/not-empty? Q: How do you test to see if a pathname is readable/writable/executable? Q: How do you test to see if a file has a size larger than zero? Q: How do you test to see if a pathname is a symlink? Q: Know the meanings of: test -z -n -r -w -x -s -e -L -f -d Q: Know how to combine tests using -a (and) and -o (or) ---------------------------------------------------------------------------- Software distribution - The tar archive: ---------------------------------------- A "tar" archive file is the Unix version of a "zip" file. Software is often distributed in tar archives. A "tar" archive (sometimes called a "tarball") contains multiple uncompressed files and directories. Tarballs may be compressed *as a whole* using gzip or options to the "tar" command: $ tar cvf tarball.tar *.c # create tarball.tar verbosely $ gzip tarball.tar # compress into tarball.tar.gz $ tar cvzf tarball.tar.gz *.c # do both of the above in one step $ tar tvf tarball.tar # verbose table of contents $ tar tzvf tarball.tar.gz # verbose table of contents $ tar xvf tarball.tar # extract contents $ tar xzvf tarball.tar.gz # extract contents Tarballs will archive entire directories if you give them directories: $ cd # go to my home directory $ tar czf /tmp/homedir.tar.gz . # archive everything into a file $ cd /some/backupdir $ tar xzf /tmp/homedir.tar.gz # extract the whole archive The name of the tar archive can be anything; the suffixes are there simply for human readers to better know what the files contain. Q: Is a tarball an archive of individual compressed files, or a compressed archive of individual files?