Week 6 Notes for DAT2330 - Ian Allen Supplement to Text, outlining the material done in class and lab. Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM![*] ([*] Read The Fine Manual) Look under the "Notes" button on the course web page for these study notes for the Linux textbook: chapter10.txt (The BASH Shell [not all]) Complete these Floppix Labs on http://floppix.ccai.com/ Floppix Lab 26: BASH Scripts Floppix Lab 21: Process Management Note: An early version of "bash" is also available on ACADAIX. -------------------------------------- Order of Shell Command Line processing -------------------------------------- The order in which the shell applies various processes (word splitting, pathname expansion, variable substitution, etc.) to each command line is partially outlined on page 358, except that it is wrong. Items 2 through 6 (tilde, parameter, variable, command, and arithmetic) happen *at the same time*, not one after the other. You cannot have a command substitution inside a variable or parameter. The full details of how the shell does command line processing for all possible circumstances would take several pages of documentation, and you don't need to know it all unless you're going to do serious shell script programming. Here's just a taste of the complexity (not to be memorized), from the man page for bash: EXPANSION Expansion is performed on the command line after it has been split into words. There are seven kinds of expansion performed: brace expansion, tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, word splitting, and pathname expansion. The order of expansions is: brace expansion, tilde expan­ sion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion. On systems that can support it, there is an additional expansion available: process substitution. Only brace expansion, word splitting, and pathname expan­ sion can change the number of words of the expansion; other expansions expand a single word to a single word. The only exceptions to this are the expansions of "$@" and "${name[@]}" as explained above (see PARAMETERS). [...] Word Splitting The shell scans the results of parameter expansion, com­ mand substitution, and arithmetic expansion that did not occur within double quotes for word splitting. For the purposes of this course, you need to know the processing order for the following features we have studied: 1. Initial word splitting (blanks, semicolons, pipes, etc.) 2. Identification and removal of I/O redirection words 3. Tilde, parameter, and variable expansion of words 4. Further word splitting (not inside double quotes) 5. Pathname expansion (globbing, wildcards) Note: The text says parameter expansion happens *before* variable expansion; this is wrong. They happen at the same time. You can't have a parameter that expands to contain a variable and then have the variable also expand, or vice-versa. Note that pathname expansion happens last, so that glob (wildcard) characters hidden inside parameters and variables will be expanded (often unintentionally). Many novice shell programmers forget this. Also note that if a glob pattern matches a filename with a blank, the blank is not treated as a special character (it doesn't generate multiple file names) because word splitting happens *before* globbing. A globbed filename containing one or more blanks is still treated as a single argument by the shell, because the shell already did the word splitting. (The same is *not* true for blanks inside unquoted variables, which do cause multiple arguments, which is why you *must* put all your variables inside double-quoted strings!) After the shell is finished processing the command line, it identifies the first word on the command line as a command name, looks for it in the search path $PATH (only if the word doesn't contain a slash), and then runs that command and waits for it to finish. Exercise: Correct the Summary in Chapter 10, p.358 so that it says that items 2 through 6 (tilde, parameter, variable, command, and arithmetic) happen at the same time. Now, put the following actions in their proper order: - shell expands $-variables and ~-variables - shell expands pathname globs (wildcards, e.g. *) - shell looks for the command name in $PATH and runs it - shell identifies and removes redirection - shell splits the command line into words - shell word-splits unquoted variable expansions Use the above order information to answer the following questions, in this order (pay careful attention to the use of single and double quotes): $ mkdir empty $ cd empty $ ls -a (Q1) What output appears here? $ x="foo*" $ echo $x (Q2) What output prints here? $ echo "$x" (Q3) What output appears here? $ echo '$x' (Q4) What output appears here? $ touch foobar $ ls (Q5) What output appears here? $ echo $x (Q6) What output prints here? $ echo "$x" (Q7) What output appears here? $ echo '$x' (Q8) What output appears here? $ echo hi >$x $ ls (Q9) What output appears here? $ echo there >"$x" $ ls (Q10) What output appears here? $ echo mom >'$x' $ ls (Q11) What output appears here? $ rm $x $ ls (Q12) What output appears here? Explain: $ mkdir empty $ cd empty $ x='a b c d' $ set | grep x= x=a b c d $ touch $x $ ls | wc 4 4 8 # why are there four file names? What names? $ rm * $ touch "$x" $ ls | wc 1 4 8 # why is it only one file name now? What name? $ rm * $ touch '$x' $ ls | wc 1 1 3 # what is the name now? Review: Order of shell processing of command line. Explain: $ mkdir empty $ cd empty $ touch aa ab ac ad ae af $ x='a* b* c* d*' $ echo $x aa ab ac ad ae af b* c* d* $ echo "$x" a* b* c* d* $ echo '$x' $x Review: Order of shell processing of command line. Explain: $ x='date >out' $ echo $x date >out $ $x Invalid character in date/time specification. $ sort $x sort: can't open date sort: can't open >out $ ls $x date not found >out not found $ sort "$x" sort: can't open date >out $ ls "$x" date >out not found $ sort '$x' sort: can't open $x $ ls '$x' $x not found Review: Order of shell processing of command line. Explain: $ touch a b '>out' $ ls >out a b $ echo * >out a b $ echo >out a b $ ls >out a b out $ cat '>out' $ cat out a b Review: Order of shell processing of command line. What is the output of this command sequence: $ foo='bar haven' $ echo foo $foo "$foo" '$foo' bar NOTE: blanks will be counted in the answer Here is an alias using a pipe to convert lower-case to upper-case: If your boss is an experienced IBM programmer, she's used to reading text in all UPPER CASE. This alias is for her: $ alias who="who | tr 'a-z' 'A-Z'" $ who Are the single quotes required around the two arguments? You can use a similar command to convert a lower-case file of IBM MVS JCL into upper-case. EXPERIMENT: Why doesn't this convert the file "myfile" to upper-case? $ alias upper="tr 'a-z' 'A-Z'" $ upper myfile Why is the file "myfile" empty after this command is run? Using the above alias, the following command line doesn't work either: $ upper myfile >new Why does this generate an error message from "tr"? (RTFM)