===================================== Unix/Linux Shell Command Line Quoting ===================================== -IAN! idallen@freenet.carleton.ca References: http://www.washington.edu/computing/training/125/quote.html http://unix.about.com/library/glossary/bldef-quoting.htm http://www.grymoire.com/Unix/Quote.html Chapter 8 and Appendix C in "Linux Shells by Example" [Quigley] ====================================== Which programs treat quotes specially? ====================================== Quotes, both double and single, are treated as special characters when fed as input to the Unix shells, usually by typing them on shell command lines or putting them inside shell scripts. Quotes are not special characters when fed as input to most other programs such as "cat" or "head": $ echo "It's a nice day" It's a nice day. <= note how shell processes quoted string $ cat "It's a nice day" <= typed input "It's a nice day" <= unchanged program output on stdout ^D $ head -1 "It's a nice day" <= typed input "It's a nice day" <= unchanged program output on stdout $ The shell treats quotes specially when they are used on a command line or in a shell script. Most other programs do not treat quotes specially. =============== Why use quotes? =============== The shells use quotes to protect other special characters from expansion by the shell. Backslashes also protect special characters from the shell; but, sometimes they are far less convenient to use. For example, one could protect all the special characters in a line of text with backslashes: $ echo \*\*\*\ It\'s\ a\ \"nice\",\ inexpen\$ive\ day. *** It's a "nice", inexpen$ive day. $ echo This\ line\ is\ a\ single\ argument\ to\ the\ \"echo\"\ command. This line is a single argument to the "echo" command. Alternately, one could use quotes to do the same thing: $ echo "*** It's a "'"nice", inexpen$ive day.' *** It's a "nice", inexpen$ive day. $ echo 'This line is a single argument to the "echo" command.' This line is a single argument to the "echo" command. Most agree that the use of quotes is easier than using a lot of backslashes. The quotes and backslashes are used by the shells to locate a string of characters to protect. The quotes and backslashes themselves are not part of the string; they are removed once the string of characters is collected. This next echo command line has six string arguments: $ echo "a b" c 'd e' f "g h" "i j" a b c d e f g h i j ^^^^ ^ ^^^^ ^ ^^^^ ^^^^ 1 2 3 4 5 6 Blanks are special characters to the shell, unless they are hidden from the shell by quoting. The shell splits the command line into arguments on any un-quoted blanks. Blanks that are quoted are not seen by the shell; they become part of the argument string passed to the command being invoked. Unquoted blanks, special to the shell, simply separate arguments. You may type one blank to separate the arguments or a hundred blanks; it makes no difference to the shell or to the arguments: $ echo a b c a b c $ echo a b c a b c The three string arguments passed to the echo command in the above two cases are identical. There are three arguments, separated by blanks. ================ Quote Processing ================ Quotes are processed by the Unix shells from left-to-right. Let's examine how this works: $ echo 'one"two"three'four"five"six"seven'eight'nine"ten one"two"threefourfivesixseven'eight'nineten The first quote found by the shell, starting from the left, is a single quote. The shell collects all the characters up to the next single quote. All these characters are protected from further analysis by the shell. The words one, two, and three are all inside this first single-quoted string. The double quotes, inside single quotes, have no special meaning to the shell. They are simply part of the string. The single quotes themselves, used to delimit the string, are not part of the string. The word four is outside of any quotes. Right after four we start a double-quoted string that extends until the next double-quote character. The word five is inside this double-quoted string. The double quotes themselves, used to delimit the string, are not part of the string. The word six is outside of any quotes. Right after six we start a double-quoted string that extends until the next double-quote character. The words seven, eight, and nine are all inside this first double-quoted string. The single quotes, inside double quotes, have no special meaning to the shell. They are simply part of the string. The double quotes themselves, used to delimit the string, are not part of the string. The word ten is outside of any quotes. The given command line argument to "echo" was built up of text that contains three quoted strings. Each quoted string connected immediately to another non-blank character, and all the blanks are inside quoted strings were protected from the shell. Thus, the "echo" command is handed only *one* command line argument. (Command line arguments are separated by blanks. There were no unquoted blanks used.) The fact that this one argument was made up of several pieces on the command line, including both quoted and unquoted parts, is invisible to the echo command. The shell did the work; the echo command gets its one argument. The quotes seen by the shell are *not* part of the argument. The quotes delimit the string; they are not part of the string. ======================== Single vs. Double Quotes ======================== Single quotes are "stronger" than double quotes. Nothing is special inside single quotes; the shell treats all the characters, no matter what they are, as part of the string being collected and it does not expand any of them inside single quotes. Inside double quotes, the shell still sees and expands some special characters. The most important character expanded by the shell even inside double quotes is the dollar sign that signals the start of a shell variable: $ echo '$SHELL' $SHELL <= single quotes prevent expansion $ echo "$SHELL" /bin/bash <= double quotes permit expansion ======================================= Quotes inside variables are not special ======================================= Due to the order of processing of the command line, all quoted strings are located and identified by the shell before any variables are expanded. This means quotes embedded inside variables are not seen as special characters by the shell: $ x='aaaaa " bbbbb' $ echo $x aaaaa " bbbbb The shell looks for and identifies quoted strings (there are none) on the above echo command line before it expands the variable $x, so the double quote inside $x appears "too late" - it is not treated as a special character by the shell. Only "exposed" quotes are treated as special on shell command lines, not quotes embedded inside variables. Exposed quotes are special; embedded quotes are not: $ mkdir empty ; cd empty; touch a b c d $ echo " * " * $ x='"' $ echo $x * $x " a b c d " The quotes embedded inside the variable $x are not treated as special characters by the shell. Note: Blanks and glob (wildcard) characters embedded inside unquoted variables *are* seen as special to the shell and may cause multiple arguments to be created: $ x="a b c" $ touch $x <= creates three files; embedded blanks are special $ touch "$x" <= creates a single file named: a b c $ y='*' $ echo "$y" <= embedded glob char is protected by quoting * $ echo $y <= unprotected glob char matches four file names! a a b c b c You must double-quote all uses of variables to prevent the embedded blanks and glob (wildcard) characters from being seen as special by the shell. ===================================== Studies in Quoting Special Characters ===================================== Understand how the shell handles quotes and blanks: $ echo hi there hi there $ echo "hi there" hi there $ echo 'hi there' hi there Explain the above three outputs. How many arguments are passed to the "echo" command in each case? The "touch" command creates empty files by name. Try this: $ touch "a b" $ ls a b $ rm a b Explain the error message that is output by the above "rm" command. How many arguments are passed to the "touch" and "rm" commands? Here are some more things to try, and to understand. $ echo "'hello'" $ echo '"hello"' $ mkdir empty ; cd empty ; touch a b c d $ echo ' * ' $ echo '" * "' $ echo '"' * '"' $ echo '"'" * "'"' $ echo ' * ' * " * " $ echo \' * \' You must be able to predict the output of each of the above command lines without having to type them in to try them. How many arguments are there to the following echo command? $ echo abc \ def \\ ghi \ \ jkl How many characters are in each of the arguments?