1.1 Adding an if statement to check the return code

Here is the same script as above, rewritten with an added if control structure that checks the result (exit code) of the mkdir and only does the other two commands if the mkdir was successful (has exit status zero):

$ cat example.sh
if mkdir foo ; then
    cd foo
    date >date.txt
fi

The if control statement in the script file runs the mkdir command and checks its return code. If the return code is good (zero), the shell runs the two indented statements between the then and the closing fi that ends the if statement. If the exit status is bad (non-zero), the shell skips the two statements in the file and does not run them.

The if statement automatically checks the return code of the mkdir command without printing the code on your screen. The if statement doesn’t need to print the return status; it simply tests it and acts on it. (You can use echo $? to see a previous return code.)

If we run this new version of the script, the shell does not execute the two commands inside the if statement if the if statement detects that the mkdir fails (with a non-zero exit code):

$ sh -u example.sh
mkdir: cannot create directory ‘foo’: File exists

$ ls -l
-rw-r--r-- 1 idallen idallen  0 Nov 19 15:25 foo
-rw-r--r-- 1 idallen idallen 53 Nov 19 15:36 example.sh

The shell does not print the return status of the commands it executes. If you want to actually see the return status of a command, you need to have the shell echo the value of the $? variable right after running the command.

2.1 The simple if … then … fi branch statement (no else)

The simplest control structure is the if statement that checks the return status of a command (or a list of commands) and only runs another command (or list of commands) if the return status of the original command is successful (zero):

if testing_command_list ; then
    zero_command_list
fi

The testing_comamnd_list is executed, and the return status is tested, then:

• If the exit status is zero (the command succeeded), the commands in the zero_command_list are executed.
• If the exit status is not zero (the command failed), the zero_command_list is not executed and the shell continues with the rest of the shell script, if any.

• Either command list can be multiple commands separated by newlines, semicolons, or logical && or || operators. If the testing_command_list is multiple commands, only the return status of the last command is tested. Any list may include commands connected by pipelines.

  if false ; false ; false ; false ; true ; then
      echo "This is true"
  fi

After the shell keyword if comes a testing_command_list, not an arithmetic or logical expression as would be found in most programming languages. The shell executes commands; it does not do arithmetic.

Here are some examples of what these simple if statements might look like inside shell scripts:

#!/bin/sh -u
if mkdir foo ; then
    cd foo
    date >date.txt
fi

#!/bin/sh -u
if fgrep "$1" /etc/passwd ; then     # search for first script argument
   echo "I found $1 in file /etc/passwd"
fi

#!/bin/sh -u
if who | fgrep "idallen" ; then
   echo "idallen is online"
   echo "Hello Ian" | write idallen
fi

Things to notice about the syntax of this control structure:

• The structure starts with if and ends with fi on a line by itself. The fi is if spelled backwards.
• There is a semicolon separating the end of the testing_command_list from the keyword then.
• The commands of the zero_command_list are indented right several spaces (or one tab stop).
• The fi keyword is lined up directly under the if keyword.
• The testing_command_list may be a pipeline or a list of several commands. Only the exit status of the last command in a pipeline or list is checked by the shell.

Always write if statements using the above form in this course.

2.2 Negating/inverting a command exit status using exclamation mark !

Any command’s return status may be logically negated/inverted (turn success to failure or failure to success) by preceding the command with an exclamation mark:

$ cd
$ echo $?
0

$ ! cd
$ echo $?
1

$ rm nosuchfile
rm: cannot remove ‘nosuchfile’: No such file or directory
$ echo $?
1

$ ! rm nosuchfile
rm: cannot remove ‘nosuchfile’: No such file or directory
$ echo $?
0

Inverting the exit status of a command is useful in an if statement when you want to execute some commands when a command fails, not when it succeeds. The syntax is to precede the testing_command_list with an exclamation mark to invert the return status:

if ! testing_command_list ; then
    nonzero_command_list      # list executes on FAILURE, not SUCCESS
fi

For example, the same mkdir script used above could be rewritten as shown below, so that the indented commands in the if statement execute when the mkdir command fails, not when it succeeds:

if ! mkdir foo ; then                 # note use of ! to invert exit status
    echo "$0: mkdir foo failed"
    exit 1
fi
cd foo
date >date.txt

The leading exclamation mark ! above turns a failure exit status from mkdir into a success status (and vice-versa) for the if statement.

The leading ! means that when mkdir fails, the non-zero exit status is inverted to a zero exit status and the nonzero_command_list executes and the script prints a failure message and exits.

Conversely, the leading ! means that when mkdir succeeds, the zero exit status is inverted to a non-zero exit status and the nonzero_command_list is not executed. The script does not exit and it continues after the fi line with the cd and date commands.

$ fgrep "nosuchstring" /etc/passwd
$ echo $?
1                                     # exit status 1 means "string not found"

$ fgrep "nosuchstring" nosuchfile
fgrep: nosuchfile: No such file or directory
$ echo $?
2                                     # exit status 2 means "error in pathname"

$ ! fgrep "nosuchstring" /etc/passwd
$ echo $?
0                                     # exit status 1 inverts to zero

$ ! fgrep "nosuchstring" nosuchfile
fgrep: nosuchfile: No such file or directory
$ echo $?
0                                     # exit status 2 also inverts to zero

if ! fgrep "nosuchstring" nosuchfile ; then
    echo "fgrep failed with exit status $?"  # WRONG: ALWAYS ZERO EXIT STATUS!
fi

2.3 The two-part if … then … else … fi branch statement

We can include an else clause inside the if statement that will contain commands to run if the testing_command_list fails (has a non-zero exit status). This gives the shell a choice of which commands to run:

if testing_command_list ; then
    zero_command_list
else
    nonzero_command_list
fi

The testing_comamnd_list is executed, and the return status is tested, then:

• If the exit status is zero (the command succeeded), only the commands in the zero_command_list are executed. The shell skips over (does not run) the commands in the nonzero_command_list (after the else keyword).
• If the exit status is not zero (the command failed), only the commands in the nonzero_command_list are executed. The shell skips over (does not run) the commands in the zero_command_list (before the else keyword).
• Any of the command lists can be multiple commands separated by newlines, semicolons, or logical && or || operators. If the testing_command_list is multiple commands, only the return status of the last command is tested. Any list may include commands connected by pipelines.

After the shell keyword if comes a testing_command_list, not an arithmetic or logical expression as would be found in most programming languages. If the exit status of the command list is zero (success), the zero branch of the if is taken, otherwise a non-zero exit status causes the non-zero branch to be taken. There are always exactly two branches: zero, and non-zero, and only the commands in one of the branches will execute, never both.

Here are some examples of what these if-else statements might look like inside shell scripts:

#!/bin/sh -u
if mkdir foo ; then
    cd foo
    date >date.txt
else
    echo 1>&2 "$0: Cannot create the foo directory; nothing done."
    exit 1         # exit the script non-zero (failure)
fi

#!/bin/sh -u
if fgrep "$1" /etc/passwd ; then    # search for first script argument
    state='found'                   # set variable indicating success
else
    state='did not find'            # set variable indicating failure
fi
echo "I $state the text '$1' in the file /etc/passwd"   # use the variable

Things to notice about the syntax of this control structure:

• The structure starts with if and ends with fi on a line by itself. The fi is if spelled backwards.
• There is a semicolon separating the end of the testing_command_list from the keyword then.
• The else keyword is on a line by itself. It separates the zero_command_list from the nonzero_command_list.
• The fi keyword is lined up directly under the else keyword that is lined up directly under the if keyword.
• The commands of the zero_command_list and the nonzero_command_list are all indented right several spaces (or one tab stop) with respect to the if, else, and fi keywords.

Always write if-else statements using the above form in this course.

2.4 Using if … else instead of ! to preserve the exit status

If you want to know the exact non-zero exit status of a command, you can’t use ! in front of the command to negate/invert the status to zero:

if ! fgrep "nosuchstring" nosuchfile ; then
    echo "fgrep failed with exit status $?"       # WRONG: ALWAYS ZERO EXIT STATUS!
fi

You can use an if-else syntax to preserve the exit code for use in $?:

if fgrep "nosuchstring" nosuchfile ; then
    :                                             # do nothing on success
else
    echo "fgrep failed with exit status $?"       # show correct $? exit status
    exit 1
fi

The shell built-in command : (also called true) does nothing.

2.5 Nested if statements: if statements inside other if statements

Often a shell script needs to test the return code of a command used in one of the branches of an if statement. For example:

#!/bin/sh -u
# First try: search for first argument in passwd file
# Second try : search for first argument in group file
#
if fgrep "$1" /etc/passwd ; then
    echo "I found '$1' in file /etc/passwd"
else
    if fgrep "$1" /etc/group ; then
        echo "I found '$1' in file /etc/group"
    else
        echo "I did not find '$1' in /etc/passwd or /etc/group"
        echo "Please try again"
    fi
fi

A nested if statement is an if or if-else statement that is contained inside one of the two branches of an outer if-else statement, as shown the example above. We inserted another complete if-else statement in the nonzero_command_list branch of the outer if statement. The second if-else statement searches for the first command line argument in the /etc/group file, but it only does the search if the argument was not found in the /etc/passwd file.

We can nest another if/else statement into the script:

#!/bin/sh -u
# First try: search for first argument in passwd file
# Second try : search for first argument in group file
# Third try : search for first argument in networks file
#
if fgrep "$1" /etc/passwd ; then
    echo "I found '$1' in file /etc/passwd"
else
    if fgrep "$1" /etc/group ; then
        echo "I found '$1' in file /etc/group"
    else
        if fgrep "$1" /etc/networks ; then
            echo "I found '$1' in file /etc/networks"
        else
            echo "I did not find '$1' in passwd, group, or networks"
            echo "Please try again"
        fi
    fi
fi

You can nest control statements as much as you like, though deeply nested structures can be hard to read and understand. Keep things simple!

4.1 Simple example: testing if a pathname is a file with -f

Here is the test helper program being used to test to see if pathnames /bin/bash and then /bin/nosuchfile are existing files:

$ test -f "/bin/bash"
$ echo $?
0                                     # zero means success (is a file)

$ test -f "/bin/nosuchfile"
$ echo $?
1                                     # non-zero means failure (is not a file)

The test command normally has no output, unless something goes wrong. Because test only sets an exit status, it doesn’t print anything on your screen. It only sets its return code, based on the tests you ask it to do. You can use echo to make the invisible exit status of test visible at the command line, by displaying the command exit status left in shell’s $? variable, as in the examples above.

You can use the test command in an if conditional control structure and check its return status just as you would use any other command:

$ cat example.sh
#!/bin/sh -u
if test -f "$1" ; then
    echo "File '$1' is a file"
    ls -l "$1"
else
    echo "'$1' is not a file"
fi

Running the above script:

$ ./example.sh /bin/bash
File '/bin/bash' is a file
-rwxr-xr-x 1 root root 1037464 Aug 31  2015 /bin/bash

$ ./example.sh /bin/nosuchfile
'/bin/nosuchfile' is not a file

In the example above, the test program silently tests to see if the pathname argument given as the first script argument is an existing file. If it is, test returns a successful (zero) exit status and the success half of the if statement is executed; otherwise, test returns failure (non-zero) and the failure half of the if statement is executed. The test program itself has no output; it only sets a return code.

Critical section: There is a small period of time between the testing of the existence of the filename and the use of the filename, and someone could, in theory, remove the file in between the testing and the using, causing ls to give an error. This is unlikely to happen, but it’s not impossible. You must be wary of these possible faults in your programming logic.

4.2 Some versions of the test command are built-in to the shell

The test program has a large number of tests and operations it can do that are essential in control statements in shell scripts. Because it is used so often and is so important, many shells (including the bash, dash, and Ubuntu sh shells) have a built-in version of test that is documented in the manual page for the shell.

The shell built-in test may be slightly different from the external one documented in the test manual page. See the manual page for your sh shell for the most accurate documentation. The test command used by sh scripts under modern versions of Ubuntu Linux is the one in the dash shell manual page: see man sh

See the test documentation for the full list of things this command can do. We will concentrate on a few key tests.

4.3 Three main categories of tests: pathname, string, numeric

The test helper command has three main categories of things it can test:

1. test pathname properties, e.g. test -f
2. test and compare strings, e.g. test -z
3. compare integers, e.g. test 4 -lt 9

Some key examples from each category follow.

4.4 Using test to test pathname properties, e.g. test -f

You can test most any property or attribute pertaining to an object in the file system. These tests are commonly used:

test -e "pathname"        # true if pathname exists (any kind of path)
test -f "pathname"        # true if pathname is a file
test -d "pathname"        # true if pathname is a directory
test -r "pathname"        # true if pathname is readable
test -w "pathname"        # true if pathname is writable
test -x "pathname"        # true if pathname is executable
test -s "pathname"        # true if pathname has size larger than zero

All the pathname tests also fail if the pathname does not exist or is not accessible (because some directory prevents access to the pathname).

Here is a script example that tests if its first argument is accessible and is a file:

#!/bin/sh -u
if test -f "$1" ; then
    echo "Pathname '$1' is an accessible file"
else
    echo "Pathname '$1' is inaccessible, missing, nor not a file"
fi

• Note that if a if a test pathname operator fails, it may also fail because you have no permission to search one of the directories in the pathname, or because the pathname simply doesn’t exist.
• Note that the opposite of “is a file” is not “is a directory”, since there are more things in the file system than just files and directories. (Check out the type of the /dev/null pathname that is neither a file nor a directory.)

See the manual page for other less common types of pathname tests.

4.5 Using test to compare text strings, e.g. test -z

Since the shell if keyword must be followed by a command name, to compare strings we must execute the test helper command to do the string comparison for us. The test helper command will do the comparison of the strings given on its command line and set its return status depending on the result of the comparison:

test -z "$1"         # true if length of argument $1 is zero (empty string)
test -n "$1"         # true if length of argument $1 is not zero
test "$1" = "foo"    # true if argument $1 and foo are the same strings
test "$1" != "foo"   # true if argument $1 and foo are not the same strings

Here is an example that tests if the first script argument is the text string --help:

#!/bin/sh -u
if test "$1" = '--help' ; then
    echo "Usage: $0 [pathname]"
    exit 3
fi

4.6 Warning: a single argument to test is a non-empty string test

WARNING: Any time the test program is given exactly one argument, it assumes the argument is preceded by -n and it tests the single argument for a non-empty string, so these are equivalent:

test "$1"               # true if length of argument $1 is not zero
test -n "$1"            # true if length of argument $1 is not zero

Because it is not an error to forget to use -n, this single-argument default to the -n string test is the cause of many shell programming mistakes. In the example below, the command does not do what it looks like it does at first glance; it does not test equality between two strings:

test "abc"="def"        # success because "abc=def" is one non-empty string argument!

If you use the above test expression in a script, it is a one-argument test -n command, not a three-argument string comparison! The resulting exit status will always be zero (success) because the single argument abc=def is not an empty string!

The correct way to test string equality is to add blanks around the equals sign to make sure the test command sees three separate arguments:

test "abc" = "def"      # correct way to test if string1 is not equal to string2

Make sure you surround all test operators with blanks on both sides so that the test command sees separate arguments, or else everything will look like a one-argument string test and will always succeed:

test "abc"="def"        # WRONG! missing blank - always succeeds (exit zero)
test -f/dev/null        # WRONG! missing blank - always succeeds (exit zero)
test -s/dev/null        # WRONG! missing blank - always succeeds (exit zero)
test -z"abc"            # WRONG! missing blank - always succeeds (exit zero)

4.7 Using test to compare integer numbers, e.g. -eq

Since the shell if keyword must be followed by a command name, to compare numbers we must execute the test helper command to do the numeric comparison for us. The test helper command will do the numeric comparison of numbers given on its command line and set its return status depending on the result of the comparison.

The test helper program can compare integers using one of six comparison operators. There are only six possible combinations for comparing two integer numbers n1 and n2:

test n1 -eq n2          # true if n1 and n2 are the same numeric value
test n1 -ne n2          # true if n1 and n2 are not the same numeric value
test n1 -lt n2          # true if n1 is less than n2
test n1 -le n2          # true if n1 is less than or equal to n2
test n1 -gt n2          # true if n1 is greater than n2
test n1 -ge n2          # true if n1 is greater than or equal to n2

Example use of a numeric test on the number of command line arguments:

if test $# -ne 2 ; then
    echo 1>&2 "$0: Expecting two arguments; found $# ($*)"
    exit 1
fi

These six numeric test operators only work on integers, not empty strings, letters, or other non-digits. If you try to compare a non-integer with one of these six operators, test issues an error message. The error message is slightly different depending on which shell is reading your shell script:

sh$ test 1 -eq a
sh: 1: test: Illegal number: a

bash$ test 1 -eq a
bash: test: a: integer expression expected

bash$ test 1 -eq ""
bash: test: : integer expression expected

Boolean Logic Warning: Note that the opposite of the condition “less than” is “greater than or equal to”, not “greater than”. Don’t make this mistake!

4.8 Comparing strings is not the same as comparing integers

Comparing numbers and comparing strings do not always give the same results:

test 0 = 00                           # fails because s1 not equal to s2
test 0 -eq 00                         # succeeds because zero equals zero

test 0 = " 0 "                        # fails because s1 not equal to s2
test 0 -eq " 0 "                      # succeeds because zero equals zero

You will often see people use a string equality test on two numbers in shell scripts, where the correct test should be the numeric equality test:

if test $# = 0 ; then ...             # should use -eq for numbers
if test $# -eq 0 ; then ...           # always use use -eq for numbers

In most cases, the results are the same, but be careful of cases where the strings may differ but the numbers may be equal. To be safe, always use one of the numeric comparison operators for integers.

4.9 Combining test expressions using logical AND -a and OR -o

You can test more than one thing in a test helper expression by separating each test expression using -o for logical OR and -a for logical AND:

test -f "path" -a -s "path"          # true if path is a file AND path is not empty
test -d "path" -o -f "path"          # true if path is a directory OR path is a file

Remember that each test expression on either side of the AND or the OR must be a complete and valid test expression:

test "$1" = 'dog' -o "$1" = 'cat'    # correct use of two expressions separated by -o
test "$1" = 'dog' -o 'cat'           # WRONG - second expression is always true

The last line, above is the OR of these two test expressions:

test "$1" = 'dog'
test 'cat'

The last expression – a single-argument test expression – is always true because the argument is not an empty string, so the logical OR of the above two expressions is also always true, which is probably not what you want.

# Example A - note the quoting of the parentheses
test -s "path" -a \( -d "path" -o -f "path" \)

# Example B - same - logical -a binds before -o
test -s "path" -a -d "path" -o -f "path"
test \( -s "path" -a -d "path" \) -o -f "path"    # same as above

4.10 Negating/Inverting test expressions using exclamation mark !

You can negate/invert the exit status of any test expression by inserting an exclamation point at the start of the test expression, e.g.

test ! -e "pathname"        # true if pathname does *not* exist
test ! -w "pathname"        # true if pathname does *not* exist or is *not* writable
test ! -s "pathname"        # true if pathname does *not* exist or has size zero

The ! negation operator applies only to the single closest expression, not to the whole expression:

test ! -f path -o -d path           # if path is not a file OR if path is a directory

In programming terms, the negation ! operator has highest precedence over all the other Boolean operators. If you want to negate an entire expression, you have to put the expression in parentheses:

test ! \( -f path -o -d path \)     # if path is not a file or directory

You can always use parentheses to make sure your logic binds the way you intend, instead of relying on default precedence.

4.11 Two ways of negating/Inverting test expressions

Negating a test expression using the exclamation mark ! may sometimes have the same effect as using an exclamation mark at the start of the whole testing_command_list to negate the whole test command (as shown above), but negating a single test expression applies just to that expression in the test command, not to the exit status of the whole test command.

These next two ways of negating (inverting) the exit status of a single test expression are the same. The first negation is done by the test command on the single expression; the second negation is done by the shell on the exit status of the test command:

if test ! -e "path" ; then ...      # true if pathname does *not* exist
if ! test -e "path" ; then ...      # same as above; done by shell

In shell programming, we prefer the first syntax, with the negation happening inside the test expression. We rarely negate the entire test command (as in the second example, above). Use the first syntax.

if test ! -e "path" -o -d "path" ; then ...    # only one expression is negated
if ! test -e "path" -o -d "path" ; then ...    # WRONG!  NOT THE SAME !

if ! test -e "path" -o -d "path" ; then ...
if test ! -e "path" -a ! -d "path" ; then ...

if not test (A or B) ; then
if test (not A and not B) ; then                   # deMorgan !(A or B) --> (!A and !B)

if test ! -e "path" -o -d "path" ; then ...    # only one expression is negated
if ! test -e "path" -a ! -d "path" ; then ...  # deMorgan

if test ! -e "path" ; then ...      # use this syntax (preferred)
if ! test -e "path" ; then ...      # DO NOT USE THIS (bad form)

if [ ! -e "path" ] ; then ...       # use this syntax (preferred)
if ! [ -e "path" ] ; then ...       # DO NOT USE THIS (bad form)

4.12 Using test with variables

Scripts often use the test helper command to test the contents of variables, so one or both of the arguments is often a double-quoted variable to be expanded inside the script:

if test "$1" = '/' ; then echo "Using ROOT directory" ; fi
if test "$1" = "$2" ; then echo "Two arguments are identical" ; fi
if test -r "$1" ; then echo "Argument $1 is a readable pathname" ; fi

Here is an example script that tests the number of positional parameters (arguments) to the script via the $# variable:

#!/bin/sh -u
if test "$#" -eq 0 ; then
    echo "$0: The script has no arguments"
fi
if test "$#" -ge 1 ; then
    echo "$0: The first argument of $# is '$1'"
fi
if test "$#" -ge 2 ; then
    echo "$0: The second argument of $# is '$2'"
fi
if test "$#" -ge 3 ; then
    echo "$0: The script has three or more arguments: $#"
fi

Running the above script:

$ ./example.sh one two three four
./example.sh: The first argument of 4 is 'one'
./example.sh: The second argument of 4 is 'two'
./example.sh: The script has three or more arguments: 4

7.1 The universal expr arithmetic helper program

The oldest and most universal arithmetic helper is the expr external command, and it is often used in older scripts inside the historic back-quote form of Command Substitution to capture its output value:

$ expr 2 + 2 \* 9
20

$ x=`expr 2 + 2 \* 9`                 # historic `...` command substitution syntax
$ echo "$x"
20

The individual arguments to the expr command constitute the arithmetic expression to be evaluated and the value is printed on standard output. Shell meta-characters such as * (GLOB) must be quoted to hide them from the shell. All numbers and operators must be individual arguments, separated by blanks, otherwise expr simply echoes the unrecognized expression back to the user without any error message and without doing any mathematics:

$ expr 2 + 2
4

$ expr 2+2
2+2

The expr command is the original shell arithmetic helper command. It is standard and works in all Unix/Linux shells ever written.

Because expr is an external command in many shells, not a shell built-in, it is slow. Using it in a loop that has to iterate hundreds or thousands of times will not be fast.

You can read more in the manual page expr(1).

7.2 The occasional let built-in command

Some later Bourne shells invented and used the let built-in helper command to do arithmetic:

$ let x=4 y=5 z=x+y
$ echo "$x $y $z"
4 5 9

• Each argument is an arithmetic expression to evaluate
• Shell variables are allowed with or without leading $
• Arithmetic expressions can involve assignment of values to variables, where = is the basic assignment operator
• See the ARITHMETIC EVALUATION section of the bash man page

The let helper command is not universally available in all Bourne shells. (In particular, the dash shell used as /bin/sh under Ubuntu does not support it!) Don’t use it in new scripts.

7.3 The modern $((...)) syntax for Arithmetic Expansion

The modern way to do arithmetic that we will use in this course does away with all helper programs. Arithmetic expressions are done directly by the shell using a special dollar-and-double-parenthesis Arithmetic Expansion syntax: $((expression))

The enclosed expression is designed for arithmetic and is not GLOB expanded. Arguments do not need surrounding spaces or special quoting and variables do not need leading $:

$ echo $(( 2 + 2 * 9 ))
20

$ echo $((2+2*9))
20

$ x=2 y=3 ; echo $((x*y))
6

• The $((expression))is an Arithmetic Expansion, like a variable expansion, and it can be used anywhere that you might use a variable.
• Just as with variable substitution, the $((expression))is is replaced by the results of the evaluated Arithmetic Expression

Example (remember blanks are optional inside the expression):

#!/bin/sh -u
if [ $# -gt 3 ] ; then
    echo 1>&2 "$0: Expecting max 3 arguments, not $# ($*)"
    echo 1>&2 "$0: Remove $(( $# - 3 )) arguments from the command line."
fi

Running the above script:

$ ./example.sh k k k k k k k k
./example.sh: Expecting max 3 arguments, not 8 (k k k k k k k k)
./example.sh: Remove 5 arguments from the command line.

7.4 Legacy $[...] syntax for Arithmetic Expansion

Some older versions of the Bourne shells use the old syntax $[expression] for Arithmetic Expansion, but this is now obsolete and deprecated:

$ echo $[2+2*9]                     # deprecated; do not use this syntax
20

10.1 Using | for multiple GLOB patterns in case statements

We can condense the script even more by using or | characters to put multiple GLOB patterns on the same line:

#!/bin/sh -u
case "$1" in
'dog' | 'cat' | 'goat' | 'pig' )         kind='animal'   ;;
'apple' | 'peach' | 'plum' | 'cherry' )  kind='fruit'    ;;
* )                                      kind='unknown'  ;;     # the "default" match
esac
echo "We classify '$1' as '$kind'."

This program is now about one-third the size of the equivalent program that used nested if statements to do the same thing.

10.2 Example: Classify a pathname

Below is an example using GLOB patterns to identify the first command-line argument in the positional parameter variable $1:

#!/bin/sh -u
case "$1" in
'' )    type='missing (empty)' ;;
/* )    type='an Absolute Pathname' ;;
*/ )    type='a Relative Pathname ending in a slash' ;;
*/* )   type='a Relative Pathname in some directory' ;;
*' '* ) type='a Relative Pathname with blank(s)' ;;
* )     type='a Relative Pathname in the current directory' ;;  # the "default" match
esac
echo "Pathname '$1' is $type"

• Most cases are GLOB patterns, so the GLOB characters must not be hidden from the shell by quotes. Do not use quotes if you want GLOB patterns to work.
• As with file name GLOB patterns, if you want a case statement GLOB pattern to match some text anywhere in the test-string, you need to put GLOB * characters at either end of the text: */*
• A trailing * in the pattern makes the text match only at the beginning of the test-string: /*
• A leading * makes the text match only at the end: */
• A GLOB pattern must be a single shell word: If you want to match spaces or special characters in the test-string, quote them all in the GLOB pattern to hide them from the shell: *' '*

Running the above script:

$ ./example.sh ''
Pathname '' is missing (empty)

$ ./example.sh /etc/passwd
Pathname '/etc/passwd' is an Absolute Pathname

$ ./example.sh a/b/c/
Pathname 'a/b/c/' is a Relative Pathname ending in a slash

$ ./example.sh foo/bar
Pathname 'foo/bar' is a Relative Pathname in some directory

$ ./example.sh "foo bar"
Pathname 'foo bar' is a Relative Pathname with blank(s)

$ ./example.sh foobar
Pathname 'foobar' is a Relative Pathname in the current directory

10.3 Example: Find the star/asterisk/*

As when matching file names on a shell command line, GLOB pattern characters in case statements must be unquoted to behave as GLOB characters. Quoting hides the GLOB characters from the shell and makes them into ordinary characters that must match the test-string exactly:

#!/bin/sh -u
case "$1" in
'*' )   msg='a single asterisk (star)' ;;
'*'* )  msg='an asterisk at the beginning of the argument' ;;
*'*' )  msg='an asterisk at the end of the argument' ;;
*'*'* ) msg='an asterisk in the middle of the argument' ;;
* )     msg='an argument with no asterisk' ;;     # the "default" match
esac
echo "You entered $msg: $1"

Running the above script:

$ ./example.sh '*'
You entered a single asterisk (star): *

$ ./example.sh '*foobar'
You entered an asterisk at the beginning of the argument: *foobar

$ ./example.sh 'foobar*'
You entered an asterisk at the end of the argument: foobar*

$ ./example.sh 'foo*bar'
You entered an asterisk in the middle of the argument: foo*bar

$ ./example.sh 'no star here'
You entered an argument with no asterisk: no star here

10.4 Example: Matching numbers by counting digits

Below is another example using more complex GLOB patterns to match the number of digits inside a number, allowing a crude method for checking number ranges:

#!/bin/sh -u
case "$1" in
0 )                     price='free' ;;           # 0
[1-9] )                 price='cheap' ;;          # 1...9
[1-4][0-9] )            price='middle' ;;         # 10...49
[5-9][0-9] )            price='upper' ;;          # 50...99
[1-9][0-9][0-9] )       price='high' ;;           # 100...999
[1-9][0-9][0-9][0-9] )  price='exorbitant' ;;     # 1000...9999
* )                     price='impossible' ;;     # the "default" match
esac
echo "We classify '$1' as '$price'."

Running the above script:

$ ./example.sh 37
We classify '37' as 'middle'.

$ ./example.sh 2348
We classify '2348' as 'exorbitant'.

$ ./example.sh crap
We classify 'crap' as 'impossible'.

Using GLOB patterns to test numeric ranges has its limitations, and doesn’t always work as nicely as one would like (since we are comparing digits and not evaluating and comparing numeric values)

$ ./example.sh 037                         # 037 is in range 10...49
We classify '037' as 'impossible'.

$ ./example.sh 3.14                        # 3.14 is in range 1...9
We classify '3.14' as 'impossible'.

Shells find and run commands; they don’t do mathematics very well.

10.5 Example: Validating an argument using a complemented character class

Since GLOB patterns can also match characters that are not in a range by adding ! inside a character class, we can use GLOB patterns to detect input that is, for example, not alphabetic:

#!/bin/sh -u
case "$1" in
'' ) echo "Empty string" ;;
*[![:alpha:]]* ) echo "Non-alphabetic '$1'" ;;
* ) echo "Alphabetic: '$1'" ;;
esac

The above script has a case GLOB pattern that matches a non-alphabetic character anywhere in the string, by using the complement (inverse) of a POSIX character class named [:alpha:] that represents alphabetic characters. If that pattern matches in the argument, we know the argument has a non-alphabetic character in it somewhere:

$ ./example.sh happy
Alphabetic: happy

$ ./example.sh abc0def
Non-alphabetic: abc0def

You can use case statements and appropriate complemented POSIX character classes to make sure that arguments contain only the types of characters you want: letters, digits, spaces, printable characters, etc.

For a list of POSIX character classes, see POSIX Character Classes.

11.1 The while … do … done loop statement

Recall the syntax of a simple if branching statement:

if testing_command_list ; then
    zero_command_list
fi

For example:

if who | fgrep "$1" ; then
   echo "User '$1' is signed on"
fi

The shell while loop is similar to a simple if statement, except that the zero_command_list is executed by the shell over and over as long as the testing_comamnd_list succeeds. Instead of then and fi, the zero_command_list of the while loop is delimited by the new keywords do and done:

while testing_command_list ; do
    zero_command_list
done

For example:

while who | fgrep "$1" ; do
   echo "User '$1' is still signed on"
   sleep 10
done

As with the if statement, the testing_comamnd_list is executed, and the return status of the last command in the list is tested, then:

• While the exit status is zero (the command succeeded), all the commands in the zero_command_list are executed, over and over.
• If the exit status becomes non-zero (the testing command fails), the zero_command_list is not executed any more and the shell continues after done with the rest of the shell script, if any.
• Either command list can be multiple commands separated by newlines, semicolons, or logical && or || operators. If the testing_command_list is multiple commands, only the return status of the last command is tested. Any list may include commands connected by pipelines.

If the testing_command_list never fails, the while loop will repeat the zero_command_list over and over, forever. To avoid the loop repeating forever, the testing_command_list should eventually return a non-zero (failing) status.

Below are some examples of what these while loop statements might look like inside shell scripts.

Example use of an integer expression to add one to a loop index variable to create 100 files with numbered names:

$!/bin/sh -u
# Create 100 files with names filename1.txt through filename100.txt
i=1
while [ $i -le 100 ] ; do
    touch "filename$i.txt"
    i=$(( i + 1 ))
done

Below is another example using a shell pipeline as the testing_comamnd_list in the while loop:

#!/bin/sh -u
# Loop while idallen is signed on, then exit.
while who | fgrep "idallen" ; do
   echo "idallen is still online"
   sleep 60
done
echo "idallen just signed off"

As you can see in the example above, the testing_comamnd_list being executed can be a shell pipeline. Only the exit status of the last command in a pipeline is used by the shell. As long as the fgrep command succeeds in finding the string idallen in the output of the who command (exit status zero), the loop will continue. When the fgrep command does not find idallen, it returns a non-zero exit status and the loop finishes and the message idallen just signed off is echoed.

Below is a similar script the works the opposite to the one above. It uses the exclamation mark ! negation operator to negate the return code of the pipeline. It waits for idallen to sign on, and loops waiting until he does. When he does sign on, the fgrep command returns a success zero status that is inverted to failure by the exclamation mark ! and the script exits with the idallen just signed on message.

#!/bin/sh -u
# Loop while idallen is NOT signed on, then exit.
while ! who | fgrep "idallen" ; do
   echo "idallen is not online yet"
   sleep 60
done
echo "idallen just signed on"
echo "Hello Ian" | write idallen

11.2 The for … do … done loop statement

Unlike the while loop that has to run a testing_comamnd_list to know when to continue the loop, the for loop does not execute any testing command. It simply iterates over a fixed list of words, one at a time. There are two kinds of for loops, one with an implicit list of words (the command line arguments) and one with an explicit list of words:

# implicit list of words come from command line arguments
for name do                           # iterates over arguments $1 $2 $3 ...
    command_list
done

# explicit list of words is supplied before semicolon
for name in word1 word2 word3 ; do    # iterates over word1 word2 word3 ...
    command_list
done

Using square brackets here to indicate optional text and ... to indicate repeated text, as in the SYNOPSIS section of man pages, we could write the for syntax like this:

for name [ in word... ; ] do
    command_list
done

• The name is the name of a variable, called the index variable. The name is given here without a leading dollar sign!
• The optional inword...; is a list of words to iterate over, one at a time. The list must end with a semicolon.
• If the list of words is omitted, the command line argument positional parameters ($1, $2, etc.) are used for the list.

The variable $name is set to the first word in the list and then the command_list is executed. Then $name is set to the second word and the command_list is executed again. This repeats for each word in the list until the list is exhausted.

An example for loop with a list of three words to iterate over:

#!/bin/sh -u
for i in dog cow pig ; do
    echo "See the $i run!"
done

Running the above script:

$ ./example.sh
See the dog run!
See the cow run!
See the pig run!

Another example without an explicit list of words uses the positional parameters (command line arguments $1, $2, etc.) as the list of words:

#!/bin/sh -u
for j do
    echo "See the $j run!"
done

Running the above script:

$ ./example.sh man nose
See the man run!
See the nose run!

The name of the index variable is arbitrary but should usually reflect the meaning of the items in the list of words:

#!/bin/sh -u
fruits=
for newfruit in apple pear plum cherry apricot banana ; do
    fruits="$fruits $newfruit"
done
echo "So many:$fruits"

Running the above script:

$ ./example.sh
So many: apple pear plum cherry apricot banana

Often the shell script iterates over a list of file names on the command line:

#!/bin/sh -u
for file do
    if [ ! -r "$file" ] ; then
        echo "Cannot read '$file'"
    fi
done

Running the above script:

$ ./example.sh /etc/*
Cannot read '/etc/group-'
Cannot read '/etc/gshadow'
Cannot read '/etc/gshadow-'
Cannot read '/etc/shadow'
Cannot read '/etc/shadow-'
Cannot read '/etc/sudoers'

Remember that the variable named at the start of a for loop does not start with a dollar sign! You only use the dollar sign to expand the variable inside the loop body.

11.3 Loop modifiers: break and continue

You can modify control flow inside loops using break and continue.

You can break out of the middle of a loop using the break statement:

#!/bin/sh
#   $0 [ filenames... ]
# Create .bak copies of all the names.
# Does not overwrite existing non-empty .bak copies.
# Skips over missing or unreadable files.
# Script terminates on copy error; does not continue.
count=0
status=0
for name do
    if [ ! -e "$name" ] ; then
        echo "$0: '$name' is inaccessible or does not exist; skipped"
        status=1
    elif [ ! -f "$name" ] ; then
        echo "$0: '$name' is not a file; skipped"
        status=1
    elif [ ! -r "$name" ] ; then
        echo "$0: '$name' is not readable; skipped"
        status=1
    elif [ -s "$name.bak" ] ; then
        echo "$0: '$name.bak' already exists; skipped"
        status=1
    elif cp -p "$name" "$name.bak" ; then
        count=$(( count + 1 ))
        echo "$count. Backed up to $name.bak"
    else
        echo 1>&2 "$0: Could not copy '$name' to '$name.bak'"
        echo 1>&2 "$0: Script terminated"
        status=1
        break
    fi
done
echo "Copied: $count"
exit "$status"

You can skip back to the top of a loop using the continue statement. In the example below, we start a variable count at zero and then loop over all the arguments counting how many are readable and adding one to the counter each time. When the loop finishes (all the arguments have been processed), we show the value of the counter.

#!/bin/sh
#  $0 [ names... ]
count=0
for name do
    if [ ! -e "$name" ] ; then
        echo "$0: '$name' is inaccessible or does not exist"
        continue                          # skip back to top of FOR loop
    fi
    if [ ! -f "$name" ] ; then
        echo "$0: '$name' is not a file"
        continue                          # skip back to top of FOR loop
    fi
    if [ ! -r "$name" ] ; then
        echo "$0: '$name' is not readable"
        continue                          # skip back to top of FOR loop
    fi
    count=$(( count + 1 ))                # add one to the counter
    echo "$count. A readable file: $name $(wc <"$name")"
done
echo "Number of readable files: $count"

13.1 Example: ErrorExit function to exit a script

This example function below echoes all its arguments onto standard error, prints a usage message, and exits the script. Putting all this common code in the function makes the script shorter, easier to read, and easier to maintain.

#!/bin/sh -u

ErrorExit () {
    echo 1>&2 "$0: $*"
    echo 1>&2 "Usage $0 [filename]"
    exit 1
}

if [ $# -ne 1 ]; then
    ErrorExit "Expecting one filename argument; found $# ($*)"
fi
if [ "$1" = "" ]; then
    ErrorExit "Argument is an empty string"
fi
if [ ! -f "$1" ]; then
    ErrorExit "Argument is nonexistent, inaccessible, or not a file: '$1'"
fi
if [ ! -r "$1" ]; then
    ErrorExit "File is not readable: '$1'"
fi
if [ ! -s "$1" ]; then
    ErrorExit "File is empty: '$1'"
fi
if cp -p "$1" "$1.bak" ; then
    echo "Backed up '$1' to '$1.bak'."
else
    echo "Failed to back up '$1' to '$1.bak'."
fi

Running the above script:

$ ./example.sh
./example.sh: Expecting one filename argument; found 0 ()
Usage ./example.sh [filename]

$ ./example.sh /bin
./example.sh: Argument is nonexistent, inaccessible, or not a file: '/bin'
Usage ./example.sh [filename]

Functions can hold code in one place that would otherwise be repeated over and over in a script. Less code is better code.

16.1 Error Messages – four rules

Error messages must obey these four rules. Error messages must:

1. Appear on standard error, not standard output: 1>&2
2. Give the name of the program (script) that is issuing the message: $0
3. Tell the user exactly what was expected (e.g. “expecting one file name”; never use the vague “expecting one argument”)
4. Display what the user actually entered. For scripts with command line arguments, variables $# and $* are often useful, e.g. “found 3 (a b c)”

Never say just “illegal input” or “invalid input” or “too many”. Always specify how many is “too many” or “too few”:

echo 1>&2 "$0: Expecting 3 file names; found $# ($*)"
echo 1>&2 "$0: Student age $student_age is not between" \
    "$min_age and $max_age"
echo 1>&2 "$0: Modify days $moddays is less than zero"

After detecting an error, the usual thing to do is to exit the script with a non-zero return code. Don’t keep processing bad data!

16.2 Summary: Four properties of Good Error Messages

Good Error Messages must:

1. Appear on standard error, not on standard output: 1>&2
2. Give the name of the program (script) that is issuing the error: $0
3. Tell the user exactly what was expected.
4. Display exactly what was supplied (that is in error).