Shell Script Debugging – using options -uv and -ux

Ian! D. Allen – www.idallen.com

Winter 2017 - January to April 2017 - Updated 2019-03-23 04:40 EDT

1 Debugging shell scripts using options -uv and -uxIndexup to index

You can have the shell display statements in a shell script as they are either read or executed using the -uv or -ux options.

2 Using sh -uv shows commands as they are readIndexup to index

The -uv option tells the shell to display each line as it is read by the script, including comments and blank lines. (Loop statements are only printed once, as they are first read.) An example:

#!/bin/sh -u
#  $0 filename directory
# Find hard links to filename located in directory
file=$1
directory=$2
inode=$( ls -id "$file" | awk '{print $1}' )
find "$directory" -inum "$inode"

Running the above script:

$ ./example.sh
./example.sh: 4: ./example.sh: 1: parameter not set

$ sh -u example.sh
example.sh: 4: example.sh: 1: parameter not set

$ sh -uv example.sh
#!/bin/sh -u
#  $0 filename directory
# Find hard links to filename located in directory
file=$1
example.sh: 4: example.sh: 1: parameter not set

Above, the use of the -uv option shows which statement the shell was about to execute when it encountered the variable with the undefined value. We can supply two arguments to the script and see how the output changes:

$ sh -uv example.sh foo bar
#!/bin/sh -u
#  $0 filename directory
# Find hard links to filename located in directory
file=$1
directory=$2
inode=$( ls -id "$file" | awk '{print $1}' )
ls: cannot access foo: No such file or directory
find "$directory" -inum "$inode"
find: missing argument to `-inum'

Above, the use of the -uv option lets us see the lines that are producing the error messages, but we don’t see the actual values interpolated by the Variable Expansions.

3 Using sh -ux expands variables and shows commands as they are executedIndexup to index

The -ux option tells the shell to display only the lines that are actually executed by the script. The lines will show the results of Variable and Command Expansions. Loop statements will print over and over each time the loop iterates.

Using -ux, we see the lines as they are executed, with the variables expanded:

$ sh -ux example.sh foo bar
+ file=foo
+ directory=bar
+ ls -id foo
+ awk {print $1}
ls: cannot access foo: No such file or directory
+ inode=
+ find bar -inum
find: missing argument to `-inum'

Above, we see that the failing ls command meant that the variable $inode had no value, leading to the error message from the find command inside the script.

3.1 Improved -ux using /bin/bash instead of /bin/shIndexup to index

On some systems, you can get better-looking -ux debugging output by using the BASH shell instead of the default /bin/sh shell.

On some systems (including Ubuntu Linux), the /bin/sh shell name is not the BASH shell; the name is linked to the smaller, faster /bin/dash shell:

$ which sh
/bin/sh
$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 May  4  2017 /bin/sh -> dash
$ file /bin/dash
/bin/dash: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 2.6.32, BuildID[sha1]=504637666875a5d526ef51acfe601c99efc99114, stripped
$ man dash
...
dash is the standard command interpreter for the system.
...

The /bin/dash shell is a POSIX-compliant shell without many of the enhancements of the larger BASH shell that are not needed for running most shell scripts. Only a few shell scripts require the enhanced features of the BASH shell:

$ head -n 1 -q $( file /bin/* /usr/bin/* | awk -F: '/shell script/ {print $1}' ) | sort | uniq -c | sort -nr
    239 #!/bin/sh
     42 #! /bin/sh
     27 #!/bin/bash
     14 #!/usr/bin/env bash
     10 #!/bin/sh -e
      6 #! /bin/bash
      4 #!/bin/sh -
      4 #! /bin/sh -e
      2 #!/bin/bash -e
      1 #!/bin/sh -u
      1 #!/bin/sh

Small and embedded systems such as routers do not include the large /bin/bash shell; they only have a minimal /bin/sh shell and they can not run shell scripts using enhanced BASH shell syntax and features.

Shell scripts in this course are specifically written to work using only the features of the universal /bin/sh shell.

That said, if you do have a BASH shell available, it gives better-looking debug output when running a shell script. Compare this output below with the output from /bin/sh (linked to /bin/dash) in the previous section:

$ bash -ux example.sh foo bar
+ file=foo
+ directory=bar
++ ls -id foo
++ awk '{print $1}'
ls: cannot access foo: No such file or directory
+ inode=
+ find bar -inum ''
find: missing argument to `-inum'

Above, the BASH shell shows empty arguments and arguments with special characters correctly single-quoted, and it shows nested Command Substitutions with double leading ++ signs, making the debug output easier to read.

Use BASH to debug your scripts, but stick with the smaller, universal /bin/sh shell to actually run them.

4 Test scripts after every lineIndexup to index

The Number One rule of writing shell scripts is:

Start Small and Add One Line at a Time!

Students who write a 10- or 100-line script and then try to test it all at once usually run out of time. An unmatched quote at the start of a script can eat the entire script until the next matching quote!

To best diagnose script problems, don’t write a bunch of script lines and try to debug the whole thing. Build up your scripts slowly and incrementally, line-by-line, adding one or two lines at a time as you build it up into its final form.

Start writing your script with the Script Header (name of interpreter, PATH, umask, comments) and some known single command such as date. If that doesn’t work, you know something fundamental is wrong, and you only have a few lines of code that you need to debug. (Is your interpreter correct? your PATH?)

Add to this simple script only one or two lines at a time, testing after each line added, so that when an error occurs you know it must be in the last line or two that you added.

Do not add 10 lines all at once to a shell script! You won’t know what you did wrong!

5 Executing scripts ./script vs. having the shell read them sh scriptIndexup to index

Remember that if you use a shell to read a shell script (e.g. sh scriptname), instead of executing it directly (./scriptname), the shell will treat all the comments at the start of the shell script as comments.

In particular, the comment that specifies the interpreter to use when executing the script (#!/bin/sh -u) will be ignored, as will all of the options listed beside that interpreter.

Only by actually executing the script will you cause the Unix kernel to use the interpreter and options given on the first line of the script. For example:

$ cat test
#!/bin/sh -u
echo 1>&2 "$0: This is '$undefined'"

$ ./test
./test: undefined: unbound variable

$ sh test                                 # missing -u option!
test: This is ''

$ sh -u test                              # this is the right way
test: undefined: unbound variable

$ csh test                                # wrong shell used!
Bad : modifier in $ ( ).

All shells treat #-lines as comments and ignore them. Only the Unix kernel treats #! specially at the start of a script, and only for executable scripts.

If you are debugging a script by handing the script to a shell on a command line, remember to use the shell and include the options specified on the #! line at the top of the script! In particular, don’t forget to include the -u option.

Author: 
| Ian! D. Allen, BA, MMath  -  idallen@idallen.ca  -  Ottawa, Ontario, Canada
| Home Page: http://idallen.com/   Contact Improv: http://contactimprov.ca/
| College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/
| Defend digital freedom:  http://eff.org/  and have fun:  http://fools.ca/

Plain Text - plain text version of this page in Pandoc Markdown format

Campaign for non-browser-specific HTML   Valid XHTML 1.0 Transitional   Valid CSS!   Creative Commons by nc sa 3.0   Hacker Ideals Emblem   Author Ian! D. Allen