% Shell Script Debugging -- using options `-uv` and `-ux` % Ian! D. Allen -- -- [www.idallen.com] % Fall 2018 - September to December 2018 - Updated 2019-03-23 04:40 EDT - [Course Home Page] - [Course Outline] - [All Weeks] - [Plain Text] Debugging shell scripts using options `-uv` and `-ux` ===================================================== You can have the shell display statements in a shell script as they are either read or executed using the `-uv` or `-ux` options. - The `-u` option tells the shell to give an error message and stop executing the script when expanding a variable that is undefined, usually because of a typing error. Without this option, all undefined variables simply and silently expand to be nothing. - The `-v` option tells the shell to display each line as it is read by the script, including comments and blank lines. (Loop statements are only printed once, as they are first read.) - The `-x` option tells the shell to display only the lines that are actually executed by the script. The lines will show the results of Variable and Command Expansions. Loop statements will print over and over each time the loop iterates. Using `sh -uv` shows commands as they are read ============================================== The `-uv` option tells the shell to display each line as it is read by the script, including comments and blank lines. (Loop statements are only printed once, as they are first read.) An example: #!/bin/sh -u # $0 filename directory # Find hard links to filename located in directory file=$1 directory=$2 inode=$( ls -id "$file" | awk '{print $1}' ) find "$directory" -inum "$inode" Running the above script: $ ./example.sh ./example.sh: 4: ./example.sh: 1: parameter not set $ sh -u example.sh example.sh: 4: example.sh: 1: parameter not set $ sh -uv example.sh #!/bin/sh -u # $0 filename directory # Find hard links to filename located in directory file=$1 example.sh: 4: example.sh: 1: parameter not set Above, the use of the `-uv` option shows which statement the shell was about to execute when it encountered the variable with the undefined value. We can supply two arguments to the script and see how the output changes: $ sh -uv example.sh foo bar #!/bin/sh -u # $0 filename directory # Find hard links to filename located in directory file=$1 directory=$2 inode=$( ls -id "$file" | awk '{print $1}' ) ls: cannot access foo: No such file or directory find "$directory" -inum "$inode" find: missing argument to `-inum' Above, the use of the `-uv` option lets us see the lines that are producing the error messages, but we don't see the actual values interpolated by the Variable Expansions. Using `sh -ux` expands variables and shows commands as they are executed ======================================================================== The `-ux` option tells the shell to display only the lines that are actually executed by the script. The lines will show the results of Variable and Command Expansions. Loop statements will print over and over each time the loop iterates. Using `-ux`, we see the lines as they are executed, with the variables expanded: $ sh -ux example.sh foo bar + file=foo + directory=bar + ls -id foo + awk {print $1} ls: cannot access foo: No such file or directory + inode= + find bar -inum find: missing argument to `-inum' Above, we see that the failing `ls` command meant that the variable `$inode` had no value, leading to the error message from the `find` command inside the script. Improved `-ux` using `/bin/bash` instead of `/bin/sh` ----------------------------------------------------- On some systems, you can get better-looking `-ux` debugging output by using the BASH shell instead of the default `/bin/sh` shell. On some systems (including Ubuntu Linux), the `/bin/sh` shell name is not the BASH shell; the name is linked to the smaller, faster `/bin/dash` shell: $ which sh /bin/sh $ ls -l /bin/sh lrwxrwxrwx 1 root root 4 May 4 2017 /bin/sh -> dash $ file /bin/dash /bin/dash: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 2.6.32, BuildID[sha1]=504637666875a5d526ef51acfe601c99efc99114, stripped $ man dash ... dash is the standard command interpreter for the system. ... The `/bin/dash` shell is a [POSIX]-compliant shell without many of the enhancements of the larger BASH shell that are not needed for running most shell scripts. Only a few shell scripts require the enhanced features of the BASH shell: $ head -n 1 -q $( file /bin/* /usr/bin/* | awk -F: '/shell script/ {print $1}' ) | sort | uniq -c | sort -nr 239 #!/bin/sh 42 #! /bin/sh 27 #!/bin/bash 14 #!/usr/bin/env bash 10 #!/bin/sh -e 6 #! /bin/bash 4 #!/bin/sh - 4 #! /bin/sh -e 2 #!/bin/bash -e 1 #!/bin/sh -u 1 #!/bin/sh Small and embedded systems such as routers do not include the large `/bin/bash` shell; they only have a minimal `/bin/sh` shell and they can not run shell scripts using enhanced BASH shell syntax and features. Shell scripts in this course are specifically written to work using only the features of the universal `/bin/sh` shell. That said, if you do have a BASH shell available, it gives better-looking debug output when running a shell script. Compare this output below with the output from `/bin/sh` (linked to `/bin/dash`) in the previous section: $ bash -ux example.sh foo bar + file=foo + directory=bar ++ ls -id foo ++ awk '{print $1}' ls: cannot access foo: No such file or directory + inode= + find bar -inum '' find: missing argument to `-inum' Above, the BASH shell shows empty arguments and arguments with special characters correctly single-quoted, and it shows nested Command Substitutions with double leading `++` signs, making the debug output easier to read. Use BASH to debug your scripts, but stick with the smaller, universal `/bin/sh` shell to actually run them. Test scripts after every line ============================= The **Number One** rule of writing shell scripts is: > Start Small and Add One Line at a Time! Students who write a 10- or 100-line script and then try to test it all at once usually run out of time. An unmatched quote at the start of a script can eat the entire script until the next matching quote! To best diagnose script problems, don't write a bunch of script lines and try to debug the whole thing. Build up your scripts slowly and incrementally, line-by-line, adding one or two lines at a time as you build it up into its final form. Start writing your script with the Script Header (name of interpreter, `PATH`, `umask`, comments) and some known single command such as `date`. If that doesn't work, you know something fundamental is wrong, and you only have a few lines of code that you need to debug. (Is your interpreter correct? your `PATH`?) Add to this simple script only one or two lines at a time, testing after each line added, so that when an error occurs you know it must be in the last line or two that you added. Do not add 10 lines all at once to a shell script! You won't know what you did wrong! Executing scripts `./script` vs. having the shell read them `sh script` ======================================================================= Remember that if you use a shell to read a shell script (e.g. `sh scriptname`), instead of executing it directly (`./scriptname`), the shell will treat all the comments at the start of the shell script as comments. In particular, the comment that specifies the interpreter to use when executing the script (`#!/bin/sh -u`) will be ignored, *as will all of the options listed beside that interpreter*. Only by actually *executing* the script will you cause the Unix kernel to use the interpreter and options given on the first line of the script. For example: $ cat test #!/bin/sh -u echo 1>&2 "$0: This is '$undefined'" $ ./test ./test: undefined: unbound variable $ sh test # missing -u option! test: This is '' $ sh -u test # this is the right way test: undefined: unbound variable $ csh test # wrong shell used! Bad : modifier in $ ( ). All shells treat `#`-lines as comments and ignore them. Only the Unix kernel treats `#!` specially at the start of a script, and only for executable scripts. If you are debugging a script by handing the script to a shell on a command line, remember to use the shell and include the options specified on the `#!` line at the top of the script! In particular, don't forget to include the `-u` option. -- | Ian! D. Allen, BA, MMath - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Course Home Page]: .. [Course Outline]: course_outline.pdf [All Weeks]: indexcgi.cgi [Plain Text]: 725_debugging_shell_scripts.txt [POSIX]: http://en.wikipedia.org/wiki/POSIX [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/