Shell Scripts – lists of commands, executable scripts, script header, command arguments and positional parameters

Ian! D. Allen – www.idallen.com

Winter 2016 - January to April 2016 - Updated 2018-11-01 17:32 EDT

1 Shells can read from keyboards or from text filesIndexup to index

A shell (e.g. bash) can be used in one of two ways:

This page discusses shell scripts.

1.1 Help on Lynda.comIndexup to index

You may find some parts of these advanced Lynda.com videos useful as you start writing your own shell scripts. These links require you to have created a free account on lynda.com via the Algonquin Lynda.com Link:

2 Create Shell Scripts to automate sysadmin tasksIndexup to index

A shell script is a list of commands, stored in a text file, that can be executed by a shell. The shell reads the commands from the file instead of reading commands from your keyboard.

If we have a set of commands that we want to run on a regular basis, we could create a text file containing those commands. The text file allows us to use a single file to hold many commands. This text file is called a Shell Script. For example:

$ cat myscript.sh                      # show the content of the file
echo "This is a shell script."
date
echo "Goodbye $USER"

$ sh myscript.sh                       # pass the file to a shell "sh"
This is a shell script
Wed Nov  2 04:47:44 EDT 2016
Goodbye idallen

Like most commands that process files, if you give the shell a file to read, it will read from that file and not from your keyboard. Unlike most other commands, shells will treat each line in the file as a command to be run.

There are two ways to run the list of commands in the script file:

2.1 Run Method 1: Using the shell to read the script fileIndexup to index

The most obvious way to run a list of commands stored in a shell script is to type the name of the shell program that we want to use to read the script file (e.g. sh or bash) and give it the script file name as a file name argument:

$ bash -u myscript.sh
$ sh -u myscript.sh

The -u argument to the shell tells the shell to report undefined variables used in the script and abort if any are used. Using -u is always good practice, in case there might be a typing mistake in the script.

With a file name argument, this script-reading copy of the shell program does not read your keyboard but instead reads the script file and executes each line in the script file as a command line. When it reaches the end of the script file, the script shell exits and you get your original interactive shell prompt back.

A separate shell process is reading the script file, not your interactive shell that is reading your keyboard. Changes made to the environment of the shell running the script (e.g. changing directories, setting variables) will not affect your interactive shell. Your interactive shell is protected from environment changes that the shell script might make as it runs.

2.2 Run Method 2: Making the script file executableIndexup to index

The other, more elegant way to run the list of commands in a script file is to make the script text file executable (only needs to be done once) and then simply type its name as a command name to the shell:

$ chmod ugo+x myscript.sh              # only needs to be done once
$ ./myscript.sh

The name of the script can be typed into your interactive shell and a second copy of the shell will execute the script by reading the script file and finding and running the commands in the script file. When that script shell reaches the end of the script file, that script shell exits and you get your original shell prompt back.

Again, a separate shell process is reading the script file, not your interactive shell that is reading your keyboard.

3 Use your own bin directory for your scriptsIndexup to index

If you put your scripts in your own directory, e.g. ~/bin and append that directory name at the end of your shell $PATH, then your script names behave just like other command names:

$ mkdir ~/bin                          # only need to do this once
$ PATH=$PATH:~/bin                     # could do this in your .bashrc
$ mv myscript.sh ~/bin/myscript        # choose a unique name
$ myscript                             # execute your script!

As with every command name typed into the shell, the shell will search for the command name myscript in all the directories in your search $PATH, finally finding it in the file ~/bin/myscript and executing it. Most people set their $PATH variable to include their personal bin/ directory at log-in time via their .bashrc.

In short:

  1. Create an executable shell script with a unique name.
  2. Move the script file to a directory that is in your search $PATH
  3. Type the name and the script runs!

As a system administrator, you can make your job easier by writing your own custom shell scripts containing lists of commands to help automate tasks. You can often bring these scripts with you when you change jobs. Sysadmin may have dozens of personal shell scripts in their own personal bin directory.

3.1 Many command names are actually shell scriptsIndexup to index

Many command names in the system are not binary executable programs; they are actually shell scripts. You can find out how many scripts are in the standard system command directories /bin and /usr/bin:

$ file /bin/* /usr/bin/* | fgrep 'script' | wc
446    3232   40941

$ file /bin/gunzip
/bin/gunzip: POSIX shell script, ASCII text executable

Above we see that the gunzip command is actually a shell script, not a binary executable program. Because it is a text file, you can read it and discover that all it does is call gzip -d with whatever arguments you give it.

4 Standard Script Header – three linesIndexup to index

Shell scripts need a consistent environment when they run. For this, course we will specify three things at the start of every shell script:

  1. The shell program and options that runs the script: /bin/sh -u
  2. The search PATH that will be used to find commands in the script: /bin:/usr/bin
  3. The umask that will be used in the script: 022

To do this, use this exact set of Standard Script Header lines as the first three lines of shell scripts you write in this course:

#!/bin/sh -u
PATH=/bin:/usr/bin ; export PATH          # (2) PATH line
umask 022                                 # (3) umask line

Each of the three lines above is described in detail below.

4.1 Line 1: Interpreter Magic Number line, or Shebang: #!Indexup to index

The very first line of every executable standard shell script in this course must be an interpreter magic number line, or shebang line (short for “shell bang” or “shell exclamation”) with this exact text (12 characters plus newline):

#!/bin/sh -u

The first two characters #! need to be the first two characters in the file, because together they form a Magic Number that tells the kernel this is an executable script file with a special format.

No additional shell comments (#) are allowed on the first line of the script, because this first shebang #! line must be executed by the Linux kernel, not by the shell, and the kernel does not understand shell comments. Use the exact line given above for all the scripts in this course.

4.1.1 The shebang #! line tells the kernel which program to runIndexup to index

The #! magic number at the start of the file must be followed by the absolute path of a binary executable program file that kernel will run when you execute this file.

This file pathname on this first line is how the kernel chooses which program will process your script file. The program specified for a shell script is almost always the standard system shell /bin/sh or (less commonly) /bin/bash:

#!/bin/sh -u

If an executable file starts with #!, the kernel will execute the program name that follows the #! and hand the file name being executed to the program as an argument. A shell program (e.g. /bin/sh) will read and execute the lines in the file as commands. For example, if the first line of a script named myscript.sh is the standard shebang line #!/bin/sh -u, then the kernel will create and then execute this command line when it executes the script file:

/bin/sh -u myscript.sh

The /bin/sh program will read and execute commands from the myscript.sh file that is its argument, using the -u shell option.

A limited number of option arguments can be supplied to the program on the shebang line, after the program name. The -u option argument to a Bourne shell program tells the shell to generate an error if the script tries to make use of a variable that’s not set and has no value. We always use -u to force the shell to detect undefined variables:

  • Undefined variables should never happen if the script is well written and tested.
  • If the error does happen, it’s better to have the shell stop processing than continue on using incorrect data.

4.1.2 The shebang #! line is a comment when the shell reads itIndexup to index

The shebang line deliberately begins with a hashtag shell comment character # and will be ignored as a shell comment when the shell program specified after the #! reads the script file.

If, instead of executing the shell script, you want to pass the shell script as an argument to a shell program, remember to also pass the options given in the shebang line. For example, if the shebang line in the file myscript.sh is #!/bin/sh -u, you must type this command line to run it properly, remembering to include the -u option:

$ /bin/sh -u myscript.sh

When you call the shell directly and pass the script as an argument, the shebang line is treated as a comment line and is ignored by the shell. You have to remember to include the options on the shebang line yourself, as shown in the above command line.

4.1.3 Any program name is possible after #!Indexup to index

Any executable program name can be specified after the #! on the shebang line, and the kernel will run that program, passing to the program the name of the file being executed as an argument. If the shebang line in file myscript.sh is:

#!/bin/sh -u      ...then the kernel executes: /bin/sh -u myscript.sh
#!/bin/bash -u    ...then the kernel executes: /bin/bash -u myscript.sh
#!/usr/bin/csh    ...then the kernel executes: /usr/bin/csh myscript.sh

Q: What would happen if you changed the first line of a script file to be:

  • #!/bin/ls -l
  • #!/usr/bin/wc
  • #!/bin/cat
  • #!/bin/rm

4.2 Line 2: Set the shell search PATHIndexup to index

The second line of the Standard Script Header sets the search PATH for the script:

PATH=/bin:/usr/bin ; export PATH          # (2) PATH line

Remember that, since the script is being executed by its own shell program, nothing you set or change inside the script will affect any shell outside the script. Settings in your login shell remain unchanged.

4.3 Line 3: Set the shell umaskIndexup to index

The third line of the Standard Script Header sets the permissions umask for the script:

umask 022                                 # (3) umask line

Remember that, since the script is being executed by its own shell program, nothing you set or change inside the script will affect any shell outside the script. Settings in your login shell remain unchanged.

5 The body of the script (after the script header)Indexup to index

The “body” of the script is the code that follows the script header.

5.1 stdin stdout stderr are unchangedIndexup to index

6 Arguments to the script on the command lineIndexup to index

6.1 Positional Parameter variables: $0, $1, $2, $#, etc.Indexup to index

6.2 Aggregate Positional Parameter variables: $* and $@Indexup to index

Shell variables $* and $@ both expand to be all of the arguments supplied on the command line. They behave differently when double quoted:

We will explore how these variables work in a separate document.

7 Sample script One – variables and positional parametersIndexup to index

Below is a small sample shell script that demonstrates the use of various shell variables, including positional parameters. Put these lines below in a file named positional.sh and make it executable and then run it.

The first three lines of the script are a copy of the standard script header:

#!/bin/sh -u
PATH=/bin:/usr/bin ; export PATH
umask 022

# The lines below are the body of this shell script:
#
myvar="howdy doody"
echo "The value of \$myvar is: $myvar"       # use backslash to hide first $

echo "The command name (the script name) is $0"
echo "The number of command line arguments is: $#"
echo "All the command line arguments are: $*"
echo "The first argument is: $1"             # fails if no arguments
echo "The second argument is: $2"            # fails if not two arguments
echo "The third argument is: $3"             # fails if not three arguments

Because the script is running the shell with the -u option, the shell will issue an error message if any variables, including positional parameter variables, are undefined. To avoid these errors, make sure you execute the above script with at least three command line arguments:

$ chmod ugo+x positional.sh               # only needs to be done once
$ ./positional.sh one two three           # give at least three arguments
The value of $myvar is: howdy doody
The command name (the script name) is ./positional.sh
The number of command line arguments is: 3
All the command line arguments are: one two three
The first argument is: one
The second argument is: two
The third argument is: three

You can also run any script file, even if it isn’t executable, by passing its file name to the shell as a file name argument:

$ sh -u positional.sh

The shebang line is not used when you run a shell script the above way, which is why we must explicitly supply the -u option to check for undefined variables.

8 Shifting positional parameters with shiftIndexup to index

The built-in shell command shift will delete the first command line argument, causing all the positional parameter numbers to shift down one:

$ cat myscript.sh
#!/bin/sh -u
echo "All arguments: $*"
echo "First argument \$1 is '$1' and second argument \$2 is '$2'; count is $#"
echo "Doing a shift"
shift
echo "All arguments: $*"
echo "First argument \$1 is '$1' and second argument \$2 is '$2'; count is $#"
echo "Doing a shift"
shift
echo "All arguments: $*"
echo "First argument \$1 is '$1' and second argument \$2 is '$2'; count is $#"

$ ./myscript.sh one two three four
All arguments: one two three four
First argument $1 is 'one' and second argument $2 is 'two'; count is 4
Doing a shift
All arguments: two three four
First argument $1 is 'two' and second argument $2 is 'three'; count is 3
Doing a shift
All arguments: three four
First argument $1 is 'three' and second argument $2 is 'four'; count is 2

As you can see, after a shift, all the arguments shift down one place. Argument $2 becomes $1 (because the first argument is gone); argument $3 becomes argument $2, etc. The shift command is most useful inside a looping control structure such as a while or for loop.

9 Comments in your own shell scripts – no “Instructor-Type” commentsIndexup to index

Many of the comments in script file examples in this course are “Instructor-Type” comments and are not appropriate for real scripts that you write. (e.g. Comments such as This is the body of the shell script or use backslash to hide first $)

“Instructor-Type” comments explain features about the syntax and structure of a shell script and are used to teach scripting to beginners. I put Instructor-Type comments in my examples because I am teaching you how to write scripts.

You would not put Instructor-Type comments in your own scripts, because the scripts you write always assume that you and your script readers already know how to write shell scripts.

Do not put “Instructor-Type” comments into the scripts that you submit for marking. Comments should address what the script is doing, not how ordinary script features work. Do not submit my Instructor-Type comments back to me again in scripts that you write.

Author: 
| Ian! D. Allen, BA, MMath  -  idallen@idallen.ca  -  Ottawa, Ontario, Canada
| Home Page: http://idallen.com/   Contact Improv: http://contactimprov.ca/
| College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/
| Defend digital freedom:  http://eff.org/  and have fun:  http://fools.ca/

Plain Text - plain text version of this page in Pandoc Markdown format

Campaign for non-browser-specific HTML   Valid XHTML 1.0 Transitional   Valid CSS!   Creative Commons by nc sa 3.0   Hacker Ideals Emblem   Author Ian! D. Allen