================================ How a Shell Script is "Executed" ================================ -IAN! idallen@idallen.ca When you type a command name or program name into a shell, e.g. "date" or "./myscript.sh" the shell will try to find an executable file to execute. (If the name you type has no slashes in it, the shell searches the directories of your $PATH to find the executable file. If the name has slashes, the shell tries to access that pathname directly.) When the shell finds an executable file, the shell forks itself into a second process and then tells the Unix/Linux kernel try to execute that file. If the file contains a machine code binary program (e.g. ls, sort, cat, grep, etc.), the kernel will load the program into memory and execute it. When the binary program finishes, that means the second process (forked by the shell) is finished and the first shell process issues another prompt for another command line, and the process repeats when you type the next command name or pathname. The kernel can *only* load into memory and execute machine code (binary) programs. The kernel *cannot* execute directly a text file, even if the text file is an executable shell script. Something else happens. If an executable file found by the shell is a text file such as a shell script (it is not a machine code binary program), one of two things will happen when the kernel tries to execute this file: 1) If the script file starts with the characters #! followed by the absolute pathname of a file containing a machine code binary program (which might be a shell, or some other binary program), the kernel execution succeeeds using that binary program. The kernel actually loads the program named in the shell script into memory, not the text of the shell script itself. (Recall that only machine code binary programs can be loaded into memory and executed.) Once the kernel starts up that specified binary program, the kernel hands the *name* of the text shell script to the binary program as its first pathname argument. Therefore, the kernel never actually *executes* a shell script - the kernel executes some specified binary program (whatever program is named in the first line of the script), and it is that program that processes the script name as its first pathname argument. See below for more details. 2) If the script file does *not* start with the #! name of a binary program to use (no #! line at the start of the file), the kernel fails in its attempt to execute the executable script file. Most shells (including all the Bourne and C shells) notice the failure. On failure, the shell forks itself into a second process, and starts a second process to read the script file and execute the commands contained in it. The shell used to read the commands in the script file is chosen based on some historic heuristics based on the first line of the file, and there is no guarantee that the shell chosen will correctly match the syntax of the script file. This is a Bad Thing. Shell scripts should always start with the name of the correct program that the kernel should execute. Do *not* let the shell use heuristics to choose which program to run (Method #2, above). Always specify the "shebang" #! line as the first line of your scripts (Method #1). --------------------------------------- How the kernel processes the first line --------------------------------------- If you correctly specify an executable binary program on the #! line at the start of your script, the kernel will load that program into memory and pass it the name of your script as its first pathname argument when you "execute" the shell script. (In fact, as mentioned earlier, you can't actually "execute" a text shell script; you can only execute the machine code binary program named in the first line of the script. That binary program will be given the name of your text script file as an argument.) Here is an example of how to (1) create a short shell script that has a correct first line, (2) make it executable, and then (3) use your shell to "execute" it: $ cat >myscript.sh <== use "cat" with output redirection #!/bin/sh -u <== 1. type in three lines of script echo "This is a shell script." <== 2. type in three lines of script date <== 3. type in three lines of script ^D <== signal EOF to "cat" $ chmod +x myscript.sh <== make it executable $ ./myscript.sh <== "execute" the script This is a shell script. Sun Feb 16 10:50:27 EST 2003 $ When you enter "./myscript.sh" on your shell command line, your shell does not search PATH to locate the executable file; because, "./myscript.sh" has a slash in it. Your shell tries to execute this pathname (a pathname in the current directory) directly. Because this executable text file starts with a correct "#!/bin/sh -u" line, the Unix kernel loads the /bin/sh shell program into memory and hands the shell the name of your script as its first pathname argument. You typed in "./myscript.sh" to your shell; but, what actually gets executed by the Unix kernel is this: /bin/sh -u ./myscript.sh The string "/bin/sh -u" names a machine code binary program to load; it comes directly from the first line of the shell script. The string "./myscript.sh" comes from what you typed on the command line to "execute" this shell script. Thus, the /bin/sh shell is passed an option argument of "-u" and the name of a script file to read. The /bin/sh shell reads the script file and executes the commands it contains. ------------------------------- Typing errors on the first line ------------------------------- The kernel is very literal about what you type - whatever you type on the #! line at the start of the script becomes the first part of the command that gets executed. If your script starts with an incorrect line such as "#!/bin/sh u" (note the missing dash), then you will likely see the following unhelpful error message when you try to execute your script: $ ./myscript.sh u: u: No such file or directory. What is happening? Well, the kernel always gets the first part of the command to execute directly from the first line of your script file; so, the kernel builds and executes the following command line (note how the kernel copied your missing dash on "u" from your script): /bin/sh u ./myscript.sh The above command line starts the /bin/sh shell and passes it two pathname arguments. The first one, "u", doesn't exist, so you get an error message. You would get exactly the same error message if you typed the above command line yourself (instead of letting the kernel build it for you out of your script file): $ /bin/sh u ./myscript.sh u: u: No such file or directory. ------------------------------ Confusing kernel error message ------------------------------ The absolute pathname after the #! characters on the first line of the script *must* name an executable machine code binary program for the kernel to load. You cannot name another shell script on the first line. If you spell the name of the executable program incorrectly, you may get a very unhelpful error message when you try to execute your shell script! $ cat >wrong.sh <== use "cat" with output redirection #!/bin/nosuchfile <== line 1. (no such program) echo "This is a shell script." <== line 2. date <== line 3. ^D <== signal EOF to "cat" $ chmod +x wrong.sh <== make it executable $ ./wrong.sh <== "execute" the script ./wrong.sh: No such file <== confusing kernel error message $ Although you can plainly see that the shell script "./wrong.sh" exists and is executable, the error message leads you to believe that the file isn't there. In fact, what the kernel is trying to tell you is that "/bin/nosuchfile" on the first line of the script is invalid - there is no such machine code binary program to load ("No such file or directory"). (Some shells say "Command not found" for this error.) Unfortunately, the error messages produced for this type of spelling error don't contain the name of the program that is causing the actual error. You have to have the experience to know that, if the file clearly exists, the problem must be with the spelling of the program named on the first line of the script. (A common error is to name the script "#!bin/sh" instead of "#!/bin/sh".) The name on the first line of the script must be an absolute pathname to a machine code binary executable program. -------------------------------- Passing Arguments via the Kernel -------------------------------- Most Unix kernels will let you squeeze a very small amount of option text onto the #! line after the name of the binary program, e.g. #!/bin/sh -u #!/bin/grep pattern #!/bin/sort -r Since it is the Unix kernel that is parsing this line and trying to find the binary program to execute, not the shell, none of the quoting or variable expansion features of a shell work here. Some kernels limit the entire line to 32 characters. Some kernels only accept exactly one option argument. "#!/bin/sh -u" appears to work everywhere. (The "-u" option to a Bourne shell tells the shell to warn you about undefined variables.) -------------------------- Any Binary Program Will Do -------------------------- The machine code binary program that you name on the first line of your shell script can be *any* binary program, not just a shell: #!/bin/cat #!/bin/sort #!/bin/wc #!/bin/head #!/bin/tail #!/bin/rm <-- be careful with this one! When you "execute" your script by typing "./myscript.sh", the kernel will load into memory the binary program specified by the first line of the script, and then give only the *name* of the script to the binary program as its first argument. (The kernel never executes a text file directly.) You should make sure that the program the kernel executes does something reasonable with its first argument. If the program is a shell program such as "#!/bin/bash", shells take an argument as a file of commands to read and execute. If the program is "#!/bin/rm", the "rm" program will unlink the name that is its first argument (which will probably delete your script!). --------- Exercises --------- - If the first line of an executable file is "#!/bin/cat", what will the output be when you execute the file? (Hint: Try it! Put a few lines of text into a file add "#!/bin/cat" as the first line, make the file executable, and then execute it. What do you see?) - If the first line of an executable file is "#!/bin/sort", what will the output be when you execute the file? (Hint: Try it!) - If the first line of an executable file is "#!/usr/bin/head -2", what will the output be when you execute the file? (Hint: Try it!) - If the first line of an executable file is "#!/usr/bin/tail -2", what will the output be when you execute the file? (Hint: Try it!) - If the first line of an executable file is "#!bin/nosuch", what will the output be when you execute the file? (Hint: Try it!) Remember this error message! You may see it again. Remember it!