Shell search PATH – finding and running commands

Ian! D. Allen – www.idallen.com

Winter 2016 - January to April 2016 - Updated 2017-02-13 01:44 EST

1 Shells find and run commandsIndexup to index

The job of a shell is to find and run commands. This file explains how the shell identifies commands names and then finds the commands to run, such as the date command:

$ date
Sat Feb 21 05:36:58 EST 2015

The shell finds the date command as an executable program file, often located in the file system as /bin/date:

$ ls -l /bin/date
-rwxr-xr-x 1 root root 59984 Nov 19 17:25 /bin/date

$ /bin/date
Sat Feb 21 05:37:12 EST 2015

The shell has an exact system for finding this executable file and executing it when you type just the name date at the start of a shell command line. This document explains how that works.

2 Identifying the command nameIndexup to index

After the shell has processed and removed I/O redirection on a command line, the first token/word remaining at the beginning (left) of the line is assumed to be a command name. The command name in both lines below is the token date:

$ date >out
$ >out date

Redirection is done first, is removed, and is never part of a command name.

2.1 Shell built-in command namesIndexup to index

A few shell built-in commands are executed directly by the shell and are not searched for and run as external programs, e.g. echo, cd, umask, pwd, history, kill, help, etc.

Some built-in commands also have external versions, so that they are both external and built-in. (Often this is because something that was very useful as an external command was added later to the shell as a built-in command so that it would run faster or have built-in features.)

Built-in commands that have no external version have no separate man pages; you can’t say man 1 cd to find a manual page since there is no external cd command. For the BASH shell you can type help to get information about these built-in commands, e.g. help cd.

3 Searching for command names in PATHIndexup to index

Any command name that is not built-in (e.g. date) is assumed to be the name of an executable program file, and the shell attempts to find an executable file with that name and run it. (Shells find and run commands.)

If the command name contains no slashes (like most command names, e.g. date), the shell looks for the executable file with that exact name in the list of directories kept in the PATH environment variable. Because PATH is a shell environment variable, you can change the list, and the list is usually exported and inherited by child processes of your shell. Directories in PATH are separated by colons, e.g. the following PATH variable contains three directories separated by two colons:

$ echo "$PATH"
/usr/local/bin:/bin:/usr/bin

When you type the command name date (with no slashes), the shell goes looking for the date executable file in the list of directories kept in the PATH environment variable, looking for an executable file named date to execute. The shell tries each directory in the PATH, left-to-right, and runs the first executable program with the matching command name that it finds.

Using the above PATH list of directories, you can see that when you type date, the first directory tried in the PATH is /usr/local/bin, so the shell looks for an executable file named /usr/local/bin/date. This pathname does not usually exist:

$ ls -l /usr/local/bin/date
ls: cannot access /usr/local/bin/date: No such file or directory

The shell next tries the second directory name in the PATH variable (/bin) and looks for an executable file named /bin/date, and this is the usual location of the date executable file:

$ ls -l /bin/date
-rwxr-xr-x 1 root root 59984 Nov 19 17:25 /bin/date

Since this file exists, the shell runs this executable program and the date appears on your screen:

$ date
Sat Mar 16 20:39:13 EDT 2013

You could get exactly the same results on your screen if you typed the full pathname of the executable date file yourself:

$ /bin/date
Sat Mar 16 20:39:43 EDT 2013

Slashes in the pathname prevent the shell from using PATH to look up the command name, so the shell executes /bin/date directly.

3.1 Only one executable file is runIndexup to index

If the same command name appears in multiple PATH directories, only the first one found is run. Even if both /bin/date and /usr/bin/date existed, the shell would stop at the first directory in PATH containing the date command and only execute that one file, once.

3.2 Command not found – not found in PATHIndexup to index

If the shell can’t find the command name as an executable file in any of the PATH directories, it tells you it can’t find the command:

$ echo "$PATH"
/usr/local/bin:/bin:/usr/bin
$ fdisk
bash: fdisk: command not found

The message doesn’t mean that there is no such command anywhere on the system, only that the shell can’t find that command name in any of the directories listed in your current PATH variable. In the above example, none of these files existed: /usr/local/bin/fdisk /bin/fdisk /usr/bin/fdisk

If you know the command really does exist, e.g. you know the path to the command is /sbin/fdisk, you could either type the full path to the command directly, or add the /sbin directory that contains fdisk to your PATH if you want the shell to find it:

$ ls -l /sbin/fdisk                      # I know where the command is
-rwxr-xr-x 1 root root 99448 Jun 17 21:21 /sbin/fdisk

$ /sbin/fdisk                            # use the full path to the command
Usage ...

$ PATH=$PATH:/sbin ; echo "$PATH"        # add /sbin to end of $PATH
/usr/local/bin:/bin:/usr/bin:/sbin
$ fdisk                                  # now the shell can find it using $PATH
Usage ...

You can set your own list of directories in the PATH environment variable. People usually set and export their own PATH environment variable in one of the shell start-up files, such as .bashrc.

3.3 Current directory in PATH (don’t do it)Indexup to index

A leading or trailing colon in PATH, or two adjacent colons (::), indicate that the current directory is one of the directories that the shell should try when looking for executable command names. This is a security risk and you must not do it:

$ echo "$PATH"
:/bin/:/usr/bin                          # EXTREME SECURITY RISK!

$ echo "$PATH"
/bin/::/usr/bin                          # MODERATE SECURITY RISK!

$ echo "$PATH"
/bin/:/usr/bin:                          # SECURITY RISK!

Do not put the current directory (or any relative path directory) in PATH – it is a security risk. You might accidentally execute a malicious program in the current directory.

For example: If you put the current directory at the start of your PATH, e.g. PATH=:/bin:/usr/bin, then when you type ls the first program that is tried is ./ls (the program named ls in the current directory) which might be a malicious program.

4 Command names with slashes avoid PATHIndexup to index

If the command name found by the shell at the beginning of the command line contains any slashes, the shell does not use PATH to find the executable file. If there are slashes, the shell executes that file pathname directly as a program and does not need to search for it:

$ /bin/foo # executes the file /bin/foo; does not search PATH
$ bar/foo  # executes the file foo in the sub-directory bar; does not search PATH
$ ./foo    # executes the foo in the current directory; does not search PATH
$ ../foo   # executes the foo in the parent directory; does not search PATH

Either the file exists (and is executable) exactly where the pathname leads, or else you get an error:

$ ./foo
bash: ./foo: No such file or directory
$ doc/file.txt
bash: doc/file.txt: Permission denied

The rule is that no PATH searching is done if a command name contains any slashes.

4.1 Executing a program in the current directoryIndexup to index

The shell will not execute a program in the current directory unless you put ./ in front of the name to avoid the PATH search:

$ cp /bin/date foo     # create a program named foo in the current directory
$ foo
bash: foo: command not found
$ ./foo
Sat Mar 16 18:21:55 EDT 2013

If you don’t use ./ in front of the name, the shell will look for foo only in the directories in PATH, and it won’t be found. You must put slashes into the name to avoid the PATH search.

Do not put the current directory in PATH – it is a security risk.

5 Examples of PATH and command namesIndexup to index

Here are examples of how the shell finds and runs commands:

$ PATH=/bin:/usr/bin ; ls
- shell tries to find "ls" in first directory in PATH: /bin
  - shell looks for /bin/ls - this exists, so it is executed

$ PATH=/bin:/usr/bin ; gcc
- shell tries to find "gcc" in first directory in PATH: /bin
  - shell looks for /bin/gcc - this is not found
- shell tries to find "gcc" in next directory in PATH: /usr/bin
  - shell looks for /usr/bin/gcc - this exists, so it is executed

$ PATH=/bin:/usr/bin ; nosuch
- shell tries to find "nosuch" in first directory in PATH: /bin
  - shell looks for /bin/nosuch - this is not found
- shell tries to find "nosuch" in next directory in PATH: /usr/bin
  - shell looks for /usr/bin/nosuch - this is not found
- shell issues message "bash: nosuch: Command not found"

$ PATH=/bin:/usr/bin ; /usr/games/fortune
- command name /usr/games/fortune contains slashes, PATH is NOT used
- shell executes /usr/games/fortune directly (if it exists)
- PATH is NOT used

$ PATH=/bin:/usr/bin ; ./foo
- command name contains slashes, PATH is NOT used
- shell executes ./foo directly (if it exists)
- PATH is NOT used

$ PATH=/xxjunkxx ; ls
- shell tries to find "ls" in first directory in PATH: /xxjunkxx
- fails because /xxjunkxx/ls does not exist
- shell issues message "bash: ls: Command not found"

Because the shell appends your typed command name to each colon-separated pathname in the PATH variable when searching for an executable file, each pathname in PATH should be a directory, not a file name:

$ PATH=/bin/ls ; ls
- shell tries to find "ls" in first directory in PATH: /bin/ls
- fails because /bin/ls/ls does not exist (/bin/ls is not a directory!)
- shell issues message "bash: ls: Command not found"

Your shell has some built-in commands that are executed directly by the shell itself and not looked up in PATH, e.g. echo, cd, umask, pwd, history

$ PATH=/xxjunkxx ; date    # fails (because /xxjunkxx/date fails)
$ PATH=/xxjunkxx ; echo hi # works because echo is built-in to shell
$ PATH=/xxjunkxx ; cd ..   # works because cd is built-in to shell
$ PATH=/xxjunkxx ; umask   # works because umask is built-in to shell
$ PATH=/xxjunkxx ; pwd     # works because pwd is built-in to shell
$ PATH=/xxjunkxx ; history # works because history is built-in to shell

7 SummaryIndexup to index

  1. Non-slash command names (the most common) are searched for in PATH
  2. A command name containing any slashes is not looked for in PATH. The pathname is executed directly (if it exists).
  3. You must put ./ in front of any program that you want to execute in your current directory, e.g. ./foo
  4. Do not put the current directory in PATH – it is a security risk.

8 PATH in Shell ScriptsIndexup to index

Put a correct PATH setting at the start of all your shell scripts and export it into the environment.

If you don’t set the PATH at the start of your script, the script will inherit the PATH from the person or program that executes your script. The inherited PATH may or may not contain the correct directories needed to find the commands used by your script – your script may fail.

Choose PATH in your script to include the system directories that contain the commands your script needs. Directories /bin and /usr/bin are almost always necessary. System scripts may need /sbin and /usr/sbin. GUI programs will need the X11 directories. Choose appropriately for the script.

Author: 
| Ian! D. Allen, BA, MMath  -  idallen@idallen.ca  -  Ottawa, Ontario, Canada
| Home Page: http://idallen.com/   Contact Improv: http://contactimprov.ca/
| College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/
| Defend digital freedom:  http://eff.org/  and have fun:  http://fools.ca/

Plain Text - plain text version of this page in Pandoc Markdown format

Campaign for non-browser-specific HTML   Valid XHTML 1.0 Transitional   Valid CSS!   Creative Commons by nc sa 3.0   Hacker Ideals Emblem   Author Ian! D. Allen