Unix/Linux and Shell Basics – miscellaneous small topics

Ian! D. Allen – www.idallen.com

Winter 2017 - January to April 2017 - Updated 2019-01-15 07:47 EST

1 Characters that Look the Same but are DifferentIndexup to index

You must pay strict attention to the exact characters used in command lines. Here are some common confusions, shown in two font styles:

A common error is to think that the option letter l after the dash in ls -l is the digit 1. It’s a letter l, not a digit 1.

2 Typing Command LinesIndexup to index

In documentation and assignments you will be asked to enter some command lines, and the lines will be written something like this:

$ date
$ toilet --gay Hello World

The above indicates that you should type the command name date at a shell prompt, followed by the Enter key, then at the next shell prompt type the command name toilet followed by some arguments --gay Hello World followed by the Enter key.

The shell prompt is indicated by the $. You do not type the prompt; the prompt is merely a documentation convention telling you that what follows to the right it is a command line that you should type at the prompt. The actual shell prompt might look more like bash-4.3$, or perhaps % or >, but we only need to show the $ in the examples.

Do not type the shell prompt when entering command lines. Enter only the command line to the right of the prompt, followed by the Enter key. The command will not be executed until the Enter key is used.

The word or string of text on the left, closest to the shell prompt, is usually the command name, e.g. date, toilet. The text strings that follow the command name, if any, are the arguments to the command, e.g. the toilet command name above has arguments --gay Hello World

The output produced by running the commands may or may not be shown in the documentation. The output, if shown, will not be preceded by the shell prompt:

$ date
Thu Jan 15 04:33:58 EST 2015

$ echo Hello World
Hello World

3 What is a shell for?Indexup to index

The function of a shell is to find and run programs. (“Programs” are also called processes, commands, or utilities.) A shell is itself a program that finds and runs other programs.

Shells also do programming kinds of things, but that programming is usually to aid in the finding and running of programs, not to do any kind of substantial mathematical or business calculations. (Shells do not do mathematics very well.)

Part of running a program involves supplying command line arguments to that program; the shell helps with that, too.

Command names typed into shells almost always resolve to be the names of executable files stored in system directories. (Some command names are not files; they are built-in to the shell, such as cd.)

For example, the shell may resolve the command name ls to run the executable file /bin/ls, passing it any arguments you type on the command line. You can control where the shell looks for command executables by setting the $PATH variable.

4 How does the shell help find and run commands?Indexup to index

We will learn many of these mechanisms in upcoming notes and lectures:

5 Basic Command Line SyntaxIndexup to index

The shell prompts you to enter command lines using a prompt. In documentation and help files, the prompt is usually assumed to be a dollar sign $ but may actually look like bash-4.3$ or something similar. Later, you will learn how to change what the prompt looks like by assigning quoted text to the PS1 variable.

Most of the time, what you enter in response to the shell prompt are command lines that start with a command name followed by some optional arguments. You type the entire line, then send it for execution by the shell by pushing the ENTER or RETURN key.

Examples of command lines (following the shell prompt):

$ date
$ figlet "Hello World"
$ toilet --gay "Big Letters"

Linux commands and file names are case sensitive, which means that typing date, Date, datE or DATE are considered different commands, and file.txt and FILE.TXT are different file names.

Almost all Linux command names are all lower-case. File names also tend to be all lower-case with no spaces, for ease of use on the command line, but with increasing use of GUI interfaces mixed-case file names containing blanks are becoming more common. You need to put file names that contain blanks or other special characters inside quote characters, e.g.

$ ls 'My Documents'

Most commands use the following syntax format where the command name is first on the line (at the left) and all the option arguments must precede (be to the left of) all other parameter arguments such as file names:

$ commandname  -options...  --options...  parameters...  [Enter]

Example:
$ ls  -ail  --full-time  /home/user  foo/bar [Enter]

Above, ls is the command name, -ail and --full-time are four options, and /home/user and foo/bar are two pathname arguments (one absolute pathname and one relative pathname). You must use the [Enter] key to submit the command to the shell. You can use [Enter] anywhere in the command line; it doesn’t have to be at the end. The [Enter] key is never shown in documentation.

As shown above, the command name and each argument have to be separated by one or more spaces. The number of additional spaces doesn’t affect the command – one space is as good as a hundred.

Usually the first piece of text on the left of a shell command line is taken to be a command name, e.g. date, ls, echo, figlet. A command name may be followed by optional space-separated arguments. Arguments may be command options followed by parameters for the command.

5.1 Verb AND ObjectIndexup to index

Most Unix/Linux commands need both a verb (what to do – the command name) and an object (what to do it on – a command argument). Here are some command line examples of verb and object:

$ cat /etc/passwd               # argument to cat is a file name
$ mail idallen@idallen.ca       # argument to mail is an email address
$ kill 1234                     # argument to kill is a process number

The following incorrect attempts at command lines are wrong:

$ /etc/passwd            # missing VERB; what are you trying to DO with this file?
$ rm                     # missing OBJECT; remove WHAT file?

Remember to tell Unix/Linux both what you want to do and to what object you wish to do it. Here are examples of different command names that all do different things to the given file name object:

$ cat /etc/passwd
$ less /etc/passwd
$ wc /etc/passwd
$ sort /etc/passwd
$ rm /etc/passwd          # fails: ordinary users do not have permission to do this

6 Most (but not all) commands take what as arguments?Indexup to index

Most – but not all – Unix/Linux commands take pathnames as arguments. Much of what people do on their computers is manipulate files and directories. This is why shells have GLOB pattern matching (wildcard) characters.

Pathnames are names that might be file names, or directory names, or some other file system object. (Unix/Linux has names that are neither files nor directories, e.g. the /dev/null pathname is a character special device.)

The shell has GLOB (wildcard) features to make matching multiple pathnames easier. e.g. try the command line: $ echo /bin/*

7 Using a Shell inside a TerminalIndexup to index

Shells run inside “terminals”, usually software terminal windows on modern GUI systems. These terminals recognize some useful unprintable control characters that aid in typing text, including command lines.

7.1 Syntax of Control Characters – ^A ^B ^C etc.Indexup to index

The documentation syntax ^C stands for the single unprintable control character CONTROL-C, also written as CTRL-C or perhaps [CTRL-C].

To generate this special character from your keyboard, hold down the CTRL key and simultaneously push the letter C key (upper or lower case; doesn’t matter).

This syntax and method applies to all the control characters, e.g. ^A, ^B, ^C, ^D, etc.

7.2 Common control characters – ^C ^D ^U ^W ^ZIndexup to index

Shells run inside terminals or terminal emulator programs such as Linux xterm, Windows PuTTY, or OSX Terminal. Here are some useful control characters that you can use in a terminal connection to a Unix/Linux machine:

^C - CTRL-C - *interrupt* (terminate) a running foreground program
^D - CTRL-D - send *End-of-File* (EOF) from your keyboard (end input)
^U - CTRL-U - erase input characters back to the start of the prompt
^W - CTRL-W - erase just the last word you typed (repeat to erase next word)
^Z - CTRL-Z - temporarily *suspend* a program; restart by typing "fg"

The above control characters are properties of the terminal, so they work for almost any program that reads your keyboard, not just shells. You can view or change any of the above characters using the stty command.

7.3 Interrupting programs with ^CIndexup to index

The usual way to interrupt a command that is executing is by using the ^C character. For example, at a shell prompt enter the command ping localhost, followed by the Enter key, and then use ^C to interrupt the ping program and get the shell prompt back:

$ ping localhost
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_req=1 ttl=64 time=0.046 ms
64 bytes from localhost (127.0.0.1): icmp_req=2 ttl=64 time=0.057 ms
64 bytes from localhost (127.0.0.1): icmp_req=3 ttl=64 time=0.055 ms
64 bytes from localhost (127.0.0.1): icmp_req=4 ttl=64 time=0.055 ms
64 bytes from localhost (127.0.0.1): icmp_req=5 ttl=64 time=0.061 ms
^C
--- localhost ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3996ms
rtt min/avg/max/mdev = 0.046/0.054/0.061/0.010 ms
$ 

Most Unix/Linux programs will terminate when they receive the ^C interrupt sent from your keyboard. When the command terminates, the shell issues another prompt for more input.

7.4 Suspending programs using ^ZIndexup to index

The ^Z character temporarily suspends a running program. The program doesn’t terminate; it just stops executing and you get the shell prompt back so that you can enter other command lines.

You can get a list of suspended programs in your shell using the command jobs and you can resume a suspended program by typing fg:

$ ping localhost
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_req=1 ttl=64 time=0.046 ms
64 bytes from localhost (127.0.0.1): icmp_req=2 ttl=64 time=0.057 ms
64 bytes from localhost (127.0.0.1): icmp_req=3 ttl=64 time=0.055 ms
64 bytes from localhost (127.0.0.1): icmp_req=4 ttl=64 time=0.055 ms
64 bytes from localhost (127.0.0.1): icmp_req=5 ttl=64 time=0.061 ms
^Z
[1] + Stopped                    ping localhost

$ echo "The ping program is suspended, waiting for fg"
The ping program is suspended, waiting for fg

$ jobs
[1] + Stopped                    ping localhost

$ fg
ping localhost
64 bytes from localhost (127.0.0.1): icmp_req=6 ttl=64 time=0.061 ms
64 bytes from localhost (127.0.0.1): icmp_req=7 ttl=64 time=0.051 ms
64 bytes from localhost (127.0.0.1): icmp_req=8 ttl=64 time=0.062 ms
64 bytes from localhost (127.0.0.1): icmp_req=9 ttl=64 time=0.052 ms
^C
--- localhost ping statistics ---
9 packets transmitted, 9 received, 0% packet loss, time 13345ms
rtt min/avg/max/mdev = 0.046/0.054/0.061/0.010 ms
$ 

Be very careful about suspending programs using ^Z. Suspending is not the same as killing or terminating!

If you accidentally suspend a program (it says Stopped), you can immediately type fg into the shell to restart it again.

Suspending a text editor (such as vim) using ^Z is a dangerous thing to do, since having multiple editors all editing the same file can cause data loss if you write out an old version of a file on top of a newer one. Do not suspend text editors unless you know what you are doing!

7.5 Bonus Control Characters – ^L ^RIndexup to index

The bash shell also recognizes some very useful control characters:

^L - CTRL-L - clear your screen; move current command line to top
^R - CTRL-R - search your current shell command history for a string

These depend on running the bash shell; they are not part of your terminal.

The ^R is very useful for searching backwards in your shell command history to find a previous command containing the search string you type. You can type multiple ^R to keep searching backwards for more command lines with the same search string.

8 Interrupting Processes and signalling End-of-File – ^C and ^DIndexup to index

The terminal emulator treats the default control characters ^C (interrupt) and ^D (end-of-file) specially:

8.1 Interrupting a Process – ^CIndexup to index

If you type a command that keeps running and doesn’t stop by itself (e.g. the ping command) and decide you want to interrupt (terminate) it, the terminal emulator lets you do this by typing a character that sends an Interrupt signal to the process.

To send an interrupt signal to a process that is running on your terminal, use the Interrupt Character, usually CTRL-C (^C). (You can program a different character; sometimes DEL is used.) The Interrupt usually throws away any pending input/output for the process and causes the process to terminate abnormally. Use it with caution.

Interrupting a process usually terminates the process. Whatever the process was doing is left incomplete and unfinished. Buffers are not flushed; files will be incomplete. An interrupted process did not finish.

8.2 Signalling End-of-File (EOF) from the keyboard – ^DIndexup to index

When a program reads a file, it naturally stops reading when the end of the file is reached – this is called End-of-File or EOF. When the program is reading from your keyboard, it doesn’t know when to stop reading unless you tell it by typing the EOF character into the terminal.

Your EOF character, signalling EOF from your keyboard, is usually CTRL-D (^D). You must type this at the beginning of a line (or type it twice). (You can program a different character; but, it is almost never done.)

Typing the EOF character tells the process reading your keyboard that you are finished typing at your terminal (EOF); but, it does not interrupt or terminate the process. The process will finish reading your keyboard and then continue with whatever it is doing. Often, that means producing some output and then exiting successfully.

(Note: Unlike DOS/Windows systems, ^D does not actually send any character to the process reading your keyboard – it simply tells the process that EOF has happened. You cannot put ^D inside a file and have it serve as an EOF. The ^D only works when typed from a keyboard inside a terminal.)

8.3 Examples of ^C and ^DIndexup to index

Here is an example showing how ^C interrupts a program before it finishes, but ^D signals an EOF and lets the program finish. In both examples below we use shell Output Redirection to save the output of the sort command into a file named out:

8.3.1 ^C Interrupting a Process (does not finish)Indexup to index

$ sort >out                              # sort keyboard input into file out
enter some lines of text and then
use ^C CTRL-C to interrupt the program
and you will get an empty file
^C
$ cat out                                # display content of file out
$                                        # nothing: empty file

8.3.2 ^D Signalling EOF (end-of-file) lets process finishIndexup to index

$ sort >out                              # sort keyboard input into file out
enter some lines of text and then
use a ^D CTRL-D End-Of-File EOF to signal end of file
and sort will sort your data and finish
^D
$ cat out                                # display content of file out
and sort will sort your data and finish
enter some lines of text and then
use a ^D CTRL-D End-Of-File EOF to signal end of file
$

Typing ^C interrupts a process, which usually terminates it abnormally. Any pending output from the process is usually lost.

Typing ^D signals EOF from your keyboard to the process, so the process knows to stop reading your keyboard and finish what it was doing.

9 Getting Out of Programs – exit, quit, etc.Indexup to index

Different Unix/Linux programs have different ways of ending or exiting from them, because they were written by different people. If you’re stuck inside a program and can’t get out to the shell again, try various things such as these:

h or help or ?                    # find out how to exit the program!

q or Q or quit                    # used to exit "less" and "more"
:q!                               # used in VI to exit without saving
x or exit
^D                                # CTRL-D - send End-Of-File - EOF
^C                                # CTRL-C - send Interrrupt signal
^\                                # CTRL-backslash - send Quit signal
logout
bye
ESC                               # the ESC or Escape key
.                                 # a period - used in some mail programs

One of the above usually works. Sometimes you can use ^Z (CTRL-Z) to suspend the process temporarily, and then type "kill %%" to terminate it. (Remember to terminate it, or it will sit there suspended, waiting forever. A process stopped with ^Z still has all its files open.)

10 Nested Shells inside ShellsIndexup to index

Your default login shell on Linux is the Bourne-Again shell (bash). You can call up other shells by name if they are installed: sh, ksh, bash, csh, tcsh, etc. Not all shells may be installed on your system.

You can call up a shell inside a shell (a “nested” shell) by typing its name at the shell prompt (e.g. type sh or even another bash). The new copy of the shell will take over reading your keyboard until you exit from it again and the previous shell returns to read your keyboard:

bash-4.3$ echo this is my login shell
this is my login shell

bash-4.3$ sh
$ echo this is a nested sh shell
this is a nested sh shell

$ exit
bash-4.3$ echo I am back to the login shell
I am back to the login shell

To exit from a shell type: exit

When you exit from the first shell in a terminal, your login shell, either the terminal window closes (if using a terminal window), or, if you are logged in directly to a Unix/Linux console, you log out from Unix.

You can see a video on sub-shells. (You may need to download this MP4 file to your computer to see the video.)

Another way to exit a shell is to type your EOF character. (Your EOF character is usualy CONTROL-D.) Some shells can be told to ignore EOF.

Remember: all Unix programs (should) have manual pages!

$ man sh
$ man bash
$ man ksh
$ man csh
$ man tcsh

Each new shell is another Unix process. To get a list of your current processes in the current terminal, use: ps

11 Shell interactive command and pathname completionIndexup to index

If you type part of a pathname on a comand line and then push the TAB key, many shells will attempt to complete the pathname typing for you.

$ echo /etc/pas<TAB>                   # shell will complete as /etc/passwd

If the pathname cannot be completed unambiguously (there is more than one name that starts with that prefix), the shell will show you a list of possible completions if you push the TAB key again:

$ echo /etc/pas<TAB><TAB>
passwd      passwd-

11.1 Command name completionIndexup to index

Similarly, if you type part of a command name and push the TAB key once or twice, the shell will complete and then list all the possible commands in your PATH that start with that prefix:

$ mkd<TAB><TAB>
mkdir        mkdiskimage  mkdosfs

Use the shell to do your typing for you – learn how to use the TAB key! (The shell always spells your pathnames correctly!)

12 Shell “history” of previous commandsIndexup to index

Most shells keep a record (“history”) of the commands you type. The history is often saved in a file when the shell exits, and is restored when you next log in. You can see the history list using the built-in history command.

Many shells allow you to use the UP-ARROW and DOWN-ARROW keys to move up and down in the command history. (Some shells use other key sequences, such as ^P and ^N.) You can use the arrow keys to move up to the command you want to re-execute, modify it as needed, then use the [Enter] key (anywhere on the line) to execute it.

Some shells let you select items from the history list by number (e.g. !123). See the History section of your shell’s man page.

Each shell process has its own separate command history. Starting a new shell (a sub-shell) will start an independent history any may not pick up the history of another shell unless that other shell saved it and your new shell loads it.

13 What is a “Bourne” shell? What is a “C” shell?Indexup to index

The shells with names sh, ksh, zsh, and bash (the “Bourne” shells) all have a common ancestry. They are all derived from the original Bourne shell sh written in the 1970s by Stephen Bourne. The programming features of these shells (if statements, for loops, etc.) all look and work the same way. This is the best shell family to study.

The shells named csh and tcsh are called the C shells and have a different history dating back to Bill Joy at Berkeley in the 1980s. Their syntax for programming is not the same as the Bourne shells. We do not cover the C shell syntax in this course; these shells are notoriously buggy and my 2002 list of tcsh bugs is still distributed with the shell.

C Shell Bugs: search the Internet for:   c shell bugs

My list of C Shell Bugs: use the above links or search the Internet for:   c shell bugs "idallen"

14 Shell command line alias troubleIndexup to index

Watch out for “helpful” system administrators that define aliases for your shells when you log in. (This is true on most versions of Unix/Linux.) The aliases may mislead you about how Unix commands actually work.

For example, the rm command does not prompt you for confirmation when removing a file. On some systems, when you log in, rm has been made to be an alias for rm -i, which does prompt. If you become accustomed to the behaviour of the alias, you will be in serious trouble when you move to a system that does not define the alias.

To avoid pre-defined aliases, sometimes you can start up a fresh copy of the shell that has no aliases defined:

$ alias
[...many aliases may print here...]

$ bash
bash$ alias
[...no more aliases here...]

The other thing you can do is execute unalias -a to remove all your aliases for the current shell. You can put this into your shell start-up file (e.g. .bashrc) to do it every time you start a new shell.

To define your own aliases, look up “aliases” in a Unix/Linux resource, e.g.:

$ alias dir='ls -lF'
$ alias cd..='cd ..'
$ alias mali=mail

Aliases defined in the current shell are not saved when the shell exits. You must put your own alias definitions in a shell start-up file to have them restored between sessions (e.g. put them into your .bashrc file).

Each shell has its own separate aliases. Starting a new shell does not get the aliases defined in other shells.

Author: 
| Ian! D. Allen, BA, MMath  -  idallen@idallen.ca  -  Ottawa, Ontario, Canada
| Home Page: http://idallen.com/   Contact Improv: http://contactimprov.ca/
| College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/
| Defend digital freedom:  http://eff.org/  and have fun:  http://fools.ca/

Plain Text - plain text version of this page in Pandoc Markdown format

Campaign for non-browser-specific HTML   Valid XHTML 1.0 Transitional   Valid CSS!   Creative Commons by nc sa 3.0   Hacker Ideals Emblem   Author Ian! D. Allen