% Unix/Linux Processes, Jobs, background, foreground, kill, killall % Ian! D. Allen -- -- [www.idallen.com] % Fall 2015 - September to December 2015 - Updated 2019-03-14 01:32 EDT - [Course Home Page] - [Course Outline] - [All Weeks] - [Plain Text] Unix/Linux Processes ==================== Programs are loaded into memory and run as individual **processes**. The operating system arranges to time-share the CPU(s) among all the processes so that each process appears to run independently and concurrently. Some programs can themselves split into multiple **threads**, each of which gets its own CPU resources. The operating system has methods to increase or decrease the amount of CPU or Input/Output (I/O) resource a process can consume. System administrators may adjust process priority to reduce the impact a very resource-intensive process is having on the system. A process usually runs with ("is owned by") the single userid and multiple group [Permissions] of the person who started it. (Recall that when you log in, you have one user permission and multiple group permissions.) Only the user who owns the process (and, of course, the `root` super-user) can terminate or adjust the priority of a process. Every process except the very first Unix process starts by **forking** itself from some **parent** process. A parent process may fork off multiple child processes, which may themselves fork off more child processes. This gives processes a tree-like hierarchical structure of multiple generations of parents and children. Each process has a numeric *process ID* or **PID** and a *parent process ID* or **PPID**. You can see these two numbers using the `l` option to the process listing command `ps`: $ ps l 18347 F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND 4 0 18347 18345 20 0 22120 2192 wait S pts/2 0:00 /bin/bash When a parent process exits (terminates) before its children, the children are orphaned and are passed to be children of Unix process Number One (1), traditionally named `init`: $ ps l 1 F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND 4 0 1 0 20 0 24684 2120 poll_s Ss ? 0:03 /sbin/init The **init** name may be a link to another program, such as `systemd`: $ ls -l /sbin/init lrwxrwxrwx 1 root root 20 Jul 20 13:13 /sbin/init -> /lib/systemd/systemd Process listing and control -- `ps` and `kill` ============================================== You can usually list and see the process IDs of *all* processes on a Unix/Linux system, but you can only *control* (terminate, pause, or adjust the priority of) your own processes (processes with the same userid as you). Process List -- `ps` -------------------- The Linux `ps` command options are a crazy mix of the BSD `ps` command options (no dashes), the incompatible SystemV UNIX `ps` command options (with dashes), plus some GNU extension options (with double-dashes). From `man ps` on Linux: This version of ps accepts several kinds of options: 1 UNIX options, which may be grouped and must be preceded by a dash. 2 BSD options, which may be grouped and must not be used with a dash. 3 GNU long options, which are preceded by two dashes. Below are some useful command lines. The big ones to remember are the two that do "all processes, all users, long format, full wide listing". $ ps # BSD: your processes in current terminal window, numeric UID $ ps u # BSD: your processes in current terminal window, text userid $ ps x # BSD: all of your processes in all windows, everywhere $ ps xl # BSD: all your processes, long format $ ps xlww # BSD: all your processes, long format, full wide listing $ ps ax # BSD: all processes for all users, numerid UID $ ps uaxww # BSD: all processes for all users, text userid, full wide listing $ ps laxww # BSD: all processes, all users, long format, full wide listing $ ps f # BSD: ascii art hierarchical display (forest) $ ps -f # UNIX: your processes in current terminal window, text userid $ ps -e # UNIX: all processes, numeric UID $ ps -ef # UNIX: all processes, text userid $ ps -elww # UNIX: all processes, long format, wide listing, numeric UID $ ps -efww # UNIX: all processes, same but with text userid instead of numeric Other process listing: `pstree`,`top`, and `atop` ------------------------------------------------- Other useful process-listing commands for a sysadmin: $ pstree # summary process tree command; similar to "ps axf" but less detail $ top # show all currently active processes, updated every few seconds $ atop # show all currently active processes, updated every 10 seconds Review ------ - Q: how do you see all your processes? - Q: how do you see all processes for all users? - Q: how do you see all active processes in real time? Terminate or Pause process -- `kill` and `killall` -------------------------------------------------- Processes can be paused (stopped) or terminated by their owners (and `root`) by sending a **signal** to the process id using the poorly-named `kill` command: kill [ -SIG ] pid ... # SIG is the signal name to send Most systems have an actual executable `kill` program in one of the `bin` directories, but `kill` is usually also built-in command to each shell so that it can use some shell-specific syntax. For a list of signals that you can send to processes, type `kill -l`. Most modern shells allow signals by name, e.g. `kill -KILL` instead of the old numeric way `kill -9` where the numbers didn't always mean the same thing on different systems. The default signal sent by `kill` is `TERM` (**Terminate**), and it allows a process to clean up before terminating. (For example, `vim` will save its buffers in a `*.swp` recovery file before exiting.) If `TERM` doesn't kill a process, other signals to try are `HUP` and finally `KILL`: $ kill 123 # default is TERM signal - same as kill -TERM 123 $ kill -HUP 123 # stronger signal (HUP = Hangup) $ kill -KILL 123 # or kill -9 123 - strongest signal - avoid using it The safest way to signal a specific process is by using `ps` to find the process **PID** and then using `kill` to send a signal to that **PID**. Some systems come with a `killall` command that can also kill processes by name (see the man page before you try this): $ killall [ -SIG ] name ... # SIG is the signal to send to name Note that the `killall` command is dangerous, since by killing processes based on name it might match unintended processes with the same name, or (if run as `root`) processes with the same name run by other users. (If an attacker decides to name an attacking program `bash`, then `killall -9 bash` would not be a good thing for a `root` user to type on the system.) Review ------ - Q: how do you terminate a process? - Q: how do you terminate a process using the strongest signal? - Q: why is the `killall` command dangerous to use, especially by `root`? Unix/Linux Shell Job Control and Background Processes ===================================================== Normally when you run a command, the shell waits until the command is finished before it prompts you to enter the next command. This waiting for the process to finish is called a shell "foreground" process or job. The foreground job is attached to your keyboard and screen (unless you have redirected input or output): $ sleep 10 # shell waits until sleep finishes before next prompt $ You can interrupt most foreground jobs using the `^C` (**Ctrl-C**) key that sends a `SIGINT` (Interrupt) signal to the foreground process being run: $ sleep 10 ^C $ If you want to run a command "in the background", without having the shell wait for it to finish, end the command with an ampersand `&`, e.g. $ sleep 99 & [1] 18920 $ The `&` starts the job without requiring the shell to wait for it. The shell gives you a prompt for your next command immediately. The background command runs independently; it can't be interrupted by the `^C` interrupt signal from the keyboard. Modern Unix/Linux shells often provide some added syntax that lets you manage background processes that are forked from your current shell. Processes forked from the current shell are called the `jobs` of the shell. You can see these jobs using the shell built-in command of the same name: $ jobs # show child processes *only of the current shell* [1]+ Stopped sudo -s (wd: ~) $ jobs -l # use -l to show the process IDs as well [1]+ 12345 Stopped sudo -s (wd: ~) Jobs of the current shell can be specified by short job numbers instead of process IDs to shell built-in commands such as `kill`: $ kill %1 # send SIGTERM to job number 1 of this shell Moving jobs background / foreground ----------------------------------- If you have already typed a command and forgot to use the `&`, you can put a foreground job into the background by typing `^Z` (**Ctrl-Z**) to send the job a signal to pause (stop or suspend) the job, followed by `bg` to put it into the background: $ sleep 99 ^Z [1]+ Stopped sleep 99 $ bg [1]+ sleep 99 & You can bring a background job into the foreground, so that the shell waits for it again (and so `^C` can interrupt it) using `fg`: $ jobs [1]+ Running sleep 99 $ fg sleep 99 Output from / input to background jobs -------------------------------------- Any output produced by a background job will appear on your terminal screen, mixing on your screen with whatever else you are reading or typing at the time. In the example below, the user started to type an `echo` command line right after starting the background job, and you can see the output from the background job appearing on the screen in the middle of typing the command: $ ( sleep 3 ; date ) & # sleep a bit then run date, in background [1] 17619 $ echo my mother is aSat Nov 8 04:07:30 EST 2014 nice person my mother is a nice person [1]+ Done ( sleep 3; date ) $ You can type `^L` at the shell to clear your screen of output from background jobs, and `^L` also works to clear your screen in `vim` command mode as well as in `less` and `more`. If a background job tries to read from your keyboard, the system will pause (stop or suspend) that job until you make it a foreground job again: $ wc & [1] 1538 $ jobs -l [1]+ 1538 Stopped (tty input) wc $ fg wc The wc command is now foreground and reading my keyboard. ^D 1 10 59 $ The shell ensures that only one job is "foreground" and reading your keyboard. All other jobs that try to read the keyboard will be paused (`Stopped`). When the shell isn't running any commands, the shell has your keyboard. How shell exit affects background jobs -------------------------------------- - Jobs running in the background when the shell exits are left running. - Jobs that are paused (`Stopped`) when the shell exits are terminated. Sending signals to jobs and processes ------------------------------------- Your shell can send signals, including stop and termination signals, to jobs of the *current* shell using job numbers instead of process numbers: $ kill %1 [1]+ Terminated sleep 99 To send signals to processes or jobs that are not started from the current shell, you first need to use `ps` to find their process numbers. For full details on jobs, see the heading **JOB CONTROL** in the man page for `bash(1)`. Keyboard signals -- `^C ^Z ^\` ------------------------------ Your terminal is configured to turn some characters you type into signals sent to foreground processes, e.g. `^C` sends an **Interrupt** signal to the current foreground process, just as if `kill -INT` had been used: $ sleep 10 ^C $ Other control characters send other signals to foreground jobs: Ctrl-C - ^C - send Interrupt signal (SIGINT) Ctrl-Z - ^Z - send Stop signal (SIGSTOP) Ctrl-\ - ^\ - send Quit signal (SIGQUIT) > You can change which characters send these signals using the `stty` > command. > > Be careful -- you can make *any* character send one of these signals, even > common letters. It is difficult to type the `stty` command to fix your > terminal when you accidentally map the letter `s` to the `SIGINT` signal! Signal Hierarchy -- start small and work up ------------------------------------------- Informally, we will order some of the more common signals based on how "strong" they are. The "strongest" signal is, of course, SIGKILL, which cannot be caught or ignored. > If SIGKILL won't end a process, the process is likely stuck on some I/O > operation inside the system. Perhaps the system will eventually clear the > I/O error, allowing the process to terminate, or perhaps you will need to > reboot the system to unjam the I/O system. Perhaps the most common, the "weakest" signal, is SIGINT, the interrupt signal that is sent when you type `^C` at the keyboard. Both the shell and the VIM editor ignore SIGINT. In between these two extremes are some other common signals that are less common and stronger than SIGINT but not as deadly as SIGKILL: SIGTERM - Terminate (the default signal for kill) SIGHUP - Hangup (connection has dropped) Your shell will ignore SIGTERM, but not SIGHUP. When you want to terminate a process, you should start with the weakest signal and, if necessary, work up to the strongest. If you start with the strongest signal, SIGKILL, the process will be terminated immediately and won't be able to flush its buffers or save its state. For example, if you send SIGTERM or SIGHUP to the VIM editor, the editor will save its state in a `.swp` file before it exits, allowing you to recover your edit session later. If you send SIGKILL to VIM, nothing is saved and everything is lost. To terminate a process, start with SIGINT or SIGTERM, and if that doesn't work, try SIGHUP, and finally use SIGKILL: 1. `kill -INT` 2. `kill -TERM` 3. `kill -HUP` 4. `kill -KILL` Privileged processes -- setuid and setgid ========================================= Some privileged **setuid** or **setgid** programs are run with the permissions of the userid and/or group of the *file* containing the program, instead of the permissions of the user and/or group of the person *running* the process. You can see the **setuid** and **setgid** bits set as an `s` in the owner or group part of the permissions in an `ls -l` listing: $ ls -l /usr/bin/passwd /usr/bin/crontab -rwsr-xr-x 1 root root 42824 Sep 12 2012 /usr/bin/passwd -rwxr-sr-x 1 root crontab 35896 Jun 19 2012 /usr/bin/crontab The above `passwd` command will start up and run with the permissions of the `root` user that owns the file, not with the permissions of the user who starts the command. The `crontab` command will run with group `crontab` permissions (the group of the file), not the group permissions of the user starting the program. Since these privileged processes are running as other users, you won't be able to send them signals (since you aren't their owner). -- | Ian! D. Allen, BA, MMath - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Course Home Page]: .. [Course Outline]: course_outline.pdf [All Weeks]: indexcgi.cgi [Plain Text]: 600_processes_and_jobs.txt [Permissions]: 500_permissions.html [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/