6.1 GRUB Device and file Naming ConventionIndex

The syntax of pathnames to files differs among operating systems. The GRUB bootloader is a mini operating system that runs before the real operating system is even running. GRUB has its own pathname syntax to access files on disk.

A full GRUB path name specification is a GRUB device name (often a partition name) followed by a file system pathname inside that partition (relative to the start of the partition).

For example the GRUB pathname (hd0,0)/grub/grub.conf can be divided into the (hd0,0) piece that names a disk (disk 0) and partition (partition 0) and the /grub/grub.conf piece that is a pathname relative to the start of that partition. This GRUB syntax is similar to DOS/Windows and its use of drive letters to name partitions.

• GRUB device names are identified by enclosing parentheses ()
• Hard disks are named hd, e.g (hd0), (hd1)
• Floppy disks are named fd, e.g. (fd0), (fd1)
• Disks and Floppies are numbered sequentially, starting at 0 (ZERO!)
• Partition numbers are given after commas, e.g. (hd0,0), (hd0,1)
• Partitions are also numbered sequentially, starting at 0 (ZERO!)
  • WARNING! Linux numbers disks starting at a! GRUB starts at ZERO.
  • WARNING! Linux numbers partitions starting at 1 (ONE)! GRUB starts at ZERO.
  • Linux sda1 is GRUB (hd0,0)
  • Linux sdc2 is GRUB (hd2,1)
• Examples:
  • (hd0) – First recognized hard disk (any type: SCSI, ATA, USB, etc.); usually corresponds to Linux whole-disk device name /dev/sda
  • (fd0) – First recognized floppy diskette; usually corresponds to Linux /dev/fd0 or Windows “A:”
  • (hd0,0) – First partition on first recognized hard disk; usually corresponds to Linux /dev/sda1 or Windows “C:”
  • (hd1,4) – First logical partition on second recognized hard disk; usually corresponds to Linux partition /dev/sdb5
• The GRUB root command can pre-set the device name for all following pathnames that lack one, similar to the function of a Linux “current directory”
  • e.g. “root (hd0,2)” followed by “kernel /linux” is the same as “kernel (hd0,2)/linux”

6.2 A sample GRUB Legacy configuration file (version 0.97)Index

All versions of GRUB use a configuration file under /boot/grub to display a menu of alternative operating systems to boot. Without a configuration file, GRUB starts in interactive mode and you have to specify and type everything yourself. With a configuration file, you can select from a menu list of choices.

Do not move or remove the GRUB configuration files! GRUB expects the files to be in a particular location on disk, and moving them may cause GRUB to fail to find them, requiring a re-install of GRUB. You can edit the GRUB configuration files in-place as long as you do not delete or move them.

Boot Menu Entry

Every title keyword in a GRUB configuration file starts a paragraph of lines that defines a “boot menu entry”. For most Linux kernels, the boot menu entry is four lines long. The CentOS GRUB configuration file grub.conf only contains one four-line Linux boot menu entry to start. (Software updates may add more, but you were told not to install any software updates.)

In GRUB Legacy, each operating system choice in the configuration file is called a boot menu entry and it starts with a title keyword followed by a descriptive name followed by some lines that say how to boot that particular operating system.

Here is a simple one-entry GRUB Legacy /boot/grub/grub.conf file:

# Sample GRUB grub.conf file (GRUB Legacy version 0.9x)
default=0
timeout=30
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
#hiddenmenu

# This sda disk has ROOT on partition sda1 and no separate /boot
# partition, so GRUB pathnames must start with /boot
title CentOS (2.6.32-431.el6.i686)
    root (hd0,0)
    kernel /boot/vmlinuz-2.6.32-431.el6.i686 root=/dev/sda1
    initrd /boot/initramfs-2.6.32-431.el6.i686.img

Comments in this file start with # and extend to the end of line.

• The part of the file before the first title section is for general GRUB configuration options – see below for a fuller description of the keywords used:
  • default: which title section to boot when the time-out happens
  • timeout: how long the menu waits before selecting the default
  • splashimage: a pretty picture graphic to put behind the GRUB menu
  • hiddenmenu: (disabled) hide the GRUB menu during the boot process
• The boot menu entry sections for Linux kernels have these basic keywords:
  • title: starts a boot menu entry and gives it a name
    • You can have many boot menu entries for many different operating systems in the same configuration file
  • root: lets you set a device prefix to be used for all following GRUB pathnames that lack a device prefix, similar to cd in the shell
    • this is not the Linux ROOT; this is a GRUB device and uses GRUB device names such as (hd0,0)
    • this device name is prefixed to all pathname references that lack a device prefix
    • in the example, the prefix is (hd0,0) and the incomplete path /boot/vmlinuz-2.6.32-431.el6.i686 has the device (hd0,0) added to the start of the path as if the path has been given as (hd0,0)/boot/vmlinuz-2.6.32-431.el6.i686
  • kernel: the Linux kernel image file followed by kernel options
  • initrd: the initial RAM disk file to be used by the kernel at boot time

• Because this system has no separate boot partition, the GRUB pathname (hd0,0)/boot/vmlinuz-2.6.32-431.el6.i686 corresponds to Linux file system pathname /boot/vmlinuz-2.6.32-431.el6.i686:

  $ ls -l /boot/vmlinuz-2.6.32-431.el6.i686 
  -rwxr-xr-x. 1 root root 3917440 Feb 21  2013 /boot/vmlinuz-2.6.32-431.el6.i686

6.3 Useful grub.conf keywordsIndex

• Global GRUB configuration options (used before any title sections):
  • default n: (optional) The nth title boot menu item, starting at 0, will be booted. Otherwise, the first (zeroth) title item is booted.
    • counts from zero – the first boot menu entry is number zero!
  • timeout 30: (optional) Number of seconds before booting the default boot menu entry.
  • hiddenmenu: hide the GRUB menu during the boot process (pretty boot)
• Individual menu options (used within the title sections):
  • title: Menu title of this boot menu entry, visible at boot time
  • kernel: Location of the Linux kernel file, followed by options
  • initrd: Location of the Linux kernel initial RAM disk file to be used by the kernel

Note: To find out about additional GRUB commands, type help at the GRUB shell prompt, and see the above Resources.

6.4 Effect of separate /boot partition on GRUB pathnamesIndex

GRUB configuration files are stored under directory /boot/grub in Linux. /boot/grub is the Linux pathname, not necessarily the GRUB pathname, because GRUB pathnames are composed of a partition name followed by a pathname inside the partition, e.g. (hd0,0)/some/path/name where /some/path/name is relative to the start of the partition.

Since partitions are mounted on directories in Linux, and those directory pathnames prefix the names inside the partition for Linux pathnames, a GRUB pathname and a Linux absolute pathname will always differ, except for the one partition that is mounted on ROOT (because prefixing an absolute pathname with / (ROOT) doesn’t change it).

Some examples will make this clearer:

# NOTICE:  You do not have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /, eg.
#          root (hd0,0)
#          kernel /boot/vmlinuz-version ro root=/dev/sda1
#          initrd /boot/initrd-[generic-]version.img

6.5 Installing GRUB – grub-installIndex

Your Linux installation already installed GRUB for you. If you have to re-install GRUB, you may find or be able to install the Linux grub-install script that will do the installation for you. (RTFM)

6.6 Installing GRUB using the GRUB shell – OPTIONALIndex

This section is OPTIONAL. It’s included here for completeness, but you won’t be asked to do any of this in this course.

You can also try to install GRUB manually, if the GRUB setup files are already stored under /boot/grub:

• As root, type grub at the command line: this will load GRUB and display the GRUB shell prompt grub>
• Type setup with two GRUB pathname arguments: the boot sector installation device and the device containing the GRUB setup files
  • the GRUB setup files (e.g. grub.conf) must be on the given partition
• To put GRUB into the MBR of the first disk:   setup (hd0) (hd0,0)
• Exit GRUB with the quit command when done

Most versions of the GRUB shell have TAB command completion, where you can type part of a device or pathname and GRUB will give you all the possible completions. The shell command-line version of GRUB is broken, and does not have this feature. The GRUB that runs at boot time is not broken and does have the TAB completion feature.

BE CAREFUL. GRUB does not ask you for confirmation! If you type the wrong thing, you will overwrite your boot sector with garbage and be unable to boot your system. You will need to boot a “live” CD and repair.

6.7 Booting without a GRUB menu – interactive GRUB commandsIndex

If the grub.conf configuration file is not present or has been moved, GRUB cannot display a boot menu. In this case it will display the GRUB interactive shell, similar to what you get by typing grub at a Linux command line.

The interactive GRUB shell has its own prompt (grub>) and its own built-in commands, some with the same names as BASH shell commands. Be clear on when you are typing into the GRUB shell and when you are typing into BASH. The same command name may do different things, depending on whether GRUB executes it or BASH executes it.

Most boot-time versions of the GRUB shell have TAB command completion, where you can type part of a device or pathname, push TAB, and GRUB will give you all the possible completions. The shell command-line version of GRUB is broken, and does not have this feature working. The GRUB that runs stand-alone at boot time is not broken and does have the TAB completion feature.

Inside the GRUB shell, you can type help for a partial list of commands you can use. Some examples:

• cat – display a text file
• geometry – display information about disk partitions
• find – find which device (partition) contains a pathname
• md5crypt – prompt for and encrypt a password

Remember that all pathnames must be GRUB pathnames, not Linux pathnames!

At the GRUB shell prompt, GRUB commands can be entered to query the system, see text files, and load and then boot a kernel image. Examples:

grub> find /grub/grub.conf
grub> cat (hd0,0)/grub/grub.conf
grub> kernel (hd0,0)/vmlinuz ro root=/dev/sda2
grub> initrd (hd0,0)/initrd
grub> boot

Note: The GRUB shell can also be accessed by following boot-time directions (usually by pressing the letter c) when the GRUB menu is presented. [Esc] brings you from the GRUB shell back to the menu. Pressing e allows you to edit boot menu entries of grub.conf. Pressing b boots the edited menu item. Some versions of GRUB allow typing a to go directly to a GRUB kernel line and edit the kernel options.

7.1 Booting Single User Mode (Maintenance Mode)Index

Single-user mode is a “half-up” state used for system repair and maintenance (especially for resetting the root password!). The system only brings up a minimal number of services (often none). The GUI is not started. Networking may not be enabled. Not all disks may be mounted. No login prompt is used; the console terminal gets a shell running as the root user.

To boot into single-user mode, add the word single as an option to the end of a kernel line in a GRUB boot menu entry and then boot that entry, e.g.:

kernel (hd0,0)/vmlinuz ro root=/dev/sda1 single

The system will boot directly into a root shell with no need to log in. Exiting this root shell will leave single-user mode and the system will finish booting into the default Run Level as recorded in the /etc/inittab file.

7.2 Booting into a different Run LevelIndex

(Run Levels are summarized below.) To start Linux with a different Run Level than the default, add the Run Level digit to the end of the kernel options line during boot, e.g. to boot Run Level 3 (which on CentOS means boot multi-user without the X11 GUI window system), use the GRUB menu to add a space and the digit 3 to the end of the kernel options:

kernel (hd0,0)/vmlinuz ro root=/dev/sda1 3

7.3 Fixing a kernel panic (missing ROOT file system)Index

If you moved your ROOT file system to a different partition but forgot to change the GRUB configuration file, your kernel will “panic” and tell you it can’t find the ROOT file system. You can reboot into the GRUB shell and interactively edit the root= option on the kernel line to point to the correct ROOT partition:

kernel (hd0,0)/vmlinuz ro root=/dev/sdb2 single

Boot in to single-user mode and edit the GRUB configuration file (and possibly the Linux /etc/fstab file) to point to the new partition, then reboot again.

• FoxTrot Kernel Panic Comic

8.1 Default Run Level in /etc/inittabIndex

• The default Run Level to use at boot time is set by the initdefault option of the /etc/inittab file. It looks like this if your default Run Level is number 3:

  id:3:initdefault:

8.2 Display Run LevelIndex

To display previous and current system Run Levels, use the runlevel command:

$ runlevel
N 3

An N will be printed as the previous Run Level right after booting, otherwise the first number will be the previous run level number. The letter S is used for single-user mode.

8.3 Change Run LevelIndex

Only root can tell the system to change Run Levels, unless you are on the system console and can type CTRL-ALT-DEL to reboot.

To change Run Levels (as root) use the telinit command with an argument of the desired Run Level, e.g. telinit 2

See the table of Run Levels in the comments in the /etc/inittab file to know what each Run Level number means.

You rarely need to explicitly tell the system to change Run Levels. The shutdown command is much better for shutting down and/or rebooting the system, because shutdown will send a warning message to all logged-in users and use a gentle series of signals to stop running services.

• You can change Run Levels while the system is running using commands such as telinit, but that command doesn’t print any warning messages.

• Usually use shutdown to shut down or reboot the system, because shutdown will send a warning message to all logged-in users.

8.4 Displaying and Controlling Run Level Services with chkconfigIndex

System services are started and stopped for each Run Level. You can see which services are on and off in each of the seven Run Levels using the chkconfig command:

$ chkconfig | wc -l
22                         # there are 22 possible services available

$ chkconfig --list sshd
sshd            0:off   1:off   2:on    3:on    4:on    5:on    6:off

$ chkconfig | grep syslog
rsyslog         0:off   1:off   2:on    3:on    4:on    5:on    6:off

$ chkconfig
auditd          0:off   1:off   2:on    3:on    4:on    5:on    6:off
crond           0:off   1:off   2:on    3:on    4:on    5:on    6:off
... many more lines ...

The chkconfig output gives a list of the known services and whether each service is supposed to be turned on or off in that Run Level.

The chkconfig command cannot tell you if the service is actually and currently on or off; it only knows what should be on or off and it does not start or stop any services itself. If you want to check whether or not a service is actually running, use the ps command to look for it.

You can use shell pipelines on the output of chkconfig, for example to find out which services are enabled to start in at least one Run Level:

$ chkconfig | fgrep ':on' | wc -l
16                         # 16 services are on in some run level

Or what services are never started in any run level:

$ chkconfig | fgrep -v ':on'
multipathd      0:off   1:off   2:off   3:off   4:off   5:off   6:off
netconsole      0:off   1:off   2:off   3:off   4:off   5:off   6:off
ntpdate         0:off   1:off   2:off   3:off   4:off   5:off   6:off
rdisc           0:off   1:off   2:off   3:off   4:off   5:off   6:off
restorecond     0:off   1:off   2:off   3:off   4:off   5:off   6:off
saslauthd       0:off   1:off   2:off   3:off   4:off   5:off   6:off

A common use of chkconfig is to change which services are supposed to be turned on or off in which Run Level (requires root permissions):

$ chkconfig --list lvm2-monitor
lvm2-monitor    0:off   1:on    2:on    3:on    4:on    5:on    6:off

$ sudo chkconfig --level 45 lvm2-monitor off

$ chkconfig --list lvm2-monitor
lvm2-monitor    0:off   1:on    2:on    3:on    4:off   5:off   6:off

You can also completely remove (delete) a service from chkconfig so that it does not appear at all in the output. (RTFM)

8.5 Displaying and Controlling Run Level Services without chkconfigIndex

If chkconfig is not available, you can still discover and change which services run in each Run Level by knowing how the System V Run Levels system works.

Services (such as the SSH server or the CRON daemon) are started using shell scripts. All the SysV-style service start-up scripts are kept in directory /etc/init.d/:

$ ls /etc/init.d/ssh* /etc/init.d/cron*
/etc/init.d/crond  /etc/init.d/sshd

$ file /etc/init.d/ssh* /etc/init.d/cron*
/etc/init.d/sshd:  Bourne-Again shell script text executable
/etc/init.d/crond: POSIX shell script text executable

$ wc /etc/init.d/ssh* /etc/init.d/cron*
 234  666 4534 /etc/init.d/sshd
 132  382 2793 /etc/init.d/crond

Symbolic links to these scripts are placed in directories named /etc/rc?.d/ for each Run Level, e.g. the services for Run Level 3 are listed as symlinks under directory /etc/rc3.d:

$ ls -l /etc/rc3.d/*ssh* /etc/rc3.d/*cron* /etc/rc3.d/*ntpdate*
lrwxrwxrwx. 1 root root 17 Oct 15 17:02 /etc/rc3.d/K75ntpdate -> ../init.d/ntpdate
lrwxrwxrwx. 1 root root 14 Oct 13 15:18 /etc/rc3.d/S55sshd -> ../init.d/sshd
lrwxrwxrwx. 1 root root 15 Oct 13 15:17 /etc/rc3.d/S90crond -> ../init.d/crond

The symbolic links starting with S and a number start a service in that Run Level; the symbolic links starting with K and a number stop (“kill”) a service in that Run Level, e.g.

/etc/rc3.d/K75ntpdate    # symlink to kill the ntpdate command
/etc/rc3.d/S55sshd       # symlink to start the SSH daemon in Run Level 3

Each of those symlinks points to a script file responsible for killing or starting the indicated service. Without chkconfig, you can change what services get killed or started by manually adding or removing symlinks from the corresponding /etc/rc?.d Run Level directories.

A service is started in a Run Level by automatically calling its SysV start-up script with a single argument of start. A service is stopped in a Run Level by automatically calling its SysV start-up script with a single argument of stop. (See below for how you can start and stop services manually from the command line.)

Most Run Level changes that cause these scripts to run happen at boot time and system shut down, though you can use the telinit command to change Run Levels if you need to. (You almost never need to. Use shutdown to shut down or reboot; don’t use telinit!)

While is is always possible to create and remove the symbolic links in the rc?.d directories by hand (using ln -s and rm), the chkconfig command is the preferred tool for listing and manipulating these symbolic links in their Run Level directories.

8.6 Resetting symlinks for a serviceIndex

Many/most of the SysV start-up scripts in /etc/init.d contain a chkconfig comment line in the comments at the start of the script. The comment line might look something like this:

$ grep 'chkconfig' /etc/init.d/crond
# chkconfig: 2345 90 60

The above shell comment line is readable by chkconfig if you give chkconfig the reset option for that service. With reset, chkconfig will read the above line and make sure there are start/stop symlinks for the service in Run Levels 2,3,4,5 at priority 90 and 60. For example:

$ sudo chkconfig crond reset

Now chkconfig --list crond will show crond enabled in Run Levels 2,3,4,5. The crond service will start at priority 90 and stop at priority 60. You can confirm this by looking for the symlinks in all the /etc/rc?.d directories:

$ chkconfig --list crond
crond           0:off   1:off   2:on    3:on    4:on    5:on    6:off

$ ls /etc/rc?.d/*crond*
/etc/rc0.d/K60crond  /etc/rc3.d/S90crond  /etc/rc6.d/K60crond
/etc/rc1.d/K60crond  /etc/rc4.d/S90crond
/etc/rc2.d/S90crond  /etc/rc5.d/S90crond

Note: You can always add the appropriate start/stop symlinks manually in the rc?.d directories if the above procedure isn’t possible because the chkconfig command is missing. Don’t depend on chkconfig!

8.7 Starting and Stopping Services using serviceIndex

Using chkconfig to change which servers are supposed to be on or off in any Run Level does not change what services are actually and currently running (or not running). A service may have died unexpectedly, or have been stopped or started manually.

System services can be started, stopped, and reset using the service command (as root) with a service name and an appropriate command operation argument such as stop, start or status:

$ sudo service crond
Usage: /etc/init.d/crond {start|stop|status|restart|condrestart|try-restart|reload|force-reload}

$ sudo service crond status
crond (pid  1597) is running...

$ sudo service crond restart
Stopping crond:                                            [  OK  ]
Starting crond:                                            [  OK  ]

$ sudo service crond status
crond (pid  8207) is running...

All Run Level services support the start and stop commands, which are used to automatically start and stop services when the system changes Run Levels. Most services also support a status command that gives the process id (pid) of the running service, e.g.

$ sudo service postfix status
master (pid  4302) is running...

$ ps uww 4302
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      4302  0.0  0.9  12524  2488 ?        Ss   05:41   0:00 /usr/libexec/postfix/master

$ sudo service crond status
crond (pid  1644) is running...

$ ps uww 1644
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      1644  0.0  0.4   5936  1152 ?        Ss   Nov27   0:00 crond

You can omit any service commands to generate a Usage message that lists what other service commands are possible for a given service:

$ sudo service crond
Usage: /etc/init.d/crond {start|stop|status|restart|condrestart|try-restart|reload|force-reload}

If your system is missing the service command, you can always execute any service script directly from the /etc/init.d/ directory, passing it an argument of what function you want to perform:

$ sudo /etc/init.d/sshd status
crond (pid  8207) is running...

$ sudo /etc/init.d/sshd restart
Stopping sshd:                                            [  OK  ]
Starting sshd:                                            [  OK  ]

Remember: chkconfig only specifies what services should be on or off in a Run Level. It does not start or stop any services. You can start or stop any services in any Run Level using the service command.

9.1 Systemd wins over Upstart, but chaos ensuesIndex

In 2013, there was still considerable controversy on the move from booting with Upstart to booting with Systemd. Fedora and Red Hat were moving to Systemd; Debian and Ubuntu were staying with Upstart; other distributions are watching the battle to see who wins.

See these references for some background on the heated discussions:

• Systemd controversy causes a fork of UDEV
  • Gentoo developers could also potentially be joined by Debian developers in what could turn out to be a groundswell of protest against the Red Hat led developments in Systemd.
• A developer complains that Systemd is breaking promises
  • We are now being told that, contrary to what was said when udev was migrated into the systemd tree, running udev without systemd is now deprecated and untested and might go away completely. How surprising, nobody ever predicted that when it migrated in, oh wait yes we did, and were assured that we were wrong, that standalone udev would always be supported for those of us who weren’t using systemd. Way to destroy your userbase’s trust in you…
• Linus Torvalds rants against the udev developers
  • Stop this crazy. FIX UDEV ALREADY, DAMMIT. […] “Two-faced lying weasel” would be the most polite thing I could say. But it almost certainly will involve a lot of cursing.
• Linus Torvalds rants against the Systemd developers
  • I also call bullshit on your “it will surely be fixed when we know what’s the right fix” excuses. […] Kay, you are so full of sh*t that it’s not funny. You’re refusing to acknowledge your bugs, you refuse to fix them even when a patch is sent to you, and then you make excuses for the fact that we have to work around *your* bugs, and say that we should have done so from the very beginning.

In February 2014, Debian voted to move to using Systemd, and Ubuntu (derived from Debian) followed shortly afterward.

• On April 2 2014, Linux Torvalds suspended key Systemd developer Kay Sievers, saying “Key[sic], I’m [expletive] tired of the fact that you don’t fix problems in the code you write, so that the kernel then has to work around the problems you cause. Greg – just for your information, I will not be merging any code from Kay into the kernel until this constant pattern is fixed”.

It remains to be seen how long this move to Systemd will take, or how it will affect non-Linux systems that can’t use Systemd Linux-only features (e.g. BSD systems such as OSX).

In March 2015, the issue is still a hot topic:

http://www.techrepublic.com/article/the-linux-camp-conflict-within/

[…] However, when one man holds such a massive responsibility for that much code (and the patches submitted therein), it only makes sense that he carry a sharp stick and tone. The problem comes when contributors begin calling out Torvalds publicly. This happened recently when Lennart Poettering called Torvalds out for encouraging hate speech and attacks. Poettering went so far to say that the Linux community is a “sick place to be in.”

Where is this coming from? Poettering is a Red Hat engineer responsible for the controversial systemd replacement for the UNIX sysvinit daemon and has been called to the carpet many times for pushing to replace a system that has worked (and worked very well) for a long time. At one point, there was even a website dedicated to Boycotting systemd (the site has been taken down). The vitriol surrounding this controversy is thick and venomous.