Updated: 2015-03-18 02:43 EDT

1 Resources and ReadingsIndexup to index

For reference:

2 Definitions: boot and boot loaderIndexup to index

3 What happens at power onIndexup to index

4 The Linux /boot directoryIndexup to index

5 Legacy DOS MBR vs. New GPT UEFI MBRIndexup to index

6 The Grand Unified Boot Loader – GRUBIndexup to index

6.1 GRUB Device and file Naming ConventionIndexup to index

The syntax of pathnames to files differs among operating systems. The GRUB bootloader is a mini operating system that runs before the real operating system is even running. GRUB has its own pathname syntax to access files on disk.

A full GRUB path name specification is a GRUB device name (often a partition name) followed by a file system pathname inside that partition (relative to the start of the partition).

For example the GRUB pathname (hd0,0)/grub/grub.conf can be divided into the (hd0,0) piece that names a disk (disk 0) and partition (partition 0) and the /grub/grub.conf piece that is a pathname relative to the start of that partition. This GRUB syntax is similar to DOS/Windows and its use of drive letters to name partitions.

6.2 A sample GRUB Legacy configuration file (version 0.97)Indexup to index

All versions of GRUB use a configuration file under /boot/grub to display a menu of alternative operating systems to boot. Without a configuration file, GRUB starts in interactive mode and you have to specify and type everything yourself. With a configuration file, you can select from a menu list of choices.

Do not move or remove the GRUB configuration files! GRUB expects the files to be in a particular location on disk, and moving them may cause GRUB to fail to find them, requiring a re-install of GRUB. You can edit the GRUB configuration files in-place as long as you do not delete or move them.

Boot Menu Entry
Every title keyword in a GRUB configuration file starts a paragraph of lines that defines a “boot menu entry”. For most Linux kernels, the boot menu entry is four lines long. The CentOS GRUB configuration file grub.conf only contains one four-line Linux boot menu entry to start. (Software updates may add more, but you were told not to install any software updates.)

In GRUB Legacy, each operating system choice in the configuration file is called a boot menu entry and it starts with a title keyword followed by a descriptive name followed by some lines that say how to boot that particular operating system.

Here is a simple one-entry GRUB Legacy /boot/grub/grub.conf file:

# Sample GRUB grub.conf file (GRUB Legacy version 0.9x)
default=0
timeout=30
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
#hiddenmenu

# This sda disk has ROOT on partition sda1 and no separate /boot
# partition, so GRUB pathnames must start with /boot
title CentOS (2.6.32-431.el6.i686)
    root (hd0,0)
    kernel /boot/vmlinuz-2.6.32-431.el6.i686 root=/dev/sda1
    initrd /boot/initramfs-2.6.32-431.el6.i686.img

Comments in this file start with # and extend to the end of line.

6.3 Useful grub.conf keywordsIndexup to index

Note: To find out about additional GRUB commands, type help at the GRUB shell prompt, and see the above Resources.

6.4 Effect of separate /boot partition on GRUB pathnamesIndexup to index

GRUB configuration files are stored under directory /boot/grub in Linux. /boot/grub is the Linux pathname, not necessarily the GRUB pathname, because GRUB pathnames are composed of a partition name followed by a pathname inside the partition, e.g. (hd0,0)/some/path/name where /some/path/name is relative to the start of the partition.

Since partitions are mounted on directories in Linux, and those directory pathnames prefix the names inside the partition for Linux pathnames, a GRUB pathname and a Linux absolute pathname will always differ, except for the one partition that is mounted on ROOT (because prefixing an absolute pathname with / (ROOT) doesn’t change it).

Some examples will make this clearer:

6.4.1 /boot is on the ROOT partitionIndexup to index

If /boot is not its own partition, then it is a directory inside the existing ROOT partition. GRUB shell command pathnames referring to the kernel and to GRUB configuration files will also be inside the ROOT partition and will therefore all start with the ROOT partition name followed by the subdirectory name /boot inside the partition, e.g. (hd0,0)/boot/grub/grub.conf refers to the Linux pathname /boot/grub/grub.conf because /boot is an ordinary directory inside the ROOT partition (hd0,0).

6.4.2 /boot is its own partitionIndexup to index

If /boot is its own mounted partition, then the GRUB files will be on this separate BOOT partition (not on the ROOT partition) that gets mounted on /boot and the GRUB pathnames (which are relative to the start of the partition, not relative to the ROOT of the file system) will therefore not start with /boot, they will start directly under the BOOT partition device name e.g. (hd0,0)/grub/grub.conf refers to the Linux pathname /boot/grub/grub.conf (because (hd0,0) is mounted on /boot and so /boot is prefixed to the part of the pathname contained inside the partition).

6.4.3 Summary of GRUB pathnamesIndexup to index

  • Separate /boot partition: (hd0,0)/grub/grub.conf
  • /boot is under ROOT: (hd0,0)/boot/grub/grub.conf

Our CentOS installation does not use a separate BOOT partition. The NOTICE comment in its GRUB configuration file tells you this:

# NOTICE:  You do not have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /, eg.
#          root (hd0,0)
#          kernel /boot/vmlinuz-version ro root=/dev/sda1
#          initrd /boot/initrd-[generic-]version.img

You must use /boot at the start of GRUB pathnames because the pathnames are in the boot subdirectory inside the main ROOT partition.

6.5 Installing GRUB – grub-installIndexup to index

Your Linux installation already installed GRUB for you. If you have to re-install GRUB, you may find or be able to install the Linux grub-install script that will do the installation for you. (RTFM)

6.6 Installing GRUB using the GRUB shell – OPTIONALIndexup to index

This section is OPTIONAL. It’s included here for completeness, but you won’t be asked to do any of this in this course.

You can also try to install GRUB manually, if the GRUB setup files are already stored under /boot/grub:

Most versions of the GRUB shell have TAB command completion, where you can type part of a device or pathname and GRUB will give you all the possible completions. The shell command-line version of GRUB is broken, and does not have this feature. The GRUB that runs at boot time is not broken and does have the TAB completion feature.

BE CAREFUL. GRUB does not ask you for confirmation! If you type the wrong thing, you will overwrite your boot sector with garbage and be unable to boot your system. You will need to boot a “live” CD and repair.

6.7 Booting without a GRUB menu – interactive GRUB commandsIndexup to index

If the grub.conf configuration file is not present or has been moved, GRUB cannot display a boot menu. In this case it will display the GRUB interactive shell, similar to what you get by typing grub at a Linux command line.

The interactive GRUB shell has its own prompt (grub>) and its own built-in commands, some with the same names as BASH shell commands. Be clear on when you are typing into the GRUB shell and when you are typing into BASH. The same command name may do different things, depending on whether GRUB executes it or BASH executes it.

Most boot-time versions of the GRUB shell have TAB command completion, where you can type part of a device or pathname, push TAB, and GRUB will give you all the possible completions. The shell command-line version of GRUB is broken, and does not have this feature working. The GRUB that runs stand-alone at boot time is not broken and does have the TAB completion feature.

Inside the GRUB shell, you can type help for a partial list of commands you can use. Some examples:

Remember that all pathnames must be GRUB pathnames, not Linux pathnames!

At the GRUB shell prompt, GRUB commands can be entered to query the system, see text files, and load and then boot a kernel image. Examples:

grub> find /grub/grub.conf
grub> cat (hd0,0)/grub/grub.conf
grub> kernel (hd0,0)/vmlinuz ro root=/dev/sda2
grub> initrd (hd0,0)/initrd
grub> boot

Note: The GRUB shell can also be accessed by following boot-time directions (usually by pressing the letter c) when the GRUB menu is presented. [Esc] brings you from the GRUB shell back to the menu. Pressing e allows you to edit boot menu entries of grub.conf. Pressing b boots the edited menu item. Some versions of GRUB allow typing a to go directly to a GRUB kernel line and edit the kernel options.

7 Kernel options: single user mode, other Run Levels, kernel panicIndexup to index

Options provided on the kernel line in a GRUB configuration file influence how the system boots and what software might be enabled. The options are given after the kernel image name on a kernel line and allow you to boot your system with different features enabled. Some versions of GRUB allow you to type a while viewing the GRUB menu, to go directly to a GRUB kernel line and edit the kernel options.

Kernel options are separated by blanks, e.g.: root=/dev/sda1 ro single txt 3

When you edit these GRUB lines at boot time, you are only changing the in-memory copy of the configuration, you are not changing the actual configuration in the GRUB configuration file on disk. To make any edit permanent, you must boot the system (single-user is fine) and actually edit the GRUB configuration file and save it.

7.1 Booting Single User Mode (Maintenance Mode)Indexup to index

Single-user mode is a “half-up” state used for system repair and maintenance (especially for resetting the root password!). The system only brings up a minimal number of services (often none). The GUI is not started. Networking may not be enabled. Not all disks may be mounted. No login prompt is used; the console terminal gets a shell running as the root user.

To boot into single-user mode, add the word single as an option to the end of a kernel line in a GRUB boot menu entry and then boot that entry, e.g.:

kernel (hd0,0)/vmlinuz ro root=/dev/sda1 single

The system will boot directly into a root shell with no need to log in. Exiting this root shell will leave single-user mode and the system will finish booting into the default Run Level as recorded in the /etc/inittab file.

7.2 Booting into a different Run LevelIndexup to index

(Run Levels are summarized below.) To start Linux with a different Run Level than the default, add the Run Level digit to the end of the kernel options line during boot, e.g. to boot Run Level 3 (which on CentOS means boot multi-user without the X11 GUI window system), use the GRUB menu to add a space and the digit 3 to the end of the kernel options:

kernel (hd0,0)/vmlinuz ro root=/dev/sda1 3

7.3 Fixing a kernel panic (missing ROOT file system)Indexup to index

If you moved your ROOT file system to a different partition but forgot to change the GRUB configuration file, your kernel will “panic” and tell you it can’t find the ROOT file system. You can reboot into the GRUB shell and interactively edit the root= option on the kernel line to point to the correct ROOT partition:

kernel (hd0,0)/vmlinuz ro root=/dev/sdb2 single

Boot in to single-user mode and edit the GRUB configuration file (and possibly the Linux /etc/fstab file) to point to the new partition, then reboot again.

8 Legacy Run Levels and ServicesIndexup to index

The System V Run Levels system was a crude way to specify groups of services (such as the Apache Web Server, or the Secure Shell Server) that were to be started and stopped together. Run Levels expect that a system boots into a fixed state, with a fixed set of disks and fixed set of services. They do not work well with systems where devices come and go after booting, e.g. dynamic USB drives, printers, cameras, hotplug disks, etc.

Since Enterprise Servers (systems with a long maintenance window) typically boot into a fixed state where devices don’t come and go, Run Levels are still a useful and simple way of configuring Enterprise systems. Enterprise systems still use Run Levels, and even the newer versions of the system boot process (using the new Upstart or Systemd) emulate traditional Run Levels. (CentOS uses Upstart to emulate Run Levels.)

The system can be in only one Run Level at a time, but can be moved to any other Run Level. Changing Run Levels cause services to be started and stopped to match what is supposed to be available in that Run Level.

Each Run Level is given a number. The broad meaning of each Run Level is usually documented in comments inside the file /etc/inittab in a table that looks similar to this (adapted from CentOS):

# 0 - Halt (Do NOT set the initdefault default run level to this)
# 1 - Single user mode
# 2 - Multi-user, without NFS (The same as 3, if you do not have networking)
# 3 - Full multi-user mode, text-only
# 4 - not used
# 5 - Full multi-user mode, with X11
# 6 - reboot (Do NOT set the initdefault default run level to this)
id:3:initdefault:

The last line of this file (initdefault) is the only active (non-comment) line in the file. It gives the Run Level number that the system will normally boot into.

As you can see above, CentOS Linux uses seven Run Levels, numbered 0 through 6, with Run Level 3 being the default Run Level for a server. (Desktop machines default to Run Level 5 with X11 graphics.)

You can tell the system to change Run Levels explicitly using commands such as telinit. The shutdown, reboot, halt, and poweroff commands also change Run Levels. Changing Run Levels will cause some system services to stop and others to start. Run Levels are not an ordered sequence; the system goes directly between Run Levels, it does not “pass through” Run Levels 3 and 4 when going from, say, Run Level 2 to Run Level 5.

The meanings of the CentOS Run Levels are given in the /etc/inittab file:

  1. Halt
    • Immediately stop all services and power off the system without any warning to the logged-in users.
    • Do not use this Run Level! Use the more gentle shutdown command instead!
    • Do not set the default Run Level initdefault to this or else you will never be able to boot your system!
  2. Single user mode
    • Stop all (or almost all) services but do not power off the system.
    • Put up a single root shell on the system console (no password needed).
    • Remote logins are not allowed since no SSH service is running.
    • Used for system maintenance and resetting a forgotten root password.
  3. Limited Multiuser mode
    • Enables most system services, but no graphical GUI or X11 services.
    • The system consoles show a text-only login: prompt.
    • Does not enable the Networking File System (NFS) service.
    • The same as Run Level 3, if you do not have networking.
  4. Full multiuser mode
    • Enables most system services, but no graphical GUI or X11 services.
    • The system consoles show a text-only login: prompt.
    • This is the default for a server machine
  5. unused
  6. full multiuser with X11 GUI
    • Enables all configured system services.
    • Full multiuser mode with the X11 GUI (graphical) window system.
    • This is the default for a Desktop machine
  7. Reboot
    • Immediately stop all services and reboot the system without any warning to the logged-in users.
    • Do not use this Run Level! Use the more gentle shutdown command instead!
    • Do not set the default Run Level initdefault to this or else you will never be able to boot your system!

Higher numbered Run Levels generally mean more and more services started, but the highest Run Level (6) is used to reboot the system without warning. Run Levels are not an ordered sequence. When going from, say, Run Level 2 to Run Level 5 the system goes directly between Run Levels; it does not “pass through” Run Levels 3 and 4.

CentOS actually uses the newer Upstart event-based services system, but hides most of that Upstart functionality behind a traditional Run Levels emulation layer. You can read about Upstart below.

8.1 Default Run Level in /etc/inittabIndexup to index

8.2 Display Run LevelIndexup to index

To display previous and current system Run Levels, use the runlevel command:

$ runlevel
N 3

An N will be printed as the previous Run Level right after booting, otherwise the first number will be the previous run level number. The letter S is used for single-user mode.

8.3 Change Run LevelIndexup to index

Only root can tell the system to change Run Levels, unless you are on the system console and can type CTRL-ALT-DEL to reboot.

To change Run Levels (as root) use the telinit command with an argument of the desired Run Level, e.g. telinit 2

See the table of Run Levels in the comments in the /etc/inittab file to know what each Run Level number means.

You rarely need to explicitly tell the system to change Run Levels. The shutdown command is much better for shutting down and/or rebooting the system, because shutdown will send a warning message to all logged-in users and use a gentle series of signals to stop running services.

8.4 Displaying and Controlling Run Level Services with chkconfigIndexup to index

System services are started and stopped for each Run Level. You can see which services are on and off in each of the seven Run Levels using the chkconfig command:

$ chkconfig | wc -l
22                         # there are 22 possible services available

$ chkconfig --list sshd
sshd            0:off   1:off   2:on    3:on    4:on    5:on    6:off

$ chkconfig | grep syslog
rsyslog         0:off   1:off   2:on    3:on    4:on    5:on    6:off

$ chkconfig
auditd          0:off   1:off   2:on    3:on    4:on    5:on    6:off
crond           0:off   1:off   2:on    3:on    4:on    5:on    6:off
... many more lines ...

The chkconfig output gives a list of the known services and whether each service is supposed to be turned on or off in that Run Level.

The chkconfig command cannot tell you if the service is actually and currently on or off; it only knows what should be on or off and it does not start or stop any services itself. If you want to check whether or not a service is actually running, use the ps command to look for it.

You can use shell pipelines on the output of chkconfig, for example to find out which services are enabled to start in at least one Run Level:

$ chkconfig | fgrep ':on' | wc -l
16                         # 16 services are on in some run level

Or what services are never started in any run level:

$ chkconfig | fgrep -v ':on'
multipathd      0:off   1:off   2:off   3:off   4:off   5:off   6:off
netconsole      0:off   1:off   2:off   3:off   4:off   5:off   6:off
ntpdate         0:off   1:off   2:off   3:off   4:off   5:off   6:off
rdisc           0:off   1:off   2:off   3:off   4:off   5:off   6:off
restorecond     0:off   1:off   2:off   3:off   4:off   5:off   6:off
saslauthd       0:off   1:off   2:off   3:off   4:off   5:off   6:off

A common use of chkconfig is to change which services are supposed to be turned on or off in which Run Level (requires root permissions):

$ chkconfig --list lvm2-monitor
lvm2-monitor    0:off   1:on    2:on    3:on    4:on    5:on    6:off

$ sudo chkconfig --level 45 lvm2-monitor off

$ chkconfig --list lvm2-monitor
lvm2-monitor    0:off   1:on    2:on    3:on    4:off   5:off   6:off

You can also completely remove (delete) a service from chkconfig so that it does not appear at all in the output. (RTFM)

8.5 Displaying and Controlling Run Level Services without chkconfigIndexup to index

If chkconfig is not available, you can still discover and change which services run in each Run Level by knowing how the System V Run Levels system works.

Services (such as the SSH server or the CRON daemon) are started using shell scripts. All the SysV-style service start-up scripts are kept in directory /etc/init.d/:

$ ls /etc/init.d/ssh* /etc/init.d/cron*
/etc/init.d/crond  /etc/init.d/sshd

$ file /etc/init.d/ssh* /etc/init.d/cron*
/etc/init.d/sshd:  Bourne-Again shell script text executable
/etc/init.d/crond: POSIX shell script text executable

$ wc /etc/init.d/ssh* /etc/init.d/cron*
 234  666 4534 /etc/init.d/sshd
 132  382 2793 /etc/init.d/crond

Symbolic links to these scripts are placed in directories named /etc/rc?.d/ for each Run Level, e.g. the services for Run Level 3 are listed as symlinks under directory /etc/rc3.d:

$ ls -l /etc/rc3.d/*ssh* /etc/rc3.d/*cron* /etc/rc3.d/*ntpdate*
lrwxrwxrwx. 1 root root 17 Oct 15 17:02 /etc/rc3.d/K75ntpdate -> ../init.d/ntpdate
lrwxrwxrwx. 1 root root 14 Oct 13 15:18 /etc/rc3.d/S55sshd -> ../init.d/sshd
lrwxrwxrwx. 1 root root 15 Oct 13 15:17 /etc/rc3.d/S90crond -> ../init.d/crond

The symbolic links starting with S and a number start a service in that Run Level; the symbolic links starting with K and a number stop (“kill”) a service in that Run Level, e.g.

/etc/rc3.d/K75ntpdate    # symlink to kill the ntpdate command
/etc/rc3.d/S55sshd       # symlink to start the SSH daemon in Run Level 3

Each of those symlinks points to a script file responsible for killing or starting the indicated service. Without chkconfig, you can change what services get killed or started by manually adding or removing symlinks from the corresponding /etc/rc?.d Run Level directories.

A service is started in a Run Level by automatically calling its SysV start-up script with a single argument of start. A service is stopped in a Run Level by automatically calling its SysV start-up script with a single argument of stop. (See below for how you can start and stop services manually from the command line.)

Most Run Level changes that cause these scripts to run happen at boot time and system shut down, though you can use the telinit command to change Run Levels if you need to. (You almost never need to. Use shutdown to shut down or reboot; don’t use telinit!)

While is is always possible to create and remove the symbolic links in the rc?.d directories by hand (using ln -s and rm), the chkconfig command is the preferred tool for listing and manipulating these symbolic links in their Run Level directories.

8.7 Starting and Stopping Services using serviceIndexup to index

Using chkconfig to change which servers are supposed to be on or off in any Run Level does not change what services are actually and currently running (or not running). A service may have died unexpectedly, or have been stopped or started manually.

System services can be started, stopped, and reset using the service command (as root) with a service name and an appropriate command operation argument such as stop, start or status:

$ sudo service crond
Usage: /etc/init.d/crond {start|stop|status|restart|condrestart|try-restart|reload|force-reload}

$ sudo service crond status
crond (pid  1597) is running...

$ sudo service crond restart
Stopping crond:                                            [  OK  ]
Starting crond:                                            [  OK  ]

$ sudo service crond status
crond (pid  8207) is running...

All Run Level services support the start and stop commands, which are used to automatically start and stop services when the system changes Run Levels. Most services also support a status command that gives the process id (pid) of the running service, e.g.

$ sudo service postfix status
master (pid  4302) is running...

$ ps uww 4302
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      4302  0.0  0.9  12524  2488 ?        Ss   05:41   0:00 /usr/libexec/postfix/master

$ sudo service crond status
crond (pid  1644) is running...

$ ps uww 1644
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      1644  0.0  0.4   5936  1152 ?        Ss   Nov27   0:00 crond

You can omit any service commands to generate a Usage message that lists what other service commands are possible for a given service:

$ sudo service crond
Usage: /etc/init.d/crond {start|stop|status|restart|condrestart|try-restart|reload|force-reload}

If your system is missing the service command, you can always execute any service script directly from the /etc/init.d/ directory, passing it an argument of what function you want to perform:

$ sudo /etc/init.d/sshd status
crond (pid  8207) is running...

$ sudo /etc/init.d/sshd restart
Stopping sshd:                                            [  OK  ]
Starting sshd:                                            [  OK  ]

Remember: chkconfig only specifies what services should be on or off in a Run Level. It does not start or stop any services. You can start or stop any services in any Run Level using the service command.

9 The Future: Upstart and/or SystemdIndexup to index

Fixed Run Levels do not work well on dynamic systems such as user Desktop, Workstation, Tablet, or Phone systems where devices and interfaces come and go after booting, e.g. USB drives, printers, cameras, hotplug disks, network connetions, etc. Run Levels can only start a fixed set of services, and there is no clean way using Run Levels to start a new service when a new device is plugged in and to stop it when the device is unplugged.

Modern Unix/Linux systems moved away from static Run Levels to use the more dynamic and complex Upstart system of starting/stopping services. Run Levels are “emulated” by Upstart, but Upstart can handle devices coming and going dynamically much better. When devices were added/removed, “events” were generated that could start and stop related system services.

Red Hat replaced the legacy System V init system (Run Levels) with the improved Upstart system for their Enterprise Linux 6.

The complexity and limitations of Upstart prompted Lennart Poettering to write a new Linux-only Systemd session manager (that didn’t work on any other Unix-like systems). Fedora 15 then moved from using Upstart to use the new Linux-only Systemd. Red Hat Enterprise Linux has also moved from Upstart to Systemd. (Yes, Upstart was used for only one release before being replaced.)

The Debian Linux community was very reluctant to use a Linux-only start-up system that wouldn’t work on non-Linux systems (such as Debian GNU/kFreeBSD or OSX), and so Debian and Ubuntu stayed with the Upstart system, not Systemd, for a year or so.

9.1 Systemd wins over Upstart, but chaos ensuesIndexup to index

In 2013, there was still considerable controversy on the move from booting with Upstart to booting with Systemd. Fedora and Red Hat were moving to Systemd; Debian and Ubuntu were staying with Upstart; other distributions are watching the battle to see who wins.

See these references for some background on the heated discussions:

In February 2014, Debian voted to move to using Systemd, and Ubuntu (derived from Debian) followed shortly afterward.

It remains to be seen how long this move to Systemd will take, or how it will affect non-Linux systems that can’t use Systemd Linux-only features (e.g. BSD systems such as OSX).

In March 2015, the issue is still a hot topic:

http://www.techrepublic.com/article/the-linux-camp-conflict-within/

[…] However, when one man holds such a massive responsibility for that much code (and the patches submitted therein), it only makes sense that he carry a sharp stick and tone. The problem comes when contributors begin calling out Torvalds publicly. This happened recently when Lennart Poettering called Torvalds out for encouraging hate speech and attacks. Poettering went so far to say that the Linux community is a “sick place to be in.”

Where is this coming from? Poettering is a Red Hat engineer responsible for the controversial systemd replacement for the UNIX sysvinit daemon and has been called to the carpet many times for pushing to replace a system that has worked (and worked very well) for a long time. At one point, there was even a website dedicated to Boycotting systemd (the site has been taken down). The vitriol surrounding this controversy is thick and venomous.

Author: 
| Ian! D. Allen  -  idallen@idallen.ca  -  Ottawa, Ontario, Canada
| Home Page: http://idallen.com/   Contact Improv: http://contactimprov.ca/
| College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/
| Defend digital freedom:  http://eff.org/  and have fun:  http://fools.ca/

Plain Text - plain text version of this page in Pandoc Markdown format

Campaign for non-browser-specific HTML   Valid XHTML 1.0 Transitional   Valid CSS!   Creative Commons by nc sa 3.0   Hacker Ideals Emblem   Author Ian! D. Allen