% Unix/Linux Partitions and File Systems - fdisk, mkfs, mount, swap % Ian! D. Allen - idallen@idallen.ca - www.idallen.com % Fall 2012 - September to December 2012 - Updated Mon Nov 26 09:56:48 EST 2012 Readings (Fedora Textbook) ========================== - Prerequisite: Disks and Partitions - Chapter 11: (`mkfs`) - Chapter 12: (`fsck`) - Show mounted disk info:   **`man df`** - Show recursive or summary disk usage:   **`man du`** - Show file system info:   **`file -s /dev/sda1`** Introduction to File Systems ============================ Creating a partition using **`fdisk`** simply divides up the space on a disk. It doesn’t by itself allow the operating system to store files there. You first have to create some infrastructure to hold content inside the partition; you have to create a **File System**. A **File System** is a way of storing content inside a disk partition. There are many kinds of file systems, each with different characteristics. Linux supports a huge number. Proprietary operating systems generally only support their own types of file systems. (Linux can access Linux, Apple, and Microsoft file systems, and dozens of others. Apple can only access Apple and Microsoft file systems. Microsoft can only access Microsoft file systems. *“Does not play well with others.”*) A file system is the way the O/S stores and retrieves data inside the partition, e.g. - create, move and delete files and directories (touch cp mkdir rmdir) - create, remove, modify filenames (ln, rm, mv) - open files for reading and writing (vi, cat, head, tail, sort) - Search for files (find) - seek within a file (grep) - list content of a directory (ls) - etc. The above basic functions are common to most operating systems, but they are implemented and managed differently from one O/S to another and perhaps even from one kind of file system to another. Creating File Systems - Three Steps =================================== The *three* steps to create a file system don’t change much from one operating system to another, but the specific details and utilities used vary greatly. The steps are: 1. Prepare the device (e.g. a disk, USB key, etc.) to receive a file system - i.e. partition the device into pieces (Linux **`fdisk`**) 2. Create the file system inside the partition - i.e. format or prepare the partition with the chosen file system (Linux **`mkfs`**) - optionally verify the file system integrity - check for flaws or errors (Linux **`fsck`**) 3. Mount the new file system to make it visible to the users - i.e. connect the file system into the existing directory tree (Linux **`mount`**) The trio is always **“`partition`”**, **“`make file system`”**, and **“`mount`”**, in that order. Let’s look at each in more detail: Step 1: Create the partition using `fdisk` ------------------------------------------ - Create, modify or change partition information on the disk - Use **`fdisk`**, Disk Druid or any other partitioning tool that can manage and create Linux partitions - Changing a partition start or end usually destroys whatever was inside the partition - back up first! - Using **`fdisk`**: see slides on Disk Partitioning - See - You must create partitions first, before creating file systems inside them Step 2: Create the file system in the partition using `mkfs` ------------------------------------------------------------ Linux organizes data inside partitions using various types of **file systems**. Linux supports a large number of different file system types, including most of the Windows file system types. You can use **`mkfs`** to create a file system to optimize the way data being stored in a partition. The **`mkfs`** command is actually a front-end for a whole set of **`mkfs`** commands, each different one selected by the **`-t`** option to **`mkfs`**. ### Syntax: Using **`mkfs`** to create a file system - `mkfs -t type [other options] device_name` - common types:   **`ext3`**, **`ext4`**, **`vfat`** - use **`vfat`** for USB keys, floppy disks, and Microsoft compatibility - the device\_name is almost always the **`/dev/`** name of an existing partition, not a whole disk - e.g.   `mkfs -t ext4 /dev/sda1`       *(first disk, first partition)* - e.g.   `mkfs -t vfat /dev/fd0`       *(first floppy disk)* - `mke2fs [options] device_name` - specialized version of **`mkfs`** specific to Linux **`ext2/ext3/ext4`** file systems - used when you want more control of the details of the file system (RTFM) - Optionally, you can check a file system for inconsistencies or errors after creating it using **`fsck`**. (Almost never done unless you suspect that your disk may have bad blocks on it.) ### Details on using **`mkfs`** - A file system must be created inside an *existing* drive/partition - You must have an existing partition before you can create a file system. - You must have a file system created before you can mount it. - The **type** defaults to the old **`ext2`** type, but for modern Linux hard disk systems you should always specify the type explicitly as either **`ext3`** or **`ext4`**. These newer file system types are journalling file systems. Do not use the old default **`ext2`** file system type, especially on large (over 400MB) disks. - The **`device_name`** is the pathname (usually absolute) of the existing device or partition that will be used, usually of the form **`/dev/sdXN`**, where X is the letter of the device (disk) being used and N is the partition number on that device. (Recall that partitions are created using the **`fdisk`** command.) - The **`mkfs`** commands do not check to see what is in the partition already. They will not ask “Are You Sure?” before they re-format a partition and destroy whatever was previously there. Be extremely careful to get the device name correct! A partition destroyed by **`mkfs`** is impossible to recover. - The **`mkfs`** commands do not care about the System ID (type) of a partition given in the partition table. You can create any type of file system in any type of partition. (For example, you can create an **`ext4`** file system on a partition labelled in the partition table as NTFS or as Swap, but this is a bad idea.) - Creating a partition with **`fdisk`** does not automatically create any type of file system in that partition. - Creating a file system with **`mkfs`** does not automatically mount or make available that file system for use in Linux. The new file system is not accessible. More on mounting file systems in the next section. ### Choosing a File System Type Your system install created “journalling” file systems on your virtual disk, using the “-t ext3” option to mkfs. Another way to request this kind of file system is to use the “-j” (Journalling) option to the original “mke2fs” command. This type of Linux journalling file system is usually called “ext3” or “ext4”, and some distributions have a small shell script named “mke3fs” that simply calls mke2fs with the “-t ext3” or “-j” option. Journalling file systems are more resistant to corruption due to sudden power loss, allowing the system to come back up more quickly by avoiding a long file system check at boot time. This does *NOT* give you permission to power off a running Linux system! Always shut down cleanly. The correct command-line for an immediate, safe system shut down is: # shutdown -h now See the man page for other options. Step 3: Mount the file system on an existing directory using mount ------------------------------------------------------------------ ![Linux File System]  Unlike Windows with its multiple drive letters, Linux has a single-ROOTed file system tree. “Mounting” attaches an existing file system found on a block device (usually a disk partition, e.g. **`/dev/hda2`**) to the Linux directory structure, e.g. onto some directory **`/boot`**. Accessing that directory, and everything under that directory, accesses the file system on that disk partition. Any number of separate file systems (stored in disk partitions) can be attached anywhere in the same Linux directory tree, resulting in one single ROOTed tree to access every file on every disk. You can detach a file system from one place in the tree and attach it somewhere else, but then the pathnames to files inside that file system would change to reflect the new mount location. File systems can be mounted from the Unix/Linux command line using the **`mount`** command. File systems can be mounted automatically at system boot time by putting their names and mount points into the **`/etc/fstab`** file. File systems on removable devices (USB keys, DVDs) can also be mounted dynamically at device insertion time using rules in system configuration files (“automounting”). (Most distribution use an existing **`/mnt/`** directory to attach file systems dynamically and temporarily to the directory structure, e.g. USB keys, CDROM and DVD, floppy disks, etc.) Desktop operating systems may also use a **`/media/`** directory for dynamic storage devices. ### Manual mount: Use the appropriate **`mount`** command - `mount` - Without any options: list all the currently mounted devices and their contained file systems. A common command usage! - `mount -a` - Mount all the file systems listed for auto mounting in the **`/etc/fstab`** file (possible for root only). - `mount [options] { device_name | directory_name }` - Mount the file system in **`/etc/fstab`** that matches the given name. - You give *either* the device\_name *or* the directory\_name of the entry in the fstab file. The system gets the other one from the fstab. - `mount [options] device_name directory_name` - Ignore **`fstab`** and mount the (file system in) the given device\_name onto the given directory\_name mount point, both of which must already exist. - You may need to create the directory\_name first. Details on using **`mount`**: - The **`device_name`** is the pathname (usually absolute) of the device containing the file system, e.g. **`/dev/sdb9`**. The device (usually a partition name, not a disk name) must have a recognized file system created inside it already, because you are mounting the file system that is inside the partition, not the partition itself. - The *directory\_name* is the pathname (often absolute) of an existing Linux directory, usually an empty directory. The file system inside the partition will be mounted on this existing directory mount point. If the mount fails, check to make sure the directory exists! **Let me repeat that: If the mount command fails, make sure that the mount point directory exists!** - Use **`mkdir`** to create a new empty directory mount point, if needed. - You can “cover up” a non-empty directory by mounting on top of it; this is not usually a good idea. - Modern **`mount`** commands are smart enough to know what type of file system resides inside a partition, but rarely you may have to specify the type using the “-t” option. - If Linux can’t automatically figure out the type, it usually means the partition has **no** file system created on it at all - you forgot to use **`mkfs`** first. - You can pass mount-time options using the “-o” option. - the options are the same options you can use in the **`/etc/fstab`** file - e.g.  `mount -o ro,noatime /dev/fd0 /mnt/floppy` - The user mounting the file system (if not root) must have access rights to the mount point directory and mount command. - Mounting a file system onto a directory that already contains files will temporarily hide those files until the file system is unmounted again. - Examples: - `mount /dev/sda1 /mnt/foo` - `mount -t vfat /dev/fd0 /mnt/dosfloppy` ### Un-mounting a mounted file system: umount - You can un-mount a file system using the (badly spelled) Linux **`umount`** command and give either the device name or the mount point, but not both: - `umount /dev/sda1` - `umount /home` - To un-mount a file system, no files must be open inside it and no programs must have it as a current directory. - If the message “device is busy” appears, a process is currently accessing the device. - A common mistake is to try (and fail) to unmount the file system of the current directory - you have to **`cd`** *all* your shells and processes out of the file system first. - Sometimes you have to find and kill processes that are using a file system as a current directory, so that you can unmount the file system. - Commands to list files open on a file system (use these as root, and give the Linux directory that is the root of the file system): - `fuser /boot` - `lsof /` ### Running a File System Check with **`fsck`** - If the system reboots and ends up in “repair file system” mode, all the file systems will be mounted “read only” so that you can repair them. - If you need to edit/modify/change any files (**`/etc/fstab`** for example) you will need to “re-mount” that file system as read-write. - To switch **`/dev/sda1`** to read-write mode use the “remount” option: - `mount –o remount,rw /dev/sda1` ### Using **`mount`** and the **`/etc/fstab`** file To ensure that a file system is mounted at every boot, add the file system to the **`/etc/fstab`** file and it will always be mounted at boot time. To learn about the format of this file:   `man 5 fstab` File systems can be listed in the **`/etc/fstab`** file to automate the mounting process. Both the mount point (the existing directory on which the file system is to be mounted) and any other options can be stored in the **`fstab`** file. The file systems can be identified in **`fstab`** in the traditional way by partition name (e.g. **`/dev/sda1`**), which is a bit risky since disk names can change depending on what is attached to your machine at boot time. Another way to identify file systems is with a Volume Label or UUID (Unique UID) identifier that is configured into the partition and that doesn’t change even if the partition name changes. You can see these Labels and UUIDs using the **`blkid`** command. You can set Labels and UUIDs into a partition using the **`tune2fs`** command. If a file system is not listed in the **`/etc/fstab`** file and you wish to mount it, then all mount information about the file system and all options need to be given on the command line. At minimum, you need the device name (partition) and the existing mount point (directory) where the file system should be mounted: - e.g.   `mount /dev/sda2 /home` - e.g.   `mount -o "ro" /dev/sda2 /home_readonly` If a file system is listed in the **`/etc/fstab`** file, then indicating either just the mount point (the directory name) or just the device name (e.g. **`/dev/sda1`**) will indicate to the **`mount`** command that the rest of the information needed for the mount should be copied from the matching entry in the **`/etc/fstab`** file: - e.g.   `mount /dev/sda1`       *(gets the directory name and options from **`fstab`**)* - e.g.   `mount /home`             *(gets the device name and options from **`fstab`**)* ### Example of an **`/etc/fstab`** file Lines in **`/etc/fstab`** starting with **`#`** are comments and are ignored: # Sample /etc/fstab taken from Fedora 12 - uses Linux Volume Manager (LVM) # DEVICE MOUNT POINT TYPE OPTIONS BACKUP FSCK /dev/mapper/VolGroup-lv_root / ext4 defaults 1 1 UUID=e55742bc-c78d-4a92-8195-d262e4eebb0e /boot ext4 defaults 1 2 /dev/mapper/VolGroup-lv_swap swap swap defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 Each line in **`/etc/fstab`** has six fields that describes a file system: 1. Field 1: Identifies the **device** (partition) to be mounted - traditionally, this a partition name under **`/dev/`** containing a file system, e.g. **`/dev/sda1`** - can be a LABEL or a UUID (Above, **`/boot`** is mounted by UUID instead ofdevice **`/dev/sda2`**) - we mount file systems by partition name, but the partition really does have to have a file system created inside it for the mount to work, and the type field has to match the file system that is in the partition 2. Field 2: Names the existing directory where that device is to be mounted - this is called the **mount point** and *must* already exist - usually, the directory should be empty (but it doesn’t have to be) - you can “cover up” a non-empty directory by mounting on top of it; not usually a good idea 3. Field 3: Names the **type** of file system inside the partition - modern Unix/Linux systems can figure this out automatically for most types of file systems if you use the keyword **`auto`** here - **`auto`** is the best idea, since if you’re wrong about the type, the system won’t boot properly - many types here refer to pseudo-file-systems that don’t have real device names, e.g. **`tmpfs`**, **`devpts`**, **`sysfs`**, **`proc`**, and sometimes the device name is given as simply **`none`** 4. Field 4: Optional mount **options** as a comma-separated list (no spaces!) - useful options (see below):   `noauto, user` - the keyword **`defaults`** uses some default options that depend on the file system type - RTFM 5. Field 5: Which file systems to **back up** - A non-zero value indicates backup priority/order - A zero means “don’t back up this file system” 6. Field 6: The order in which **file system checks** are done at boot time - The **`fsck`** program uses this to run file system checks in order - The root should always be checked first - value 1 - Use value 2 or larger for any others that need to be checked - No value or a value of 0 means no file system check File systems do not need to be listed in the **`/etc/fstab`** file to be mounted - you can always mount anything from the command line. Having a file system listed in the **`fstab`** means you only need to specify the **`device`** name or the **`mount point`** to the **`mount`** command; the other information for the mount will be read from the **`/etc/fstab`** file. ### Some common options in **`fstab`** used by **`mount`** - **`auto`** - will be mounted when **`mount`** is used with the -a option (and so mounted at boot time) (default) - **`noauto`** - will be ignored by the -a option (not mounted at boot time) - **`exec`** - permit execution of binaries (default) - **`ro`** - mount the file system read-only (read, execute, no writing) - **`rw`** - mount the file system read-write (default) - **`user`** - let a non-root user mount and unmount this file system - **`nouser`** - only root can mount and unmount this file system (default) - **`defaults`** - use default options:   **`auto, exec, rw, nouser`** - Defaults may vary with distribution! Be careful! Virtual Memory - Swap Partitions ================================ Another use for disk partitions is to hold memory pages used to implement Unix/Linux Virtual Memory. These are called **swap** partitions or the **`swap area`**, and they, too need some minimal structure created to be used by the system. Usually, you find only one swap partition in a system, but systems with a lot of memory may have more. Again, there are the usual three steps: Partition, Create, and Connect Step 1: Create the partition using fdisk ---------------------------------------- - Same as creating any other kind of partition - use **`fdisk`** - Set the partition type ID to be “Linux Swap” to document that this is a swap partition. Many systems will automatically connect and use swap partitions if they have the correct type ID. Step 2: Initialize (format) the swap area using mkswap ------------------------------------------------------ - Make a Linux partition into a swap partition - `mkswap [options] device_name` - e.g.   `mkswap /dev/sdb9` - rarely need any options - This destroys whatever else was in the partition! - you don’t get asked to confirm - type it right the first time The **`device_name`** is the pathname (usually absolute) of the existing swap partition that will be used. These commands do not care about the System ID (type) of a partition given in the partition table. You can initialize as swap any type of partition. Most Linux systems at boot time will automatically use as swap all partitions that have a type of “Linux swap”, if they are initialized as swap. You can add more swap space to a running system using the above two commands. To display the currently active swap partitions, omit the **`device_name`** and use only the **`-s`** or **`--summary`** option to the **`swapon`** command: - Display currently active swap partitions:   `swapon -s` Step 3: Connect the swap area to the system using swapon -------------------------------------------------------- - Use **`swapon`** to activate the swap area you just created with **`mkswap`** - `swapon device_name` - e.g.   `swapon /dev/sdb9` - To list the currently active swap areas: - `swapon -s`         *(list all currently active swap areas)* - Use **`swapoff`** to de-activate (stop using) a swap area: - `swapoff device_name` - e.g.   `swapoff /dev/sdb9` - de-activates that swap partition and allows the partition to be re-formatted and used for something else - the “-a” option will disconnect *all* swap areas (not usually a good idea) - At boot time, partitions in **`/etc/fstab`** marked with type **`swap`** will be automatically connected and used as swap areas - You can manually ask that all **`fstab`** swap partitions be connected using: - `swapon -a`       *(connect and enable all known swap partitions)* - the **`swapoff`** command has an identical option to *disconnect* all active swap areas -- | Ian! D. Allen - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [Linux File System]: file_system.png "Linux File System" [Plain Text]: 720_partitions_and_file_systems.txt [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/