Winter 2014 - January to April 2014 - Updated 2014-04-12 22:37 EDT
Do not print this assignment on paper!
- On paper, you will miss updates, corrections, and hints added to the online version.
- On paper, you cannot follow any of the hyperlink URLs that lead you to hints and course notes relevant to answering a question.
- On paper, scrolling text boxes will be cut off and not print properly.
23h59 (11:59pm) Saturday April 12, 2014 (end of Week 13)
sudo
and sysadmin account)Do not print this assignment on paper! On paper, you cannot follow any of the hyperlink URLs that lead you to hints and course notes relevant to answering a question.
rsync
file transfer program.This is an overview of how you are expected to complete this assignment. Read all the words before you start working.
Do not print this assignment on paper. On paper, you cannot follow any of the hyperlink URLs that lead you to hints and course notes relevant to answering a question.
You will create filesystem structure in your CLS home directory containing various directories and files. You will also make changes in your own Linux Virtual Machine running CentOS 6.5. You can use the Checking Program to check your work as you do the tasks. You can check your work with the checking program as often as you like before you submit your final mark. Some task sections below require you to finish the whole section before running the checking program; you may not always be able to run the checking program successfully after every single task step.
When you are finished the tasks, leave the files and directories in place on both the CLS and your own Linux Virtual Machine as part of your deliverables. Do not delete any assignment work until after the term is over! Assignments may be re-marked at any time on the CLS; you must have your term work available on the CLS right until term end.
Since we also do manual marking of student assignments, your final mark may not be the same as the mark submitted using the current version of the Checking Program. We do not guarantee that any version of the Checking Program will find all the errors in your work. Complete your assignments according to the specifications, not according to the incomplete set of mistakes detected by the Checking Program.
The previous term’s course notes are always available on the Internet here: CST8207 GNU/Linux Operating Systems I.
All the current and previous terms notes files are also stored on the CLS. You can learn about how to read and search these files using the command line on the CLS under the heading Copies of the CST8207 course notes near the bottom of the page Course Linux Server. The current CST8177 term notes are searchable there, too!
All references to the “Source Directory” below are to the CLS directory
~idallen/cst8177/14w/assignment10/
and that name starts with a
tilde character followed by a userid with no intervening slash.
The leading tilde indicates to the shell that the pathname starts with
the HOME directory of the account idallen
(seven letters).
You do not have permission to list the names of all the files in the Source Directory, but you can access any files whose names you already know.
Review your work from CST8207 GNU/Linux Operating Systems I:
fdisk, mkfs, mount, /etc/fstab
I recommend that once you have booted your CentOS VM, you connect
to it and work using a remote login session (e.g. ssh
or PuTTY
)
where copy-and-paste works and where you can have multiple simultaneous
connections into the VM. The VMware console is not friendly.
If you can’t get an SSH (PuTTY or ssh
) connection working into your
Linux VM, see the Network Diagnostics page.
Note that SSH sessions (and whatever you are doing inside them) do not survive across a VMware suspend. Make sure you save your editor files and exit your SSH session before you pause or suspend your virtual machine. (Editor sessions that run inside the VMware console do survive across suspend and resume, since they don’t depend on a network connection.)
Advanced users may look into the various virtual terminal programs such as
tmux
andscreen
that do allow you to suspend and resume your sessions even from a remote login.
Do a Remote Login to the Course Linux Server (CLS) from any existing computer, using the host name appropriate for whether you are on-campus or off-campus.
Create the CLS directory ~/CST8177-14W/Assignments/assignment10
Create the check
symbolic link needed to run the Checking
Program, as described in the section
Part II - Check and Submit below.
Run Part II - Check and Submit to verify your work so far.
sudoers
group.~/CST8177-14W/Assignments/assignment10
(the same
hierarchy as you have already made on the CLS).This CentOS assignment10
directory in your sysadmin account is the
base directory for all pathnames in this assignment. Store your CentOS
files and answers below in this sysadmin base assignment10
directory.
Run the Fetch and Checking Program to verify your work so far.
sdb
IndexYou will add a second hard disk to your CentOS Virtual Machine, and partition it. The procedure for adding a hard disk to an actual physical computer is different only in the steps that take place while the machine is powered off. Any step carried out while the machine is running would be the same for physical machines as it is for virtual machines. The console of a physical machine is its actual keyboard and monitor, but in the case of a VM, the console is the VMware window of the machine.
Most of the system admin commands in this assignment access the raw disk
and will require you to prefix the actual command name with sudo
to gain root
permissions (unless you are in single-user mode and
therefore running everything as root
).
If you get “permission denied” errors, you forgot to use sudo
.
If your CentOS Virtual Machine is not already powered off, login and use the correct command to power off the virtual machine.
With your CentOS machine still powered off, use the VMware
Settings menu for your CentOS VM to add to your VM a virtual
10GB
hard disk, accepting defaults for everything except the size.
(You did similar work in CST8207 adding a VMware disk; review
the notes.)
After adding the new disk, power on your VM, then login as your system administrator user.
root
since the root
account
password is locked.Ensure the /proc/partitions
file contains the second disk you added.
10485760
.10485760/1024/1024
to confirm the number of gigabytes.sdb1
or sdb2
or other sdb
partitions, this
is not a new disk with no partition table. Get help.When the second disk is correct, copy /proc/partitions
to file
partitions_before.txt
in your CentOS sysadmin base directory (6 lines,
20 words). Remember: all files should eventually be moved to under
your sysadmin base directory on CentOS for marking.
Verify that the three-letter device name for the second disk also
exists under the /dev
directory. Put a long (ls -l
)
listing of all names under /dev
that start with the first two
letters of the new disk name into file sd_all.txt
in your base directory.
fdisk
IndexFirst, you must have added a new 10GB
hard drive in VMware and
rebooted, as described above. Log in to the machine.
Run (always with root
privileges) fdisk -cul /dev/sdb
and make sure you see Disk /dev/sdb: 10.7 GB
with no
errors and no partitions listed under it.
$ sudo fdisk -cul /dev/sdb
Disk /dev/sdb: 10.7 GB, 10737418240 bytes
If you don’t see 10.7 GB
, then shut down, delete the disk,
recreate the disk, and reboot until your 10GB disk install works.
Make sure you only change things on this new
sdb
disk in this section! Thesda
disk is your Linux ROOT disk; if you damage it you will need to recover back to your snapshot. Make sure you have a snapshot to go back to!
In the man
page for the fdisk
command, locate and make a note
of two option letters:
Run the command fdisk
devicename
, where
devicename
is the absolute path of the device corresponding to the
new disk under /dev
. This will start the fdisk
program, just
as you did in CST8207 Fdisk Command.
fdisk
starts, read the WARNING
about DOS-compatible mode.fdisk
program.fdisk
command, this time inserting the two option letters
you found in the man
page. (Keep the same device name.)WARNING
should be gone when you start fdisk
with
those two options. Always use these two options on CentOS.
(Other versions of fdisk
use these options as defaults.)Inside fdisk
use the command to display the partition table and
verify that the disk you are working on is the 10GiBi disk with no
partition table. Use fdisk
commands to partition the new disk as
follows:
fdisk
on the wrong disk.
Make sure you use fdisk
on the new disk device name!+sizeM
where size
is the number of megabytes).fdisk
tells you as it quits whether you need to
reboot for the new partition table to take effect. Do what
it says.Copy the new version of /proc/partitions
(showing the new
partitions you just created) to partitions_after.txt
in your
sysadmin base directory.
/dev
directory.Use the diff
command to find the differences between the old and new
partitions_{before,after}.txt
and redirect the results to
partitions_diff.txt
in your sysadmin base directory.
Examine the differences file, and verify that your new partitions are the only differences. You should see four additional lines in the new partition file, corresponding to the four partitions you created:
6a7,10
> 8 17 512000 sdb1
> 8 18 1 sdb2
> 8 21 409600 sdb5
> 8 22 9561088 sdb6
If your numbers differ, perhaps you forgot to use the fdisk
options that turn off DOS-compatibility mode and switch to using
sectors instead of cylinders. Delete and start over.
/home
directory to its own filesystemIndexYou will create an
ext4
filesystem on the primary partition of the new hard disk. Then, in single user mode, you will migrate the contents of the/home
directory to that new filesystem. You will configure the/etc/fstab
so that the new filesystem will be automatically mounted on/home
, with the option for giving the users disk space quotas.
lsof
package. RTFM to see what it does.Take a snapshot of your CentOS VM.
Run the command file -s /dev/
somedevice
to check the type of the device special file
somedevice
that corresponds to your new primary partition on
your new disk. Because it has nothing on it, you should see nothing
but unknown data
:
/dev/sdb1: data
Create an ext4
filesystem on the only primary partition on the new disk.
Again check the type of the device that corresponds to your new
primary partition on your new disk. It should show an ext4
filesystem:
/dev/sdb1: Linux rev 1.0 ext4 filesystem data (extents) (huge files)
Also check the type of /dev/sda1
that has an active (in
use) ext4
filesystem. Note the warning “needs journal recovery”
indicating this filesystem is open and being modified:
/dev/sda1: Linux rev 1.0 ext4 filesystem data (needs journal recovery) (extents) (large files) (huge files)
Also check the type of /dev/sda2
, that is not an ext4
filesystem.
/home
firstYou are about to make a copy of all the files in the /home
directory.
now
./home
directory. On a real system, you would run a full
back-up before you attempted this. You don’t have a back-up system
running on CentOS, but since the /home
is small, you can create a
tar
archive.Close down any remote login sessions you are running into your
CentOS machine. Exit all PuTTY and SSH connections. These
connections will not work when you shut down to single-user, but
they could leave processes running that might interfere with
moving the /home
directory.
From the console (the actual VMware window, not a remote PuTTY or an
SSH login that will be disconnected) take the system down to single
user mode using shutdown now
to do so. (Do not halt the machine!)
1 S
whoami
and confirm that you are always the root
user when
running in single-user mode. Be careful!Create a compressed tar
archive of /home
and save it under the
HOME directory of the root
account (which is not the ROOT directory)
using the name home.tar.gz
and use file
to confirm that it is a
compressed file:
home.tar.gz: gzip compressed data, from Unix, last modified: ...
The archive should contain everything under the /home
directory.
newusers
command in a previous assignment./home
is empty directories or duplicate files.You will copy the files in /home
to the new partition. First record
the file names so you can make sure the copy works:
Record a recursive, sorted listing of all of the pathnames of your
/home
directory using find /home | sort
and redirecting the
output to a file named home_before.txt
in root
’s
home directory. Your file should contain more than 130 lines,
one for each pathname in the /home
directory.
Ensure no processes are using the /home
directory or
any files under it, with lsof +D /home
/home
is not yet a mount point, you need to use the
+D
option to include every directory under /home
./home
directory./home
./home
, you probably forgot
to exit all remote sessions before going into single-user
mode. Kill all the processes that are using /home
.lsof
command isn’t found, you missed an earlier step.
Exit single-user to multi-user, install the package, and return
to single-user.)You must finish correctly the remaining steps in this section before you reboot, or your sysadmin account will be missing its HOME directory and you will get an error message about that when you log in. You must completely finish the remaining steps in this section correctly to regain log-in access to your sysadmin files in your HOME directory. Do not shut down or reboot your machine in this section, since the reboot will cause the
/home
directory to unmount and all your HOME directories, including the one for your sysadmin account, won’t work. You might want to take another snapshot here before you continue.You can safely use VMware to PAUSE or SUSPEND your CentOS VM in the middle of this work, just don’t shut it down and reboot until you finish this section.
You know from lsof
that nothing is using the /home
directory.
Rename the existing /home
directory to /old_home
/home
has been renamed and is therefore missing.Re-create a new empty /home
directory that will be used as a mount
point for the new filesystem you just created, above.
Mount onto the empty /home
directory the new 500 MB ext4
filesystem that you created earlier.
(Review the mount
command in CST8207 Partitions and File Systems.)
Run the mount
command and confirm that you can see /dev/sdb1
mounted on the /home
directory. Do not proceed until this is true:
/dev/sdb1 on /home type ext4 (rw)
The df -h
command will also show /dev/sdb1
mounted on
/home
, with approximately these sizes:
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 485M 11M 449M 3% /home
Verify that there is a lost+found
directory under /home
now,
because /home
is now a file system mount point instead of just a
plain directory. Do not accidentally delete this directory, or else
the system won’t have a place to put orphan files! (If you delete it,
read the man page for the mklost+found(8)
command and recreate it.)
Use the copy command with the archive option to copy the contents
of the old /old_home
directory to the new 500 MB /home
filesystem.
/old_home
into
/home
and do not copy the name /old_home
in to /home
!/home
and confirm that you do
not see the old_home
directory name there.lost+found
directory inside /home
.Record the list of all pathnames in /home
again, in the
same way, sorted, except redirect the output into a new
file home_after.txt
also in root
’s home directory.
Record the differences between the two home_{before,after}.txt
files, in a file named home_diff.txt
also in root
’s
home directory. (The files should differ by exactly one line;
the new HOME directory has one additional directory in it that wasn’t
in the original. We’ve already told you what its name is.)
Add a record to the /etc/fstab
file so the new /home
filesystem is
mounted automatically, with default options and added quota options for
both users (usrquota
) and groups (grpquota
).
/etc/fstab
.0
) for the sixth field (fsck
pass number).
(RTFM for fstab
and note that using pass number 1
is reserved
for the ROOT file system.)Use the mount
command with the remount
option to remount the
/home
filesystem according to the new options given in /etc/fstab
man mount
and look for the remount
section under the
-o
flag (options).mount
command doesn’t read the new quota options from
/etc/fstab
, then you are specifying both the device and the
mount point to the mount
command, which means it won’t read
the file to get the new options. Don’t do it that way.mount
command has other errors, do not continue. Fix it!Use the mount
command to verify that /home
is now remounted with
the two quota options that you set in /etc/fstab
:
/dev/sdb1 on /home type ext4 (rw,usrquota,grpquota)
Use su -l
to temporarily become your sysadmin account and
verify that your sysadmin HOME directory is valid and contains its
usual files.i
Unmount /home
and then mount it again, relying on the fstab
to
provide the device name:
# umount /home ; mount /home ; mount
You should see no errors, and mount
should show /home
mounted
with the quota options again.
At this point you have verified that the new /home
directory is working.
Your system could be safely shut down and rebooted, but let’s clean
up first.
Return from single-user to runlevel 3 by typing exit
at the
single-user shell. The system will boot multi-user into the
default run level.
Log in using your account (you may use SSH again) and verify that you are in runlevel 3 by running the command that displays the system’s runlevel.
Normally, you would remove the /old_home
directory, and everything
beneath it to free up space on the /
filesystem, reaping the
rewards of moving the /home
directory to its own filesystem;
however, leave the /old_home
directory in place for marking
purposes. Do not remove /old_home
.
Move into your sysadmin base directory the tarball and all
of the *.txt
files you created in the home directory for root
, and
then change the owner and group of those files from root
to yourself
(your sysadmin user whose name is of the form abcd0001
).
sudo
. Why? (Hint: Who is running the shell
that is doing the GLOB expansion before executing sudo
?)Run the Fetch and Checking Program to verify your work so far.
You will add a third hard disk to your CentOS Virtual Machine, and partition it.
Remember to take snapshots often. You can delete the older snapshots if everything works.
Use VMware to add a 2G hard disk to your virtual machine.
Ensure the /proc/partitions
file contains the third disk you added.
2097152
.2097152/1024/1024
to confirm the number of gigabytes.When the third disk is correct, copy /proc/partitions
to file
partitions_LVM_before.txt
in your sysadmin base directory.
Verify that the three-letter device name for the third disk also
exists under the /dev
directory. Repeat the command you used
to create sd_all.txt
and redirect the new output into file
sd_all2.txt
(note the digit 2) in your sysadmin base directory.
(See above for how you created sd_all.txt
.)
Partition the third disk as follows:
WARNING
message about DOS compatibility, you
forgot to use the correct options. Start over.Copy the new version of /proc/partitions
(showing the new
partition you just created) to partitions_LVM_after.txt
in your
sysadmin base directory.
/dev
directory.Use the diff
command to find the differences between the old and new
partitions_LVM_{before,after}.txt
and redirect the results to
partitions_LVM_diff.txt
in your sysadmin base directory.
Examine the differences file, and verify that your new partition is the only difference. You should see only one additional line in the new partition file, corresponding to the single partition you created:
11a12
> 8 33 2096128 sdc1
If your numbers differ, perhaps you forgot to use the fdisk
options that turn off DOS-compatibility mode and switch to using
sectors instead of cylinders. Delete and start over.
You will use LVM capabilities and disk partitions to create the volume group
VolGroup00
and the logical volumeLogVol00
with anext4
filesystem. Then you will extendVolGroup00
using the first logical partition on the first disk you added (the 400MB logical partition). Then you’ll grow theLogVol00
logical volume, and then you’ll grow the filesystem that resides on that volume. Each step of the way, you’ll record the state before, then after, and record the difference.
Display the list of physical volumes (PV) on your VM with pvdisplay
Make the only partition on the third hard disk into a physical volume with
the command pvcreate
devicename
where
devicename
is the absolute path of the device name of the first
(only) partition of the third disk.
dev_is_mpath: failed to get device for 8:33
which you may safely ignore.Physical volume "/dev/sdc1" successfully created
Run the command pvdisplay
with one argument that is the absolute
path of the device name, to show the attributes of this physical volume.
"/dev/sdc1" is a new physical volume of "2.00 GiB"
PV name
is /dev/sdc1
VG name
is blankCreate a new volume group (VG) named VolGroup00
, containing the
/dev/sdc1
physical volume, with the command
vgcreate VolGroup00 /dev/sdc1
Volume group "VolGroup00" successfully created
vgremove
followed by the name.Run the pvdisplay
command again with the same device name and
verify that that physical volume (PV Name) now belongs to the
VolgGroup00
volume group (VG Name).
Run the vgdisplay
command and verify that there are
511 Total PE and 511 Free PE (physical extents) totalling 2GiB.
Create a new logical volume (LV) named LogVol00
, occupying all the
space of VolGroup00
with the command
lvcreate -l 100%FREE -n LogVol00 VolGroup00
Logical volume "LogVol00" created
lvremove
followed by the name.Run vgdisplay
again and verify that now the volume group has
zero free physical extents; all the space is allocated.
Run lvdisplay
and find the LV Path
of the logical volume you
created. This is the name of the device on which you can make a
filesystem, and it should be a pathname under /dev
that includes
your volume group name and your logical volume name.
ls -l
on this device name.LV Path
isn’t actually a device; it’s a
symbolic link to an actual device.Use ls -li
on the absolute path of the actual LV device name (not
the symbolic link) and redirect the output into file dm-0.txt
in
your sysadmin base directory. (The inode number must be included, and the
type of the device must be b
and not l
.)
Using the LV Path
device name, create an ext4
filesystem on
that device.
As you did after creating the ext4
file system on the second
disk, check the type of the actual LV device that corresponds to
LV Path. It should show an ext4
filesystem:
/dev/dm-0: Linux rev 1.0 ext4 filesystem data (extents) (large files) (huge files)
If you see symbolic link
, you forgot to dereference the symlink
to find the real device pathname.
Create a mount point (an empty directory) named /mnt/lvm_fs
to
be used for the new ext4
file system.
Mount the new LV Path ext4
filesystem on the /mnt/lvm_fs
mount point.
Run the mount
command and confirm that you can see LV Path
mounted on the /mnt/lvm_fs
directory. Do not proceed until this is true:
/dev/mapper/VolGroup00-LogVol00 on /mnt/lvm_fs type ext4 (rw)
The df -h
command will also show the same:
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00 2.0G 35M 1.9G 2% /mnt/lvm_fs
Verify that there is a lost+found
directory under /mnt/lvm_fs
because it is a file system mount point.
Record your LVM status by running these commands exactly as shown:
pvdisplay
into a file named pv_before.txt
vgdisplay
into a file named vg_before.txt
lvdisplay
into a file named lv_before.txt
df -h
into a file named df_before.txt
Create another physical volume (PV), using the same method as you used for the first physical volume you created:
file -s
should say the partition is empty (shows data
).Physical volume "/dev/sdb5" successfully created
file -s
should now say the partition is
LVM2 (Linux Logical Volume Manager)
Display the attributes of that physical volume (using its device name), as you did before.
"/dev/sdb5" is a new physical volume of "400.00 MiB"
PV name
is /dev/sdb5
VG name
is blankAdd the new physical volume to the VolGroup00
volume group,
with the command vgextend VolGroup00 /dev/sdb5
Volume group "VolGroup00" successfully extended
Display all the volume groups (only one) and redirect the output
into vg_after.txt
and then use diff
to note the key changes with
respect to vg_before.txt
:
99
Grow the LogVol00
logical volume to consume all the space you
just added to its volume group, with the command
lvextend -l+99 /dev/VolGroup00/LogVol00
-l
is a “dash el” option letter, not “minus one”99
in -l+99
is the number of Free PE from above.Logical volume LogVol00 successfully resized
Display all the logical volume groups (only one) and redirect the
output into lv_after.txt
and then use diff
to note the key
changes with respect to lv_before.txt
:
Display all the volume groups (only one) again and note:
With the /dev/VolGroup00/LogVol00
filesystem sill mounted on
/mnt/lvm_fs
, grow the ext4
filesystem to use all the free space
on the underlying logical volume group by running
the command resize2fs /dev/VolGroup00/LogVol00
Performing an on-line resize
and
The filesystem on /dev/VolGroup00/LogVol00 is now 624640 blocks long.
As you did before, run the command that reports filesystem disk
space usage (in “human” readable form) and redirect the output to
df_after.txt
and then use diff
to note the key changes with
respect to df_before.txt
for the /mnt/lvm_fs
file system:
4c4
< /dev/mapper/VolGroup00-LogVol00 2.0G 35M 1.9G 2% /mnt/lvm_fs
---
> /dev/mapper/VolGroup00-LogVol00 2.4G 35M 2.2G 2% /mnt/lvm_fs
There may be small changes in other file system sizes as well.
You can see the mounted /mnt/lvm_fs
file system has about 400 MB
more space. The space was added without taking the machine down
for maintenance. This is an important feature of LVM and ext4
file
systems: They can be resized “live” without any down-time. If the Linux
kernel supports disk hot-plug, you can even add disks to the machine
without any down-time.
Add a line to /etc/fstab
to mount the volume group as shown above.
/etc/fstab
.0
) for the sixth field (fsck
pass number).umount /mnt/lvm_fs ; mount /mnt/lvm_fs
Record your LVM status by running these commands exactly as shown:
pvdisplay
into a file named pv_after.txt
vgdisplay
into a file named vg_after.txt
lvdisplay
into a file named lv_after.txt
df -h
into a file named df_after.txt
Run the Fetch and Checking Program to verify your work so far.
rsync
IndexThe rsync
command is an intelligent form of copy command that only
transfers data if the data isn’t already there. You will practice using
rsync
between your CentOS VM and its loop-back network adapter, which we
will call the Backup machine (even though it’s really the same machine).
A trivial change to the remote host name lets you transfer files to any
machine on the Internet that lets you run rsync
.
The modern
rsync
command uses an underlying SSH protocol to actually transfer the data, so any configuration you have done for SSH (such as private keys, host aliases, or SSH agents) applies torsync
as well.
Take a snapshot of your CentOS VM. You can never have too many snapshots.
Log in to CentOS as your sysadmin account.
Make sure your 2.4G LVM volume group is mounted on /mnt/lvm_fs
/mnt/lvm_fs
instead.Create a new account user named backup
with these options:
Backup Account
/mnt/lvm_fs/backup
Give the new backup
account a really short password (because you
will be typing it a lot in this assignment).
Tip: The root
user can give an account a “too short”
password if you persist:
$ sudo passwd backup
Changing password for user backup.
New password:
BAD PASSWORD: it is WAY too short
BAD PASSWORD: is a palindrome
Retype new password:
passwd: all authentication tokens updated successfully.
Verify your new backup
account:
ssh backup@localhost id
UID_MIN
and GID_MIN
in /etc/login.defs
ssh backup@localhost pwd
/mnt/lvm_fs/backup
Have you taken a snapshot recently?
Install the rsync
package.
As a simple test, use rsync
to transfer a single file to the Backup
machine using the standard three archive, verbose, and
hard-links options as follows:
$ date >foo
$ rsync -avH foo backup@localhost:
Note the trailing colon (:
) character after the host name in
the destination pathname! Since nothing follows the colon,
the same file name will be used on the remote machine.
The three options -avH
are standard sysadmin use for this
command and are almost always used, just as sysadmin must always
use the -p
option to both cp
and scp
to preserve modes and times.
You should see: sent 121 bytes received 31 bytes
Confirm that the HOME directory of the backup
account now contains
an exact copy of file foo
backup
account is not
under the usual /home
directory. Look in the right place.Confirm that the foo
file in the backup
account has exactly
the same time and date as the one in your own account:
$ sudo diff foo /mnt/lvm_fs/backup/foo
$ sudo ls -l foo /mnt/lvm_fs/backup/foo
The output of ls
must show identical sizes, times, and dates.
See your instructor if you can’t get this one-file transfer working.
Repeat the exact same rsync
command.
sent 49 bytes received 12 bytes
rsync
protocol.Touch foo
and repeat the same rsync
again.
sent 92 bytes received 37 bytes
Redirect a new date
into foo
and repeat the same rsync
again.
sent 121 bytes received 31 bytes
Remove the foo
file and reverse the rsync
to restore a local copy
from the remote Backup machine:
$ rm foo
$ rsync -avH backup@localhost:foo .
:
) character after the host name in
the source pathname, followed by a relative pathname!.
) to copy into the current directory
as a destination pathname! The same file name will be used.-avH
are standard sysadmin use for this
command and are almost always used, just as you must always
use the -p
option to both cp
and scp
to preserve modes and times.sent 30 bytes received 122 bytes
foo
is restored into the current directory.The rsync
command only does the least amount of work needed to make
the remote file or directory the same as the local one (or vice-versa).
Typing backup@localhost
is too much work. You can shorten that.
If necessary, create directory .ssh
in your HOME directory and
remove all permissions for group or other.
Put the following four lines into file config
in the above
.ssh
directory:
Host backup back bk b
Hostname localhost
HostKeyAlias localhost
User backup
Remove all permissions for group or other from the config
file.
Now try these; all should work using the above SSH Host
aliases:
$ rsync -avH foo backup:
$ rsync -avH foo back:
$ rsync -avH foo bk:
$ rsync -avH foo b:
$ rsync -avH b:foo .
You can now use the short SSH host and user alias b:
instead
of typing backup@localhost:
as either a source or destination
host name.
Use rsync
with the standard three sysadmin options to send your entire
HOME directory to the Backup machine under remote directory test1
.
Always use the relative path on the remote machine.
Use rsync
with the added dry-run option so that you can see
what pathnames are being copied. When the pathnames look
correct (see below), remove the dry-run option.
You may use the optional SSH host alias b:
as part of the destination
pathname, if you created it above, otherwise you need to use
the full backup@localhost:
name.
As noted in your rsync
course notes PDF,
be careful how you specify the source pathname for
your HOME directory. You must ensure that every local file
/home/abcd0001/
foo
transfers
to the Backup machine HOME directory as
test1/
foo
and not as abcd0001/test1/
foo
.
If rsync
displays pathnames that begin with your userid,
such as this:
sending incremental file list
created directory test1
abcd0001/
abcd0001/.bash_history
[...etc...]
then your source pathname is NOT correct. If you are not
using the dry-run option, you have to remove the
abcd0001
directory from the backup
account and re-read
your rsync
course notes PDF.
The pathnames transferred should look similar to this:
sending incremental file list
created directory test1
./
.bash_history
[...etc...]
Only when the pathnames look correct should you remove the
rsync
dry-run option and actually transfer the files.
Make sure there is no extra abcd0001
directory under the
test1
directory in the backup
account HOME directory.
Compare a local file and a backed-up file to make sure they are the same, including the time and date:
$ pwd ; echo ~backup
/home/abcd0001/CST8177-14W/Assignments/assignment10
/mnt/lvm_fs/backup
$ sudo diff do.sh ~backup/test1/CST8177-14W/Assignments/assignment10/do.sh
$ sudo ls -l do.sh ~backup/test1/CST8177-14W/Assignments/assignment10/do.sh
Make sure the files are exactly the same.
The output of ls
must show identical sizes, times, and dates.
Put the exact rsync
command line you used into file
rsync_home_test1.txt
in your sysadmin base directory.
Test that you can restore an existing file from the Backup machine to the
/tmp
directory on the local machine. Compare the tmp
copy to
the original file. The two files should be exactly the same,
including the time and date:
$ rsync -avH backup@localhost:test1/CST8177-14W/Assignments/assignment10/do.sh /tmp/foo
$ diff /tmp/foo ~/CST8177-14W/Assignments/assignment10/do.sh
$ ls -l /tmp/foo ~/CST8177-14W/Assignments/assignment10/do.sh
b:
in the source
pathname, if you created it above.assignment10/do.sh
rsync
says failed: No such file or directory
then verify
that the file exists where you think it should be under the
test1
directory in the backup
HOME directory.ls
must show identical sizes, times, and dates.Repeat the exact same rsync
command to the test1
directory that
you did in Step 1 above and that you saved in the
rsync_home_test1.txt
file: sh rsync_home_test1.txt
rsync_home_test1.txt
file should
be the only file that has changed.)speedup is
line at the bottom of the rsync
verbose output that tells you how much faster it was to compare
files and not have to transfer any of the files that were
already there.Change to your sysadmin base directory. (Perhaps you are already there?)
In your sysadmin base directory, touch your existing rsync_home_test1.txt
file and repeat the full HOME directory backup again.
In your sysadmin base directory:
rsync_home_test1.txt
to rsync_base_test1.txt
rsync
with the dry-run option to attempt to update just
the current sysadmin base directory (not your whole HOME
directory) to the corresponding remote sysadmin base directory on the
Backup machine.
rsync
must be simply .
(the
current, sysadmin base directory) not your HOME directory.rsync
will propose
to update only one single file to the remote machine – the new
rsync_base_test1.txt
file.rsync
proposes to transfer all the pathnames in the
current directory, then you have the destination directory wrong.rsync
with the dry-run option says only one file
will be updated from this sysadmin base directory to the
remote sysadmin base directory, remove the dry-run option and
update the Backup machine with the current directory.
rsync
command line you used into file
rsync_base_test1.txt
in your sysadmin base directory.Remove just the one file rsync_base_test1.txt
from the remote
sysadmin base directory as backed up under the test1
directory in the
Backup account, like this:
$ sudo rm ~backup/test1/CST8177-14W/Assignments/assignment10/rsync_base_test1.txt
From your sysadmin base directory repeat the exact same base-directory-only
rsync
command that you saved in the
rsync_base_test1.txt
file: sh rsync_base_test1.txt
rsync_base_test1.txt
Since rsync
can transfer a lot of files in a very short time, always do
a dry-run rsync
before doing the real thing, just to make sure that
you have the pathnames correct! As it says in the rsync
course notes
PDF, the source pathname syntaxes foo
and
foo/.
are NOT the same, and it’s usually foo/.
that you want
to use as a source pathname! Always use the dry-run option first!
Run the Fetch and Checking Program to verify your work so far.
rsync
to other machinesYou can probably see that using rsync
to send files to another machine
is simply a matter of choosing the remote userid and machine name for
the SSH login:
$ date >foo
$ rsync -avH foo backup@localhost:
$ rsync -avH foo abcd0001@cst8177.idallen.ca:
Of course, you need an SSH account on the remote machine, and rsync
must be installed there.
Tip: You might choose to back up your CentOS sysadmin account HOME directory to a backup directory in your account on the CLS every now and then.
Warning: An incorrect use of
rsync
to the CLS can overwrite important files on the CLS. Somersync
options may even delete files. Always use the dry-run option to see whatrsync
proposes before actually lettingrsync
do the transfer.
If you find yourself locked out of a Linux machine, and you have access to the console, booting into single user mode will will often not require a password, and in single-user mode you can change passwords or perform various other repair tasks. (Some systems do password-protect single-user mode, in which case you would need to boot a “live” or “rescue” CD to reset your
root
password.)
Use the correct command to reboot your CentOS VM, and when you see the
GNU GRUB
menu and the countdown timer, halt the GRUB countdown by
pressing the space bar or an arrow key.
Use GRUB to edit your boot options so you boot into single user mode. (Refer to last term’s CST8207 Booting and GRUB.)
Verify that you are in single user mode: when you issue the command
runlevel
, the output should be either N S
or unknown
Note that you are running as root
and can change the password of
any user in single-user mode, including the root
password.
Put the output of the command ps auxww
into a file named
ps_auxww.txt
in your sysadmin base directory, and change the
ownership and group of this file to your ordinary sysadmin user.
(Don’t leave root
-owned files in ordinary user accounts!)
Exit this single-user shell, which will allow the system to boot into the default runlevel.
Log in (using SSH if possible) and verify you’re in the default
runlevel by issuing the runlevel
command.
S 3
Run the Fetch and Checking Program to verify your work so far.
If you find a Linux machine is unbootable, and you have console access, you may be able to rescue it by booting the machine from a “Live CD”. You will use the CentOS installation DVD to boot into “rescue” mode, which is a “Live CD” mode.
Shut down or power down your CentOS VM gracefully using the proper command.
Attach the CentOS Installation ISO image file to your VMware virtual DVD drive, connect it, and make sure it will be connected at Power On. (You did exactly this when you first installed CentOS.)
Access the VMware Settings for your Virtual Machine and increase the RAM to at least 1024MB. (The installer and Rescue mode needs more RAM for the graphics than the server-style CentOS machine.)
Boot into the VMware BIOS of your virtual machine, as follows:
*.vmx
file*.vmx
file to give a longer
pause on the VMware BIOS screen: bios.bootDelay = "60000"
In the VMware BIOS menu, use the keyboard to change the “Boot” settings so that the CD/DVD drive is before the hard disk in the boot order, if it isn’t already.
Save and Exit the VMware BIOS to boot from the virtual CD/DVD drive that contains the CentOS installation ISO image file.
If the ISO image boots, you will see the blue CentOS Welcome boot
screen below with the large CentOS-6
banner on the screen. The
Welcome menu includes the item Rescue installed system
. Use the
arrow keys to select the Rescue line and boot it by pressing Enter:
Follow the instructions on the screen, choosing the defaults, to finish booting into rescue mode, allowing it to search for and mount your Linux system when it asks you:
No
/mnt
that will be used to
mount and access your Linux installation.Continue
and not Read-Only
for your Linux
installation, since we need to write on the file system.At the three-item menu that starts with shell Start shell
,
choose the first item (start a shell).
When you finally have a bash
root
prompt, cat
the password file
to see that this is not your own CentOS system running. It is
the Rescue system, with its own Rescue password file.
Running df
will confirm that your CentOS ROOT partition /dev/sda1
is mounted on directory /mnt/sysimage
and your CentOS HOME partition
/dev/sdb1
is mounted on directory /mnt/sysimage/home
Run ls -l
on all the HOME directories and note that they
all have numeric owners and groups.
Save a copy of the LiveCD’s password file, preserving timestamps,
permissions, etc., to the file livecd_passwd.txt
in your
sysadmin base directory in your mounted CentOS system.
/home
when mounted on the Rescue CD! Read all the words above.)sum
of your livecd_passwd.txt
file should be 63933 2
The owner and group of the livecd_passwd.txt
file you just
created is currently root
. Try (and fail) to change the file to be
owned by your CentOS system admin account.
chown: invalid user: '
abcd0001
'
chown
fail when run from the Rescue CD?
Why is your userid invalid
(doesn’t exist)?Run the command chroot /mnt/sysimage
to obtain a root
shell
running with that directory (your CentOS ROOT) as its ROOT directory.
As long as you remain in the chroot
shell, this directory will
be ROOT.
Now when you cat
the password file, you will see the password
file relative to the new chroot
ROOT directory, which is your
CentOS ROOT directory, so you see your CentOS password file, not
the LiveCD password file.
chroot
shell will behave as if
they used your CentOS file system as the ROOT.grub-install
at
this point, or do any other repairs to your CentOS Linux file system.In the chroot
shell you are running, confirm that you can now see
your livecd_passwd.txt
file in your CentOS sysadmin base directory
using its usual path with respect to the usual ROOT directory:
# cd /home/abcd0001/CST8177-14W/Assignments/assignment10
# ls -l livecd_passwd.txt
Note that the file is still owned by root
.
Change the owner and group of the livecd_passwd.txt
file to your
system admin account. It will succeed this time.
chown
succeed in the chroot
shell
but fail before using chroot
?Exit the chroot
shell back to the Rescue CD shell prompt.
At the Rescue CD shell prompt, use a command to shut down the machine.
With the machine powered off:
256MB
.Power on your CentOS VM. You should see a familiar GNU GRUB
menu.
Boot from local drive
.When your CentOS has rebooted, log back in as your system admin account (using SSH if possible, since it’s nicer than the console).
Change the ownership and group (to you) of any remaining root
owner
or group files anywhere in your system admin account. (If you’ve
done your work carefully, there should be nothing owned by root
.)
root
-owned files as an indication that someone has
broken into the system. Don’t leave root
-owned
files in your own CentOS sysadmin account.root
files in your CLS
assignment directories. This is intentional: don’t delete these!Run the Fetch and Checking Program to verify your work so far.
That is all the tasks you need to do.
Check your work a final time using the Fetch and Checking Program and save the output as described below. Submit your mark following the directions below.
Optional: Keeping your main CentOS Virtual Machine snapshot, remove any intermediate snapshots you no longer require, to free up disk space. - Be careful not to remove your current work!
Summary: Do some tasks, then run the Fetch and checking program to verify your work as you go. You can run the Fetch and checking program as often as you want. When you have the best mark, upload the marks file to Blackboard.
Since I also do manual marking of student assignments, your final mark may not be the same as the mark submitted using the current version of the Checking Program. I do not guarantee that any version of the Checking Program will find all the errors in your work. Complete your assignments according to the specifications, not according to the incomplete set of the mistakes detected by the Checking Program.
The checking program resides on the Course Linux Server, but your work is on your CentOS Virtual Machine. There is a Fetch program that you must download and use on your CentOS Virtual Machine to copy information from your CentOS Virtual Machine to your account on the CLS so that the checking program can check it on the CLS.
Once the Fetch program has fetched these files from your Virtual Machine to the CLS, you can run the checking program on the CLS to check what is saved in the files. When you make changes on your CentOS Virtual Machine, you need to run the Fetch program again on CentOS to update the saved files on the CLS.
Simply running the checking program on the CLS will not update the saved files on the CLS. You must run the Fetch program on your CentOS VM when you make changes on your CentOS Virtual Machine.
Do all the following steps on your CentOS Virtual Machine. Read through the whole list before you start typing anything. An example of what to type is given below the descriptions that follow.
Failure to read all the words will lock your account out of the CLS.
root
account
(same userid as Blackboard).CST8177-14W/Assignments/assignment10
(use the same directory
hierarchy as you already have in your own account on the CLS).
This is your base directory for this assignment.curl
to get a copy of the Fetch program
from the given URL into a file named do.sh
.
Make sure you have a file named do.sh
in your sysadmin base
directory. You only need to download this once per assignment.curl
program.$ whoami ; hostname ; pwd
abcd0001 # your userid, not abcd0001
abcd0001 # your userid, not abcd0001
/home/abcd0001/CST8177-14W/Assignments/assignment10
$ url=http://teaching.idallen.com/cst8177/14w/notes/data/assignment10do.sh
$ curl -A mozilla "$url" >do.sh
[... make sure you scroll right to read the full web URL above ...]
[... various download statistics print here ...]
$ fgrep -i 'error' do.sh # make sure no errors (no output)
$ head -n1 do.sh # make sure it's a shell script
#!/bin/sh -u
You must run the do.sh
script you just downloaded. You must run
the script as the root
user with the USER
environment variable
set to your own CLS account userid. (Do not use abcd0001
;
use your own.) Failure to set the USER=
variable as shown below
will cause your account to be locked out of the CLS.
As shown below, use sudo
and sh
to run the do.sh
script you
just downloaded to CentOS with the USER
environment variable set
to your own CLS account userid (as stored in the USER
variable).
$ echo "$USER" ; pwd
abcd0001 # your userid, not abcd0001
/home/abcd0001/CST8177-14W/Assignments/assignment10
$ sudo USER=$USER sh do.sh
This do.sh
script runs a Fetch program that will connect from
your CentOS machine to the CLS using your account name in the USER
variable. It will copy selected files from your CentOS machine
to your assignment10
directory on the CLS. It will then run the
checking program on the CLS to check your work. You will need to
answer one question about your IP address, and then wait and type
in your CLS password, as shown below:
---------------------------------------------------------------------------
abcd0001: FETCH version 3. Connecting to CLS as USER='abcd0001' using ssh
---------------------------------------------------------------------------
abcd0001: Use local Algonquin IP cst8177-alg.idallen.ca [y/N/?]? n
abcd0001: Please wait; using ssh to connect to user 'abcd0001' on cst8177.idallen.ca ...
*** COURSE LINUX SERVER ***
abcd0001@cst8177.idallen.ca's password: # enter your CLS password
---------------------------------------------------------------------------
idallen-ubuntu assignment10fetch_server.sh version 8 run by abcd0001.
Please wait; collecting info from abcd0001 Virtual Machine
---------------------------------------------------------------------------
VM files collected into CST8177-14W/Assignments/assignment10/abcd0001.tar.bz on CLS.
Now running checking program for abcd0001 on CLS:
[... checking program output appears here ...]
tar
archive in your account under
assignment10
on the CLS and then runs the checking program on the CLS.
If you only run the checking program on the CLS, it won’t update the
files from your CentOS VM and it will just check the existing files
saved under assignment10
on the CLS..bashrc
file or
world-writable files on the CLS). These errors are on the CLS, not
on your CentOS machine.When you are done with your assignment, you need to run the checking program one last time on the CLS (not from CentOS) and submit the output file, as follows:
Do all this on the Course Linux Server when you are ready to submit:
There is a Checking Program named assignment10check
in the
Source Directory on the CLS. Create a Symbolic Link to this
program named check
under your new assignment10
directory on the
CLS so that you can easily run the program to check your work and
assign your work a mark on the CLS. Note: You can create a symbolic
link to this executable program but you do not have permission to
read or copy the program file.
Execute the above “check” program on the CLS using its symbolic link. (Review the Search Path notes if you forget how to run a program by pathname from the command line.) This program will check your fetched CentOS work, assign you a mark, and display the output on your screen. (You may want to paginate the long output so you can read all of it.)
Remember: The checking program does not fetch new files to the CLS from your CentOS VM. You must run the Fetch program on your CentOS VM to update the fetched files on the CLS so that the checking program can mark them on the CLS.
You may run the “check” program as many times as you wish, to correct mistakes and get the best mark. Some task sections require you to finish the whole section before running the checking program at the end; you may not always be able to run the checking program successfully after every single task step.
When you are done with checking this assignment, and you like what
you see on your screen, redirect the output of the Checking Program
into the text file assignment10.txt
under your assignment10
directory on the CLS. Use the exact name assignment10.txt
in your
assignment10
directory. Case (upper/lower case letters) matters.
Be absolutely accurate, as if your marks depended on it. Do not
edit the file.
YOUR MARK for
Transfer the above assignment10.txt
file from the CLS to your local
computer and verify that the file still contains all the output from
the checking program. Do not edit this file! No empty files, please!
Edited or damaged files will not be marked. You may want to refer
to your File Transfer notes.
YOUR MARK for
Upload the assignment10.txt
file under the correct Assignment
area on Blackboard (with the exact correct name) before the due
date. Upload the file via the assignment10 “Upload Assignment”
facility in Blackboard: click on the underlined assignment10
link in Blackboard. Use “Attach File” and “Submit” to upload
your plain text file.
No word-processor documents. Do not send email. Use only “Attach File”. Do not enter any text into the Submission or Comments boxes on Blackboard; I do not read them. Use only the “Attach File” section followed by the Submit button. If you need to comment on any assignment submission, send me email.
You can upload the file more than once; I only look at the most recent. You must upload the file with the correct name; you cannot correct the name as you upload it to Blackboard.
Verify that Blackboard has received your submission: After using the Submit button, you will see a page titled Review Submission History that will show all your submissions.
You will also see the Review Submission History page any time you already have an assignment attempt uploaded and you click on the underlined assignment10 link.
You cannot delete an assignment attempt, but you can always upload a new version. I only mark the latest version.
Your instructor may also mark files in your directory in your CLS account after the due date. Leave everything there on the CLS. Do not delete any assignment work from the CLS until after the term is over!
I do not accept any assignment submissions by email. Use only the Blackboard Attach File. No word processor documents. Plain Text only.
Use the exact file name given above. Upload only one single file of Linux-format plain text, not HTML, not RTF, not MSWord. No fonts, no word-processing. Linux plain text only.
NO EMAIL, WORD PROCESSOR, PDF, RTF, or HTML DOCUMENTS ACCEPTED.
No marks are awarded for submitting under the wrong assignment number or for using the wrong file name. Use the exact 16-character, lower-case name given above.
WARNING: Some inattentive students don’t read all these words. Don’t make that mistake! Be exact.
READ ALL THE WORDS. OH PLEASE, PLEASE, PLEASE READ ALL THE WORDS!