% CST8207 Assignment 09 – tar, syslog, processes, mail, crontab, at % Todd Kelley, Ian! D. Allen – - [www.idallen.com] % Fall 2014 - September to December 2014 - Updated Sun Mar 22 01:50:23 EDT 2015 Due Date and Deliverables ========================= > **Do not print this assignment on paper!** > > - On paper, you will miss updates, corrections, and hints added to the > online version. > - On paper, you cannot follow any of the [hyperlink URLs] that lead you > to hints and course notes relevant to answering a question. > - On paper, scrolling text boxes will be cut off and not print properly. - **Due Date**: `12h00 (noon) Monday November 24, 2014 (start of Week 13)` - You have more than two weeks to do this assignment, but your next assignment will be available soon and will overlap this assignment. Start work on this now! Don’t delay! - Late assignments or wrong file names may not be marked. Please be accurate and punctual. - **Available online** - Version 1 – 05:00 Saturday November 8, 2014 (preview release; not finished; no checking script yet) - Version 2 – 08:35 Monday November 10, 2014 (no checking script yet) - Version 3 – 20:30 Thursday November 13, 2014 (checking script ready) - **Prerequisites** - All [Class Notes][hyperlink URLs] since the beginning of term. - All your previous [Assignments] and [Worksheets]. - An ability to **READ ALL THE WORDS** to work effectively. - **Deliverables** 1. One plain text file uploaded to Blackboard according to the steps in the [Checking Program] section below. 2. Directory structure and files created and left for marking on the [Course Linux Server] (**CLS**).\ **Do not delete any assignment work from the CLS until after the term is over!** **WARNING:** Some inattentive students upload Assignment #9 into the Assignment #8 upload area. Don’t make that mistake! Be exact. Purpose of this Assignment ========================== > **Do not print this assignment on paper!** On paper, you cannot follow any > of the hyperlink URLs that lead you to hints and course notes relevant to > answering a question. This assignment is based on your weekly [Class Notes]. 1. Working with [`tar` archives] 2. Working with [Unix/Linux Processes] 3. Working with [system log files] 4. Using [`cron` and `at`] 5. [Reading eMail] on the CLS. Remember to **READ ALL THE WORDS** to work effectively and not waste time. Introduction and Overview ========================= This is an overview of how you are expected to complete this assignment. Read all the words before you start working. For full marks, follow these directions exactly. 1. Complete the **Tasks** listed below. 2. Verify your own work before running the **Checking Program**. 3. Run the **Checking Program** to help you find errors. 4. Submit the output of the **Checking Program** to Blackboard before the due date. 5. **READ ALL THE WORDS** to work effectively and not waste time. You will create file system structure in your CLS home directory containing various directories and files. You can use the **Checking Program** to check your work as you do the tasks. You can check your work with the checking program as often as you like before you submit your final mark. **Some task sections below require you to finish the whole section before running the checking program; you may not always be able to run the checking program successfully after every single task step.** When you are finished the tasks, leave these files and directories in place on the CLS as part of your deliverables. **Do not delete any assignment work from the CLS until after the term is over!** Assignments may be re-marked at any time on the CLS; you must have your term work available on the CLS right until term end. The Source Directory -------------------- All references to the “Source Directory” below are to the CLS directory `~idallen/cst8207/14f/assignment09/` and that name starts with a *tilde* character `~` followed by a user name with no intervening slash. The leading tilde indicates to the shell that the pathname starts with the HOME directory of the account `idallen` (seven letters). You do not have permission to list the names of all the files in the Source Directory, but you can access any files whose names you already know. Searching the course notes on the CLS ------------------------------------- All course notes are available on the Internet and also on the CLS. You can learn about how to read and search these CLS files using the command line on the CLS under the heading *Copies of the CST8207 course notes* near the bottom of the page [Course Linux Server]. Tasks ===== - Do the following tasks in order, from top to bottom. - These tasks must be done in your account on the [Course Linux Server]. - **READ ALL THE WORDS!** and do not skip steps. - Run the [Checking Program] to grade your work, then upload the file to Blackboard. - Your instructor will also mark on the due date the work you do in your account on the CLS. Leave all your work on the CLS and do not modify it. - **Do not delete any assignment work from the CLS until after the course is over.** Set Up – The Base Directory on the CLS -------------------------------------- 1. Do a [Remote Login] to the [Course Linux Server] (**CLS**) from any existing computer, using the host name appropriate for whether you are on-campus or off-campus. 2. Make the CLS directory `~/CST8207-14F/Assignments/assignment09`, in which you will create the files and scripts resulting from the following tasks. 3. Create the `check` symbolic link needed to run the **Checking Program**, as described in the section [Checking Program] below. > **This `assignment09` directory is the [Base Directory] for most > pathnames in this assignment. Store your files and answers in this [Base > Directory].** Use the symbolic link to run the [Checking Program] to verify your work so far. Part A – Disk Usage, `tar` Archive and Listing ---------------------------------------------- You need to know [Disk Usage] and [Package Management][`tar` archives] to do this task. Optional: Read the mouse-over text in this [`tar`-related comic] from the [XKCD] webcomic. 1. On the CLS go to your `Assignments` directory for this course. Stay in this directory for this entire **Part A** task. 2. Put the sum total of disk blocks in your `assignment02` subdirectory into a new file `02blocks.txt` in your [Base Directory]. - The file will be one line long, containing two words: a number and a relative pathname containing no slashes. 3. Create a compressed `tar` archive in your [Base Directory] of your `assignment02` subdirectory named `assignment02_`*YYYYMMDD*`.tar.gz` (no spaces), where *YYYYMMDD* is the numeric year-month-day date of the final exam in this course. - The archive (output file) must be created in your [Base Directory]. - Do *NOT* try to create an archive inside the same directory that you are saving into the archive! - The pathnames in the `tar` file must not include any parent directories of the `assignment02` directory you were asked to save. 4. Look up the option to `ls` that gives “the allocated size of each file, in blocks” and use that option (and only that option) to display the size and name of the `tar` archive you just created. - The output will be one line long, containing two words: a number and a relative pathname containing one slash. - Append the one line of output (a size and a relative pathname) to the disk blocks file you created earlier. - Look at the two lines in the disk blocks file and note how the compressed `tar` archive is much smaller (fewer disk blocks) than the original disk space used. 5. Generate a verbose listing of your `tar` archive file, showing the contents of the archive including all the owners and date/time stamps, into file `tar_tvzf.txt` under your [Base Directory]. Did you read **All The Words**, especially the words in point #1, above? Run the [Checking Program] on the CLS to verify your work so far. Part B – Process Listing ------------------------ You need to know [Processes and Jobs][Unix/Linux Processes] to do this task. 1. Place a full list of all processes for all users, **BSD** format, all users, text user name (not numeric UID), full wide listing (not truncated at all), into file `psbsd.txt` in your [Base Directory]. It should be “at least” 105 lines and 9KB. The header line and one of the very long lines will be a `dhclient` line similar to this (use a text-searching command to find it in the output): USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 7053 0.0 0.0 7264 496 ? Ss Oct23 0:00 dhclient3 -e IF_METRIC=9999 -pf /var/run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases -1 eth0 2. Place a full list of all processes for all users, **UNIX** (System V) format, text user name (not numeric UID), full wide listing (not truncated at all), into file `psunix.txt` in your [Base Directory]. It should be “at least” 105 lines and 7KB. The header line and one of the very long lines will be a `dhclient` line similar to this (use a text-searching command to find it in the output): UID PID PPID C STIME TTY TIME CMD root 7053 1 0 Oct23 ? 00:00:00 dhclient3 -e IF_METRIC=9999 -pf /var/run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases -1 eth0 3. Pick off the first line (the header line) of the `psbsd.txt` file and put the one line into file `mine.txt`. (`1 11 73 mine.txt`) - There is a command that can do this easily. - Make sure you pick the header line from the right file. 4. Find all lines in `psbsd.txt` that contain your userid anywhere in the line and append those lines to the `mine.txt` file. (Some of the lines in this file may be very long.) > As discussed in class, we will create a symbolic link named > `vi thesis` that will actually run a different command. The `ps` > command will show the name of the symlink, not the name of the program that > is the target being run using the symlink. 5. Create a symbolic link named `vi thesis` in your [Base Directory] that points to (has as a target) the absolute pathname of the system `sleep` command. - **Hints:** You need to find which absolute pathname the shell uses to run the `sleep` command name. See “Commands related to PATH” in [Search Path]. You need to remember how to create a [Symbolic Link]. 6. Confirm that your symlink is working by running a checksum on the `vi thesis` name and making sure it prints the same checksum number as a checksum of the real `sleep` program file. - If you see `vi thesis: No such file or directory`, you didn’t get the symbolic link correct. - If you see `vi: No such file or directory`, you forgot to hide the blank in the name from the shell. See [Quoting]. 7. Run your new `vi thesis` command (that name is in the current directory) as a background job with a single argument of `999` (that the command will interpret as a number of seconds to sleep). Your shell will print a shell background job number and process ID similar to: `[1] 12345` - **Hints:** You need to remember how to execute a command in the current directory. You need to know how to hide blanks (in the command name) from the shell. If you got the symlink correct, you can use the TAB key to ask the shell to complete the command name in the current directory. You do not need any “seconds” qualifier on the `999` number; the number defaults to seconds. 8. Confirm that your `vi thesis` command is running by asking the shell for a listing of jobs of the shell that includes the process ID of the jobs. It will look similar to this (note the process ID is included): [1]+ 12345 Running ./vi\ thesis 999 & Redirect the above output into file `thesis.txt` in your [Base Directory]. 9. Run the `ps` command and you should see output similar to this: PID TTY TIME CMD 12345 pts/5 00:00:00 vi thesis 15000 pts/5 00:00:00 ps 22460 pts/5 00:00:00 bash Note the `vi thesis` command, which is really `sleep`, and could be made to be any command you like if you change the symlink target. (The command name shown by `ps` is not necessarily accurate.) 10. Kill the job, using the special shell syntax for its job number. When you next push the `[Enter]` key, you should see: `[1]+ Terminated ./vi\ thesis 999` Run the [Checking Program] on the CLS to verify your work so far. Part C – System Log Files ------------------------- You need to know [System Log Files][system log files] to do this task. 1. What is the actual name of the **syslog** program on the CLS? Search for and extract the one line from each file `psbsd.txt` and `psunix.txt` that contains this name and redirect the results (two lines, one from each file) into file `syslog.txt`. (The result will be 2 lines, 21 words.) 2. The system authentication log file is named `auth.log` in the system log directory. Generate an `ls` long listing of this file using the full absolute pathname, and put the results of the `ls` (the output of `ls`) into file `authlog.txt`. (The result of the `ls` command should be 1 line, 9 words, at least 60 characters.) (Optional: You may find it convenient to set a shell variable to this absolute pathname, so that you can use the short variable name rather than the long absolute pathname in the rest of these exercises. Or, you could make a short symbolic link to this file.) 3. Use one command to put a list (one line) of your numeric uid, your userid, your numeric gid, your group name, and your additional group names into the file `id.txt`. The result will be 1 (long) line, 3 words. (See [Permissions] for the command to use to do this. Do not edit the output of the command.) 4. Look at the contents of the `id.txt` and `authlog.txt` files. Note that your account is in a group that matches the group of the system `auth.log` file, giving you **group permissions** on this file. Using a text editor, put the matching group name (three characters), the symbolic group permissions (three characters), and the octal group permissions (one digit), onto three lines in file `myperms.txt`. (The result will be 3 lines, 3 words, 10 characters. Do you remember why a line containing seven alphanumeric characters on three lines is counted as a 10-character file on Linux? Review [Line End Differences].) 5. The system `auth.log` file contains over 1,118,500 lines. Use a command to extract just the first line (one line) from the head of this file and redirect that one line into new file `authhead.txt`. (The result will be 1 line 14 words 111 chars.) The time and date at the start of this line is when this copy of the log file was started. 6. If you count the number of lines in the system `auth.log` containing the exact text string `Failed password`, the count is more than 5,500 lines. Of those lines (the lines containing that text string), extract just lines 100 through 110 (inclusive) and put those 11 lines into file `failedpass.txt`. The word count for the file will be `11 158 1213`. Every line should contain the text string somewhere. Run the [Checking Program] on the CLS to verify your work so far. Part D – Crontab ---------------- You need to know [Crontab and At][`cron` and `at`] to do this task. 1. Create a personal `crontab` entry that uses a command to update the (access and) modification time on file `crontab.txt` in your [Base Directory] (not in your `HOME` directory) every minute of every day. Use the shortest relative pathname to your `crontab.txt` file. Do not use an absolute pathname. Verify that the time of the file changes every minute. **Hints:** If it doesn’t work, read your Linux EMail for EMail messages from the **Cron** daemon showing possible errors. See [Reading eMail] for help. The single working `crontab` line should be seven words: five for the date/time, a command name, and a relative file name. 2. List your personal `crontab` (one entry, with perhaps some comment lines) and redirect the output into file `crontab1.txt` in your [Base Directory]. 3. Delete your personal `crontab`. 4. Create a personal `crontab` entry that redirects the current date and time into the same file `crontab.txt` at exactly `9:37PM` on the `20th` day of every month. Also use a relative pathname, not an absolute pathname. 5. List your personal `crontab` (one entry, with perhaps some comment lines) and redirect the output into file `crontab2.txt`. Do not delete this personal `crontab` entry; leave it for marking. Run the [Checking Program] on the CLS to verify your work so far. Part E – At Job Reminder ------------------------ You need to know [Crontab and At][`cron` and `at`] to do this task. 1. Create an `at` job that sends your Algonquin Live account an empty EMail message with the exact five-word subject `CST8207 Final Exam 8AM Today` at 1 AM on the date of your final exam in this course. To send an empty message, redirect standard input from the null device for this command line. **Hints:** See the [Crontab and At][`cron` and `at`] course notes or RTFM to find out how to specify both a time and a date. See [Throwing away input] for shell syntax to redirect standard input. No pipes are needed. See [Sending eMail] for help in sending EMail with a subject line. (Once you get the line working, you may want to also throw away the standard output from `mail` that says “Null message body; hope that’s ok”.) 2. Again, check the queue of `at` jobs and make sure the scheduled time is correct. Leave this job queued on the CLS for marking. 3. Display all your queued `at` jobs and redirect the output into file `atjob.txt`. (You will only have one job – one line.) (If you have more than one line, delete the other jobs first.) Run the [Checking Program] on the CLS to verify your work so far. When you are done ----------------- That is all the tasks you need to do. Read your CLS Linux EMail and remove any messages that may be waiting. See [Reading eMail] for help. Check your work a final time using the [Checking Program] and save the output as described below. Submit your mark following the directions below. Checking, Marking, and Submitting your Work =========================================== **Summary:** Do some tasks, then run the checking program to verify your work as you go. You can run the checking program as often as you want. When you have the best mark, upload the marks file to Blackboard. > Since I also do manual marking of student assignments, your final mark may > not be the same as the mark submitted using the current version of the > [Checking Program]. I do not guarantee that any version of the [Checking > Program] will find all the errors in your work. Complete your assignments > according to the specifications, not according to the incomplete set of the > mistakes detected by the [Checking Program]. 1. There is a [Checking Program] named `assignment09check` in the [Source Directory] on the CLS. Create a [Symbolic Link] to this program named `check` under your new [Base Directory] on the CLS so that you can easily run the program to check your work and assign your work a mark on the CLS. Note: You can create a symbolic link to this executable program but you do not have permission to read or copy the program file. 2. Execute the above `check` program on the CLS using its symbolic link. (Review the [Search Path] notes if you forget how to run a program by pathname from the command line.) This program will check your work, assign you a mark, and display the output on your screen. (You may want to paginate the long output so you can read all of it.) You may run the `check` program as many times as you wish, to correct mistakes and get the best mark. **Some task sections require you to finish the whole section before running the checking program at the end; you may not always be able to run the checking program successfully after every single task step.** 3. When you are done with checking this assignment, and you like what you see on your screen, **redirect** the output of the [Checking Program] into the text file `assignment09.txt` under your [Base Directory] on the CLS. Use that *exact* name. Case (upper/lower case letters) matters. Be absolutely accurate, as if your marks depended on it. - Do not edit the output file. Submit it exactly as given. - Make sure the file actually contains the output of the checking program! - The file should contain near the bottom a line starting with: `YOUR MARK for` - Really! **MAKE SURE THE FILE HAS YOUR MARKS IN IT!** 4. Transfer the above `assignment09.txt` file from the CLS to your local computer and verify that the file still contains all the output from the checking program. Do not edit this file! No empty files, please! Edited or damaged files will not be marked. You may want to refer to your [File Transfer] notes. - Do not edit the output file. Submit it exactly as given. - Make sure the file actually contains the output of the checking program! - The file should contain near the bottom a line starting with: `YOUR MARK for` - Really! **MAKE SURE THE FILE HAS YOUR MARKS IN IT!** 5. Upload the `assignment09.txt` file from your local computer to the correct Assignment area on Blackboard (with the exact name) before the due date: 1. On your local computer use a web browser to log in to Blackboard and go to the Blackboard page for this course. 2. Go to the Blackboard *Assignments* area for the course, in the left side-bar menu, and find the current assignment. 3. Under *Assignments*, click on the underlined **assignment09** link for this assignment. a) If this is your first upload, the *Upload Assignment* page will open directly; skip the next sentence. b) If you have already uploaded previously, the *Review Submission History* page will be open and you must use the *Start New* button at the bottom of the page to get to the *Upload Assignment* page. 4. On the *Upload Assignment* page, scroll down and beside *Attach File* use *Browse My Computer* to find and attach your assignment file from your local computer. Make sure the assignment file has the correct name on your local computer before you attach it. 5. After you have attached the file on the *Upload Assignment* page, scroll down to the bottom of the page and use the *Submit* button to actually upload your attached assignment file to Blackboard. Use only *Attach File* on the *Upload Assignment* page. Do not enter any text into the *Text Submission* or *Comments* boxes on Blackboard; I do not read them. Use only the *Attach File* section followed by the *Submit* button. If you need to comment on any assignment submission, send me [EMail]. You can revise and upload the file more than once using the *Start New* button on the *Review Submission History* page to open a new *Upload Assignment* page. I only look at the most recent submission. You must upload the file with the correct name from your local computer; you cannot correct the name as you upload it to Blackboard. 6. **Verify that Blackboard has received your submission**: After using the *Submit* button, you will see a page titled *Review Submission History* that will show all your uploaded submissions for this assignment. Each of your submissions is called an *Attempt* on this page. A drop-down list of all your attempts is available. a) Verify that your latest *Attempt* has the correct 16-character, lower-case file name under the *SUBMISSION* heading. b) The one file name must be the *only* thing under the *SUBMISSION* heading. Only the one file name is allowed. c) No *COMMENTS* heading should be visible on the page. Do not enter any comments when you upload an assignment. d) **Save a screen capture** of the *Review Submission History* page on your local computer, showing the single uploaded file name listed under *SUBMISSION*. If you want to claim that you uploaded the file and Blackboard lost it, you will need this screen capture to prove that you actually uploaded the file. (To date, Blackboard has never lost an uploaded file.) You will also see the *Review Submission History* page any time you already have an assignment attempt uploaded and you click on the underlined **assignment09** link. You can use the *Start New* button on this page to re-upload your assignment as many times as you like. You cannot delete an assignment attempt, but you can always upload a new version. I only mark the latest version. 7. Your instructor may also mark files in your directory in your CLS account after the due date. Leave everything there on the CLS. **Do not delete any assignment work from the CLS until after the term is over!** - I do not accept any assignment submissions by EMail. Use only the Blackboard *Attach File*. No word processor documents. Plain Text only. - Use the *exact* file name given above. Upload only one single file of Linux-format plain text, not HTML, not RTF, not MSWord. No fonts, no word-processing. Linux plain text only. - **NO EMAIL, WORD PROCESSOR, PDF, RTF, or HTML DOCUMENTS ACCEPTED.** - No marks are awarded for submitting under the wrong assignment number or for using the wrong file name. Use the exact 16-character, lower-case name given above. - WARNING: Some inattentive students don’t read all these words. Don’t make that mistake! Be exact. **READ ALL THE WORDS. OH PLEASE, PLEASE, PLEASE READ ALL THE WORDS!** -- | Todd Kelley, and | Ian! D. Allen - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [hyperlink URLs]: indexcgi.cgi#Important_Notes__alphabetical_order_ [Assignments]: indexcgi.cgi#Assignments [Worksheets]: indexcgi.cgi#Worksheets__not_for_hand_in_ [Checking Program]: #checking-marking-and-submitting-your-work [Course Linux Server]: 070_course_linux_server.html [`tar` archives]: 520_package_management.html [Unix/Linux Processes]: 600_processes_and_jobs.html [system log files]: 580_system_log_files.html [`cron` and `at`]: 630_crontab_at_job_scheduler.html [Reading eMail]: 630_crontab_at_job_scheduler.html#reading-email-from-cron-and-at-jobs [Remote Login]: 110_remote_login.html [Base Directory]: #set-up-the-base-directory-on-the-cls [Disk Usage]: 457_disk_usage.html [`tar`-related comic]: http://xkcd.net/1168/ [XKCD]: http://xkcd.com/ [Search Path]: 400_search_path.html [Symbolic Link]: 460_symbolic_links.html [Quoting]: 440_quotes.html [Permissions]: 500_permissions.html [Line End Differences]: 015_file_transfer.html#text-file-line-end-differences [Throwing away input]: 200_redirection.html#throwing-away-inputoutput-using-devnull [Sending eMail]: 070_course_linux_server.html#sending-email-from-the-cls [Source Directory]: #the-source-directory [File Transfer]: 015_file_transfer.html [EMail]: mailto:idallen@idallen.ca [Plain Text]: assignment09.txt [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/