% CST8207 Assignment 11 - processes, syslog, crontab, at, mail, tar % Ian! D. Allen – – [www.idallen.com] % Winter 2014 - January to April 2014 - Updated Mon Mar 31 12:24:33 EDT 2014 Due Date and Deliverables ========================= > **Do not print this assignment on paper!** > > - On paper, you will miss updates, corrections, and hints added to the > online version. > - On paper, you cannot follow any of the [hyperlink URLs] that lead you > to hints and course notes relevant to answering a question. > - On paper, scrolling text boxes will be cut off and not print properly. - **Due Date**: `10h00 (10am) Monday April 7, 2014 (start of Week 13)` - **NEW DUE DATE** - Late assignments or wrong file names may not be marked. Please be accurate and punctual. - **Available online** - Version 1 – 10:40am March 21, 2014 - Version 2 – 10:50am March 25, 2014 – fixed 1 13 111 for `auth.log` and 1 10 86 for `denyhosts` *(thanks to Sam)* - Version 3 – 12:25pm March 31, 2014 – new due date shown above - **Prerequisites** - All [Class Notes][hyperlink URLs] since the beginning of term. - All your previous [Assignments]. - An ability to **READ ALL THE WORDS** to work effectively. - **Deliverables** 1. One text file uploaded to Blackboard according to the steps in the [Checking Program] section below. 2. Directory structure created and left for marking on the [Course Linux Server] (**CLS**).\ **Do not delete any assignment work from the CLS until after the term is over!** **WARNING:** Some inattentive students upload Assignment #11 into the Assignment #10 upload area. Don’t make that mistake! Be exact. Purpose of this Assignment ========================== > **Do not print this assignment on paper!** On paper, you cannot follow any > of the hyperlink URLs that lead you to hints and course notes relevant to > answering a question. This assignment is based on your weekly [Class Notes]. 1. Working with [Unix/Linux Processes] 2. Using [`cron` and `at`] Remember to **READ ALL THE WORDS** to work effectively and not waste time. Introduction and Overview ========================= This is an overview of how you are expected to complete this assignment. Read all the words before you start working. 1. Complete the **Tasks** listed below. 2. Verify your own work before running the **Checking Program**. 3. Run the **Checking Program** to help you find errors. 4. Submit the output of the **Checking Program** to Blackboard before the due date. 5. **READ ALL THE WORDS** to work effectively and not waste time. Practice controlling [Processes and Jobs][Unix/Linux Processes]. Search [System Log Files]. Schedule jobs using [Crontab and At][`cron` and `at`]. Review minimal permissions from [Worksheet #08]. You will create file system structure in your CLS home directory containing various directories and files. You can use the **Checking Program** to check your work as you do the tasks. You can check your work with the checking program as often as you like before you submit your final mark. **Some task sections below require you to finish the whole section before running the checking program; you may not always be able to run the checking program successfully after every single task step.** When you are finished the tasks, leave the files and directories in place on both the CLS as part of your deliverables. **Do not delete any assignment work until after the term is over!** Assignments may be re-marked at any time on the CLS; you must have your term work available on the CLS right until term end. The Source Directory -------------------- All references to the “Source Directory” below are to the CLS directory `~idallen/cst8207/14w/assignment11/` and that name starts with a *tilde* character followed by a userid with no intervening slash. The leading tilde indicates to the shell that the pathname starts with the HOME directory of the account `idallen` (seven letters). You do not have permission to list the names of all the files in the Source Directory, but you can access any files whose names you already know. Tasks ===== - Do the following tasks in order, from top to bottom. - These tasks must be done in your account on the [Course Linux Server]. - Your instructor will mark on the due date the work you do in your account on the CLS. Leave all your work on the CLS and do not modify it. - **Do not delete any assignment work from the CLS until after the course is over.** - **READ ALL THE WORDS!** and do not skip steps. Set Up – The Base Directory on the CLS -------------------------------------- 1. Do a [Remote Login] to the [Course Linux Server] (**CLS**) from any existing computer, using the host name appropriate for whether you are on-campus or off-campus. 2. Make the directory `~/CST8207-14W/Assignments/assignment11`, in which you will create the files and scripts resulting from the following tasks. 3. Create the `check` symbolic link needed to run the **Checking Program**, as described in the section [Checking Program] below. **This `assignment11` directory is the *base* directory for most pathnames in this assignment. Store your files and answers below in this *base* `assignment11` directory.** Run the [Checking Program] on the CLS to verify your work so far. Part A – Process Listing ------------------------ Review [Processes and Jobs][Unix/Linux Processes]. 1. Place a full list of all processes for all users, **BSD** format, all users, text user name (not numeric UID), full wide listing (not truncated at all), into file `psbsd.txt` in your `assignment11` directory. It should be “at least” 105 lines and 9KB. The header line and one of the very long lines will be a `dhclient` line similar to this (use a text-searching command to find it in the output): USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 4312 0.0 0.0 7268 328 ? Ss Mar15 0:00 dhclient3 -e IF_METRIC=9999 -pf /var/run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases -1 eth0 2. Place a full list of all processes for all users, **UNIX** (System V) format, text user name (not numeric UID), full wide listing (not truncated at all), into file `psunix.txt` in your `assignment11` directory. It should be “at least” 105 lines and 7KB. The header line and one of the very long lines will be a `dhclient` line similar to this (use a text-searching command to find it in the output): UID PID PPID C STIME TTY TIME CMD root 4312 1 0 Mar15 ? 00:00:00 dhclient3 -e IF_METRIC=9999 -pf /var/run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases -1 eth0 3. Pick off the first line (the header line) of the `psbsd.txt` file and put the one line into file `mine.txt`. (`1 11 73 mine.txt`) 4. Find all lines in `psbsd.txt` that contain your userid anywhere in the line and append those lines to the `mine.txt` file. Run the [Checking Program] on the CLS to verify your work so far. Part B – System Log Files ------------------------- Review [System Log Files]. 1. What is the actual name of the **syslog** program on the CLS? Search for and extract the one line from each file `psbsd.txt` and `psunix.txt` that contains this name and redirect the results (two lines, one from each file) into file `syslog.txt`. (The result will be 2 lines, 21 words.) 2. The system authentication log file is named `auth.log` in the system log directory. Generate an `ls` long listing of this file using the full absolute pathname, and put the results of the `ls` (the output of `ls`) into file `authlog.txt`. (The result of the `ls` command should be 1 line, 9 words, at least 60 characters.) (Optional: You may find it convenient to set a shell variable to this absolute pathname, so that you can use the short variable name rather than the long absolute pathname in the rest of these exercises. Or, you could make a short symbolic link to this file.) 3. Use one command to put a list (one line) of your numeric uid, your userid, your numeric gid, your group name, and your additional group names into the file `id.txt`. The result will be 1 (long) line, 3 words. (See [Permissions] for the command to use to do this. Do not edit the output of the command.) 4. Look at the contents of the `id.txt` and `authlog.txt` files. Note that your account is in a group that matches the group of the system `auth.log` file, giving you **group permissions** on this file. Using a text editor, put the matching group name (three characters), the symbolic group permissions (three characters), and the octal group permissions (one digit), onto three lines in file `myperms.txt`. (The result will be 3 lines, 3 words, 10 characters. Do you remember why a line containing seven alphanumeric characters on three lines is counted as a 10-character file?) 5. The system `auth.log` file contains over 1,858,190 lines. Use a command to extract just the first line (one line) from the head of this file and redirect that one line into new file `authhead.txt`. (The result will be 1 line 13 words 111 chars.) The time and date at the start of this line is when this copy of the log file was started. 6. If you count the number of lines in the system `auth.log` containing the exact text string `Failed password`, the count is more than 4,060 lines. Of those lines (the lines containing that text string), extract just lines 100 through 110 (inclusive) and put those 11 lines into file `failedpass.txt`. The word count for the file will be `11 164 1214`. Every line should contain the text string somewhere. 7. The CLS is running the `denyhosts` package to blacklist attacking machines. It logs its activity in the expected `denyhosts` log file. There are over 340 lines containing the exact three-word string `new denied host` in this file. Of those lines containing that string, select just the one line that was logged on March 5, 2014 and redirect that one line into file `badhost.txt` (word count `1 10 86`). **Hint:** You did a similar search in [Assignment #06]. Use a pipeline with two text-finding commands to find the only line that matches **both** the date and the desired text. You will need to search for the date in the format that it appears in the log file. 8. In the `auth.log` file search for all lines containing the IP address of the attacking machine (that IP address is the one IP address in the one line you just put in `badhost.txt`) to find out why that IP address was blacklisted. Redirect the output into file `badauth.txt`. The result will be at least 11 lines 137 words 1234 characters. (It may be more, if they attack again!) Every line of output will contain that attacking IP address somewhere in the line. 9. Warning! Do not run the `whois` command more than absolutely necessary, since too many uses may result in us being cut off from using the `whois` database! Run the command and save the output in to a temporary file, then extract the needed information from that temporary file rather than running the command again. Run the `whois` command (once!) and find out where this attacking IP address originates. Put the four output lines from `whois` containing the string `address:` into file `badaddr.txt`. The result will count as `4 21 188`. Feel free to explore some other attacking IP addresses in the `denyhosts` log file to learn what it’s like having a machine exposed on the Internet. (But please be selective about using the `whois` command!) Run the [Checking Program] on the CLS to verify your work so far. Part C – Crontab and At Scheduling ---------------------------------- Review [Crontab and At][`cron` and `at`]. 1. Create a personal `crontab` entry that uses a command to update the modification time on file `crontab.txt` in your `assignment11` directory (not in your `HOME` directory) every minute of every day. Use the shortest relative pathname to your `crontab.txt` file. Do not use an absolute pathname. Verify that the modification time of the file changes every minute. If it doesn’t work, read your Linux email for email messages from the **Cron** daemon showing possible errors. See [Reading eMail] for help. 2. List your personal `crontab` (one entry) and redirect the output into file `crontab1.txt`. 3. Delete your personal `crontab`. 4. Create a personal `crontab` entry that updates the modification time on the same file `crontab.txt` at exactly 7:34 PM on the 20th day of every month. (Also use a relative pathname, not an absolute pathname.) 5. List your personal `crontab` (one entry) and redirect the output into file `crontab2.txt`. Do not delete this `crontab` entry; leave it for marking. 6. Create an `at` job that updates the modification time on the file `atjob.txt` (in your `assignment11` directory, not in your `HOME` directory) one or two minutes from now. (Check the current system time and add a minute or two. Look at the queue of `at` jobs and make sure the scheduled job time is correct.) Make sure that the file time changes one or two minutes from now. If it doesn’t work, check the queue of `at` jobs and make sure the job isn’t queued later than you think it is. Also, read your Linux email for email messages from the **At** daemon showing possible errors. See [Reading eMail] for help. 7. Make `assignment11` your current directory and then create an `at` job that updates the modification time on the file `atjob.txt` (in your current directory, not in your `HOME` directory) at the start date and time of your CST8207 final exam. See the [Crontab and At][`cron` and `at`] course notes or RTFM to find out how to specify both a time and a date. 8. Again, check the queue of `at` jobs and make sure the scheduled time is correct. Leave this job queued on the CLS for marking. 9. Display all your queued `at` jobs and redirect the output into file `atjob.txt`. (You will only have one job – one line.) (If you have more than one line, delete the other jobs first.) 10. Read your CLS Linux email and remove any messages that may be waiting. See [Reading eMail] for help. Run the [Checking Program] on the CLS to verify your work so far. Part D – `tar` archive ---------------------- 1. On the CLS go to your `Assignments` directory. 2. Create a compressed `tar` archive of your `assignment11` directory named `assignment11_`*YYYYMMDD*`.tgz` (no spaces), where *YYYYMMDD* is the date you created the archive backup. - Do *NOT* try to create an archive inside the same directory that you are saving into the archive! - The pathnames in the `tar` file must not include any parent directories of the `assignment11` directory you were asked to save. 3. Move the archive you just created into your `assignment11` directory. 4. Generate a verbose listing of your `tar` archive file, showing the contents of the archive including all the owners and date/time stamps, into file `tar_tvzf.txt` under your `assignment11` directory. Optional: Read the mouse-over text in this [`tar`-related comic] from the [XKCD] webcomic. Run the [Checking Program] on the CLS to verify your work so far. When you are done ----------------- That is all the tasks you need to do. Check your work a final time using the [Checking Program] and save the output as described below. Submit your mark following the directions below. Checking, Marking, and Submitting your Work =========================================== **Summary:** Do some tasks, then run the checking program to verify your work as you go. You can run the checking program as often as you want. When you have the best mark, upload the marks file to Blackboard. > Since I also do manual marking of student assignments, your final mark may > not be the same as the mark submitted using the current version of the > [Checking Program]. I do not guarantee that any version of the [Checking > Program] will find all the errors in your work. Complete your assignments > according to the specifications, not according to the incomplete set of the > mistakes detected by the [Checking Program]. 1. There is a [Checking Program] named `assignment11check` in the [Source Directory] on the CLS. Create a [Symbolic Link] to this program named `check` under your new `assignment11` directory on the CLS so that you can easily run the program to check your work and assign your work a mark on the CLS. Note: You can create a symbolic link to this executable program but you do not have permission to read or copy the program file. 2. Execute the above “check” program on the CLS using its symbolic link. (Review the [Search Path] notes if you forget how to run a program by pathname from the command line.) This program will check your fetched CentOS work, assign you a mark, and display the output on your screen. (You may want to paginate the long output so you can read all of it.) You may run the “check” program as many times as you wish, to correct mistakes and get the best mark. **Some task sections require you to finish the whole section before running the checking program at the end; you may not always be able to run the checking program successfully after every single task step.** 3. When you are done with checking this assignment, and you like what you see on your screen, **redirect** the output of the [Checking Program] into the text file `assignment11.txt` under your `assignment11` directory on the CLS. Use the *exact* name `assignment11.txt` in your `assignment11` directory. Case (upper/lower case letters) matters. Be absolutely accurate, as if your marks depended on it. Do not edit the file. - Make sure the file actually contains the output of the checking program! - The last text line of the file should begin with: `YOUR MARK for` - Really! **MAKE SURE THE FILE HAS YOUR MARKS IN IT!** 4. Transfer the above `assignment11.txt` file from the CLS to your local computer and verify that the file still contains all the output from the checking program. Do not edit this file! No empty files, please! Edited or damaged files will not be marked. You may want to refer to your [File Transfer] notes. - Make sure the file actually contains the output of the checking program! - The last text line of the file should begin with: `YOUR MARK for` - Really! **MAKE SURE THE FILE HAS YOUR MARKS IN IT!** 5. Upload the `assignment11.txt` file under the correct Assignment area on Blackboard (with the exact name) before the due date. Upload the file via the **assignment11** “Upload Assignment” facility in Blackboard: click on the underlined **assignment11** link in Blackboard. Use “**Attach File**” and “**Submit**” to upload your plain text file. No word-processor documents. Do not send email. Use only “Attach File”. Do not enter any text into the **Submission** or **Comments** boxes on Blackboard; I do not read them. Use only the “**Attach File**” section followed by the **Submit** button. If you need to comment on any assignment submission, send me [email]. You can upload the file more than once; I only look at the most recent. You must upload the file with the correct name; you cannot correct the name as you upload it to Blackboard. 6. **Verify that Blackboard has received your submission**: After using the *Submit* button, you will see a page titled *Review Submission History* that will show all your submissions. a) Verify that your latest submission has the correct 16-character, lower-case file name beside the *Attached Files* heading. b) The *Submission Field* and *Student Comments* headings must be **empty**. (I do not read them.) c) **Save a screen capture** showing the uploaded file name. If there is an upload missing, you will need this to prove that you uploaded the file. (Blackboard has never lost a file.) You will also see the *Review Submission History* page any time you already have an assignment attempt uploaded and you click on the underlined **assignment11** link. You cannot delete an assignment attempt, but you can always upload a new version. I only mark the latest version. 7. Your instructor may also mark files in your directory in your CLS account after the due date. Leave everything there on the CLS. **Do not delete any assignment work from the CLS until after the term is over!** - I do not accept any assignment submissions by email. Use only the Blackboard *Attach File*. No word processor documents. Plain Text only. - Use the *exact* file name given above. Upload only one single file of Linux-format plain text, not HTML, not RTF, not MSWord. No fonts, no word-processing. Linux plain text only. - **NO EMAIL, WORD PROCESSOR, PDF, RTF, or HTML DOCUMENTS ACCEPTED.** - No marks are awarded for submitting under the wrong assignment number or for using the wrong file name. Use the exact 16-character, lower-case name given above. - WARNING: Some inattentive students don’t read all these words. Don’t make that mistake! Be exact. **READ ALL THE WORDS. OH PLEASE, PLEASE, PLEASE READ ALL THE WORDS!** -- | Ian! D. Allen - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [hyperlink URLs]: indexcgi.cgi#XImportant_Notes__alphabetical_order_ [Assignments]: indexcgi.cgi#XAssignments [Checking Program]: #checking-marking-and-submitting-your-work [Course Linux Server]: 070_course_linux_server.html [Unix/Linux Processes]: 600_processes_and_jobs.html [`cron` and `at`]: 630_crontab_at_job_scheduler.html [System Log Files]: 580_system_log_files.html [Worksheet #08]: worksheet08.html [Remote Login]: 110_remote_login.html [Permissions]: 500_permissions.html [Reading eMail]: 630_crontab_at_job_scheduler.html#reading-email-from-cron-and-at-jobs [`tar`-related comic]: http://xkcd.net/1168/ [XKCD]: http://xkcd.com/ [Source Directory]: #the-source-directory [Symbolic Link]: 460_symbolic_links.html [Search Path]: 400_search_path.html [File Transfer]: 015_file_transfer.html [email]: mailto:idallen@idallen.ca [Plain Text]: assignment11.txt [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/