% CST8207 Assignment 11 -- tar, syslog, processes, mail, crontab, at, shell script % Ian! D. Allen -- -- [www.idallen.com] % Winter 2018 - January to April 2018 - Updated 2018-04-11 14:05 EDT - [Course Home Page] - [Course Outline] - [All Weeks] - [Plain Text] Due Date and Deliverables ========================= > **Do not print this assignment on paper!** > > - On paper, you will miss updates, corrections, and hints added to the > online version. > - On paper, you cannot follow any of the [hyperlink URLs] that lead you > to hints and course notes relevant to answering a question. > - On paper, scrolling text boxes will be cut off and not print properly. - **Due Date**: `23h59 (11:59pm) Friday April 20, 2018 (end of Week 13)` - This is the last non-bonus assignment for the term. - Late assignments or wrong file names may not be marked. Please be accurate and punctual. - **Available online** - Version 1 -- 05:30 March 15, 2018 - Version 2 -- 14:15 April 11, 2018 -- extended due date; see above. - **Prerequisites** - All [Class Notes][hyperlink URLs] since the beginning of term. - All your previous [Assignments] and [Worksheets]. - An ability to **READ ALL THE WORDS** to work effectively. - **Deliverables** 1. One plain text file uploaded to Blackboard according to the steps in the [Checking Program] section below. 2. Use [Remote Login] to connect to the [Course Linux Server] (**CLS**) and use commands in [The Unix/Linux Shell] to create directory structure and files for marking on the **CLS**.\ **Do not delete any assignment work from the CLS until after the term is over!** **WARNING:** Some inattentive students upload Assignment #11 into the Assignment #10 upload area. Don't make that mistake! Be exact. Purpose and Background ====================== This assignment is based on your weekly [Class Notes][All Weeks] and covers these topics: 1. Working with `tar` archives from [Tar and Gzip] 2. Working with [Processes and Jobs] 3. Working with [System Log Files] (syslog) 4. Using the [Crontab and At Job Schedulers] (`cron` and `at`) 5. [Sending EMail] and [Reading EMail] on the CLS. 6. Writing some very basic [Shell Scripts]. (No flow control.) How to complete this Assignment =============================== For full marks, follow these directions exactly: 1. These tasks must be done in your account via [Remote Login] to the [Course Linux Server]. 2. Do the tasks in order, from top to bottom. Do not skip steps. Most tasks are independent, but some depend on successful completion of a previous task. 3. **READ ALL THE WORDS** in each task before you begin the task, especially all the **Hints**, Notes, and links. 4. Verify your own work before running the **Checking Program**. You won't have a Checking Program at your job interview and the **Checking Program** is not guaranteed to check everything. 5. Run the **Checking Program** at the end of the task to grade your work and help you find some of your errors. A perfect mark from the **Checking Program** does *not* mean your answers are correct. 6. When you are done with this Assignment, submit the output of the **Checking Program** to Blackboard before the due date, following the directions given at the end of this Assignment. Notes on doing assignment work ------------------------------ 1. You can use the **Checking Program** to check your work **after** you have completed each task. Most task sections below require you to **finish the whole task section before running the Checking Program**. You may not always be able to run the **Checking Program** successfully in the middle of a task or after every single task sub-step. The assignment tells you where you can safely check your work. 2. You will create file system structure in your CLS home directory containing various directories and files. When you are finished the tasks, leave the files and directories in place on the CLS as part of your deliverables for your instructor to verify. Assignments may be re-marked at any time on the CLS; you must have your term work available on the CLS right until term end. **Do not delete any assignment work until after the term is over!** 3. You can modify your work and check it with the **Checking Program** as often as you like before you submit your final mark to Blackboard. You can upload your marks to Blackboard as many times as you like before the due date. Partial marks are accepted. 4. Your instructor will also mark on the due date the work you do in your account on the CLS. Leave all your work on the CLS and do not modify it after you have submitted your final mark to Blackboard. 5. You must keep a list of command names used each week and write down what each command does, as described in the [List of Commands You Should Know]. Without that list to remind you what command names to use, you will find future assignments very difficult. Searching the course notes on the CLS ------------------------------------- All course notes are available on the Internet and also on the CLS. You can learn about how to read and search these CLS files using the command line on the CLS under the heading *Copies of the CST8207 course notes* near the bottom of the page [Course Linux Server]. You also learned how to search the notes in [Assignment #05 HTML]. The Source Directory -------------------- All references to the **Source Directory** below are to the CLS directory `~idallen/cst8207/18w/assignment11/` and that name starts with a *tilde* character `~` followed by a user name with no intervening slash. The leading tilde indicates to the shell that the pathname starts with the HOME directory of the account `idallen` (seven letters). You do not have permission to list the names of all the files in the Source Directory, but you can access any files whose names you already know. Tasks ===== Have you completed all the prerequisites, before attempting these tasks? Set Up -- The Base Directory on the CLS --------------------------------------- 1. Do a [Remote Login] to the [Course Linux Server] (**CLS**) from any existing computer, using the host name appropriate for whether you are on-campus or off-campus. **All work in this assignment must be done on the CLS.** 2. Create the `assignment11` directory in your usual `Assignments` directory. **This `assignment11` directory is called the [Base Directory] for most pathnames in this assignment. Store your files and answers in this [Base Directory], not in your HOME directory or anywhere else.** ### `check` 3. Create the `check` symbolic link needed to run the **Checking Program**, as you did in a previous assignment and as described in the section [Checking Program] below. **Hints:** See your previous assignment for hints on doing the above. Use the symbolic link to run the [Checking Program] to verify your work so far. Part A -- Disk Usage, `tar` Archive and Listing ----------------------------------------------- You need to know [Disk Usage] and [Tar and Gzip] to do this task. > Warning: Shell command completion will often add a slash suffix when you > TAB-complete a directory name or a symlink that points to a directory name, > e.g. the shell will TAB-complete `/etc` into `/etc/` when you use the TAB > key. This is mostly helpful, but will give very different results when you > TAB-complete a symlink that points to a directory. If `/tmp/foo` is a > symlink to a directory, then `ls -l /tmp/foo` shows the symlink > itself but `ls -l /tmp/foo/` is the same as `ls -l /tmp/foo/.` > and it shows the contents of the entire directory, not just the symlink > itself. Be careful when using TAB completion! 1. Make your [Base Directory] your current directory and do *not* change directories for this entire task. All recorded pathnames must be *relative* to the [Base Directory]. Later in this assignment you will need to copy all the command lines you use in this part into a script file. All command lines must create and use pathnames *relative* to the current (base) directory. ### `dumaze11` 2. In the current directory (the [Base Directory]), create a directory named `dumaze11` (and those are two digits at the end of the name). ### *abcd0001* 3. Without changing directories, create a symbolic link in that `dumaze11` directory that is the name of your 8-character CLS userid. The symlink should point to (have a target of) the absolute path of the `maze` directory that is in the [Assignment #03] **Source Directory**. The symbolic link will have a size of exactly 43 characters (the absolute path of the `maze` directory), e.g. for userid *abcd0001* the left part of the symlink long listing would look similar to this: lrwxrwxrwx 1 abcd0001 abcd0001 43 Mar 20 10:00 dumaze11/abcd0001 -> The rest of the symlink (not shown) is the absolute path of the [Assignment #03] `maze` directory. You probably have the symlink correct if `ls dumaze11/`*abcd0001* (use your own accout name) shows over 1600 (non-hidden) pathnames from the maze. ### `mzblocks.txt` 4. Use redirection to save a long listing of the one symlink (only the one symlink, not the whole directory) into a new file `mzblocks.txt` in the `dumaze11` directory. The new file will contain one line, 11 words, 112 characters. Do not use any options other than the one that produces a long listing of the symlink. 5. Display the sum total of disk blocks in the hidden `dumaze11/`*abcd0001*`/.0` sub-directory inside the maze. Do not change directories to do this! - Use your *own* symlink and userid in the name, not *abcd0001* - The number of blocks printed should be larger than 5900. - The *relative* pathname beside the number should include your symlink pathname exactly as given above. - You will need this exact *relative* pathname in the next questions. - If you get the wrong pathname, re-read the first sentence of Item #1, above about not changing directories. ### `mzblocks.txt` 6. **Append** the above one-line of output (the number of disk blocks) to the existing `mzblocks.txt` file. - The file will now be two lines long. The second line will contain two words: a number larger than 5900 and a *relative* pathname containing two slashes. - If you get the wrong pathname, re-read the first sentence of Item #1, above about not changing directories. ### *YYYYMMDD*`.tar.gz` 7. Use a single command to create a *gzip*-style compressed `tar` archive in the `dumaze11` directory containing the contents of the `.0` directory from above. Use the *relative* pathname from above as the source of the files to archive. Name the new archive *YYYYMMDD*`.tar.gz` (no spaces) under `dumaze11`, where *YYYYMMDD* is the numeric year-month-day date of the Final Exam in this course (a Saturday in April). - Do not let the `tar` command display the file names as they are added to the archive; add the files silently, not verbosely. - The *gzip*-style compressed archive will be approximately 19,000 bytes. - Generate a table of contents of this archive and show only the first 10 lines on your screen. (Don't show all the lines!) - All the pathnames in the `tar` archive file must be *relative* paths with `dumaze11/abcd0001/.0/` at the beginning of every name. - The archive should contain about 1400 pathnames. (Generate a table of contents and confirm this.) - If you get the wrong pathnames, re-read the first sentence of Item #1, above about not changing directories. ### *YYYYMMDD*`.tar.bz2` 8. Create a *bzip2*-style compressed `tar` archive in the `dumaze11` directory of the same `.0` directory from above. Use the same name as for the *gzip* archive, but use the file extension `.bz2` instead of the `.gz` extension. - Use the correct options to generate the *bzip2*-style compressed archive, and also use the correct output name. - The *bzip2*-style compressed archive will be approximately 6400 bytes. If it's much larger than that, you didn't use the right options to create it. - Generate a table of contents of this archive and show only the first 10 lines on your screen. (Don't show all the lines!) - All the pathnames in the `tar` archive file must be *relative* paths with `dumaze11/abcd0001/.0/` at the beginning of every name. - The archive should contain about 1400 pathnames. (Generate a table of contents and confirm this.) - If you get the wrong pathnames, re-read the first sentence of Item #1, above about not changing directories. 9. Look up the option to `ls` that gives just "the allocated size of each file, in blocks" and use that option (and only that option) to display the size and name of the two `tar` archives you just created in the `dumaze11` directory. - A GLOB pattern will be helpful to generate the two archive pathnames in the `dumaze11` directory as arguments to `ls`. - The output will be one line long, four words, with each tar archive name (relative pathnames) preceded by its size in blocks. ### `mzblocks.txt` 10. **Append** the output of the above command line (the sizes and *relative* pathnames) to the existing disk blocks file you created earlier. The file will now be four lines long. The last three lines will contain exactly two blank-separated words each. - Note how the output of `ls` changes to be on separate lines when output is to a file instead of directly to your screen. This is one of the few commands that does this. - Look at the last three lines in the disk blocks file and note how the compressed `tar` archives are much smaller (fewer disk blocks) than the original disk space used. - Note how the newer *bzip2* compression algorithm is much better at compressing this file (fewer disk blocks) than the older *gzip* algorithm. **Hint:** Go back and re-read the first step #1 in this task if you don't have the right *relative* pathnames in your output. ### `tartable.txt` 11. Generate a verbose listing of the table of contents of your *gzip*-style `tar` archive file, showing the contents of the archive including all the owners and date/time stamps, but don't display it directly on your screen since it's over 1400 lines long. (Generate the verbose listing and verify that it outputs over 1400 lines by counting them with a pipe.) Once you know you can generate the verbose listing, save just the first five and last five lines of the verbose listing into file `tartable.txt` under your `dumaze11` directory. The file word count will be `10 62 806` and the first line and last line should look similar to this (where *abcd0001* is replaced by your userid): drwxr-xr-x idallen/idallen 0 2001-01-01 00:00 dumaze11/abcd0001/.0/ -rw-r--r-- idallen/idallen 156 2001-01-01 00:00 dumaze11/abcd0001/.0/.okek0001*txt **Hints:** You will need to use one command pipeline to generate the first five lines into the output file, and then use a second command pipeline to generate the last five lines and *append* them to the output file to make a total of ten lines in the file. If the checking program says you have unprintable characters in your file, you have not used the right command to generate a verbose listing of a `tar` file; re-read the notes on how to use `tar` given at the top of this Part A section. 12. The *bzip2* compression algorithm is better than the *gzip* compression algorithm; *bzip2* produces smaller compressed files. Answer this question: **True or False:** Because *bzip2* is smaller than *gzip*, generating the verbose table of contents of the *bzip2*-style `tar` archive file will produce fewer lines than the table of contents of the *gzip*-style archive. **Append** your one-word answer `true` or `false` to the `tartable.txt` file. (The file will now contain 11 lines.) *(The checking program will not check this answer. Your instructor will check the answer and mark it after you hand in your assignment.)* Run the [Checking Program] on the CLS to verify your work so far. If you have errors, go back and re-read the first step #1 in this task. Part B -- The Mystery Puzzle File --------------------------------- You need to know [Tar and Gzip] to do this task. After you have run the [Checking Program] at least once, you will find created for you in the [Source Directory] a puzzle file in a `puzzle` subdirectory named with your userid `puzzle/`*abcd0001*`.mystery` where *abcd0001* is replaced by *your own userid*. This file contains many layers of compression and `tar` and `zip` archiving. ### `puzzle.txt` 1. Extract the ASCII text file from deep inside this puzzle file and save the ASCII text as file `puzzle.txt` in your [Base Directory]. **Hints:** Copy the file to your account and then repeatedly unpack the compression and `tar` and `zip` archiving until you find an ASCII text file. The file has many layers (ten or more)! The `file` command will be helpful to identify each nested layer of the puzzle. You will need to fix the permissions and possibly the file extensions on some of the nested files as you try to uncompress and extract them. I will demonstrate one way of decoding this puzzle in class -- see your class notes. Run the [Checking Program] on the CLS to verify your work so far. Part C -- Process Listing ------------------------- You need to know [Processes and Jobs] to do this task. ### `psBSD.txt` 1. Place a list of your own processes in **BSD** format, text user name (not numeric UID) into file `psBSD.txt` in your [Base Directory]. Only a few lines will display and your userid will be at the start of every line. The first header line of the output must look like this (this is **BSD** format): USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND ### `psBSD.txt` 2. **Append** a full list of all processes for all users, **BSD** format, text user name (not numeric UID), full wide listing (not truncated at all) to the same file `psBSD.txt`. The file should now be more than 100 lines and 10KB. One of the very long lines will be a `dhclient` line similar to this (use a text-searching command to find it in the output): root 1026 0.0 0.0 16128 888 ? Ss 02:51 0:00 /sbin/dhclient -1 -v -pf /run/dhclient.enp0s3.pid -lf /var/lib/dhcp/dhclient.enp0s3.leases -I -df /var/lib/dhcp/dhclient6.enp0s3.leases enp0s3 -e IF_METRIC=9999 If you don't see the full line width, re-read the question and use all the options. ### `psmine.txt` 3. Pick off the first line (the header line) of the `psBSD.txt` file and put the one line into the file `psmine.txt` - There is a command that can do this easily. - The file will be 1 line, 11 words, 73 characters. - Save the one BSD header line in the file, not the word count! 4. Find all lines in `psBSD.txt` that contain your userid anywhere in the line and **append** those lines to the `psmine.txt` file. (Some of the lines in this file may be very long.) Run the [Checking Program] on the CLS to verify your work so far. Part D -- System Log Files -------------------------- You need to know [System Log Files] to do this task. ### `syslog.txt` 1. What is the actual name of the **syslog** daemon program that is running on the CLS? Search for and extract the one line from `psBSD.txt` that contains this name and redirect the results (one line) into file `syslog.txt` (one line, 12 words). ### `authlogi.txt` 2. The system authentication log file is named `auth.log` in the system log directory. Generate an `ls` long listing showing inode number of this file using the full absolute pathname and put the output into file `authlogi.txt`. (The file should be one line, 10 words, starting with an inode number, showing the permissions, owner, group, etc.) ### `al` 3. Create an absolute symbolic link in your [Base Directory] to the above system log file. Name the symlink `al` -- a nice short name. You might find using this short symlink useful in subsequent items in this assignment that require you to look in this log file. ### `authlogi.txt` 4. **Append** a long listing of the above symlink to the `authlogi.txt` file. (The file will now contain two lines, 21 words.) ### `meIDs.txt` 5. Use one command name (no arguments) to put a list (one line) of your numeric UID, your userid, your numeric GID, your group name, and your additional group names into the file `meIDs.txt`. The result will be 1 (long) line, 3 words. See [Permissions] for the command to use to do this. Do not edit the output of the command. ### `meperms.txt` 6. Use a text editor (or other means) to enter just the numeric UID for your account into file `meperms.txt` (one number). (The UID number was displayed by the command you used in the previous step.) ### `meperms.txt` 7. Look at the contents of the `meIDs.txt` and `authlogi.txt` files. Note that your account is in a group that matches the group of the system `auth.log` file, giving you **group permissions** on this file. Using a text editor, *append* the matching group name (one word), the symbolic group permissions (three characters), and the octal group permissions (one digit), onto three more lines in file `meperms.txt`. (The appended result will make the file 4 lines and 4 words, with one word on each line in the file.) ### `authhead.txt` 8. The system `auth.log` file for this term contains over 1,115,505 lines (as of March 2018). (Make sure this is true before you continue.) Use a command to extract just the first line (one line) from the beginning of this file and redirect that one line into new file `authhead.txt`. (The result will be 1 line 13 words 108 chars.) The time and date at the start of this line is when this copy of the log file was started. ### `authhead.txt` 9. **Append** the same information that you put in the last line of the `authlogi.txt` file above (about the `al` symlink) to the `authhead.txt` file, making the file two lines long. ### `failedpass.txt` 10. If you count the number of lines in the system `auth.log` containing the exact text string `Failed password`, the count is more than 9,300 lines as of March 2018. (Make sure this is true before you continue.) Of those lines (the lines containing that exact text string), use a command pipeline to extract just lines 111 through 120 (inclusive) and put only those lines into file `failedpass.txt`. Every line should contain the above exact text string somewhere. **Hint:** You solved a similar problem in [Worksheet #05 PDF] and on your midterm and practice tests. The correct output should contain 140 words and 1078 bytes. All the selected log lines have a date on January 16 after 11am. Run the [Checking Program] on the CLS to verify your work so far. Part E -- Crontab ----------------- This section assumes that you have no personal `crontab` created. If you have created one, remove it before you begin and work all the way to the end of this Part before you check your work. You need to know [Crontab and At Job Schedulers] and [Differences between text files][Tar and Gzip] to do this task. Re-read the [Notes on doing assignment work] before you continue. You cannot check your work in the middle of this task part. ### `minutepid.txt` 1. Create a personal `crontab` entry, often called a **cron job**, that echoes the value of the [shell process ID variable] into file `minutepid.txt` in your [Base Directory] every minute of every day. The command in your crontab should redirect the command output into the shortest *relative* pathname to the `minutepid.txt` file. After you create your crontab, verify that the pid number in the file changes every minute. (You may have to wait up to a minute before the cron runs your job for the first time.) **Hints:** If it doesn't work, read your Linux EMail for EMail messages from the **Cron** daemon suggesting possible errors. See [Reading EMail] for help. The single working `crontab` line should have five fields for the date/time, an echo command with a single shell variable argument, and redirection into a *relative* pathname into your [Base Directory], not into your HOME directory. Do not use an absolute pathname. Do *NOT* check your work yet. ### `crontab_list1.txt` 2. Use a command to list your personal `crontab` that you just created (one entry, with perhaps some comment lines) and redirect the output into file `crontab_list1.txt` in your [Base Directory]. This `crontab` entry is the one that runs every minute. 3. Delete your personal `crontab` (the one that runs every minute). ### `differ.txt` 4. Use a command and two absolute pathnames that shows the [Differences between text files][Tar and Gzip] between the [Linux password file][Permissions] and its backup copy. (The backup copy has the same name with a dash "`-`" appended.) A few lines should display, or perhaps none if the files are identical. 5. When you have the password file line difference command correct, create a personal `crontab` entry that runs that same command and redirects the output into the file `differ.txt` in your [Base Directory] at your [Personal Crontab Time]. Also use a *relative* pathname to the output file, not an absolute pathname. ### `crontab_list2.txt` 6. List your personal `crontab` (just one entry running at your [Personal Crontab Time], with perhaps some comment lines) and redirect the output into file `crontab_list2.txt`. Do not delete this personal `crontab` entry; leave it for marking. Make sure your displayed `cron` job is scheduled to run at your [Personal Crontab Time]. Run the [Checking Program] on the CLS to verify your work so far. Re-read the [Notes on doing assignment work] if you are trying to check your work in the middle of a task instead of at the end of a task. Part F -- At Job Reminder ------------------------- Re-read the [Notes on doing assignment work] if you are trying to check your work in the middle of a task instead of at the end of a task. You need to know [Crontab and At Job Schedulers] to do this task. 1. Create an `at` job that prints the list of logged-in users on the system, one per line, at your [Personal Crontab Time] in the year 2020. You can use any of several commands from this course to show the list of logged-in users one per line; see the [List of Commands You Should Know]. **Hints:** You need to get the order of the date correct on the `at` command line; see the [Crontab and At Job Schedulers] course notes or RTFM to find out how to specify both a time and a date for an `at` job. No pipes or redirection or files are needed for this command in the `at` job; it's just running one single command name. 2. Display your list of `at` jobs to confirm the correct scheduling date and time in 2020. 3. Create an `at` job that runs the command that prints the log messages in the kernel ring buffer. Schedule the job at your [Personal Crontab Time] in the year 2020. (See the [List of Commands You Should Know].) **Hints:** No pipes are needed for this `at` job; it's just one command name that displays the in-memory kernel log buffer. Look in your weekly command list for the command name you need. 4. Create an `at` job that echoes the five words of text `CST8207 Final Exam 10:30am Tomorrow` as a mail message to your Algonquin Live EMail account at 1 PM on the day before your final exam in this course. (Your final exam is on a Saturday in April 2018.) The EMail message body sent at 1pm must have the exact message text above and the exact subject line of the EMail must be (five different words): `CST8207 Final Exam on Saturday` **Hints:** A pipe will be needed to connect the one-line output of `echo` with the standard input of the mail program. See [Sending EMail] for help in sending EMail with a subject line. Did you use the exact (different) words given to you for the message and Subject texts? Did you test that your pipeline works (sends you email) before you copied the pipeline into the `at` job for later execution? Make sure that your pipeline actually works before you put it into the `at` job! 5. Check the queue of `at` jobs and make sure the scheduled times are correct for all three jobs. 6. Delete just the (first) `at` job that shows the list of users. ### `atjob.txt` 7. Display both your queued `at` jobs and redirect the output into file `atjob.txt`. You will only have two jobs -- two lines. If you have more than two lines, delete the other jobs. 8. Leave these two jobs queued on the CLS for marking. Run the [Checking Program] on the CLS to verify your work so far. Re-read the [Notes on doing assignment work] if you are trying to check your work in the middle of a task instead of at the end of a task. Part G -- Shell Scripts ----------------------- You need to know basic [Shell Scripts] to do this task. ### Properties of all scripts a. *None* of the scripts in this assignment need control flow statements. They are all simple linear scripts. b. Each script below must begin with the Standard Script Header. See the class notes. c. Though some of the Standard Script Header is executable code, in the descriptions below we don't count those lines, or any comment or blank lines, in the size of the script. We only count the *new* lines of executable code that you write. > For example, a "one-line script" is really several lines of header, a blank > line, a block of several comment lines that [Document Your Script], another > blank line, and then your **one line** of actual script code. The > description below calls this a *one line script*, even though it may > contain a dozen lines. d. Make sure that each of your script files is executable, so that it can be executed as `./scriptname.sh` from the shell command line. e. Build up each script by adding a few lines and testing what you have added; don't write the whole thing and try to debug it! f. Run the given example tests on your scripts to make sure they work. Sample output for each of the scripts is given, so that you may check your work as you proceed. g. Make sure your script handles *all* of the sample inputs given, especially the inputs containing shell metacharacters. (System crackers often attack your system using special characters as input.) h. The examples below do not fully test your script; you will need to try other examples to make sure your scripts work properly for all possible inputs, especially inputs with blanks and shell meta-characters. i. Remember to [double quote all variable expansions] to prevent GLOB and blank expansion that can cause syntax errors and other unwanted problems in your script. j. Scripts must be documented following the rules in [Document Your Script]. k. If you are having problems with your script and are getting error messages from the shell, review [Shell Script Debugging]. ### Checking only one of your scripts Normally the [Checking Program] checks all the scripts. This can be slow if you are only interested in the check output for one script that you are working on. You can check just one or more individual scripts by giving the script names as arguments to the Checking Program: $ ./check partA.sh # only check this script $ ./check partA.sh showargs.sh # only check these two scripts Do not submit for marking the output of checking only a few scripts! ### Shell script: `partA.sh` Topic: **Simple linear shell script; no parameters or control statements** You need to understand basic [Shell Scripts] to do this. No flow control statements are needed. 1. Write an executable Bourne shell script named `partA.sh` that does automatically all the manual work you did creating files in Part A of this assignment. The script must create and use the new output directory name `dumaze99` (in the current directory) instead of the original directory name `dumaze11`. The script should show no output on your screen; it should create all its files with no output on your screen. **Hints and Notes:** a) Do not change directories inside the script. b) The first three executable lines of the script file should be the three lines of the **Standard Script Header**. You can insert `#` comment lines after the first line, but not before it. c) Make the script file editable and executable for only the owner (you). Group can read the file but not write or execute it. No other permissions. **Hint:** Review [Permissions]. d) The script should recursively and silently remove any existing `dumaze99` from the current directory at the start of the script. (You used a command to do this silent recursive removal in almost every new Lab section of every Worksheet.) e) Create a new directory named `dumaze99` in the current directory. f) Run sufficient commands to create the symlink and four files from Part A, inside the new `dumaze99` directory. g) Do *not* change directories inside your script file. h) Use the new `dumaze99` directory for all output. i) The working script must *not* show any output on your screen. > Having already done Part A, your `bash` shell history already contains all > the command lines you need for your script. All you need to do is adapt > each command line to use the `dumaze99` directory instead of the original > directory. My solution script contained three lines of **Standard Script > Header** followed by about ten command lines to create the directory, the > symlink, and the four files from Part A. Make sure you test run your script > before you run the Checking Program! Add comments and a KEY line to [Document Your Script]. Run the [Checking Program] on the CLS to verify your work so far. ### Shell script: `showargs.sh` Topic: **Arguments on the command line and positional parameters:** You need to understand basic [Shell Scripts] with argument parameters to do this. No flow control statements are needed. Create a three-line script named `showargs.sh` that prints to the screen (standard output) exactly three lines: 1. **Line 1:** The name of the script, using the correct shell variable. 2. **Line 2:** The number of arguments given to the script, preceded by a short description telling what this output number means. In the examples below, replace `nargsxxx` with your description. The number must appear on the same line as the description. 3. **Line 3:** All the arguments themselves, preserving blanks, preceded by a short description telling what this output is. In the examples below, replace `argdescyyy` with your description. The arguments must appear on the same line as the description. **Make sure all the examples below work before you run the Checking Program!** Examples: $ ./showargs.sh ./showargs.sh # from the correct shell variable nargsxxx: 0 # use your own words for nargsxxx argdescyyy: # use your own words for argdescyyy $ ./showargs.sh one two 'three four' '*' ./showargs.sh # from the correct shell variable nargsxxx: 4 argdescyyy: one two three four * $ ./showargs.sh foo bar >out $ cat out ./showargs.sh # from the correct shell variable nargsxxx: 2 argdescyyy: foo bar $ ./showargs.sh /bin/* >out $ head -n 2 out ./showargs.sh # from the correct shell variable nargsxxx: 151 # number may differ $ ./showargs.sh /usr/bin/* >out $ head -n 2 out ./showargs.sh # from the correct shell variable nargsxxx: 1782 # number may differ **Hints and Notes:** 1. Review [Properties of all Scripts], above. 2. Write your own words to replace the descriptive text *nargsxxx* and *argdescyyy* above. Explain in your own words what the output is. 3. GLOB characters must *not* expand when output by the script. 4. The standard output is always exactly *three* lines, no more, no less. You should check this yourself to make sure. Add comments and a KEY line to [Document Your Script]. Run the [Checking Program] on the CLS to verify your work so far. Part H -- Personal `bin` directory ---------------------------------- You need to know [Shell Scripts], [Search Path], [Start-Up Files], and [Symbolic Links] to do this task. ### `bin` 1. Create your own personal `bin` directory in which to keep your own personal Linux commands. Use the exact name `bin` and create it in your HOME directory. 2. **Append** your `bin` directory path to the end (tail) of your search PATH. Do this at login time in your shell start-up file so that it applies to all shells. **Hints:** Review [Search Path], [Start-Up Files], and *Working with your search PATH* in [Assignment #07 HTML]. ### `partA` 3. Create a *relative* symlink named `partA` in your `bin` directory. The symlink should point to (have a *target* of) your `partA.sh` script file that you just created. **Hints:** Review [Symbolic Links]. The *relative* symlink target from the `bin` directory to your script will contain four slashes. 4. Make sure that you can now type your new command name `partA` at your shell to execute your `partA.sh` shell script. ### `showargs` 5. Repeat the above for the `showargs.sh` script, naming the symlink `showargs` in your `bin` directory. 6. Make sure that you can now type your new command name `showargs` at your shell to execute your `showargs.sh` shell script. Run the [Checking Program] on the CLS to verify your work so far. When you are done ----------------- That is all the tasks you need to do. Read your CLS Linux EMail and remove any messages that may be waiting. See [Reading EMail] for help. Check your work a final time using the [Checking Program] below and save the standard output of that program into a file as described below. Submit that file (and only that one file) to Blackboard following the directions below. Your instructor will also mark the [Base Directory] in your account on the due date. Leave everything there on the CLS. Do not delete anything. When you are done, log out of the CLS before you close your laptop or close the PuTTY window, by using the shell `exit` command: $ exit Document Your Script ==================== You must document your script with comment lines before you submit it. Script comment lines start with the comment or *hashtag* character `#` and extend to the end of the line. You can (and must) use more than one comment line in your script. Add at least five (or more) comment lines to each script containing the following five types of information, in the following order: 1. The assignment number (copied exactly from the top of the assignment page, i.e. `Assignment 11`). 2. The question number and script name, e.g. `4.8.4 showargs.sh` 3. Your name, your 9-digit student number, and your Algonquin email address. 4. The one-line Signing Key for this script file, generated by running the Checking Program with a first argument of `-s` and a second argument of the script name as you did in **Part A** of [Assignment #09 HTML], e.g. `./check -s showargs.sh` The Signing Key comment line must start with `# KEY:` and will be about 80-89 characters long. 5. A brief summary in your own words of what the script does. The summary can be one or more comment lines long. The comments will be read and marked by your professor after you have submitted your lab; the Checking Program cannot evaluate the quality of what you write. Poor comments means poor marks. Obey these rules for your script comments: 1. Use your own words to *describe* your script; don't copy mine. Your description might document any special features that are worth noting and remembering. 2. The block of five or more comment lines must appear below the standard script header and above your actual script code. 3. A blank line must separate the block of comment lines from the script header above it and another blank line must separate the block of comments from the script code below it. 4. Each comment line except the `KEY` line should be **less than 80 characters long**, to fit on a standard terminal screen nicely. Use multiple comment lines starting with `#` rather than making one huge long comment line. 5. The comments will be read and marked by your professor after you have submitted your lab; the Checking Program cannot evaluate the quality of the documentation that you write. Poor comments means poor marks. Here is a sample comment block for a hypothetical assignment number 99: # Assignment 99 # 1.2.3 foo.sh # Ian Allen 123456789 abcd0001@algonquinlive.com # KEY: foo.sh ==w/XdTMtcDMygDVTN0/zADMx8vY3AjM4Q3cj9Paz5ycnJXY39Gaz9PNwMzM4MDM5QTMV # This is a script that demonstrates how to frob the widjet. # If there are no widjets to frob, the script prints an # error message end exits with status 2. Otherwise exit zero. Make sure you do the correct placement of the comment block in the script file, as described above! The comments will be read and marked by your professor after you have submitted your lab; the Checking Program cannot evaluate the quality of the documentation that you write. Poor comments means poor marks. Checking, Marking, and Submitting your Work =========================================== **Summary:** Do some tasks, then run the **Checking Program** to verify your work as you go. You can run the **Checking Program** as often as you want. When you have the best mark, upload the single file that is the output of the **Checking Program** to Blackboard. > Since I also do manual marking of student assignments, your final mark may > not be the same as the mark submitted using the current version of the > **Checking Program**. I do not guarantee that any version of the **Checking > Program** will find all the errors in your work. Complete your assignments > according to the specifications, not according to the incomplete set of the > mistakes detected by the **Checking Program**. ### `check` 1. There is a **Checking Program** named `assignment11check` in the [Source Directory] on the CLS. Create a symbolic link named `check` in your [Base Directory] that links to the above [Checking Program] in the [Source Directory], as you did in a previous assignment. 2. Execute the above **Checking Program** as a command line on the CLS. The checking program will check your work, assign you a mark, and display the output on your screen: $ ./check | less If the **Checking Program** is not yet ready, it will say `NOT FINISHED YET` and `DO NOT SUBMIT THIS FILE`. No mark is shown; do not submit the file. Wait until the checking program is finished (it gives you a mark) before you save and submit your marks. You may run the **Checking Program** as many times as you wish, allowing you to correct mistakes and get the best mark. **Some task sections require you to finish the whole section before running the *Checking Program* at the end; you may not always be able to run the *Checking Program* successfully after every single task step.** 3. When you are done with this assignment, and you like the mark displayed on your screen by the **Checking Program**, you must **redirect** only the standard output of the **Checking Program** into the text file `assignment11.txt` in your [Base Directory] on the CLS, like this: $ ./check >assignment11.txt $ less assignment11.txt - Use standard output redirection with that *exact* `assignment11.txt` file name. - Use that *exact* name. Case (upper/lower case letters) matters. - Be absolutely accurate, as if your marks depended on it. - Do not edit the output file; the format is fixed. - Make sure the file actually contains the output of the **Checking Program**! - The file should contain, near the bottom, a line starting with: `YOUR MARK for` - Really! **MAKE SURE THE FILE HAS YOUR MARKS IN IT!** 4. Transfer the above single file `assignment11.txt` (containing the output from the **Checking Program**) from the CLS to your local computer. - You may want to refer to the [File Transfer] page for how to transfer the file. - Verify that the file still contains all the output from the **Checking Program**. - Do not edit or open and save this file on your local computer! Edited or damaged files will not be marked. Submit the file exactly as given. - The file should contain, near the bottom, a line starting with: `YOUR MARK for` - Really! **MAKE SURE THE FILE YOU UPLOAD HAS YOUR MARKS IN IT!** 5. Upload the `assignment11.txt` file from your local computer to the correct Assignment area on Blackboard (with the exact name) before the due date: 1. On your local computer use a web browser to log in to Blackboard and go to the Blackboard page for this course. 2. Go to the Blackboard *Assignments* area for the course, in the left side-bar menu, and under there find **assignment11** 3. Under *Assignments*, click on the underlined **assignment11** link for this assignment. a) If this is your first upload, the *Upload Assignment* page will open directly; skip the next sentence. b) If you have already uploaded previously, the *Review Submission History* page will be open and you must use the *Start New* button at the bottom of the page to get to the *Upload Assignment* page. 4. On the *Upload Assignment* page, scroll down and beside *Attach File* use *Browse My Computer* to find and attach your `assignment11.txt` file from your local computer. Make sure the assignment file has the correct name on your local computer before you attach it. Attach *only* your `assignment11.txt` file for upload. Do not attach any other file names. 5. After you have attached the `assignment11.txt` file on the *Upload Assignment* page, scroll down to the bottom of the page and use the *Submit* button to actually upload your attached `assignment11.txt` file to Blackboard. 6. Submit the file exactly as uploaded from the CLS. 7. Do not submit an empty file. Do not submit any other file names. **Make sure the file name on Blackboard is correct!** Use only *Attach File, Browse My Computer* on the *Upload Assignment* page. Do not enter any text into the *Write Submission* or *Add Comments* boxes on Blackboard; I do not read them. Use only the *Attach File, Browse My Computer* section followed by the *Submit* button. If you need to comment on any assignment submission, send me [EMail]. You can revise and upload the file more than once using the *Start New* button on the *Review Submission History* page to open a new *Upload Assignment* page. I only look at the most recent submission. You must upload the file with the correct name from your local computer; you cannot correct the name as you upload it to Blackboard. **Make sure the file name on Blackboard is correct!** 6. **Verify that Blackboard has received your submission**: After using the *Submit* button, you will see a page titled *Review Submission History* that will show all your uploaded submissions for this assignment. Each of your submissions is called an *Attempt* on this page. A drop-down list of all your attempts is available. a) Verify that your latest *Attempt* has the correct 16-character, lower-case file name under the *SUBMISSION* heading. b) The one file name must be the *only* thing under the *SUBMISSION* heading. Only the one file name is allowed. c) No *COMMENTS* heading should be visible on the page. Do not enter any comments when you upload an assignment. d) Click on the *Download* button to open and view the file you just uploaded. **MAKE SURE THE FILE YOU JUST UPLOADED HAS YOUR MARKS IN IT!** e) **Save a screen capture** of the *Review Submission History* page on your local computer, showing the single uploaded file name listed under *SUBMISSION*. If you want to claim that you uploaded the file and Blackboard lost it, you will need this screen capture to prove that you actually uploaded the file. (To date, Blackboard has never lost an uploaded file.) f) Make sure you have used *Submit* and not *Save as Draft*. I cannot mark draft assignments. Make sure you *Submit*. You will also see the *Review Submission History* page any time you already have an assignment attempt uploaded and you click on the underlined **assignment11** link. You can use the *Start New* button on this page to re-upload your assignment as many times as you like. You cannot delete an assignment attempt, but you can always upload a new version. I only mark the latest version. 7. Your instructor may also mark files in your directory in your CLS account after the due date. Leave everything there on the CLS. **Do not delete any assignment work from the CLS until after the term is over!** - I do not accept any assignment submissions by EMail. Use only the Blackboard *Attach File, Browse My Computer*. No word processor documents. Plain Text only. - Use the *exact* file name given above. Upload only one single file of Linux-format plain text, not HTML, not RTF, not MSWord. No fonts, no word-processing. Linux plain text only. - **NO EMAIL, WORD PROCESSOR, PDF, RTF, or HTML DOCUMENTS ACCEPTED.** - No marks are awarded for submitting under the wrong assignment number or for using the wrong file name. Use the exact 16-character, lower-case name given above. - **WARNING:** Some inattentive students don't Read All The Words. Don't make that mistake! Be exact. **READ ALL THE WORDS. OH PLEASE, PLEASE, PLEASE READ ALL THE WORDS!** Appendix I: Your Personal Crontab Time ====================================== This section shows you how to calculate your Personal Crontab Time for use in your cron and at jobs. You need to know your nine-digit student number and how to calculate the arithmetic [modulus] of a number. > 1. Take your 9-digit student number and remove the first three digits > (probably `040`), leaving six digits. Use these last six digits as > follows: > 2. Take the first two of those six digits as a number, [modulo][modulus] > 12, and then add 1, giving a number between 1 and 12. This is your > month number. > 3. Take the next (middle) two of those six digits as a number, > [modulo][modulus] 24, giving a number between 0 and 23. This is your > hour number. > 4. Take the last two of those six digits as a number, [modulo][modulus] > 60, giving a number between 0 and 59. This is your minute number. > 5. Take the same last two of those six digits as a number, > [modulo][modulus] 28, and then add 1, giving a number between 1 and 28. > This is your day-of-the-month number. For example, if your nine-digit student number were `123456789`: 1. Remove the first three digits `123`, leaving the last six digits `45 67 89` 2. Using the first two digits `45`, the month would be `(45 mod 12) + 1 = 10` (October) 3. Using the next two digits `67`, the hour would be `67 mod 24 = 19` (7pm) 4. Using the last two digits `89`, the minute would be `89 mod 60 = 29` 5. Using the last two digits `89`, the day of the month would be `(89 mod 28) + 1 = 6` The Personal Crontab Time for student number `123456789` is October 6 at 19h29 (7:29pm). **Exercise:** Show that the Personal Crontab Time for student number `987654321` is June 22 at 19h21 (7:21pm): 1. last six digits = `65 43 21` 2. month = `6` (June) 3. hour = `19` (7pm) 4. minute = `21` 5. day of month = `22` which is June 22 at 19h21 (7:21pm). -- | Ian! D. Allen, BA, MMath - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Course Home Page]: .. [Course Outline]: course_outline.pdf [All Weeks]: indexcgi.cgi [Plain Text]: assignment11.txt [hyperlink URLs]: indexcgi.cgi#Important_Notes__alphabetical_order_ [Assignments]: indexcgi.cgi#Assignments [Worksheets]: indexcgi.cgi#Worksheets__not_for_hand_in_ [Checking Program]: #checking-marking-and-submitting-your-work [Remote Login]: 110_remote_login.html [Course Linux Server]: 070_course_linux_server.html [The Unix/Linux Shell]: 120_shell_basics.html [Tar and Gzip]: 525_tar_gzip_diff.html [Processes and Jobs]: 600_processes_and_jobs.html [System Log Files]: 580_system_log_files.html [Crontab and At Job Schedulers]: 630_crontab_at_job_scheduler.html [Sending EMail]: 070_course_linux_server.html#sending-email-from-the-cls [Reading EMail]: 630_crontab_at_job_scheduler.html#reading-email-from-cron-and-at-jobs [Shell Scripts]: 700_shell_scripts.html [List of Commands You Should Know]: 900_unix_command_list.html [Assignment #05 HTML]: assignment05.html [Base Directory]: #set-up-the-base-directory-on-the-cls [Disk Usage]: 457_disk_usage.html [Assignment #03]: assignment03.html [Source Directory]: #the-source-directory [Permissions]: 500_permissions.html [Worksheet #05 PDF]: worksheet05.pdf [Notes on doing assignment work]: #notes-on-doing-assignment-work [shell process ID variable]: 320_shell_variables.html [Personal Crontab Time]: #appendix-i-your-personal-crontab-time [Document Your Script]: #document-your-script [double quote all variable expansions]: 440_quotes.html#double-quote-all-variable-expansions [Shell Script Debugging]: 725_debugging_shell_scripts.html [Properties of all Scripts]: #properties-of-all-scripts [Search Path]: 400_search_path.html [Start-Up Files]: 350_startup_files.html [Symbolic Links]: 460_symbolic_links.html [Assignment #07 HTML]: assignment07.html [Assignment #09 HTML]: assignment09.html [File Transfer]: 015_file_transfer.html [EMail]: mailto:idallen@idallen.ca [modulus]: https://en.wikipedia.org/wiki/Modulo_operation [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/