% CST8207 Assignment 05 - GLOB, aliases, options, and start-up files % Ian! D. Allen - idallen@idallen.ca - www.idallen.com % Winter 2013 - January to April 2013 - Updated Wed May 8 21:04:31 EDT 2013 Due Date and Deliverables ========================= - **Due Date**: `11h00 (11am) Monday March 4, 2013 (start of Week 8)` - Late assignments or wrong file names may not be marked. Be accurate. - **Available online**: - Version 01: 07h30 Friday February 15, 2013 - **Prerequisites**: - The current [Class Notes] - an ability to **READ ALL THE WORDS** to work effectively - **Deliverables**: 1. One text file uploaded to Blackboard according to the steps in the [Checking Program] section below. 2. Directory structure created and left for marking on the [Course Linux Server] (**CLS**).\ **Do not delete any assignment work from the CLS until after the term is over!** **WARNING:** Some inattentive students upload Assignment #5 into the Assignment #4 upload area. Don’t make that mistake! Be exact. Purpose of this Assignment ========================== This assignment is based on your weekly [Class Notes]. 1. Learn to use the VI/VIM text editor. 2. Use VI/VIM to create your profile files with sysadmin aliases and options. 3. Practice with GLOB and I/O redirection in [Lab Worksheet #05 ODT]. 4. Use GLOB patterns and redirection to select files. 5. Use `grep` patterns to search the course notes for keywords. 6. Copy 100 files based on a complex GLOB pattern. 7. Transfer and submit a file to Blackboard. Remember to **READ ALL THE WORDS** to work effectively and not waste time. Introduction and Overview ========================= This is an overview of what you will do in this assignment. Read all the words before you start working. Complete the numbered [Tasks] listed below on the [Course Linux Server] (**CLS**). Run a [Checking Program] to verify your work as you finish each task. Submit your marks to Blackboard. You will create file system structure in your CLS home directory containing various directories and files. You can use the [Checking Program] to check your work as you do the tasks. You can check your work with the checking program as often as you like before you submit your final mark. (Some tasks sections below require you to finish the whole section before running the checking program; you may not always be able to run the checking program successfully after every single task step.) When you are finished the tasks, leave these files, directories, and links in place on the CLS as part of your deliverables. **Do not delete any assignment work from the CLS until after the term is over!** Assignments may be re-marked at any time on the CLS; you must have your term work available on the CLS right until term end. Tasks ===== - Do all this work logged in to your account on the CLS. - Do the following numbered tasks in order, from top to bottom. **READ ALL THE WORDS!** and do not skip steps. - Your instructor will mark on the due date the work you do in your account on the CLS. Leave all your work on the CLS and do not modify it. **Do not delete any assignment work from the CLS until after the course is over.** The Source Directory -------------------- All references to the “Source Directory” below are to the directory `~idallen/cst8207/13w/assignment05/` and that name starts with a *tilde* character “`~`” followed by a userid with no intervening slash. The leading tilde indicates to the shell that the pathname starts with the HOME directory of the account `idallen`. Set Up ------ 1. Create the following directory structure in your CLS HOME directory and record (for study purposes) the series of Unix commands you used to create it. Spelling and capitalization must be exactly as shown: CST8207-13W/ `-- Assignments `-- assignment05 2. Make a directory named `warez` under your new `assignment05` directory. 3. Make a directory named `osfiles` under your new `assignment05` directory. Run the [Checking Program] to verify your work so far. Creating profile files using VIM -------------------------------- 1. Review the [Class Notes]: - [The VI (VIM) Text Editor] - [Shell Variables - Basics (non-scripting)] - [Login and Shell Startup Files, aliases, options: .bash\_profile and .bashrc] - [List of Commands] 2. Print the front side of the recommended [VIM reference card PDF]. 3. Complete the command-line `vimtutor` VIM tutorial (in any language). Make use of the front of the recommended VIM reference card. Make notes as you do the tutorial so that you know how to edit a file. See also [Bonus Assignment #6] for more VIM practice, if you have extra time and want bonus marks. You do not need to do the bonus assignment. 4. Use VIM to create your `.bash_profile` and `.bashrc` files with the suggested content described in the page [Startup Files][Login and Shell Startup Files, aliases, options: .bash\_profile and .bashrc]. Read the man pages for each command to know what the options to each command mean. The `cp -p` option is very useful for sysadmin. **Do not set any options or aliases in your `.bashrc` that you do not understand!** If you don’t know the meaning of a setting, don’t use it. You can RTFM in the `bash` man page for all BASH settings, and RTFM in command man pages to learn about options to commands. 5. Using the `PS1` variable from [Worksheet #2 HTML], set your prompt to include your user name, your computer name, and the basename of your current working directory. Put this setting at the bottom of your `.bashrc` file. 6. Verify that nothing prints on your screen after you enter your password when you run the non-interactive shell connection using `ssh localhost true` (as described in the section on [Non-interactive shells and PS1]: $ ssh localhost true *** COURSE LINUX SERVER *** user@localhost's password: $ There must be **no** output on your screen after you enter your password. Your instructor will mark the `.bashrc` and `.bash_profile` files in your account on the assignment due date. Leave them there on the CLS. Do not delete anything. Run the [Checking Program] to verify your work so far. Learning to use GLOB and Redirection ------------------------------------ 1. Review the [Class Notes]: - [Shell GLOB patterns (wildcard pathname matching)] - [Unix Shell I/O Redirection (including Pipes)] - [List of Commands] 2. Do the [Lab Worksheet #05 ODT] before continuing. - Record all your answers for later quizzes. Searching for files using GLOB ------------------------------ 1. In your HOME directory, create two symbolic links to the old and new course notes for CST8207 using the method described in [Copies of the CST8207 Course Notes]. (The old notes must be term `12f` and the new notes must be term `13w`.) 2. In your HOME directory, use the `echo` command with a shell GLOB pattern to match all pathnames under `oldnotes/` that end in `.txt` and display all the names on your screen. (Do not use `grep`.) The shell will find 68 file names ending in `.txt`, and the `echo` command will display those 68 names on your screen. Hint: The example in the notes already uses this GLOB pattern to generate file names for the `grep` command. Use `echo` not `grep` to display the pathnames. 3. When the `echo` output on your screen is correct, pipe the output into the `wc` command and confirm that it says: `1 68 1995`. 4. In your HOME directory, use the `echo` command with a shell GLOB pattern to match pathnames under `oldnotes/` that contain the word `symbolic` anywhere in the file name and display the names on your screen. (Do not use `grep`.) The shell will find two file names, one ending in `.html` and the other in `.txt`, and the `echo` command will display those two names on your screen. 5. When the `echo` output on your screen is correct, pipe the output into the `wc` command and confirm that it says: `1 2 65`. 6. Remove the pipe and `wc` and redirect the `echo` output into a new file named `echofound` under your existing `assignment05` directory. The output of `wc echofound` should still be `1 2 65`. 7. In your HOME directory, use the `echo` command with a shell GLOB pattern to match pathnames under `oldnotes/` that contain the word `vi` anywhere in the file name and end in the extension `.pdf`. The shell will find two file names, each ending in `.pdf`. 8. When the `echo` output on your screen is correct, pipe the output into the `wc` command and confirm that it says: `1 2 59`. 9. Remove the pipe and `wc` and *append* the `echo` output onto the end of the existing file `echofound` you created previously, adding to what is already in the file. The output of `wc echofound` should now be `2 4 124` showing that the file contains both lines. Run the [Checking Program] to verify your work so far. Searching for patterns in the course notes ------------------------------------------ As mentioned in [Lab Worksheet #03 HTML], choose which searching command you use depending on whether special characters are being used in the search string. 1. In your HOME directory, create two symbolic links to the old and new course notes for CST8207 using the method described in [Copies of the CST8207 Course Notes], unless you already have these links created. Note the example use of `grep` with GLOB patterns to match `*.txt` files in these notes directories. What do the `-i` and `-l` options to `grep` do? Know when to use them, and when not to use them. 2. In your HOME directory, use the `grep` command with a shell GLOB pattern to match all pathnames under `oldnotes/` that end in `.txt` and search for the word `Einstein` in those files. One line of text from inside one of the weekly notes files will display. (Hint: This is almost identical to the example use of `grep` referred to above.) 3. When the `grep` output on your screen is correct, pipe the output into the `wc` command and confirm that it says: `1 8 125`. 4. Remove the pipe and `wc` and redirect the `grep` output into a new file named `grepfound` under your existing `assignment05` directory. The output of `wc grepfound` should still be `1 8 125`. 5. In your HOME directory, use the `grep` command with a shell GLOB pattern to match all pathnames under `oldnotes/` that end in `.txt` and search for the word `nano` in those files. Two lines of text will display. 6. When the `grep` output on your screen is correct, pipe the output into the `wc` command and confirm that it says: `2 11 149`. 7. Remove the pipe and `wc` and *append* the `grep` output onto the end of the existing file `grepfound` you created previously, adding to what is already in the file. The output of `wc grepfound` should now be `3 19 274` showing that the file contains all three lines. Run the [Checking Program] to verify your work so far. The cracker WAREZ files ----------------------- The “story” here is that a cracker has hidden a bunch of WAREZ files in a directory on the server. (See .) Your job is to take a copy of these files, and only these files, for use in a court case. You must not copy any other files, only the WAREZ files. 1. Under the [Source Directory] there is a directory named `start`. Hidden deeper under this directory is one single directory containing about **99,900** files. Be careful about typing `ls` in this directory without using any output pagination pipe - the amount of output may flood your terminal window for some time. Exactly 100 of these files in that one directory have names that contain your userid (lower-case) followed somewhere later by the string `warez`, where `warez` may appear in any combination of upper- and lower-case letters, e.g. `warez`,`Warez`,`wArez`,`waREz`, etc. Some sample file names for userid `abcd0001` might look like these (note that the `warez` word *must* always follow the userid in all the file names): - `HhUtfgYtyGhjJADGekCAkgtZEKsTGKdYZZabcd0001ADGekCwaREZZaFSrXJnxGex` - `zynabcd0001uKVUFOsCXaGFWZPECbYWVFKzynuKWaREZv` Using one single copy command, copy all 100 (exactly 100) of these cracker files (and no others) into your own directory named `warez` that you created above. Make sure you preserve the modify times of the copied files, as you did in a previous lab. (Hint: Use a shell GLOB pattern to match the right file names. Do not use a pipe or `find` to select the file names. The shell can do it all with the right GLOB pattern.) You can check you work using: `$ \ls warez | wc` which should show a count of 100 lines. (The backslash before `ls` turns off any alias you might have defined for `ls` that would affect the output.) All the files should have their original modify dates preserved - verify this. 2. Somewhere under that same `start` directory are exactly three non-empty files whose names contain your userid (lower-case) somewhere in the name. Find these three non-empty files and copy them (preserving modify times) to the `osfiles` directory that you created above. You will find your userid mentioned inside each file, but because the files are not all Unix/Linux text files, some of the text content may not display correctly. Hint: What command finds files based on size and a basename that can be a GLOB-style pattern? You have used this command many times this term. See [Worksheet #2 HTML]. You can use your mouse to copy-and-paste the three file names you need to copy. You can also read this optional material on a better way to [Use Find -exec And Xargs]. 3. In your `osfiles` directory, determine which operating system created each of the three non-empty files. Rename the Unix/Linux file to be `unix`, the Windows file to be `windows` and the Macintosh file to be `macintosh`. Hint: In the Week 2 preparation for [Assignment #2 File Transfer Practice] you used a command that can determine file type to identify the text inside a file created with Windows Notepad. You will also find this command listed under Week 2 in the [List of Commands] in your notebook. Use this command and the notes on [Text File Line End Differences] to identify the special line endings of the Windows and Macintosh files. Your instructor will mark the `assignment05` directory in your account on the due date. Leave them there on the CLS. Do not delete anything. Run the [Checking Program] to verify your work so far. When you are done ----------------- That is all the tasks you need to do. Verify your word counts and then mark your work: Checking, Marking, and Submitting your Work =========================================== **Summary:** Do some tasks, then run the checking program to verify your work as you go. You can run the checking program as often as you want. When you have the best mark, upload the marks file to Blackboard. 1. There is a [Checking Program] named `assignment05check` in the [Source Directory] on the CLS. You can execute this program by typing its (long) pathname into the shell: `$ ~idallen/cst8207/13w/assignment05/assignment05check` 2. Execute the above “check” program. This program will check your work, assign you a mark, and display the output on your screen. (You may want to paginate the long output so you can read all of it.) You may run the “check” program as many times as you wish, to correct mistakes and get the best mark. Some tasks sections require you to finish the whole section before running the checking program at the end; you may not always be able to run the checking program successfully after every single task step. 3. When you are done with checking this assignment, and you like what you see on your screen, redirect the output of the [Checking Program] into the text file `assignment05.txt` under your `assignment05` directory. Use the *exact* name `assignment05.txt` in your `assignment05` directory. You only get *one* chance to get the name correct. Case (upper/lower case letters) matters. Be absolutely accurate, as if your marks depended on it. Do not edit the file. 4. Transfer the above `assignment05.txt` file from the CLS to your local computer and verify its contents. Do not edit this file! No empty files, please! Edited or damaged files will not be marked. You may want to refer to your [File Transfer] notes. 5. Submit the `assignment05.txt` file under the correct Assignment area on Blackboard (with the exact name) before the due date. Upload the file via the **assignment05** “Upload Assignment” facility in Blackboard: click on the underlined **assignment05** link in Blackboard. Use “**Attach File**” and “**Submit**” to upload your plain text file. No word-processor documents. Do not send email. Use only “Attach File”. Do not enter any text into the **Submission** or **Comments** boxes on Blackboard; I do not read them. Use only the “**Attach File**” section followed by the **Submit** button. (If you want to send me comments about your assignment, use email.) 6. Your instructor may also mark the `assignment05` directory in your CLS account after the due date. Leave everything there on the CLS. **Do not delete any assignment work from the CLS until after the term is over!** Use the *exact* file name given above. Upload only one single file of plain text, not HTML, not MSWord. No fonts, no word-processing. Plain text only. Did I mention that the format is plain text (suitable for VIM/Nano/Pico/Gedit or Notepad)? **NO EMAIL, WORD PROCESSOR, PDF, RTF, or HTML DOCUMENTS ACCEPTED.** No marks are awarded for submitting under the wrong assignment number or for using the wrong file name. Use the exact name given above. WARNING: Some inattentive students don’t read all these words. Don’t make that mistake! Be exact. **READ ALL THE WORDS. OH PLEASE, PLEASE, PLEASE READ ALL THE WORDS!** -- | Ian! D. Allen - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [Checking Program]: #checking-marking-and-submitting-your-work [Course Linux Server]: 010_course_linux_server.html [Lab Worksheet #05 ODT]: worksheet05.odt [Tasks]: #tasks [The VI (VIM) Text Editor]: 300_vi_text_editor.html [Shell Variables - Basics (non-scripting)]: 320_shell_variables.html [Login and Shell Startup Files, aliases, options: .bash\_profile and .bashrc]: 350_startup_files.html [List of Commands]: 900_unix_command_list.html [VIM reference card PDF]: vi_refcard_front.pdf [Bonus Assignment #6]: assignment06.html [Worksheet #2 HTML]: worksheet02.html [Non-interactive shells and PS1]: 350_startup_files.html#non-interactive-shells-and-ps1 [Shell GLOB patterns (wildcard pathname matching)]: 190_glob_patterns.html [Unix Shell I/O Redirection (including Pipes)]: 200_redirection.html [Copies of the CST8207 Course Notes]: 010_course_linux_server.html#copies-of-the-cst8207-course-notes [Lab Worksheet #03 HTML]: worksheet03.html [Source Directory]: #the-source-directory [Use Find -exec And Xargs]: 185_find_and_xargs.html [Assignment #2 File Transfer Practice]: assignment02.html#cls-login-and-file-transfer-practice [Text File Line End Differences]: 015_file_transfer.html#text-file-line-end-differences [File Transfer]: 015_file_transfer.html [Plain Text]: assignment05.txt [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/