3.1 The Source DirectoryIndex

All references to the “Source Directory” below are to the CLS directory ~idallen/cst8207/14w/assignment04/ and that name starts with a tilde character ~ followed by a userid with no intervening slash. The leading tilde indicates to the shell that the pathname starts with the HOME directory of the account idallen (seven letters).

You do not have permission to list the names of all the files in the Source Directory, but you can access any files whose names you already know.

4.1 Worksheet and PromptIndex

These worksheets prepare you to do the numbered tasks listed below by executing commands via Remote Login to the Course Linux Server.

You can download Libre Office (or Open Office) for Windows to edit the Worksheet *.odt files and save your answers. (Or you can print the PDF and write your answers on paper.) Do not use MSWord.

Record and save all your worksheet answers for study and quizzes!

See Assignment #02 for how best to fill in the worksheets.

1. Use LibreOffice to do the worksheets Worksheet #04 ODT and Worksheet #05 ODT before this assignment. Record and save all your worksheet answers for study and quizzes!

2. Set your prompt every time you log in to the CLS, so that the prompt changes to tell you you the basename of your current working directory, as you did in the worksheets and the previous assignment. Hint: You can use your saved shell history to set the prompt very quickly at login time, as demonstrated in lectures.

4.2 Set Up – The Base DirectoryIndex

1. Create the following directory structure starting in your CLS HOME directory and record (for study purposes) the series of Unix commands you used to create it. (You do not have to create any directories that you have already created in a previous assignment.) Spelling and capitalization must be exactly as shown:
   1. Create the CST8207-14W directory in your CLS HOME directory.
   2. Create the Assignments directory in the CST8207-14W directory.
   3. Create the assignment04 directory in the Assignments directory.

The correct directory tree will look similar to the ASCII tree diagram below. The spelling and capitalization must be exactly as shown.

CST8207-14W
`-- Assignments
    `-- assignment04

This assignment04 directory is the base directory for most pathnames in this assignment. Store your files and answers below in this base assignment04 directory.

Run the Checking Program to verify your work so far.

4.3 Working with directories and filesIndex

You may find it easier to type if you make the base directory you created above (assignment04) your current directory for this section.

1. Create the following directory structure in the base directory and record (for study purposes) the series of Unix commands you used to create it. Spelling and capitalization must be exactly as shown:
   1. Create a directory named base1 in your base directory.
   2. Under directory base1 create two new directories named bdir1 and bdir2 (four letters and one digit each).
   3. Under directory bdir1 create new directory bdir1-1 (four letters, two digits, and one dash).
   4. Under directory bdir2 create new directory bdir2-1 (four letters, two digits, and one dash).
   Note: You can create the entire directory tree above using one single command with one option and two pathname arguments, as you did at the end of Section 4 in Worksheet #02 HTML. System administrators like to work efficiently – they learn how to do things quickly.

The correct directory tree under the base directory will look similar to the ASCII tree diagram below. The spelling and capitalization must be exactly as shown.

base1
|-- bdir1
|   `-- bdir1-1
`-- bdir2
    `-- bdir2-1

2. Make a directory named warez under your base directory. You will put files here later.

3. Make a directory named osfiles under your base directory. You will put files here later.

4. Make the base1 directory your current directory and then use a command to recursively generate a list of all pathnames under your current directory. (You used this recursive command many times in the last section at the end of Worksheet #02 HTML. Do not use the tree or ls commands. Use the current directory as your starting directory.) The recursive output of all pathnames under your current directory will be exactly five lines long and will contain this exact line in the output somewhere (along with another four lines):

   ./bdir2/bdir2-1

   Make sure the exact line above appears as one of the five lines! The word count of this command output must be: 5 5 50 and if the count is wrong, re-read all the words in ths step.

5. When the recursive output above on your screen looks correct, redirect the five lines of output of the command into the file named base1find.txt in the base directory (not in the current directory). (That file name has one digit in it.) The word count of this file must also be: 5 5 50.

6. RTFM and find the single-letter option to the copy program that preserves mode, ownership, and timestamps when you copy. (Make a note of this option in your own course notes.)

7. Under your base directory create another directory base2 that contains a copy of everything you created under base1, with the timestamps preserved. Hint: The fastest way to do this is with a single recursive copy command, as you did in Worksheet #03 HTML, but add the option to preserve timestamps.

8. Create an empty file named MT in each of the five directories from base2 down. You will create five empty files, one in each of five directories. (You can do this with one single command name and five relative pathnames.)

9. Make the base2 directory your current directory and then use a command to recursively generate a list of all pathnames under your current directory. The recursive output of all pathnames under your current directory will be exactly ten lines long and will contain this exact line in the output somewhere (along with another nine lines):

   ./bdir2/bdir2-1/MT

   Make sure the exact line above appears as one of the ten lines! The word count of this command output must be: 10 10 115 and if the count is wrong, re-read all the words in this step.

10. When the recursive output above on your screen looks correct, redirect the ten lines of output of the command into the file named base2find.txt in your base directory (not in the current directory). The word count of this file must also be: 10 10 115.

11. Under your base directory create another directory base3 that contains a copy of everything that you created under base2, with timestamps preserved. Hint: See the previous Hint!

12. Under base3 rename each of the MT files to have the number of the immediate parent directory that contains it, e.g. rename base3/MT to be base3/MT3 and bdir2-1/MT to be bdir2-1/MT2-1, etc. (You don’t know enough scripting yet to do this with a script; do each file manually.)

13. Make the base3 directory your current directory and then use a command to recursively generate a list of all pathnames under your current directory. The recursive output of all pathnames under your current directory will be exactly ten lines long and will contain this exact line in the output somewhere (along with another nine lines):

    ./bdir2/bdir2-1/MT2-1

    Make sure the exact line above appears as one of the ten lines! The word count of this command output must be: 10 10 124 and if the count is wrong, re-read all the words in this step.

14. When the recursive output above on your screen looks correct, redirect the ten lines of output of the command into the file named base3find.txt in your base directory (not in the current directory). The word count of this file must also be: 10 10 124

Run the Checking Program to verify your work so far.

4.4 Finding multiple files in a maze using a GLOB patternIndex

1. Under the Source Directory there is a name maze that is a symbolic link that leads to the same directory maze used in a previous assignment: Assignment #02 HTML.

   Use the absolute pathname of the maze name in the Source Directory as an argument to ls along with an option that shows the long information about the pathname. You probably have the right option if the first word of the output is lrwxrwxrwx, indicating that maze is a symbolic link. Save the output (one line) into file mazeinfo.txt under your base directory.

2. Again, in a manner similar to your previous assignments, you must find files in this maze, but the symbolic link requires some special handling because the command that finds files does not follow symbolic link arguments on the command line without using some options:

   1. You must choose one of these approaches to reach the maze:

      1. Use the maze pathname from the previous assignment, not the pathname in the Source Directory that is the symbolic link. OR
      2. Use an option to the finding command that makes it follow symbolic links while processing the command line arguments. Hint: RTFM and do not use the -L option. OR
      3. Make the maze your current directory and then recursively search the current directory. (A current directory can never be a symbolic link – it must be a real directory.)

   2. Important: Look for the string quote in the course notes web page on Searching for and finding files by name, size, use, modify time, etc. Read all the paragraphs containing this word and remember the importance of quoting. You will need this information for this next question:

   3. In this maze, use a single command (not a pipeline) to recursively find all pathnames with a basename that contains your eight-character userid anywhere in the name. For example, if your userid were abcd0001 then you might output pathnames containing basenames such as abcd0001, abcd0001YYY, XXXabcd0001, and XXXabcd0001YYY where XXX and YYY can be anything (zero or more characters). You must use a single command that finds files by using a pattern basename to find these pathnames. Do not try to use cd and ls to find the names in the maze; the maze is very big. (You have previously used this command many times without a pattern for a basename. You should find exactly 31 pathnames.)

3. Modify the same single command you used to find the names above and add the option that makes the command show the full information about the names (including permissions, owner, size, date, etc.) instead of just the pathname. (You know this option from your answers in Worksheets 2 and 3 and from reading the More features section of the Searching for and finding files by name notes.) You will know you have the right option if the output of the command shows 31 lines and approximately 352 words.

4. Pipe the pathname output of the above command (31 lines of output) into a sorting program and put the sorted output into file mazefound.txt under your base directory. The sorted file will still contain 31 lines and approximately 352 words.

Run the Checking Program to verify your work so far.

4.5 Using shell GLOB patterns to select namesIndex

1. Make your HOME directory your current directory.

2. In your HOME directory, create two symbolic links to the old and new course notes for CST8207 using the ln -s command and option and the method described in Copies of the CST8207 Course Notes. (The old notes must be term 13f and the new notes must be term 14w in the pathnames you use.)

3. Do a long listing of the new oldnotes symlink and verify that it looks similar to this (but the userid and time will differ):

   lrwxrwxrwx 1 abcd0001 abcd0001 52 Jan 27 07:37 oldnotes -> /home/idallen/public_html/teaching/cst8207/13f/notes

   You should be able to do ls oldnotes and see all the course notes file names from last term. If not, remove and redo the symlink.

4. In your HOME directory, use the echo command with a single shell GLOB pattern to match all pathnames under the symbolic link oldnotes/ that end in .txt and display all the names on your screen. The shell will find 86 pathnames ending in .txt, and the echo command will display those 86 names on your screen in one long line. One of the last names on your screen should look exactly like this:

   oldnotes/worksheet08.txt

   Make sure you see 86 pathnames. Use a command to count the words to be sure you have 86.

   Hints: No pipeline is required, just use the echo command and one GLOB pattern argument containing the symlink oldnotes. This use of a GLOB pattern on a command line is illustrated in Copies of the CST8207 Course Notes. The example in the notes uses the given GLOB pattern to generate pathnames to the ls command and count them. Follow the example but use echo instead of ls, and display the pathnames on your screen instead of counting them (don’t use any pipes).

5. When the echo output on your screen is correct (86 names), redirect the output into file textfound.txt under your base directory. The file must contain 86 names on one line.

6. Again in your HOME directory, use the echo command with a shell GLOB pattern to match all pathnames under oldnotes/ that contain the word symbolic anywhere in the file name and display the names on your screen. The shell will find two pathnames, one ending in .html and the other in .txt, and the echo command will display those two names on your screen.

7. When the echo output on your screen is correct (two names), redirect the output into file symfound.txt under your base directory. The file must contain two names on one line.

8. Again in your HOME directory, use the echo command with a shell GLOB pattern to match pathnames under oldnotes/ that contain the word vi anywhere in the file name and end in the extension .pdf at the end. The shell will find two pathnames, each ending in .pdf at the end, and the echo command will display those two names on your screen.

9. When the echo output on your screen is correct (two names), change the command name from echo to ls and add an option to show the full, long information about the pathnames. You should see two long lines on your screen, showing the full file information for each of the two files.

10. Now redirect the long output on your screen into file vifound.txt under your base directory. The file must contain two lines.

Run the Checking Program to verify your work so far.

4.6 Searching for text inside files (e.g. course notes)Index

As mentioned in Worksheet #03 HTML, choose which text search command you use depending on whether special characters are being used in the search string. We almost always use the fixed-string fgrep command in this introductory course. You will learn regular expressions and the grep command in later terms.

Always verify that the correct output appears on your screen before you redirect the output into a file. You can only redirect what you can see.

1. Search for lines containing your login userid in the password file and redirect the output into file mypasswd.txt in your base directory. You should find exactly one line.

2. Search for lines containing the special character asterisk (*) in the file starfile.txt in the Source Directory. The word count of the correct output should be: 3 22 116

3. When you have the correct output on your screen, redirect that output into file star.txt under your base directory. The word count of the file should still be: 3 22 116

4. In your HOME directory, create two symbolic links to the old and new course notes for CST8207 using the method described in Copies of the CST8207 Course Notes, unless you have already created these links earlier in this assignment.

   In the same notes section, see the example use of fgrep with GLOB patterns to match *.txt files in these old and new notes directories.

   These text-searching commands grep and fgrep are case-sensitive by default – searching for abc won’t find ABC unless you use an option to ignore case distinctions. (What option? RTFM)

5. In the course notes from last year, search all the .txt files for the word Filezilla (spelled exactly as shown, case-sensitive). Only two lines of text should display.

   Hint: You will need to use the same GLOB pattern you used earlier to match all the .txt files under oldnotes. This time, use the GLOB pattern with fgrep so that fgrep searches all those file names. If you see more than two lines of output, you are likely using options that make the search case-insensitive. Don’t do that.

6. Repeat the above, but add the fgrep option that ignores case distinctions. (RTFM) Now, 11 lines are found in five different files.

7. Redirect the 11 lines of output into a file named filezilla.txt under your base directory (not in the current directory).

Run the Checking Program to verify your work so far.

4.7 The cracker WAREZ 100 filesIndex

The “story” here is that a malicious cracker has hidden a bunch of WAREZ files in a directory on the server. (See https://en.wikipedia.org/wiki/Warez.) Your job is to take a copy of these files, and only these files, for use in a court case. You must not touch or copy any other files, only the WAREZ files.

1. Under the Source Directory there is a directory named start. Hidden (really hidden) deeper under this directory is one single directory containing over 112,000 files. Be careful about typing ls in this directory without using any output pagination pipe – the amount of output may flood your terminal window for some time and even ^C may take a minute or two to interrupt the command! One way to avoid flooding your screen is by using ls | wc to count how many pathnames would be output on your screen before you do just ls.

   When you find this huge directory, make this huge directory your current directory, so that you can experiment with the GLOB pattern you need more easily.

   Exactly 100 files in this one (huge) directory have names that contain your userid (lower-case) followed somewhere later by the string warez, where warez may appear in any combination of upper- and lower-case letters, e.g. warez,Warez,wArez,waREz, etc. Any amount of text may appear before your userid, between your userid and the warez, and after the warez.

   Some sample file names for userid abcd0001 might look like these (note that the warez word must always follow the userid in all the required file names):

   • HhUtfgYtyGhjJADGekCAkgtZEKsTGKdYZZabcd0001ADGekCwaREZZaFSrXJnxGex
   • zynabcd0001uKVUFOsCXaGFWZPECbYWVFKzynuKWaREZv

   Using one single copy command and a GLOB expression, copy all 100 (exactly 100) of these cracker files (and no others) into your own directory named warez that you created in the base directory, above. Make sure you preserve the modify times of the copied files, as you did in a previous lab. (In this simulation, all the files are empty.)

   Hints: Use a shell GLOB pattern to match the right file names. The shell can do it all with one copy command using the right GLOB pattern for the source files, as you did in Worksheet #04 HTML. Always use echo and word count to see if your file names are correct before you try to use the GLOB expression in a copy command. Do not use a pipe or find to select the file names. Use only the copy command with a GLOB pattern for the source files.

   When you succeed, you can check your work by doing a recursive listing of your warez directory and counting the number of names that were copied. All the files should have their original modify dates preserved – verify this.

Run the Checking Program to verify your work so far.

4.8 Three different O/S Text FilesIndex

1. Somewhere under that same start directory are exactly three non-empty files whose names contain your userid (lower-case) somewhere (anywhere) in the name.

   Find these three non-empty (size larger than zero) files and copy them (preserving modify times) to the osfiles directory that you created above. You will find your userid mentioned inside each file, but because the files are not all Unix/Linux text files, some of the text content may not display correctly on your terminal screen.

   Hints: What command finds files based on size and a basename that can be a GLOB-style pattern? You have used this command many times this term. See the end of Worksheet #02 HTML and the notes on Finding Files. The less command is better than cat when displaying files containing strange (e.g. unprintable) characters.

   Since there are only three file names, you can use your mouse to copy-and-paste the three file names you need to copy. You can also read this optional material on a better way to Use find -exec and xargs (advanced use, optional).

2. In your osfiles directory, determine which operating system created each of the three non-empty files. Rename the Unix/Linux file to be unix, the Windows file to be windows and the Macintosh file to be macintosh.

   Hints: In Assignment #1 you used a command that can determine file type to identify the text inside a cal.txt file. You will also find this command listed under Week 01 in the List of Commands in your notebook. Use this command and the notes on Text File Line End Differences to identify the special line endings of the Windows and Macintosh files.

Your instructor will mark the base directory in your account on the due date. Leave everything there on the CLS. Do not delete anything.

Run the Checking Program to verify your work so far.

4.9 Appending to filesIndex

1. Count the lines, words, and characters in the file services under the /etc directory and put the count in file wc.txt under your base directory. (Use the absolute pathname of the services file when you count and do not use any pipes.) The file wc.txt should contain one line.

2. Extract just the first line of the same services file and append this one line to the end of the wc.txt file, so that the file wc.txt now has two lines in it (the word count and the first line of services). Hint: You know a command that shows lines at the start of a file. Review your work in Worksheet #05 HTML and the notes on Redirection and Pipes.

3. Append the count of the lines, words, and characters in the file protocols under the /etc directory to the end of file wc.txt, so that the wc.txt file now has three lines in it. (Use the absolute pathname of the protocols file when you count and do not use any pipes.)

4. Extract just the last line of the same protocols file and append this one line to the end of the wc.txt file, so that the file wc.txt now has four lines in it. Hint: You know a command that shows lines at the end of a file.

   Confirm that the word count of the file gives 4 20 140. If you see the right number of lines but the other values differ, go back and re-read all the words in the parenthesized sentences abovek, especially the sentences that start with “Use the”.

Run the Checking Program to verify your work so far.

4.10 When you are doneIndex

That is all the tasks you need to do.

Check your work a final time using the Checking Program and save the output as described below. Submit your mark following the directions below.