% CST8177 Assignment 02 - PATH and Simple Shell Scripts % Todd Kelley Richard Donnelly Ian! D. Allen - idallen@idallen.ca - www.idallen.com % Winter 2013 - January to April 2013 - Updated Sat Feb 2 17:58:32 EST 2013 Due Date and Deliverables ========================= - **Due Date**: `23h59 (11:59pm) Saturday February 2, 2013 (end of Week 4)` - **NEW DUE DATE** - Late assignments or wrong file names may not be marked. Be accurate. - **Available online**: - Version 01 - 09:00 Monday January 21, 2013 - Version 02 - 09:10 Monday January 21, 2013 - Version 03 - 15:45 Monday January 21, 2013 - Version 04 - 01:20 Saturday January 26, 2013 - **Prerequisites**: - previous term notes: [CST8207 GNU/Linux Operating Systems I] - current term notes: [Course Notes] - previous lab work - an ability to **READ ALL THE WORDS** to work effectively - **Deliverables**: 1. One text file uploaded to Blackboard according to the steps in the [Checking Program] section below. 2. Directory structure and files created and left for marking on the [Course Linux Server] (**CLS**).\ **Do not delete any assignment work from the CLS until after the term is over!** Purpose of this Assignment ========================== 1. More review of basic Linux command line operations and utilities, e.g. from [CST8207 GNU/Linux Operating Systems I]. 2. Explore the PATH mechanism of the shell 3. Verify why putting the current directory in your PATH is undesirable 4. Create simple shell scripts Remember to **READ ALL THE WORDS** to work effectively and not waste time. Introduction and Overview ========================= This is an overview of what you will do in this assignment. Read all the words before you start working. Complete the [Tasks] listed below on the [Course Linux Server] (**CLS**). Run a [Checking Program] to verify your work as you go. Submit your marks. Task 1 guides you through the process of exploring the PATH mechanism of the shell. In the process, you will gain some practice working with shell variables. The remainining tasks involve creating simple shell scripts. You can use the [Checking Program] to check your work after you create each script. You can check your work with the checking program as often as you like before you submit your final mark. When you are finished the tasks, leave the files and directories in place as part of your deliverables. **Do not delete any assignment work until after the term is over!** Assignments may be re-marked at any time; you must have your term work available right until term end. This lab involves the simplest of scripting techniques, to be built upon in future labs. You will draw on your knowledge of the material from the prerequisite course [CST8207 GNU/Linux Operating Systems I]; however, you will probably need to refresh your memory of various topics by referring to the CST8207 course notes and the Linux man pages. Your lab instructor is there to help you, but he will want you to have tried consulting the notes and man pages first. The prevous term’s course notes are available on the Internet here: [CST8207 GNU/Linux Operating Systems I]. All the notes files are also on the CLS. You can learn about how to read and search these files using the command line on the CLS under the heading *Copies of the CST8207 course notes* near the bottom of the page [Course Linux Server]. Remember to **READ ALL THE WORDS** to work effectively and not waste time. Tasks ===== - Do the following tasks in order, from top to bottom. - These tasks must be done in your account on the [Course Linux Server]. - **READ ALL THE WORDS!** and do not skip steps. The Source Directory -------------------- All references to the “Source Directory” below are to the directory `~idallen/cst8177/13w/assignment02/` and that name starts with a *tilde* character followed by a userid with no intervening slash. Change your password from the default ------------------------------------- System technicians must be highly security-conscious. The CLS is directly on the Internet and is being attacked every minute of the day. Passwords for accounts are being tested for weaknesses. If you have not already changed your password from the default given to you by your instructor, do so now using the following guidelines. Accounts with unchanged passwords will be disabled next week. 1. Read “Guidelines for strong passwords” in 2. Now read this XKCD comic: 3. Change your password to one that is more secure than the one you were given. **Find a way to remember your new password.** If you forget your password, contact your Linux instructor to have it reset. The CLS is on the public Internet; security is important. Choose your password carefully. Accounts that do not have their passwords changed before the due date of this assignment will be disabled. See your instructor. Exploring the search PATH mechanism ----------------------------------- NOTE: in this task, you will change the `PATH` variable temporarily for the current shell session only. If at any time you want to undo these temporary changes, you can simply `exit` the shell that has the changes (and perhaps then start a new shell, or re-login if you exited your login shell). Do **NOT** edit your `.bashrc` to make permanent changes that affect future login sessions, unless explicitly told to do so. 1. Create the new directory `assignment02` under the existing `Assignments` directory in your CLS account. (You should have created the `Assignments` directory in a previous assignment.) 2. Start a second copy of `bash` (a nested shell). (Watch the video on “subshells” from the [Course Notes]). You can use the `ps` command to see that you have two copies of `bash` running. Re-read the NOTE, above. 3. Display the value of the shell environment variable `PATH` on your screen and verify that the directory `/bin` is in the list of directories in `PATH`. (Hint: There are several ways to see the value of an environment variable. You could use a command that shows single lines of text on your screen.) (See your instructor if `/bin` is not in your `PATH`.) 4. Use a command to discover which of your `PATH` directories contains the `ls` program. Use redirection to put the output of this command (the absolute path to the `ls` command) in file `lspath` under your new `assignment02` directory. You will need to know this pathname later. 5. Store the current value of your `PATH` variable in a new shell variable named `oldpath` by entering `oldpath=$PATH` (no blanks). Make sure that both `PATH` and `oldpath` have the same values. (Some commands that can display the value of environment variable `PATH` cannot be used to display the value of local non-environment variable `oldpath`.) Since setting the value of a shell variable doesn’t save it on disk, the saved value is temporary. The value will be lost if you exit this shell and start a new shell. If you exit this shell before completing this section of the assignment, you will lose the value of `oldpath` and will have to re-save it. 6. Put the value of the shell variable `oldpath` into a new file named `oldpath` under your new `assignment02` directory. (Hint: Use output redirection. It’s easy and accurate.) 7. Remove all write permissions from the `oldpath` file, so that you don’t accidentally overwrite it. 8. Set your `PATH` variable to have no value. The method shown in class was to type nothing after the equals sign - just push **Enter** after the equals: `PATH=` \ Display the new (empty) value of `PATH` on your screen. 9. Try to run any non-builtin command (for example, `ls, who, date`, etc.) and notice that the attempt fails because the shell cannot find a program with that name in your current (empty) `PATH`. Type the `ls` command name and use the correct syntax to redirect just the shell error message into a file called `path_error` in your `assignment02` directory. Verify that the error message is in the file. (How can you verify this if no commands can be found using the current shell?) 10. Run the `ls` program using its absolute pathname (the one you discovered earlier). Note that you do not need any directories in your `PATH` to run a command using a pathname (containing slashes) instead of a command name (without slashes). The `PATH` is not used to look for command names containing slashes. 11. Issue a shell built-in command (for example, `echo, pwd, shopt`) and verify that built-in commands are part of the shell process itself and do not need to be found in `PATH` directories. Redirect the output of shell built-in `shopt` into a file called `assignment02/builtin` 12. Restore your `PATH` variable to its original value by restoring its value from the `oldpath` variable. This is the opposite of what you did earlier to save the value. If you make a mistake, re-read the NOTE, above. 13. Display your `PATH` variable to verify it is restored (has the same value as `oldpath`). 14. Try a few non-builtin commands to verify they are found with the restored `PATH` in place. Creating your first shell script -------------------------------- 1. Create the directory `bin` in your HOME directory as a place to keep your own personal commands. 2. Create the directory `Assignments/assignment02` under your HOME directory if you have not already done so. 3. In the `assignment02` directory, create the file `myfirst.sh`, containing the following two lines: a) The first line of the file should be exactly these 12 characters: `#!/bin/sh -u` (plus one more character for the newline at the end of the line). b) The second line of the file should be a command that displays the exact text: `*** It's Not a "Micro$oft" World ***` on the screen. Make sure you count the asterisks and get the quotes and dollar sign correct. 4. Make the `myfirst.sh` file executable for only the owner (you). Group and others can read the file but not execute it. You have just finished creating a very simple Linux command – a script that prints some text to its standard output. 5. Execute the script. You will note that, just as with the `check` script from an earlier assignment, the shell cannot find `myfirst.sh` as an executable command name (no slashes). You need to use a relative pathname (with slashes) to execute this file. When the output is correct, redirect the output into file `worldout` in the `assignment02` directory. 6. We will give our new command a more command-like name, without the file extension. We will at the same time put the new name into our private `bin` directory that we just created: To do that, create a hard link from your `myfirst.sh` script to the name `myfirst` (no extension) inside your `bin` directory. You now have one file inode with two names. 7. Make `assignment02` your current directory. Now, without changing directories again, check the inode numbers of both names to verify that they are hard links to the same inode. Use one command with two relative pathnames to check. When the output is correct, save the output into file `myinodes` in the `assignment02` (current) directory. 8. Change to your HOME directory. Without changing directories again, execute your new `myfirst` command that is inside your `bin` directory using a relative pathname (from your HOME). 9. In a manner similar to adding “`.`” to the start of your `PATH` (see the last of the “03” lecture slides), add the absolute pathname of your new `bin` directory to the *right end* (not the *start*) of the `PATH` variable of your current shell. Remember how you did this – you will need to do it again later in your `.bashrc` file. 10. Display your `PATH` to confirm that the absolute path of your `bin` directory is on the right end (not at the start). Do *not* add the directory more than once – it should only appear once in your `PATH` variable. If you need to start over, exit this shell and then start another one. 11. If you have done the previous steps correctly, you can now type the command name `myfirst` (no slashes, no file extension) and the shell will find your `myfirst` script in your `bin` directory and execute it (because your `bin` directory is now in the `PATH` for this shell). 12. When you have your own `myfirst` command working using your modified `PATH` varaible, save a copy of your modified `PATH` variable in file `binpath` in your `assignment02` directory. Make sure there are no duplicate directories in `PATH` before you save it. 13. Create a backup copy of your `.bashrc` file, in case you need to start over. 14. Make the necessary changes to your `.bashrc` file to add the absolute path of your `bin` directory to the right end of your `PATH` variable, exactly as you did earlier with your interactive shell. Save the file. 15. Log out and log in. Check your `PATH` variable to make sure the absolute path of your `bin` directory is at the right end of your `PATH`. Verify that your `myfirst` command still works. Putting the current directory in `PATH` is a bad idea ----------------------------------------------------- 1. Create a fake `ls` command by *copying* the script you created above to a file in your home directory and name the new file `ls`; make sure you use an option to the copy command that preserves the mode (permissions) and timestamps of the file. (RTFM: You should have an alias that adds this option already!) 2. Edit that new file, changing the text message to read: `You are running fake ls with all your privileges - this script could now remove your files!` 3. Make the new fake `ls` command executable if it is not already. You have just finished setting a trap for anyone who has the current directory `.` near the beginning their `PATH` and runs `ls` while in your home directory. 4. Run the `ls` command and note that you ran the real `ls` command, because your `PATH` does not include the current directory. You do not have `.` at the start of your `PATH` so you didn’t run the fake `ls` in the current directory. 5. As you did earlier, save the current value of your `PATH` variable in an `oldpath` variable. 6. Make yourself a victim: temporarily put the current directory `.` at the front of your `PATH` variable – see the last of the 03 lecture slides for how to do this. 7. Display the new value of your `PATH` variable and verify that `.` is the first directory in the sequence. 8. Change into your home directory (where the fake `ls` command resides) and run the `ls` command as you normally would. Notice that you’ve run a fake ls command as yourself, and it has privileges to do anything your user can do. 9. Imagine you are running as root and you have made the mistake of putting the current directory in your path as we have done here. Imagine further that you run the `ls` command after changing into the home directory of a user that has placed a fake `ls` command there. Because you’re `root`, that fake command runs with `root` privileges – and it can destroy your system. **NEVER PUT THE CURRENT DIRECTORY IN YOUR PATH!** 10. Using the saved `oldpath` variable, restore your PATH variable to its former proper value, removing the current directory. Creating a script ----------------- In this task you will create a script that prints out the number of processes each user is currently running on the system. You will create a series of scripts, each being an enhancement of the previous one, until you have the final product. NOTE: The command that gives you a “snapshot of the current processes” has several kinds of options, some with dashes and some without, because it is really two or three commands merged into one program. You sometimes find that mixing options from the different types (with and without dashes) gives error messages. It’s best not to mix option types. See the first few paragraphs of the man page. 1. Run a command that prints out a “full-format listing” and “select all processes” running on the system. Search in the man page for the terms “Select all processes” and “full-format” to discover the correct two dash option letters. The output will be several hundred lines long, with every line starting with the userid that owns that process. The lines will look something like this: root 29556 555 0 Jan17 ? 00:00:00 /sbin/udevd --daemon cst8207b 26103 26102 0 02:15 pts/2 00:00:00 -bash 2. Create a new two-line script file `assignment02/processes.sh` by copying your existing `myfirst.sh` file and changing the second line of the new file to use the process listing command you used from a previous step. Do *NOT* change the first line. 3. Run the new script `processes.sh` to make sure it works. It should generate the full-format, all-processes listing for you. 4. Recall from class that the command `awk '{print $1}'` reads from standard input and prints just the first space-delimited column from that input, and notice the output of the process listing command inside your new script is conveniently in space- delimited columns. Run a command pipeline that feeds the output from your `processes.sh` script into this `awk` command line that selects just the first leading userid column. You should end up with a list of several hundred userids on your screen. 5. Copy `processes.sh` to `processes_users.sh` in the same directory, preserving the mode and timestamps as before. (System administrators always preserve the mode and timestamps of copied files! That’s why we have an alias for the copy command!) 6. Change the second line in the new `processes_users.sh` file to be a command pipeline prints just the first column of the process listing. (Add the previously-mentioned `awk` command to the end of the line to make it a command pipeline, as you did in a previous step.) Do not change the first line. 7. Run the new script `processes_users.sh` to make sure it works. It should generate a list of several hundred userids. 8. Pipe the output of your new `processes_users.sh` script into the `sort` command. The output will be the list of several hundred userids in sorted order. 9. Copy `processes_users.sh` to `processes_sorted.sh` in the same directory, preserving the mode and timestamps as before. (System administrators always preserve the mode and timestamps of copied files! That’s why we have an alias for the copy command!) 10. Enhance the command pipeline in the new `processes_sorted.sh` file, adding a further pipeline into the `sort` command so that the one column of userids comes out sorted. 11. Run the new script `processes_sorted.sh` to make sure it works. It should generate a list of several hundred userids in sorted order. 12. There is a command that can count the number of occurrences of adjacent input lines, displaying each **unique** line preceded by the *count* of the number of times that line appeared. To recall this **unique** command name and its option to count adjacent lines, review the [Command List] from last term and the many counting **Examples of pipes** in the page [Redirection]. Pipe the output of your new `processes_sorted.sh` script into this uniqueness command with its counting option. The output will be counts of the number of processes being run by each unique userid. Instead of hundreds of lines of output, you will see a few dozen lines, depending on how many users are logged in. The lines will all look like this: 3 postfix 78 root 13. Copy `processes_sorted.sh` to `processes_counted.sh`. (From here on, we assume you will always preserve the mode and timestamps when you copy a file, and we won’t keep reminding you. Make sure you have your copy alias defined!) 14. Enhance the command pipeline in the new `processes_counted.sh` script file to show the counts of processes run by each unique userid. 15. Run the new script `processes_counted.sh` to make sure it works. It should generate a list of more than a dozen unique userids, each preceded by a count. We can now see that there are two odd things in the output that are not account userids: - `102` – a number, not a userid (this appears to be a bug in the process listing program) - `UID` – the column heading from the output of the process listing 16. Copy `processes_counted.sh` to `processes_counted2.sh`. 17. Enhance the command pipeline in the new `processes_counted2.sh` file to eliminate the bogus `UID` column heading in the output. There are at least two ways to do this; pick either way and make the change: - One way is to insert a pipeline filter that removes any line containing the string `UID`. (Hint: the command that searches inside files for lines matching a pattern has an option to invert the match and only show non-matching lines.) - Another way is to see if the process listing command has an option to *print no header line*. (Hint: It does!) - **DO NOT** use any removal method that relies on the number of lines output or the order of the lines output, since the number of lines and the user names in those lines will change from day to day. 18. Run the new script `processes_counted2.sh` to make sure it works. It should generate a list of more than a dozen userids in sorted order, with no bogus `UID` in the output. - Re-read the note above about how **NOT** to remove the `UID` line! 19. Copy `processes_counted2.sh` to `processes_counted_header.sh`. 20. Enhance the new `processes_counted_header.sh` script so that it prints out its own header line before it generates the output. The header line should be the exact text “`NumProc Username`” (16 characters, including the single space, with upper-case for the `N`, `P`, and `U`) and you can optionally output some dashed underlining under it as well, to make the heading stand out. Line up the blank in the title with the blank in the two columns of output. The output will look similar to this, though the actual numbers and accounts may differ: NumProc Username ------- -------- 1 102 2 avahi 1 colord 6 cst8207b ... etc ... 21. Run the new script `processes_counted_header.sh` to make sure it works. It should generate a header line, some optional underlining, followed by a list of more than a dozen userids in sorted order, with no bogus `UID` in the output. - Re-read the note above about how **NOT** to remove the `UID` line! 22. The long script name `processes_counted_header.sh` is helpful for knowing what the script does, but it’s much too long for a command name. Make a hard link from `processes_counted_header.sh` into your `bin` directory and name the link `pch`. 23. Because your `bin` directory is in your `PATH`, you should now be able to run `pch` as a command name with no slashes. You have written your second Linux command script. Check your work a final time using the `assignment02check` program symlink. Submit your final mark following the directions below. Checking, Marking, and Submitting your Work =========================================== **Summary:** Do some tasks, then run the checking program to verify your work as you go. You can run the checking program as often as you want. When you have the best mark, upload the marks file to Blackboard. 1. There is a [Checking Program] named `assignment02check` in the [Source Directory] on the CLS. Create a symbolic link to this program named `check` under your new `assignment02` directory so that you can easily run the program to check your work and assign your work a mark. Note: You can create a symbolic link to this executable program but you do not have permission to read or copy the program file. To verify the symbolic link, try executing it. 2. Execute the above “check” program using its symbolic link. (Review the [CST8207 Search Path] notes if you forget how to run a program by pathname from the command line.) This program will check your work, assign you a mark, and display the output on your screen. (You may want to paginate the long output so you can read all of it.) You may run the “check” program as many times as you wish, to correct mistakes and get the best mark. 3. When you are done with checking this assignment, and you like what you see on your screen, redirect the output of the [Checking Program] into the text file `assignment02.txt` under your `assignment02` directory. Use the *exact* name `assignment02.txt` in your `assignment02` directory. You only get *one* chance to get the name correct. Case (upper/lower case letters) matters. Be absolutely accurate, as if your marks depended on it. Do not edit the file. 4. Transfer the above `assignment02.txt` file from the CLS to your local computer and verify its contents. Do not edit this file! No empty files, please! Edited or damaged files will not be marked. You may want to refer to this term’s updated [File Transfer] notes. 5. Submit the `assignment02.txt` file under the correct Assignment area on Blackboard (with the exact name) before the due date. Upload the file via the **assignment02** “Upload Assignment” facility in Blackboard: click on the underlined **assignment02** link in Blackboard. Use “**Attach File**” and “**Submit**” to upload your plain text file. No word-processor documents. Do not send email. Use only “Attach File”. Do not enter any text into the **Submission** or **Comments** boxes on Blackboard; I do not read them. Use only the “**Attach File**” section followed by the **Submit** button. (If you want to send me comments about your assignment, use email.) 6. Your instructor may also mark the `assignment02` directory in your CLS account after the due date. Leave everything there on the CLS. **Do not delete any assignment work from the CLS until after the term is over!** Use the *exact* file name given above. Upload only one single file of plain text, not HTML, not MSWord. No fonts, no word-processing. Plain text only. Did I mention that the format is plain text (suitable for VIM/Nano/Pico/Gedit or Notepad)? **NO EMAIL, WORD PROCESSOR, PDF, RTF, or HTML DOCUMENTS ACCEPTED.** No marks are awarded for submitting under the wrong assignment number or for using the wrong file name. Use the exact name given above. WARNING: Some inattentive students don’t read all these words. Don’t make that mistake! Be exact. **READ ALL THE WORDS. OH PLEASE, PLEASE, PLEASE READ ALL THE WORDS!** -- | Todd Kelly / Richard Donnelly and | Ian! D. Allen - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [CST8207 GNU/Linux Operating Systems I]: ../../../cst8207/12f [Course Notes]: indexcgi.cgi [Checking Program]: #checking-marking-and-submitting-your-work [Course Linux Server]: 000_course_linux_server.html [Tasks]: #tasks [Command List]: ../../../cst8207/12f/notes/900_unix_command_list.txt [Redirection]: ../../../cst8207/12f/notes/200_redirection.html [Source Directory]: #the-source-directory [CST8207 Search Path]: ../../../cst8207/12f/notes/400_search_path.html [File Transfer]: 220_file_transfer.html [Plain Text]: assignment02.txt [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/