% CST8207 Week 05 Notes -- I/O redirection and Midterm Test 1 % Ian! D. Allen -- -- [www.idallen.com] % Fall 2015 - September to December 2015 - Updated 2015-10-12 11:32 EDT - [Course Home Page] - [Course Outline] - [All Weeks] - [Plain Text] Readings, Assignments, Labs, Tests, and ToDo ============================================ Read (at least) these things (All The Words) -------------------------------------------- 1. [Week 05 Notes HTML] -- this file -- **Read All The Words** 2. [Unix Shell I/O Redirection (including Pipes)] 3. [Linux and Sysadmin News in the World] 4. [List of Commands You Should Know] -- keep a notebook! 5. [Video Tutorials on Lynda.com] -- tagged by week number - Create your [lynda.com] account and watch these sections of [Unix for Mac OS X Users] - **3. Working with Files and Directories** - Unix text editors 6m 39s - `vi/vim, nano` - `head, tail` - Searching for files and directories 6m 32s - `find` using "wildcard" (GLOB) patterns - same as shell GLOB - **6. Directing Input and Output 20m 39s** - Standard input and standard output 1m 24s - Directing output to a file 4m 13s - Appending to a file 2m 44s - Directing input from a file 5m 28s - Piping output to input 4m 40s - Suppressing output 2m 10s - **7. Configuring Your Working Environment 41m 28s** - Setting command aliases 6m 59s - saving aliases in your `.bashrc` file Assignments this week --------------------- Check the due date for each assignment and put a reminder in your agenda, calendar, and digital assistant. - Read All The Words, Do, and then Submit via Blackboard: - [Assignment #03 HTML] -- working with files: PS1, find, mkdir, mv, cp - Bonus (optional) [Assignment #04 HTML] -- the VIM text editor - [Assignment #05 HTML] -- using GLOB patterns and redirection - Really do **Read All The Words**. You don't get a second chance to get it right. ### Worksheets Form a small study group to do the worksheets. Each person tries the example given, and you make sure you all get the same answers. Worksheets teach you how commands work. Worksheets are not for hand-in; they are not worth marks. > The worksheets are available in four formats: Open Office (ODT), PDF, HTML, > and Text. Only the Open Office format allows you "fill in the blanks" in > the worksheet. The PDF format looks good but doesn't allow you to type into > the blanks in the worksheet. The HTML format is crude but useful for quick > for viewing online. Do **NOT** open the ODT files using any Microsoft products; they will mangle the format and mis-number the questions. Use the free Libre Office or Open Office programs to open these ODT documents. On campus, you can [download Libre Office here]. - [Worksheet #02 ODT] -- Using standard Linux commands I - [Worksheet #02 PDF] - [Worksheet #02 HTML] - `PS1, cd, find, less, ls, man, mkdir, passwd, pwd, rmdir` - [Worksheet #03 ODT] -- Using standard Linux commands II - [Worksheet #03 PDF] - [Worksheet #03 HTML] - `cat, clear, cp, find, grep, history, less, man, mv, rm, sleep, touch` - [Worksheet #04 ODT] -- shell GLOB patterns and Aliases - [Worksheet #04 PDF] - [Worksheet #04 HTML] - shell GLOB patterns (wildcards), Aliases: `alias, sum, unalias` - [Worksheet #05 ODT] -- shell I/O redirection (including pipes) - [Worksheet #05 PDF] - [Worksheet #05 HTML] - shell I/O redirection, `date, head, nl, tail, tr, wc` - [Worksheet #06 HTML] -- *Optional* Bonus VIM Text Editor Practice - This is an *optional* worksheet for a BONUS assignment using `vim` - Optional command-line VIM tutorial: the `vimtutor` program on the CLS. - Bonus (optional) [Assignment #04 HTML] -- the VIM text editor - Read [The VI (VIM) Text Editor] Lab work this week ------------------ - Read [The VI (VIM) Text Editor] and optionally do the bonus [Assignment #04 HTML]. - For Assignment 5 do [Worksheet #04 PDF] -- shell GLOB patterns and Aliases - Do [Assignment #05 HTML] -- using GLOB patterns and redirection Upcoming tests -------------- Read the [Test Instructions] (all the words) before your midterm tests. 1. First Midterm test: 45 minutes; in lecture class Friday in Week 5 (Oct 9) 2. Second Midterm test: 45 minutes; in lecture class Friday in Week 9 (Nov 6) Tests take place in class in your one-hour lecture hour, not in your lab period. ### Midterm Test #1 -- October 9 - Midterm #1 takes place on Friday, October 9 (Week 5) in your scheduled lecture hour (not in your lab period). - For full marks, you must read the [Test Instructions] before the test for important directions on how to enter your answers, your lab (not lecture) section number, and the test version number on the question sheet and the mark-sense forms. - There may be more questions on the test than you can answer in the time allowed; answer the ones you know, first. - A set of practice questions and answers for the first midterm test is posted: [Practice Tests and Answers]. - The tests use the semicolon `;` to separate multiple commands on the same line, to save space and paper, e.g. three separate commands can be written on one line like this: `date ; echo "hi" ; ls` - Blackboard has some quizzes taken randomly from the practice test. See below. ### Quizzes: Midterm #1 Quiz This quiz is one of several quizzes in this course. See last week for the details. From the Class Notes link on the Course Home Page ================================================= - [Course Home Page] - [All Weeks] - Review last week. Did you do everything assigned last week? From the Classroom Whiteboard/Chalkboard ======================================== - **Take notes in class!** Your in-class notes would go here. - Are you making notes from the worksheets on how each command works? - What do the options used in the worksheets mean, for each command? - Don't copy and use options that you don't understand! - The stupid `nano` text editor. (Learn `vim` instead!) - Students who did not change their passwords (see [Assignment #02]) will have had their CLS accounts disabled. - There is information on [Automatic Backups on the CLS] that explains how to get back a file you've deleted or changed. - The [VI Text Editor][The VI (VIM) Text Editor] creates hidden recovery files as you edit that can get included in recursive path listings: `.*.swp` - [Verify your GLOB patterns before you use them!] - Don't [help other students fail the course]! Why take notes in class? ------------------------ **Take notes in class!** A student writes: > But for the note taking, I personally find that without my laptop in front > of me I am more focused on the teacher. Also with taking the notes by hand > I find that the information that is presented to me in the lectures is > remembered much easier. > > But I do have my ipad in front of my with the class notes. This also helps > me follow along with the examples that you present in class. > > Also sometimes I read over the class notes after the class and fix my notes > with anything that was missed. - [Do your lecture notes look like this?] - [Does your Linux notebook look like this?] - [Are you drawing file system trees (PDF)?] Read All The Words ------------------ - When I say "read all the words in a question, especially the Hints" - When I say "don't use Word to open the worksheets" - When I say "do the worksheet to know how to do the assignment" - When I say "keep a notebook with descriptions of each command" - When I say "don't work on a problem longer than 15 minutes before trying something else, such as getting help" Missing lectures: save one hour to spend three ---------------------------------------------- Students who save an hour by not attending a lecture usually end up spending three hours trying to answer assignment questions that I gave the answers to in the lecture. Work smarter. Fifteen minute rule -- REMINDER ------------------------------- **READ THIS:** This is a repeat of what I told you back in [Week 3]: Your time as a student is valuable. If you come up against a tough problem and make no progress in fifteen minutes despite your best efforts: 1. get help from a professor or lab instructor 2. get help from classmates 3. take a break 4. put it on hold and work on something else This happens to all of us, and we need to watch out for it when it happens. Be especially careful if you find yourself trying to solve a problem with online searches, since those searches will return answers irrelevant to the assignment. Read and search the course notes; don't waste time searching the entire Internet. Wasting your time with online searches -------------------------------------- I watch students trying to solve assignment problems using online searches for answers. They may end up using commands we haven't even covered yet: > Sorry, I dont know what `sudo` does; was getting help online to find files. If you look at the [List of Commands You Should Know] that is given in the weekly readings every week, you will see that we never use the `sudo` command, so you should not be using it. Don't use commands that you haven't learned yet. Also, you must never use any command that you don't understand, even if you get it from the course notes or a worksheet. You risk deleting all your files if you use commands you don't understand. If you use online help, you will find thousands of complex wrong ways to do things. If you read the course notes, not **Google**, you will find a simple, right way. You only need to use the commands we've learned so far in the course notes and the worksheets. That is all. - Don't use strange commands and/or options you find on the Internet. Stick with the commands in the web notes and assignments and listed in the weekly notes under [List of Commands You Should Know]. Working Smart under Linux ------------------------- Growing up under Mac/Windows, you are accustomed to having to "go to" a folder to make any changes to files in that folder. This leads to some inefficient behaviour under Unix/Linux, because Unix/Linux can operate on any file in any folder without having to "go" there: The "Windows/Mac" way to create a file under `a/b/c/d/date.txt`: $ mkdir a $ cd a $ mkdir b $ cd b $ mkdir c $ cd c $ mkdir d $ cd d $ date >date.txt $ cd The Unix/Linux way: $ mkdir -p a/b/c/d $ date >a/b/c/d/date.txt The "Windows/Mac" way to rename a file under `a/b/c/d/date.txt` to be `old.txt`: $ cd a $ cd b $ cd c $ cd d $ mv date.txt old.txt $ cd The Unix/Linux way: $ mv a/b/c/d/date.txt a/b/c/d/old.txt The "Windows/Mac" way to delete `a/b/c/d/old.txt`: $ cd a $ cd b $ cd c $ cd d $ rm old.txt $ cd The Unix/Linux way: $ rm a/b/c/d/old.txt Using Unix/Linux pathnames, you can work faster, and all your commands are saved in your history for modification and re-use. Useful Command to extract fields from lines: `awk` -------------------------------------------------- The oddly-named `awk` command can extract a field, by field number, from one or more input lines. See [using `awk` to select fields]. Current Working Directory Stack ------------------------------- The shell built-in command `pushd` works like `cd` to change the current directory, but it saves the previous directory on a stack and lets you type `popd` to return to the previous directory. You may find this saves you some typing. See the shell man page or use the `help` built-in. Attacks on the Course Linux Server ---------------------------------- - Count the attacks: `fgrep -c "refused connect" /var/log/auth.log` - Another way to count the lines (less efficient): `fgrep "refused connect" /var/log/auth.log | wc` - Only show the last 10 lines: `fgrep 'refused connect' /var/log/auth.log | tail` - See the current list of *evil* host IPs: `less /etc/hosts.evil` Locked out of Course Linux Server ================================= Don't do this: Sep 30 12:33:50 Invalid user Patrick from 99.240.239.9 Sep 30 12:33:58 Failed password for invalid user Patrick from 99.240.239.9 Sep 30 12:34:35 Failed password for invalid user Patrick from 99.240.239.9 Sep 30 12:35:11 Failed password for invalid user Patrick from 99.240.239.9 Sep 30 12:35:14 Failed password for invalid user Patrick from 99.240.239.9 Sep 30 12:44:13 refused connect from cpec8be198f774b-cmbc140127c8b0.cpe.net.cable.rogers.com (99.240.239.9) Or this: Oct 4 21:11:01 Failed password for pate0377 from 99.224.164.198 Oct 4 21:11:10 Failed password for pate0377 from 99.224.164.198 Oct 4 21:11:27 Failed password for pate0377 from 99.224.164.198 Oct 4 21:11:48 Invalid user pate0037 from 99.224.164.198 Oct 4 21:12:10 Failed password for invalid user pate0037 from 99.224.164.198 Oct 4 21:12:37 refused connect from cpe00fc8d22a843-cm00fc8d22a840.cpe.net.cable.rogers.com (99.224.164.198) Or this: Oct 4 23:34:43 Invalid user martr0991 from 99.245.238.68 Oct 4 23:35:00 Failed password for invalid user martr0991 from 99.245.238.68 Oct 4 23:35:12 Failed password for invalid user martr0991 from 99.245.238.68 Oct 4 23:35:24 Failed password for invalid user martr0991 from 99.245.238.68 Oct 4 23:35:39 Failed password for invalid user martr0991 from 99.245.238.68 Oct 4 23:36:46 Invalid user mart091 from 99.245.238.68 Oct 4 23:36:57 Failed password for invalid user mart091 from 99.245.238.68 Oct 4 23:37:07 Failed password for invalid user mart091 from 99.245.238.68 Or this: Oct 4 20:27:51 Invalid user lavi2078 from 70.26.160.194 Oct 4 20:28:18 Failed password for invalid user lavi2078 from 70.26.160.194 Oct 4 20:28:29 Failed password for invalid user lavi2078 from 70.26.160.194 Oct 4 20:28:47 Failed password for invalid user lavi2078 from 70.26.160.194 Oct 4 23:43:39 Invalid user lavi2078 from 70.26.160.194 Oct 4 23:43:55 Failed password for invalid user lavi2078 from 70.26.160.194 Oct 4 23:45:09 refused connect from bas1-jockvale05-1176150210.dsl.bell.ca (70.26.160.194) Or this: Oct 5 01:36:50 Invalid user from 70.55.30.201 Oct 5 01:37:03 Failed password for invalid user from 70.55.30.201 Oct 5 01:37:06 Failed password for invalid user from 70.55.30.201 Oct 5 01:37:30 Failed password for invalid user from 70.55.30.201 Oct 5 01:37:32 Failed password for invalid user from 70.55.30.201 Oct 5 01:38:17 refused connect from bas2-kanata16-1178017481.dsl.bell.ca (70.55.30.201) Lock-out is not an excuse for non-submission -------------------------------------------- A student writes me the night before the assignment is due: > I locked myself out of the CLS. I cant hand in my assignment because of > this. You can. As it says in the [Course Linux Server] notes, you can submit your assignment from any other IP address. If you're locked out of home, go somewhere else to submit. You can always submit from school. Working from home is a privilege. If you abuse the privilege and lock yourself out, go to school. Go VOTE -- a public service message =================================== ![[Students are VOTING!]][1] ![[Stakes are high for young people: VOTE!]][2] ![[Students are VOTING!]][3] ![[Rick Mercer says young people must VOTE (link to CBC radio)]][4]  ![Take Notes in Class] -- | Ian! D. Allen - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Course Home Page]: .. [Course Outline]: course_outline.pdf [All Weeks]: indexcgi.cgi [Plain Text]: week05notes.txt [Week 05 Notes HTML]: week05notes.html [Unix Shell I/O Redirection (including Pipes)]: 200_redirection.html [Linux and Sysadmin News in the World]: 950_linux_world.html [List of Commands You Should Know]: 900_unix_command_list.html [Video Tutorials on Lynda.com]: 910_lynda_index.html [lynda.com]: http://algonquincollege.com/onlineresources/mobileStudent/lynda.htm [Unix for Mac OS X Users]: http://www.lynda.com/Mac-OS-X-10-6-tutorials/Unix-for-Mac-OS-X-Users/78546-2.html [Assignment #03 HTML]: assignment03.html [Assignment #04 HTML]: assignment04.html [Assignment #05 HTML]: assignment05.html [download Libre Office here]: 050_course_introduction.html#install-libreoffice-or-openoffice-into-windows [Worksheet #02 ODT]: worksheet02.odt [Worksheet #02 PDF]: worksheet02.pdf [Worksheet #02 HTML]: worksheet02.html [Worksheet #03 ODT]: worksheet03.odt [Worksheet #03 PDF]: worksheet03.pdf [Worksheet #03 HTML]: worksheet03.html [Worksheet #04 ODT]: worksheet04.odt [Worksheet #04 PDF]: worksheet04.pdf [Worksheet #04 HTML]: worksheet04.html [Worksheet #05 ODT]: worksheet05.odt [Worksheet #05 PDF]: worksheet05.pdf [Worksheet #05 HTML]: worksheet05.html [Worksheet #06 HTML]: worksheet06.html [The VI (VIM) Text Editor]: 300_vi_text_editor.html [Test Instructions]: 000_test_instructions.html [Practice Tests and Answers]: PRACTICE_TEST_README.html [Assignment #02]: assignment02.html [Automatic Backups on the CLS]: 070_course_linux_server.html#automatic-backups-on-the-cls [Verify your GLOB patterns before you use them!]: 190_glob_patterns.html#verifying-glob-patterns-before-using-them [help other students fail the course]: 050_course_introduction.html#plagiarism-and-working-together [Do your lecture notes look like this?]: data/kf_link_redirect.png [Does your Linux notebook look like this?]: data/rn_notebook.jpg [Are you drawing file system trees (PDF)?]: data/ah_diagram.pdf [Week 3]: week03notes.html#fifteen-minute-rule [using `awk` to select fields]: 187_selecting_fields_awk.html [Course Linux Server]: 070_course_linux_server.html [Students are VOTING!]: http://www.facebook.com/munsu35/ [1]: data/student_vote.jpg "VOTE" [Stakes are high for young people: VOTE!]: http://www.straight.com/news/553881/mike-taylor-five-reasons-why-stakes-are-high-young-canadians-election [2]: data/go_vote.jpg "VOTE" [3]: data/youth_vote_disaster.png "VOTE" [Rick Mercer says young people must VOTE (link to CBC radio)]: http://www.cbc.ca/news/canada/rick-mercer-on-the-2015-election-1.3256919 [4]: http://scontent-yyz1-1.xx.fbcdn.net/hphotos-xft1/v/t1.0-9/12038228_10153562822799361_7419787984021110970_n.jpg?oh=6434035a11ad73335ab6941dfebdb620&oe=56A6D21E "VOTE" [Take Notes in Class]: data/remember.jpg "Take Notes in Class" [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/