% CST8207 Week 04 Notes -- stdout, stderr, and I/O redirection (including pipes) % Ian! D. Allen -- -- [www.idallen.com] % Fall 2015 - September to December 2015 - Updated 2015-10-09 04:29 EDT - [Course Home Page] - [Course Outline] - [All Weeks] - [Plain Text] Readings, Assignments, Labs, Tests, and ToDo ============================================ ![[Dilbert -- Linux Advanced Technology]][1] Read (at least) these things (All The Words) -------------------------------------------- 1. [Week 04 Notes HTML] -- this file -- **Read All The Words** 2. [Unix Shell I/O Redirection (including Pipes)] 3. [Linux and Sysadmin News in the World] 4. [List of Commands You Should Know] -- keep a notebook! 5. [Video Tutorials on Lynda.com] -- tagged by week number - Create your [lynda.com] account and watch these sections of [Unix for Mac OS X Users] - **3. Working with Files and Directories** - Unix text editors 6m 39s - `vi/vim, nano` - `head, tail` - Searching for files and directories 6m 32s - `find` using "wildcard" (GLOB) patterns - same as shell GLOB - **6. Directing Input and Output 20m 39s** - Standard input and standard output 1m 24s - Directing output to a file 4m 13s - Appending to a file 2m 44s - Directing input from a file 5m 28s - Piping output to input 4m 40s - Suppressing output 2m 10s - **7. Configuring Your Working Environment 41m 28s** - Setting command aliases 6m 59s - saving aliases in your `.bashrc` file Assignments this week --------------------- Check the due date for each assignment and put a reminder in your agenda, calendar, and digital assistant. - Read All The Words, Do, and then Submit via Blackboard: - [Assignment #03 HTML] -- working with files: PS1, find, mkdir, mv, cp - Bonus (optional) [Assignment #04 HTML] -- the VIM text editor - Really do **Read All The Words**. You don't get a second chance to get it right. ### Worksheets Form a small study group to do the worksheets. Each person tries the example given, and you make sure you all get the same answers. Worksheets teach you how commands work. Worksheets are not for hand-in; they are not worth marks. > The worksheets are available in four formats: Open Office (ODT), PDF, HTML, > and Text. Only the Open Office format allows you "fill in the blanks" in > the worksheet. The PDF format looks good but doesn't allow you to type into > the blanks in the worksheet. The HTML format is crude but useful for quick > for viewing online. Do **NOT** open the ODT files using any Microsoft products; they will mangle the format and mis-number the questions. Use the free Libre Office or Open Office programs to open these ODT documents. On campus, you can [download Libre Office here]. - [Worksheet #02 ODT] -- Using standard Linux commands I - [Worksheet #02 PDF] - [Worksheet #02 HTML] - `PS1, cd, find, less, ls, man, mkdir, passwd, pwd, rmdir` - [Worksheet #03 ODT] -- Using standard Linux commands II - [Worksheet #03 PDF] - [Worksheet #03 HTML] - `cat, clear, cp, find, grep, history, less, man, mv, rm, sleep, touch` - [Worksheet #04 ODT] -- shell GLOB patterns and Aliases - [Worksheet #04 PDF] - [Worksheet #04 HTML] - shell GLOB patterns (wildcards), Aliases: `alias, sum, unalias` - [Worksheet #05 ODT] -- shell I/O redirection (including pipes) - [Worksheet #05 PDF] - [Worksheet #05 HTML] - shell I/O redirection, `date, head, nl, tail, tr, wc` - [Worksheet #06 HTML] -- *Optional* Bonus VIM Text Editor Practice - This is an *optional* worksheet for a BONUS assignment using `vim` - Optional command-line VIM tutorial: the `vimtutor` program on the CLS. - Bonus (optional) [Assignment #04 HTML] -- the VIM text editor - Read [The VI (VIM) Text Editor] Lab work this week ------------------ - Continue to work on [Worksheet #02 PDF] and [Worksheet #03 PDF] so you can do [Assignment #03 HTML]. - Read [The VI (VIM) Text Editor] and optionally do the bonus [Assignment #04 HTML]. - For Assignment 5 do [Worksheet #04 PDF] -- shell GLOB patterns and Aliases Upcoming tests -------------- Read the [Test Instructions] (all the words) before your midterm tests. 1. First Midterm test: 45 minutes; in lecture class Friday in Week 5 (Oct 9) 2. Second Midterm test: 45 minutes; in lecture class Friday in Week 9 (Nov 6) Tests take place in class in your one-hour lecture hour, not in your lab period. ### Midterm Test #1 -- October 9 - Midterm #1 takes place on Friday, October 9 (Week 5) in your scheduled lecture hour (not in your lab period). - For full marks, you must read the [Test Instructions] before the test for important directions on how to enter your answers, your lab (not lecture) section number, and the test version number on the question sheet and the mark-sense forms. - There may be more questions on the test than you can answer in the time allowed; answer the ones you know, first. - A set of practice questions and answers for the first midterm test is posted: [Practice Tests and Answers]. - The tests use the semicolon `;` to separate multiple commands on the same line, to save space and paper, e.g. three separate commands can be written on one line like this: `date ; echo "hi" ; ls` - Blackboard has some quizzes taken randomly from the practice test. See below. ### Quizzes: Midterm #1 Quiz This quiz is one of several quizzes in this course. Each midterm and final exam will have an associated quiz. See the course outline for the mark weight of all course quizzes, midterm tests, and exams. The quizzes are open-book, but the midterm tests and final exam are closed-book (as will be your job interview). This quiz is based on the [Midterm #1 Practice Test] questions that are posted in the [Class Notes][All Weeks]. The quiz is 10 questions long and you see the answers right after you submit the quiz. You can take the quiz as many times as you like. Every time you take the quiz, you get a random set of ten questions from the practice test. You will not see all the practice questions by doing quizzes; to see all the practice questions, you must do all the questions in the actual PDF practice test posted in the Course Notes. This quiz closes just before the Final Exam in this course; quizzes submitted after the Final Exam begins may not count toward your best score. Your mark for this quiz is the average of your five best quiz scores. Examples: - Your best scores: 10 10 10 10 10 10 10 9 8 7 7 4 - Your quiz mark: (10+10+10+10+10)/50 = 100% - Your best scores: 10 10 10 10 9 8 7 7 4 3 - Your quiz mark: (10+10+10+10+9)/50 = 98% - Your best scores: 10 10 10 9 8 7 7 4 3 2 - Your quiz mark: (10+10+10+9+8)/50 = 94% - Your best scores: 10 8 8 8 8 5 4 4 4 - Your quiz mark: (10+8+8+8+8)/50 = 84% - Your best scores: 10 10 8 (only three quizzes submitted) - Your quiz mark: (10+10+8+0+0)/50 = 56% Your score will be taken from the five best scores. The more times you do the quiz, the more likely you are to have a set of excellent best scores. You must have five perfect quiz scores to get a perfect averaged quiz mark. Missing quizzes (fewer than five) count as zeroes. Only the five best scores are averaged. (Blackboard averages all the scores and thus displays your score incorrectly.) This quiz closes just before the Final Exam in this course; quizzes submitted after the Final Exam begins may not count toward your best score. See the **Assignments and Quizzes** section in the Blackboard left side-bar for your course. **Note:** *Blackboard displays the quiz mark incorrectly, since it cannot calculate "best 5" and instead averages the marks of **all** your quiz attempts. Your quiz mark is actually the average of your five **best** attempts, not all the attempts as shown by Blackboard. Ignore the Blackboard mark; it is wrong.* From the Class Notes link on the Course Home Page ================================================= - [Course Home Page] - [All Weeks] - Review last week. Did you do everything assigned last week? From the Classroom Whiteboard/Chalkboard ======================================== - **Take notes in class!** Your in-class notes would go here. - Are you making notes from the worksheets on how each command works? - What do the options used in the worksheets mean, for each command? - Don't copy and use options that you don't understand! - The stupid `nano` text editor. (Learn `vim` instead!) - Using GLOB patterns as `-name` arguments to `find` - Remember to quote the `find` GLOB patterns to hide them from the shell! - GLOB patterns are *NOT* needed in Assignment 3. See Assignment 5. - Confused about redirection and the copy and move commands? These are wrong: - `cp cal.txt >calnew,txt # error message from cp` - `mv cal.txt ../../ >cal.txt # what is now in cal.txt ?` - No example in any of the notes or worksheets ever uses redirection and `cp` or `mv` together. These commands do not produce anything on standard output that is worth redirecting. Usually, the commands produce no output at all. Do not use redirection here. - Confused about pathnames and blanks on the command line? This is wrong: - `cp file.txt ../../../ file.copy` - Blanks separate arguments. You are giving the copy command three pathnames, where you should have only two. - Stop using `cd` Windows People! Use pathnames! This is very inefficient: $ cd CST8207-15F $ cd Assignments $ cd assignment03 $ cd topdir $ cd other $ cat foo.txt This is all you need to do (using the TAB key to help you): $ cat CST8207-15F/Assignments/assignment03/topdir/other/foo.txt - Stop using `cd` Windows People! Use pathnames! This is very inefficient: $ cd dir1 $ cd subdir $ touch file $ cd .. $ cd .. $ cd dir2 $ cd subdir $ touch file $ cd .. $ cd .. [...etc...] This is all you need to do (using the TAB key to help you): $ touch dir1/subdir/file dir2/subdir/file [...etc...] - If a pathname doesn't work, use `ls` to find out why. Usually it's because you're spelling it wrong. Use the TAB key and the shell will complete the pathname for you, with no spelling errors. Commands -------- - `head, tail, fgrep, wc` - Count the pathnames (5 lines on the CLS): - `ls /bin/*sh | wc` - Count the number of lines containing `idallen` (over 500 lines on the CLS): - `fgrep 'idallen' /etc/passwd | wc` - More efficient way to count matching lines using an option: - `fgrep -c 'idallen' /etc/passwd` - Show only the last 10 matching lines: - `fgrep 'refused connect' /var/log/auth.log | tail` - `fgrep 'password' /var/log/auth.log | tail` - `fgrep 'idallen' /etc/passwd | tail` - Show your 10 most recent login sessions to the CLS (use your own userid): - `last abcd0001 | head` - using GLOB patterns in `find -name` expressions: `find /etc -name '*pass*'` - GLOB patterns are *NOT* needed in Assignment 3. See Assignment 5. ![[Rick Mercer says young people must VOTE (link to CBC radio)]][2] ![[Students are VOTING!]][3]  ![Take Notes in Class] -- | Ian! D. Allen - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Course Home Page]: .. [Course Outline]: course_outline.pdf [All Weeks]: indexcgi.cgi [Plain Text]: week04notes.txt [Dilbert -- Linux Advanced Technology]: http://dilbert.com/strip/2007-01-25 [1]: http://assets.amuniversal.com/94aaa2106cc801301d50001dd8b71c47 "Dilbert -- Linux Advanced Technology" [Week 04 Notes HTML]: week04notes.html [Unix Shell I/O Redirection (including Pipes)]: 200_redirection.html [Linux and Sysadmin News in the World]: 950_linux_world.html [List of Commands You Should Know]: 900_unix_command_list.html [Video Tutorials on Lynda.com]: 910_lynda_index.html [lynda.com]: http://algonquincollege.com/onlineresources/mobileStudent/lynda.htm [Unix for Mac OS X Users]: http://www.lynda.com/Mac-OS-X-10-6-tutorials/Unix-for-Mac-OS-X-Users/78546-2.html [Assignment #03 HTML]: assignment03.html [Assignment #04 HTML]: assignment04.html [download Libre Office here]: 050_course_introduction.html#install-libreoffice-or-openoffice-into-windows [Worksheet #02 ODT]: worksheet02.odt [Worksheet #02 PDF]: worksheet02.pdf [Worksheet #02 HTML]: worksheet02.html [Worksheet #03 ODT]: worksheet03.odt [Worksheet #03 PDF]: worksheet03.pdf [Worksheet #03 HTML]: worksheet03.html [Worksheet #04 ODT]: worksheet04.odt [Worksheet #04 PDF]: worksheet04.pdf [Worksheet #04 HTML]: worksheet04.html [Worksheet #05 ODT]: worksheet05.odt [Worksheet #05 PDF]: worksheet05.pdf [Worksheet #05 HTML]: worksheet05.html [Worksheet #06 HTML]: worksheet06.html [The VI (VIM) Text Editor]: 300_vi_text_editor.html [Test Instructions]: 000_test_instructions.html [Practice Tests and Answers]: PRACTICE_TEST_README.html [Midterm #1 Practice Test]: practicetest1.pdf [Rick Mercer says young people must VOTE (link to CBC radio)]: http://www.cbc.ca/news/canada/rick-mercer-on-the-2015-election-1.3256919 [2]: http://scontent-yyz1-1.xx.fbcdn.net/hphotos-xft1/v/t1.0-9/12038228_10153562822799361_7419787984021110970_n.jpg?oh=6434035a11ad73335ab6941dfebdb620&oe=56A6D21E "VOTE" [Students are VOTING!]: http://www.facebook.com/munsu35/ [3]: data/student_vote.jpg "VOTE" [Take Notes in Class]: data/remember.jpg "Take Notes in Class" [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/