% CST8207 Week 06 Notes -- Midterm Analysis, Search PATH, Quoting, Work Stoppage % Ian! D. Allen -- -- [www.idallen.com] % Fall 2017 - September to December 2017 - Updated 2017-11-26 21:47 EST - [Course Home Page] - [Course Outline] - [All Weeks] - [Plain Text] Possible cessation of classes next week (Oct 16) ================================================ College faculty want quality education issues addressed; their offer to Management from September 27 is summarized here and available here (PDF download). Faculty have prepared a list of [Frequently Asked Questions for students]. Management walked away from the bargaining table on October 10, saying their current offer is the final offer and they will not bargain further to avoid a strike. They want the faculty bargaining team to present their latest offer directly to faculty for a vote. The Management commentary and offer is here (PDF download). You will note that the two sides are not addressing the same issues. The faculty response to the Management final offer is here: A faculty strike has been called for early on Monday, October 16. More information is available in the [FAQ for students][Frequently Asked Questions for students] and [on the faculty union web site] and via [Work Stoppage Resources]. It's not clear whether access to the Course Linux Server will continue in the event of a faculty strike. The College may lock out access, though since Blackboard will remain accessible, I hope the CLS will, too. Access to off-campus course web pages will continue. Readings, Assignments, Labs, Tests, and ToDo ============================================ Read (at least) these things (All The Words) -------------------------------------------- 1. [Week 06 Notes HTML] -- this file -- **Read All The Words** 2. [Search PATH] -- how the shell finds commands 3. [Quoting] -- hiding characters from the shell 4. For next week: [Unix/Linux File System] -- directories, inodes, hard links 5. [List of Commands You Should Know] 6. [Video Tutorials on Lynda.com] -- tagged by week number 7. [Linux and Sysadmin News in the World] - Create your [lynda.com] account and watch these sections of [Unix for Mac OS X Users] - these videos are for *all* Unix/Linux students, not just Mac OSX users - **3. Working with Files and Directories** - Naming files 5m 41s - use quotes to surround names with blanks or special characters - Hard links 5m 30s - Symbolic links 6m 36s - **5. Commands and Programs** - The PATH variable 4m 13s - Disk information commands 6m 8s - df, du, - **7. Configuring Your Working Environment 41m 28s** - Setting the PATH variable 6m 10s - using double quotes instead of single quotes Assignments and lab work this week ---------------------------------- Check the due date for each assignment and put a reminder in your agenda, calendar, and digital assistant. Just like in the Real World, not all due dates are on the same days or at the same times. - Review last week. Did you do everything assigned last week? - Read All The Words, Do, and then Submit via Blackboard: - [Assignment #05 HTML] -- GLOB wildcard patterns, finding files using GLOB, redirection and pipes - Coming soon: [Assignment #07 HTML] -- start-up files, environment, search PATH, quoting, hard and soft links, disk usage - OPTIONAL: Read [The VI (VIM) Text Editor] and optionally do the bonus [Assignment #04 HTML]. - Really do **Read All The Words**. You don't get a second chance to get it right. - Do Bonus (optional) [Assignment #06 HTML] about Midterm #1 - There is a checking program available to check your file format for this bonus assignment, but only people who Read All These Words will know about it. Wrong file format means no marks. *Do not redirect the output of this checking program!* Lab work this week ------------------ - Finish work on the ODT or PDF [Worksheet #04 HTML] and [Worksheet #05 HTML] so you can do [Assignment #05 HTML] -- GLOB wildcard patterns, finding files using GLOB, redirection and pipes - These all require you to understand [Finding Files], [Shell GLOB patterns] (wildcard pathname matching) and [Shell I/O Redirection] (including Pipes). - [Assignment #07 HTML] is about start-up files, environment, search PATH, quoting, hard and soft links, disk usage - This assignment will require material not yet covered in lectures. - OPTIONAL: Do Bonus [Assignment #06 HTML] about Midterm #1 - There is a checking program available to check your file format for this bonus assignment, but only people who Read All These Words will know about it. Wrong file format means no marks. *Do not redirect the output of this checking program!* - OPTIONAL: Read [The VI (VIM) Text Editor] and optionally do the bonus [Assignment #04 HTML]. ### Worksheets Worksheets are preparation for your assignments. You can't do the assignments without having done the worksheets first, and you can't do the worksheets without having first read the Course Notes: 1. Read the web notes. (Please: **Read All The Words**) 2. Do the relevant Worksheet(s). 3. Do the relevant Assignment(s). Make notes from the worksheets on how each command works. What do the options used in the worksheets mean, for each command? (See the weekly [Command List][List of Commands You Should Know].) Form a small study group to do the worksheets. Each person tries the example given, and you make sure you all get the same answers. Worksheets are not for hand-in; they are not worth marks; the assignments test your knowledge of the lectures and worksheets. > The worksheets are available in four formats: Open Office (ODT), PDF, HTML, > and Text. Only the Open Office format allows you "fill in the blanks" in > the worksheet. The PDF format looks good but doesn't allow you to type into > the blanks in the worksheet. The HTML format is crude but useful for quick > for viewing online. Do **NOT** open the Worksheet ODT files using any Microsoft products; they will mangle the format and mis-number the questions. Use the free Libre Office or Open Office programs to open these ODT documents. On campus, you can [download Libre Office here]. - [Worksheet #04 ODT] -- shell GLOB patterns and Aliases - [Worksheet #04 PDF] - [Worksheet #04 HTML] - shell GLOB patterns (wildcards), Aliases: `alias, sum, unalias` - [Worksheet #05 ODT] -- shell I/O redirection (including pipes) - [Worksheet #05 PDF] - [Worksheet #05 HTML] - shell I/O redirection, `date, head, nl, tail, tr, wc` - [Worksheet #06 HTML] -- *Optional* Bonus VIM Text Editor Practice - This is an *optional* worksheet for a BONUS assignment using `vim` - Optional command-line VIM tutorial: the `vimtutor` program on the CLS. - Bonus (optional) [Assignment #04 HTML] -- the VIM text editor - Read [The VI (VIM) Text Editor] Worksheets prepare you for the upcoming assignments. Upcoming tests and exams ------------------------ This course has two midterm tests (10%, 15%) and one final exam (40%). ![Bring your favourite marker to your test] > Note that your upcoming Midterm Test #2 is worth more than five assignments > put together. Don't spend all your time doing an assignment at the expense > of studying the [Practice Tests and Answers] for the Midterm Test. - Put these dates below into your phone! - Read the [Test Instructions] (all the words) before your tests and exam. - Use the [Algonquin Registered Name Game] link in the [Test Instructions] to test the spelling of your name before the test. - I don't answer questions about the instructions during the test. Ask me in a lab period before the test. - You must write the test in the lecture class in which you are registered. ### Second Midterm test -- November 2 or 3 -- 15% Your Second Midterm test is 45 minutes long and contains approximately 45 multiple-choice questions similar to those found in the two preceding [Practice Tests and Answers]. Do *both* practice tests before the Second Midterm test! - Time: in your one-hour lecture class on Thursday or Friday in Week 9 (November 2 or 3). - General seating; sit anywhere. - 45 minutes for 45 multiple-choice questions - Do *both* the practice tests before the Midterm test! Midterm Test #1 Analysis ======================== > I spent many hours correcting your errors and omissions on your mark-sense > forms. Many of you lost marks for that. Before your next test, carefully > re-read the posted [Test Instructions]. Penalties go up for making errors > on the next tests. Here are some statistics for the first midterm test: - 211 students are registered in the course. - Of the 211, 8 did not write the test, leaving 203 who did. - Of the 203, 47 did not enter a valid test version code on the Scantron form. (I had to enter all those test version codes for you, so that you could get some marks.) - Of the 203, 26 did not enter their own name to match the name on file with the Algonquin Registrar office. (These students didn't check their names using the Name Game before the test.) - Of the 203, 18 did not enter their own student number correctly. - Of the 203, 9 did not enter their own name or their own student number correctly, so the test couldn't be marked. (I had to fix those names for you so that you could get some marks.) - Of the 203, 24 did not put their full name on their question sheet. (18 left the name field blank. I had to fix those for you, so that you could get your question sheet back for the bonus assignment.) - Of the 203, 31 got question #45 wrong even though the answer was given in the **Test Instructions** printed at the start of the test. (12 people used the same, wrong, answer as the practice test.) - **Read All The Words** ! The 203 class scores: 100 100 97.8 97.8 95.6 95.6 95.6 95.6 93.3 93.3 93.3 93.3 93.3 93.3 93.3 92.4 91.1 90.8 90.2 88.9 88.9 88.9 88.9 88.9 88.9 86.7 86.7 86.7 86.7 86.7 86.2 84.4 84.4 84.4 84.4 84.4 84.4 84.4 84.4 84 83.6 82.3 82.2 82.2 82.2 82.2 82.2 82.2 81.4 81.4 80.2 80 80 79.4 78.1 77.8 77.8 77.8 77.8 77.8 77.6 76 76 75.6 75.6 75.6 74.8 73.3 73.3 73.3 73.3 73.3 72.6 72.6 71.8 71.1 71.1 71.1 70.4 69.7 68.9 68.9 68.9 67.6 67.6 66.8 66.7 66 66 64.4 64.4 64.4 64.4 63.8 62.2 62.2 62.2 62.2 60 60 60 60 60 60 59.1 58.2 58.2 57.8 57.8 57.3 57.2 55.6 54.9 54.9 54.9 54.9 53.9 53.3 53.3 53.3 53.3 52.8 52.8 52.8 52.8 52.8 52.2 51.7 51.1 51.1 51.1 50.7 49.1 48.9 48.9 48.9 48.9 48.9 48.9 48.6 47.2 47.1 46.7 45.3 44.2 43 42.2 42.2 41.8 40.1 40 40 40 40 40 38.1 38.1 38 38 38 38 37.8 37.4 36.6 36.1 34.8 33.3 33.3 33.3 33.3 32.5 31.1 30.7 30.7 29.6 28.9 28 27.2 26.7 26.7 26.7 26.7 24.4 24.4 24.2 23.7 23.2 22.5 22.2 22.2 20.9 20.3 19 19 19 17.8 17.2 16.7 14.8 12.7 9.7 8.4 4.4 - You can see the class errors in [Midterm Test #1 PDF]. Notes from the Classroom ======================== - **Take notes in class!** Keep a pad open on your desk. - ASCII-Art Weather. Widen your terminal to 125 characters or more and type this at any shell prompt: `curl wttr.in/ottawa` - See also - A [Command Line Primer for Beginners] - Uses a newbie command: `cat file | grep pattern` - Use `grep pattern file` instead! - Uses incorrect `-mtime 5` when it should be `-mtime -5` - Don't alias system command names such as `ls` or you won't be able to do the practice tests or worksheets! - [First 5 Commands When I Connect on a Linux Server] Commands Used ------------- - Keep a notebook with a [List of Commands][List of Commands You Should Know] in it. - You need to write down yourself what each command *does*. - Check the updated list of commands each week. - I will check for this list in your lab periods. - Bring your notes to class! Stop wasting time looking up commands. - Are you making notes from the worksheets on how each command works? - What do the options used in the worksheets mean, for each command? - Don't copy and use options that you don't understand! - Useful command: `hostname` - Needed for [Assignment #07 HTML] - Three useful options to the `wc` command (often used in tests): - `wc -l` - `wc -w` (Needed for your second midterm test.) - `wc -c` Marking Scheme for Assignment #06 ================================= Bonus (optional) [Assignment #06 HTML] may be done after you get your marks email and question sheet back for Midterm #1. You can use this bonus work to optionally improve your mark on the first midterm test. See the assignment for details. Your question sheet will be brought to the lab section you wrote on the question sheet, if it is a valid lab section. See the Blackboard course Announcements for details (under "bonus Assignment 6 details"). Searching the course notes ========================== I received an email like the text below from a student who was working on a problem for over two hours: > I have spent almost 2 hours on Task 4.6 and I still can't find the 100 > files. \[...\] I think the problem is searching for warez case-insensitive. > I know we learned this before, but I just can't find it in my notes. This is not good use of student time for two reasons: 1. Don't spend more than 15 minutes on a problem! Your time as a student is valuable. See the [Course Introduction: fifteen minute rule] 2. I showed you how to search the course notes for keywords in [Assignment #05 HTML]: - In section [3.2 Searching the course notes on the CLS] - In section [4.5 Searching for text inside files (e.g. course notes)] You have a search tool that will find keywords inside the course notes. Please use it and save yourself time. Student Feedback -- How to succeed in Linux =========================================== I encourage you to read the course notes. Don't take my word for it: Note from a student in 17W: From: Damien Houle - CST8207-17W Subject: Wonderful Website I wanted to make sure your aware of how much the students and myself from last semester appreciate your awesome website. Many of us have used it on multiple occasions throughout our level 2 course for our current assignments because it has everything we need, as long as we READ ALL THE WORDS! I really miss your labs and lectures and realize now how spoiled I was to have you as my teacher and Linux mentor. Hope the semester offers you as much fun as our class was although i doubt it. Best regards Damien Houle (aka Linux people) Note from a student in 17W: Dear Professor Allen, I was working with the mid-term practice last weekend, and it doesn't goes very well. And then I go back to watch the class notes(I know! I should do that in the first place) because I realize that I missed lots of the information. Then I found that the pages of course notes are super helpful. It helps me a lot! I thought the pages are too much links and too complex when I just taking a quick look in the first time, but I was wrong. It's so impressive that the contents are strong logical, everything is explained in detail but not complicated, easy to understand. Most of the things does not take me a long time to understand. I really should read the pages earlier!!! It does takes me many time to adapt the new learning style as a foreign student, but I am feeling better right now. Hopefully, it is not too late for the test. Anyway, just want to tell you that the course pages are very helpful. And thank you, This is a great work for us! Getting a job reference from your professor =========================================== Note from another former student: I just wanted to thank you again for the reference that you me when applying for a government position last year. I ended up getting the job and I am part of a great team. Thanks again, If you want a good reference from me, get great marks and keep your attendance up-to-date. Copying from other students -- plagiarism ========================================= Here is a note from a Student who was charged with plagiarism (academic fraud) in a previous year: Professor, After everything, I would just like to thank you for not giving us any harsher consequences. I really appreciate it, it's probably the best Christmas gift after this stressful weekend I could get. And what might be weird for you, I'd like to thank you for that you found this plagiarism out because I have a lesson in my mind I will never forget. I'm still very embarrassed [...] but these are the consequences I have to face right now. Thank you once again and I wish you Merry Christmas and Happy New Year. One other student involved in the above case failed the course. Review the class notes on [Plagiarism] if you copy the work of other students or if you let your work be copied by other students. I know who you are and we're going to have a chat about your behaviour soon. Learning and using VIM via Web Browser ====================================== A user found a keyboard-driven web browser (Vimperator) that led him to using VIM, the editor it was based on: : > "I become quickly fond of Vimperator -- it complemented my current workflow > with the mouse very well. I could use my other hand to close tabs with 'd' > etc. Over time my surfing got really efficient and that made me curious -- > if Vimperator can make surfing the web this good, then perhaps that weird > Vim editor isn't that bad after all? So, I read about Vim and started using > it. It took me literally hours before I was hooked with Vim. I was mind > blown! > > And it was after that moment I truly understood how powerful and efficient > a keyboard driven workflow can be. The terminal is not just some niche > thing nerds use to look cool." Learn that editor! Tutoring ======== Many students find that hiring a personal tutor helps them get through the first term. Financial assistance is available. See the **Tutoring** heading in the [Course Introduction]. Attacks on the CLS ================== You are not paranoid if they really are out to get you! See the [Week 03 Notes HTML] for command pipelines to count the SSH attacks on the CLS. HTTP attacks on Course Linux Server ----------------------------------- In addition to the SSH attacks, machines on the Internet receive attacks using bogus HTTP requests. These unsuccessful attacks below attempt to use a chain of parent directories to fetch a file out of the file system: [Dec 13 13:50:57 2015] [client 54.201.251.33] Invalid URI in request GET /../../ratings.html HTTP/1.1 [Dec 13 13:51:01 2015] [client 54.201.251.33] Invalid URI in request GET /../../index.html HTTP/1.1 [Apr 26 17:09:59 2016] [client 67.79.37.250] Invalid URI in request GET /../../../../../../../mnt/mtd/yqcP HTTP/1.1 [May 02 12:21:45 2016] [client 222.235.67.147] Invalid URI in request GET /../../../../../../../mnt/mtd/IJSj HTTP/1.1 [Jun 27 17:54:40 2016] [client 222.198.128.207] Invalid URI in request GET /../../../../../../../mnt/mtd/3qS9 HTTP/1.1 [Oct 07 09:34:11 2016] [client 64.137.234.42] Invalid URI in request GET /../../../../../../../mnt/mtd/ahMk HTTP/1.1 [Oct 15 06:49:57 2016] [client 177.142.99.103] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [Oct 19 20:49:33 2016] [client 189.100.188.31] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [Oct 23 00:43:39 2016] [client 191.189.55.9] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [Nov 02 21:35:09 2016] [client 201.37.223.72] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [Jan 07 20:51:33 2017] [client 201.17.154.181] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [May 10 21:22:29 2017] [client 187.67.167.31] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 The Apache server blocked all the above attacks. ![Take Notes in Class] -- | Ian! D. Allen, BA, MMath - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Course Home Page]: .. [Course Outline]: course_outline.pdf [All Weeks]: indexcgi.cgi [Plain Text]: week06notes.txt [Frequently Asked Questions for students]: http://www.collegefaculty.org/student_qa [on the faculty union web site]: http://locallines.org/ [Work Stoppage Resources]: http://www.algonquincollege.com/strikeinfo/ [Week 06 Notes HTML]: week06notes.html [Search PATH]: 400_search_path.html [Quoting]: 440_quotes.html [Unix/Linux File System]: 450_file_system.html [List of Commands You Should Know]: 900_unix_command_list.html [Video Tutorials on Lynda.com]: 910_lynda_index.html [Linux and Sysadmin News in the World]: 950_linux_world.html [lynda.com]: http://algonquincollege.com/onlineresources/mobileStudent/lynda.htm [Unix for Mac OS X Users]: http://www.lynda.com/Mac-OS-X-10-6-tutorials/Unix-for-Mac-OS-X-Users/78546-2.html [Assignment #05 HTML]: assignment05.html [Assignment #07 HTML]: assignment07.html [The VI (VIM) Text Editor]: 300_vi_text_editor.html [Assignment #04 HTML]: assignment04.html [Assignment #06 HTML]: assignment06.html [Worksheet #04 HTML]: worksheet04.html [Worksheet #05 HTML]: worksheet05.html [Finding Files]: 180_finding_files.html [Shell GLOB patterns]: 190_glob_patterns.html [Shell I/O Redirection]: 200_redirection.html [download Libre Office here]: 050_course_introduction.html#install-libreoffice-or-openoffice-into-windows [Worksheet #04 ODT]: worksheet04.odt [Worksheet #04 PDF]: worksheet04.pdf [Worksheet #05 ODT]: worksheet05.odt [Worksheet #05 PDF]: worksheet05.pdf [Worksheet #06 HTML]: worksheet06.html [Bring your favourite marker to your test]: data/sniff_markers.jpg "Bring your favourite marker to your test" [Practice Tests and Answers]: PRACTICE_TEST_README.html [Test Instructions]: 000_test_instructions.html [Algonquin Registered Name Game]: http://cst8207.idallen.ca/~idallen/namegame.cgi [Midterm Test #1 PDF]: midterm1_17f.pdf [Command Line Primer for Beginners]: http://lifehacker.com/5633909/who-needs-a-mouse-learn-to-use-the-command-line-for-almost-anything [First 5 Commands When I Connect on a Linux Server]: http://www.linux.com/blog/first-5-commands-when-i-connect-linux-server [Course Introduction: fifteen minute rule]: 050_course_introduction.html#fifteen-minute-rule-dont-waste-your-time [3.2 Searching the course notes on the CLS]: assignment05.html#searching-the-course-notes-on-the-cls [4.5 Searching for text inside files (e.g. course notes)]: assignment05.html#searching-for-text-inside-files-e.g.course-notes [Plagiarism]: 050_course_introduction.html#plagiarism-and-working-together [Course Introduction]: 050_course_introduction.html [Week 03 Notes HTML]: week03notes.html [Take Notes in Class]: data/remember.jpg "Take Notes in Class" [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/