% CST8207 Week 06 Notes -- Search PATH, Quoting, File System, Inodes, Hard Links % Ian! D. Allen -- -- [www.idallen.com] % Winter 2017 - January to April 2017 - Updated 2017-03-06 00:42 EST - [Course Home Page] - [Course Outline] - [All Weeks] - [Plain Text] Readings, Assignments, Labs, Tests, and ToDo ============================================ Read (at least) these things (All The Words) -------------------------------------------- 1. [Week 06 Notes HTML] -- this file -- **Read All The Words** 2. [Search PATH] -- how the shell finds commands 3. [Quoting] -- hiding characters from the shell 4. [Unix/Linux File System] -- directories, inodes, hard links 5. [List of Commands You Should Know] 6. [Linux and Sysadmin News in the World] 7. [Video Tutorials on Lynda.com] -- tagged by week number - Create your [lynda.com] account and watch these sections of [Unix for Mac OS X Users] - these videos are for *all* Unix/Linux students, not just Mac OSX users - **3. Working with Files and Directories** - Naming files 5m 41s - use quotes to surround names with blanks or special characters - Hard links 5m 30s - Symbolic links 6m 36s - **5. Commands and Programs** - The PATH variable 4m 13s - Disk information commands 6m 8s - df, du, - **7. Configuring Your Working Environment 41m 28s** - Setting the PATH variable 6m 10s - using double quotes instead of single quotes Assignments this week --------------------- Check the due date for each assignment and put a reminder in your agenda, calendar, and digital assistant. Just like in the Real World, not all due dates are on the same days or at the same times. - Review last week. Did you do everything assigned last week? - Read All The Words, Do, and then Submit via Blackboard: - Bonus (optional) [Assignment #04 HTML] -- the VIM text editor - [Assignment #05 HTML] -- GLOB wildcard patterns, finding files using GLOB, redirection and pipes - Bonus (optional) [Assignment #06 HTML] about Midterm #1 - [Assignment #07 HTML] -- start-up files, environment, search PATH, quoting, hard and soft links, disk usage - Really do **Read All The Words**. You don't get a second chance to get it right. - Do Bonus (optional) [Assignment #06 HTML] about Midterm #1 - There is a checking program available to check your file format for this bonus assignment, but only people who Read All These Words will know about it. Wrong format means no marks. Lab work this week ------------------ - Finish work on the ODT or PDF [Worksheet #04 HTML] and [Worksheet #05 HTML] so you can do [Assignment #05 HTML] -- GLOB wildcard patterns, finding files using GLOB, redirection and pipes - These all require you to understand [Finding Files], [Shell GLOB patterns] (wildcard pathname matching) and [Shell I/O Redirection] (including Pipes). - [Assignment #07 HTML] is about start-up files, environment, search PATH, quoting, hard and soft links, disk usage - This assignment requires material not yet covered in lectures. - Do Bonus (optional) [Assignment #06 HTML] about Midterm #1 - There is a checking program available to check your file format for this bonus assignment, but only people who Read All These Words will know about it. Wrong format means no marks. *Do not redirect the output of this checking program!* - Read [The VI (VIM) Text Editor] and optionally do the bonus [Assignment #04 HTML]. Worksheets ---------- Worksheets are preparation for your assignments. You can't do the assignments without having done the worksheets first, and you can't do the worksheets without having first read the Course Notes: 1. Read. 2. Worksheets. 3. Assignment. Form a small study group to do the worksheets. Each person tries the example given, and you make sure you all get the same answers. Worksheets are not for hand-in; they are not worth marks; the assignments test your knowledge of the lectures and worksheets. > The worksheets are available in four formats: Open Office (ODT), PDF, HTML, > and Text. Only the Open Office format allows you "fill in the blanks" in > the worksheet. The PDF format looks good but doesn't allow you to type into > the blanks in the worksheet. The HTML format is crude but useful for quick > for viewing online. Do **NOT** open the Worksheet ODT files using any Microsoft products; they will mangle the format and mis-number the questions. Use the free Libre Office or Open Office programs to open these ODT documents. On campus, you can [download Libre Office here]. - [Worksheet #04 ODT] -- shell GLOB patterns and Aliases - [Worksheet #04 PDF] - [Worksheet #04 HTML] - shell GLOB patterns (wildcards), Aliases: `alias, sum, unalias` - [Worksheet #05 ODT] -- shell I/O redirection (including pipes) - [Worksheet #05 PDF] - [Worksheet #05 HTML] - shell I/O redirection, `date, head, nl, tail, tr, wc` - [Worksheet #06 HTML] -- *Optional* Bonus VIM Text Editor Practice - This is an *optional* worksheet for a BONUS assignment using `vim` - Optional command-line VIM tutorial: the `vimtutor` program on the CLS. - Bonus (optional) [Assignment #04 HTML] -- the VIM text editor - Read [The VI (VIM) Text Editor] Worksheets prepare you for the upcoming assignments. Upcoming tests and exams ------------------------ ![Bring your favourite marker to your test] This course has two midterm tests and one final exam. - Put these dates below into your phone! - Read the [Test Instructions] (all the words) before your tests and exam. - Use the **Name Game** link (in the [Test Instructions]) to test the spelling of your name before the test. - I don't answer questions about the instructions during the test. Ask me in a lab period before the test. ### Second Midterm test -- 10am March 17 -- 15% Your Second Midterm test is 45 minutes long and contains approximately 45 multiple-choice questions similar to those found in the two preceding [Practice Tests and Answers]. Do *both* practice tests before the Second Midterm test! - Time: 10h00-10h45 (10am to 10:45am) Friday March 17 (Week 9) - Location: C346 (the usual lecture classroom) - General seating; sit anywhere. - 45 minutes for 45 questions - Do *both* the practice tests before the Midterm test! *Midterm tests take place at 10am in your 50-minute lecture class, not in your lab period.* ### The Final Exam -- 8am April 28 -- 40% The Final Exam is three hours long and contains approximately 180 multiple-choice questions similar to those found in the three preceding [Practice Tests and Answers]. Do *all three* practice tests before the Final Exam! - Time: 08h00-11h00 (8am to 11am) Friday April 28 (Week 15) - Location: CA-105 A,B,C (across the Woodroffe pedestrian overpass) - This exam has **assigned seating** -- see your email for your assigned seat. - 180 minutes (three hours) for 180 questions - Do *all three* practice tests before the Final Exam! - Three Hours! Take a bathroom break before you start the exam! All three practice tests will be posted under [Practice Tests and Answers]. The Final Exam is comprehensive of the whole course; you need to do *all three* practice tests for the Final Exam. Midterm Test #1 Analysis ======================== Here are the final statistics for the first midterm test: - 94 students are registered in the course. - Of the 94, 3 did not write the test, leaving 91 who did. - Of the 91, 39 did not enter a valid test version code. - Of the 91, 13 did not enter their own name correctly. - Of the 91, 4 did not enter their own student number correctly. - Of the 91, 19 did not put their name on their question sheet. - Of the 91, 20 got question #45 wrong even though the answer was given in the **Test Instructions** printed at the start of the test. - The 91 class scores: 97.8 95.6 93.3 93.3 88.9 84.9 84.4 84.4 84.4 82.2 80 77 75.6 74.8 74.8 73.3 72.6 71.1 71.1 71.1 70.4 70.4 68.9 68.9 68.2 68.2 66.7 66.7 66.7 65.3 64.4 64.4 64.4 63.8 63.2 61.6 61.6 61 60 58.2 57.8 57.8 57.2 56.6 55.6 55.6 55 55 53.3 52.3 51.1 51.1 50.6 47.9 46.2 44.4 41.8 41.8 41.8 41.4 41.4 40 40 39.6 39.2 37.4 37 37 36.7 35.6 35.2 35.2 33.3 33 32.7 31.1 28.9 28.3 28.3 28 26.4 26.4 26.1 25.9 21.8 19.6 19.6 19.4 19.4 17.4 15.6 - You can see the class errors in [Midterm Test #1 PDF]. > I spent several hours correcting your errors on the mark-sense forms. > Before your next test, you need to re-read the [Test Instructions]. > Penalties go up for making errors on the next test. Notes from the Classroom ======================== - **Take notes in class!** Keep a pad open on your desk. - Next week is Study Break Week. Get caught up on your worksheets! - We are under attack! Try these reports on the CLS: - `fgrep '72.142.19.154' /var/log/auth.log | fgrep 'Failed'` - `fgrep 'Failed password for invalid user ' /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -n` - `fgrep 'Failed password for ' /var/log/auth.log | awk '{print $9}' | sort | uniq -c | sort -n` - ASCII-Art Weather. Widen your terminal to 125 characters or more and type this at any shell prompt: `curl wttr.in/ottawa` - See also - A [Command Line Primer for Beginners] - [First 5 Commands When I Connect on a Linux Server] Commands Used ------------- - Keep a notebook with a [List of Commands][List of Commands You Should Know] in it. - You need to write down yourself what each command *does*. - Check the updated list of commands each week. - Bring your notes to class! Stop wasting time looking up commands. - I will check for this list in your lab periods. - Are you making notes from the worksheets on how each command works? - What do the options used in the worksheets mean, for each command? - Don't copy and use options that you don't understand! - Useful command: `hostname` - Needed for [Assignment #07 HTML] - Three useful options to the `wc` command (often used in tests): - `wc -l` - `wc -w` (Needed for your second midterm test.) - `wc -c` - Know (for the assignment) the three commands that are [directory operations] that manipulate file names; they don't touch the file data and don't need any permissions on the file data. Student Feedback ---------------- Note from a student on Tuesday: Dear Professor Allen, Hello! I was working with the mid-term practice last weekend, and it doesn't goes very well. And then I go back to watch the class notes(I know! I should do that in the first place) because I realize that I missed lots of the information. Then I found that the pages of course notes are super helpful. It helps me a lot! I thought the pages are too much links and too complex when I just taking a quick look in the first time, but I was wrong. It's so impressive that the contents are strong logical, everything is explained in detail but not complicated, easy to understand. Most of the things does not take me a long time to understand. I really should read the pages earlier!!! It does takes me many time to adapt the new learning style as a foreign student, but I am feeling better right now. Hopefully, it is not too late for the test. Anyway, just want to tell you that the course pages are very helpful. And thank you, This is a great work for us! Best Regards. Note from a former student: Hey Ian, I just wanted to thank you again for the reference that you me when applying for a government position last year. I ended up getting the job and I am part of a great team. Thanks again, If you want a good reference from me, get great marks and keep your attendance up-to-date. Copying from other students --------------------------- Here is a note from an International Student who was charged with plagiarism (academic fraud) in a previous year: Professor, After everything, I would just like to thank you for not giving us any harsher consequences. I really appreciate it, it's probably the best Christmas gift after this stressful weekend I could get. And what might be weird for you, I'd like to thank you for that you found this plagiarism out because I have a lesson in my mind I will never forget. I'm still very embarrassed [...] but these are the consequences I have to face right now. Thank you once again and I wish you Merry Christmas and Happy New Year. One other student involved in the above case failed the course. Review the class notes on [Plagiarism] if you copy the work of other students or if you let your work be copied by other students. I know who you are and we're going to have a chat about your behaviour soon. Marking Scheme for Assignment #06 ================================= Bonus (optional) [Assignment #06 HTML] may be done after you get your marks email and question sheet back for Midterm #1. You can use this bonus work to optionally improve your mark on the first midterm test. See the assignment for details. Learning and using VIM via Web Browser ====================================== A user found a keyboard-driven web browser (Vimperator) that led him to using VIM, the editor it was based on: : > "I become quickly fond of Vimperator -- it complemented my current workflow > with the mouse very well. I could use my other hand to close tabs with 'd' > etc. Over time my surfing got really efficient and that made me curious -- > if Vimperator can make surfing the web this good, then perhaps that weird > Vim editor isn't that bad after all? So, I read about Vim and started using > it. It took me literally hours before I was hooked with Vim. I was mind > blown! > > And it was after that moment I truly understood how powerful and efficient > a keyboard driven workflow can be. The terminal is not just some niche > thing nerds use to look cool." Learn that editor! HTTP attacks on Course Linux Server =================================== In addition to the SSH attacks, the [Course Linux Server] receives attacks using bogus HTTP requests. These unsuccessful attacks below attempt to use a chain of parent directories to fetch a file out of the file system: # zfgrep '../..' /var/log/apache2/error.log* /var/log/apache2/error.log:[Wed Oct 19 20:49:33 2016] [error] [client 189.100.188.31] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 /var/log/apache2/error.log.1:[Sat Oct 15 06:49:57 2016] [error] [client 177.142.99.103] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 /var/log/apache2/error.log.14.gz:[Mon Jun 27 17:54:40 2016] [error] [client 222.198.128.207] Invalid URI in request GET /../../../../../../../mnt/mtd/3qS9 HTTP/1.1 /var/log/apache2/error.log.2.gz:[Fri Oct 07 09:34:11 2016] [error] [client 64.137.234.42] Invalid URI in request GET /../../../../../../../mnt/mtd/ahMk HTTP/1.1 /var/log/apache2/error.log.22.gz:[Mon May 02 12:21:45 2016] [error] [client 222.235.67.147] Invalid URI in request GET /../../../../../../../mnt/mtd/IJSj HTTP/1.1 /var/log/apache2/error.log.23.gz:[Tue Apr 26 17:09:59 2016] [error] [client 67.79.37.250] Invalid URI in request GET /../../../../../../../mnt/mtd/yqcP HTTP/1.1 /var/log/apache2/error.log.42.gz:[Sun Dec 13 13:50:57 2015] [error] [client 54.201.251.33] Invalid URI in request GET /../../ratings.html HTTP/1.1 /var/log/apache2/error.log.42.gz:[Sun Dec 13 13:51:01 2015] [error] [client 54.201.251.33] Invalid URI in request GET /../../index.html HTTP/1.1 The Apache server blocked all the above attacks. Tutoring ======== Many students find that hiring a personal tutor helps them get through the first term. Financial assistance is available. See the **Tutoring** heading in the [Course Introduction]. ![Take Notes in Class] -- | Ian! D. Allen, BA, MMath - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Course Home Page]: .. [Course Outline]: course_outline.pdf [All Weeks]: indexcgi.cgi [Plain Text]: week06notes.txt [Week 06 Notes HTML]: week06notes.html [Search PATH]: 400_search_path.html [Quoting]: 440_quotes.html [Unix/Linux File System]: 450_file_system.html [List of Commands You Should Know]: 900_unix_command_list.html [Linux and Sysadmin News in the World]: 950_linux_world.html [Video Tutorials on Lynda.com]: 910_lynda_index.html [lynda.com]: http://algonquincollege.com/onlineresources/mobileStudent/lynda.htm [Unix for Mac OS X Users]: http://www.lynda.com/Mac-OS-X-10-6-tutorials/Unix-for-Mac-OS-X-Users/78546-2.html [Assignment #04 HTML]: assignment04.html [Assignment #05 HTML]: assignment05.html [Assignment #06 HTML]: assignment06.html [Assignment #07 HTML]: assignment07.html [Worksheet #04 HTML]: worksheet04.html [Worksheet #05 HTML]: worksheet05.html [Finding Files]: 180_finding_files.html [Shell GLOB patterns]: 190_glob_patterns.html [Shell I/O Redirection]: 200_redirection.html [The VI (VIM) Text Editor]: 300_vi_text_editor.html [download Libre Office here]: 050_course_introduction.html#install-libreoffice-or-openoffice-into-windows [Worksheet #04 ODT]: worksheet04.odt [Worksheet #04 PDF]: worksheet04.pdf [Worksheet #05 ODT]: worksheet05.odt [Worksheet #05 PDF]: worksheet05.pdf [Worksheet #06 HTML]: worksheet06.html [Bring your favourite marker to your test]: data/sniff_markers.jpg "Bring your favourite marker to your test" [Test Instructions]: 000_test_instructions.html [Practice Tests and Answers]: PRACTICE_TEST_README.html [Midterm Test #1 PDF]: midterm1_17w.pdf [Command Line Primer for Beginners]: http://lifehacker.com/5633909/who-needs-a-mouse-learn-to-use-the-command-line-for-almost-anything [First 5 Commands When I Connect on a Linux Server]: http://www.linux.com/blog/first-5-commands-when-i-connect-linux-server [directory operations]: 455_links_and_inodes.html [Plagiarism]: 050_course_introduction.html#plagiarism-and-working-together [Course Linux Server]: 070_course_linux_server.html [Course Introduction]: 050_course_introduction.html [Take Notes in Class]: data/remember.jpg "Take Notes in Class" [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/