% CST8207 Week 05 Notes -- awk fields, variables, start-up files .bashrc, Midterm Test #1 % Ian! D. Allen -- -- [www.idallen.com] % Fall 2017 - September to December 2017 - Updated 2017-10-10 13:59 EDT - [Course Home Page] - [Course Outline] - [All Weeks] - [Plain Text] Readings, Assignments, Labs, Tests, and ToDo ============================================ Read (at least) these things (All The Words) -------------------------------------------- 1. [Week 05 Notes HTML] -- this file -- **Read All The Words** 2. [Selecting Fields with `awk`] -- extracting fields from input lines and columns from input files 3. [Shell Variables] -- quoting, local and environment variables, `USER HOME SHELL PATH TERM $$` 4. [Start-Up Files] -- Login and Shell Startup Files, aliases, options: `.bash_profile` and `.bashrc` 5. [Practice Tests and Answers] -- practice questions for your midterm test. 6. [List of Commands You Should Know] 7. [Video Tutorials on Lynda.com] -- tagged by week number 8. [Linux and Sysadmin News in the World] - Create your [lynda.com] account and watch these sections of [Unix for Mac OS X Users] - these videos are for *all* Unix/Linux students, not just Mac OSX users - **3. Working with Files and Directories** - Unix text editors 6m 39s - `vi/vim, nano` - `head, tail` - Naming files 5m 41s - use quotes to surround names with blanks or special characters - **5. Commands and Programs** - The PATH variable 4m 13s - **7. Configuring Your Working Environment 41m 28s** - Profile, login, and resource files 9m 11s - Setting command aliases 6m 59s - saving aliases in your `.bashrc` file - Setting and exporting environment variables 4m 54s - Setting the PATH variable 6m 10s - using double quotes instead of single quotes Assignments this week --------------------- Check the due date for each assignment and put a reminder in your agenda, calendar, and digital assistant. Just like in the Real World, not all due dates are on the same days or at the same times. - Review last week. Did you do everything assigned last week? - Read All The Words, Do, and then Submit via Blackboard: - [Assignment #03 HTML] -- working with files and pathnames: `PS1 find mkdir mv cp rm touch file` - OPTIONAL: Read [The VI (VIM) Text Editor] and optionally do the bonus [Assignment #04 HTML]. - [Assignment #05 HTML] will be out soon -- using GLOB wildcard patterns, selecting files using GLOB, redirection and pipes - Really do **Read All The Words**. You don't get a second chance to get it right. Lab work this week ------------------ - Study the first [Practice Tests and Answers] for your upcoming Midterm Test #1 on Thursday or Friday. - Complete and save the ODT or PDF versions of [Worksheet #04 HTML] and [Worksheet #05 HTML] in preparation for Midterm Test #1 and for doing [Assignment #05 HTML] - These worksheets require you to understand [Finding Files], [Shell GLOB patterns] (wildcard pathname matching) and [Shell I/O Redirection] (including Pipes). - Get ready for upcoming [Assignment #05 HTML] -- GLOB wildcard patterns, selecting files using GLOB, redirection and pipes - This assignment requires you to understand the material in the web pages and Worksheets 4 and 5. - OPTIONAL: Read [The VI (VIM) Text Editor] and optionally do the bonus [Assignment #04 HTML]. ### Worksheets Worksheets are preparation for your assignments. You can't do the assignments without having done the worksheets first, and you can't do the worksheets without having first read the Course Notes: 1. Read. 2. Worksheets. 3. Assignment. Make notes from the worksheets on how each command works. What do the options used in the worksheets mean, for each command? (See the weekly [Command List][List of Commands You Should Know].) Form a small study group to do the worksheets. Each person tries the example given, and you make sure you all get the same answers. Worksheets are not for hand-in; they are not worth marks; the assignments test your knowledge of the lectures and worksheets. > The worksheets are available in four formats: Open Office (ODT), PDF, HTML, > and Text. Only the Open Office format allows you "fill in the blanks" in > the worksheet. The PDF format looks good but doesn't allow you to type into > the blanks in the worksheet. The HTML format is crude but useful for quick > for viewing online. Do **NOT** open the Worksheet ODT files using any Microsoft products; they will mangle the format and mis-number the questions. Use the free Libre Office or Open Office programs to open these ODT documents. On campus, you can [download Libre Office here]. These first two worksheets require you to have read [File System and Pathnames]: - [Worksheet #02 ODT] -- Using standard Linux commands I - [Worksheet #02 PDF] - [Worksheet #02 HTML] - `PS1, cd, find, less, ls, man, mkdir, passwd, pwd, rmdir` - [Worksheet #03 ODT] -- Using standard Linux commands II - [Worksheet #03 PDF] - [Worksheet #03 HTML] - `cat, clear, cp, find, fgrep, history, less, man, mv, rm, sleep, touch` - [Worksheet #04 ODT] -- shell GLOB patterns and Aliases - [Worksheet #04 PDF] - [Worksheet #04 HTML] - shell GLOB patterns (wildcards), Aliases: `alias, sum, unalias` - [Worksheet #05 ODT] -- shell I/O redirection (including pipes) - [Worksheet #05 PDF] - [Worksheet #05 HTML] - shell I/O redirection, `date, head, nl, tail, tr, wc` - [Worksheet #06 HTML] -- *Optional* Bonus VIM Text Editor Practice - This is an *optional* worksheet for a BONUS assignment using `vim` - Optional command-line VIM tutorial: the `vimtutor` program on the CLS. - Bonus (optional) [Assignment #04 HTML] -- the VIM text editor - Read [The VI (VIM) Text Editor] Worksheets prepare you for the upcoming assignments. Upcoming tests and exams ------------------------ This course has two midterm tests (10%, 15%) and one final exam (40%). ![Bring your favourite marker to your test] > Note that your upcoming Midterm Test #1 is worth more than three > assignments put together. Don't spend all your time doing an assignment at > the expense of studying the [Practice Tests and Answers] for the Midterm > Test. - Put these dates below into your phone! - Read the [Test Instructions] (all the words) before your tests and exam. - Use the [Algonquin Registered Name Game] link in the [Test Instructions] to test the spelling of your name before the test. - I don't answer questions about the instructions during the test. Ask me in a lab period before the test. - You must write the test in the lecture class in which you are registered. 1. **First Midterm test:** 45 minutes; in your one-hour lecture class on Thursday or Friday in Week 5 (October 5 or 6). 2. **Second Midterm test:** 45 minutes; in your one-hour lecture class on Thursday or Friday in Week 9 (November 2 or 3). Tests take place in your 50-minute lecture class, not in your lab period. Each midterm test is 45 minutes long and contains approximately 45 multiple-choice questions similar to those found in [Practice Tests and Answers]. You must write the test in the lecture class in which you are registered. Your first midterm test takes place this week. **See the [Week 04 Notes HTML] for all the test and quiz information.** Notes from the Classroom ======================== - **Take notes in class!** Keep a pad open on your desk. - This week, we learn more about the shell, including how to save settings so that they are set when you log in. - You will need to learn a text editor. I recommend VIM. - Note that your upcoming Midterm Test #1 is worth more than three assignments put together. Don't spend all your time doing an assignment at the expense of studying the [Practice Tests and Answers] for the Midterm Test. Draw file system diagrams ------------------------- See last week's note about drawing file system diagrams: [Week 03 Notes HTML] Commands Used ------------- > Keep a notebook with a [List of Commands][List of Commands You Should Know] > in it. > > - You need to write down yourself what each command *does*. > - Check the updated list of commands each week. > - I will check for this list in your lab periods. > - Bring your notes to class! Stop wasting time looking up commands. - Are you making notes from the worksheets on how each command works? - What do the options used in the worksheets mean, for each command? - Don't copy and use options that you don't understand! - Using GLOB patterns as `-name` arguments to `find` - Remember to quote the `find` GLOB patterns to hide them from the shell! - GLOB patterns are needed in Assignment 5. Tutoring ======== Many students find that hiring a personal tutor helps them get through the first term. Financial assistance is available. See the **Tutoring** heading in the [Course Introduction]. SSH Attacks on the CLS ====================== You are not paranoid if they really are out to get you! See the [Week 03 Notes HTML] for command pipelines to count the attacks on the CLS. HTTP attacks on Course Linux Server ----------------------------------- In addition to the SSH attacks, machines on the Internet receive attacks using bogus HTTP requests. These unsuccessful attacks below attempt to use a chain of parent directories to fetch a file out of the file system: [Dec 13 13:50:57 2015] [client 54.201.251.33] Invalid URI in request GET /../../ratings.html HTTP/1.1 [Dec 13 13:51:01 2015] [client 54.201.251.33] Invalid URI in request GET /../../index.html HTTP/1.1 [Apr 26 17:09:59 2016] [client 67.79.37.250] Invalid URI in request GET /../../../../../../../mnt/mtd/yqcP HTTP/1.1 [May 02 12:21:45 2016] [client 222.235.67.147] Invalid URI in request GET /../../../../../../../mnt/mtd/IJSj HTTP/1.1 [Jun 27 17:54:40 2016] [client 222.198.128.207] Invalid URI in request GET /../../../../../../../mnt/mtd/3qS9 HTTP/1.1 [Oct 07 09:34:11 2016] [client 64.137.234.42] Invalid URI in request GET /../../../../../../../mnt/mtd/ahMk HTTP/1.1 [Oct 15 06:49:57 2016] [client 177.142.99.103] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [Oct 19 20:49:33 2016] [client 189.100.188.31] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [Oct 23 00:43:39 2016] [client 191.189.55.9] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [Nov 02 21:35:09 2016] [client 201.37.223.72] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [Jan 07 20:51:33 2017] [client 201.17.154.181] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [May 10 21:22:29 2017] [client 187.67.167.31] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 The Apache server blocked all the above attacks. ![Take Notes in Class] -- | Ian! D. Allen, BA, MMath - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Course Home Page]: .. [Course Outline]: course_outline.pdf [All Weeks]: indexcgi.cgi [Plain Text]: week05notes.txt [Week 05 Notes HTML]: week05notes.html [Selecting Fields with `awk`]: 187_selecting_fields_awk.html [Shell Variables]: 320_shell_variables.html [Start-Up Files]: 350_startup_files.html [Practice Tests and Answers]: PRACTICE_TEST_README.html [List of Commands You Should Know]: 900_unix_command_list.html [Video Tutorials on Lynda.com]: 910_lynda_index.html [Linux and Sysadmin News in the World]: 950_linux_world.html [lynda.com]: http://algonquincollege.com/onlineresources/mobileStudent/lynda.htm [Unix for Mac OS X Users]: http://www.lynda.com/Mac-OS-X-10-6-tutorials/Unix-for-Mac-OS-X-Users/78546-2.html [Assignment #03 HTML]: assignment03.html [The VI (VIM) Text Editor]: 300_vi_text_editor.html [Assignment #04 HTML]: assignment04.html [Assignment #05 HTML]: assignment05.html [Worksheet #04 HTML]: worksheet04.html [Worksheet #05 HTML]: worksheet05.html [Finding Files]: 180_finding_files.html [Shell GLOB patterns]: 190_glob_patterns.html [Shell I/O Redirection]: 200_redirection.html [download Libre Office here]: 050_course_introduction.html#install-libreoffice-or-openoffice-into-windows [File System and Pathnames]: 160_pathnames.html [Worksheet #02 ODT]: worksheet02.odt [Worksheet #02 PDF]: worksheet02.pdf [Worksheet #02 HTML]: worksheet02.html [Worksheet #03 ODT]: worksheet03.odt [Worksheet #03 PDF]: worksheet03.pdf [Worksheet #03 HTML]: worksheet03.html [Worksheet #04 ODT]: worksheet04.odt [Worksheet #04 PDF]: worksheet04.pdf [Worksheet #05 ODT]: worksheet05.odt [Worksheet #05 PDF]: worksheet05.pdf [Worksheet #06 HTML]: worksheet06.html [Bring your favourite marker to your test]: data/sniff_markers.jpg "Bring your favourite marker to your test" [Test Instructions]: 000_test_instructions.html [Algonquin Registered Name Game]: http://cst8207.idallen.ca/~idallen/namegame.cgi [Week 04 Notes HTML]: week04notes.html [Week 03 Notes HTML]: week03notes.html [Course Introduction]: 050_course_introduction.html [Take Notes in Class]: data/remember.jpg "Take Notes in Class" [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/