% CST8207 Week 04 Notes – redirection, text editors, nano, vim % Ian! D. Allen - - [www.idallen.com] % Winter 2014 - January to April 2014 - Updated Tue Apr 22 12:47:03 EDT 2014 Readings, Assignments, Labs, and ToDo ===================================== - Read (at least) these things (All The Words): - [Week 04 Notes HTML] - this file - **Read All The Words** - [Unix Shell I/O Redirection (including Pipes)] - [List of Commands You Should Know] - [Video Tutorials on Lynda.com] - Using your [lynda.com] account, watch [Unix for Mac OS X Users] - **3. Working with Files and Directories** - Unix text editors 6m 39s - `vi/vim, nano` - `head, tail` - **6. Directing Input and Output 20m 39s** - Standard input and standard output 1m 24s - Directing output to a file 4m 13s - Appending to a file 2m 44s - Directing input from a file 5m 28s - Piping output to input 4m 40s - Suppressing output 2m 10s - **7. Configuring Your Working Environment 41m 28s** - Setting command aliases 6m 59s - saving aliases in your `.bashrc` file Midterm Test #1 ---------------- - Midterm #1 takes place Wednesday, February 12 in your lecture. - You must read the [Test Instructions] first. - [224 practice Midterm Test #1 questions] are available along with an answer key in the [Class Notes]. - A Quiz on these 224 questions is available on Blackboard. See below: Quizzes: Midterm #1 Quiz ------------------------- This is one of several quizzes in this course. Each midterm and final exam will have an associated quiz. See the course outline for the mark weight of all course quizzes, midterm tests, and exams. The quizzes are open-book, but the midterm tests and final exam are closed-book. This quiz is based on the [224 practice Midterm Test #1 questions] that are posted in the [Class Notes]. Each quiz is 10 questions long and you see the answers right after you submit the quiz. You can take the quiz as many times as you like. Every time you take the quiz, you get a different random set of ten questions. This quiz closes just before the Final Exam in this course; after the Final Exam begins you will not be able to submit any more quiz attempts. Your mark for this quiz is the average of your five best quiz scores. Examples: - Your best scores: 10 10 10 10 10 10 10 9 8 7 7 4 - Your quiz mark: (10+10+10+10+10)/50 = 100% - Your best scores: 10 10 10 10 9 8 7 7 4 3 - Your quiz mark: (10+10+10+10+9)/50 = 98% - Your best scores: 10 10 10 9 8 7 7 4 3 2 - Your quiz mark: (10+10+10+9+8)/50 = 94% - Your best scores: 10 8 8 8 8 5 4 4 4 - Your quiz mark: (10+8+8+8+8)/50 = 84% - Your best scores: 10 10 8 (only three quizzes submitted) - Your quiz mark: (10+10+8+0+0)/50 = 56% Your score will be taken from the five best scores. The more times you do the quiz, the more likely you are to have a set of excellent best scores. You must have five perfect quiz scores to get a perfect averaged quiz mark. Missing quizzes (fewer than five) count as zeroes. Only the five best scores are averaged. This quiz usually closes just before the Final Exam in this course; after the Final Exam begins you will not be able to submit any more quiz attempts. (Extensions are sometimes granted; check Blackboard for the exact final due date.) See the “Quizzes” section in the Blackboard left side-bar for CST8207. Note: Blackboard averages the marks of *all* your quiz attempts. Your quiz mark is actually the average of your five *best* attempts, not all the attempts. Assignments this week --------------------- Check the due date for each assignment and put a reminder in your agenda, calendar, and digital assistant. - Read All The Words, Do, and then Submit via Blackboard: - [Assignment #02 HTML] – simple file system commands on the CLS - [Assignment #04 HTML] – simple commands, copy, GLOB and redirection - Really do **Read All The Words**. You don’t get a second chance to get it right. Lab work this week ------------------ The worksheets are available in four formats: Open Office, PDF, HTML, and Text. Only the Open Office format allows you “fill in the blanks” in the worksheet. The PDF format looks good but doesn’t allow you to type into the blanks in the worksheet. The HTML format is crude but useful for quick for viewing online. - [Worksheet #02 HTML] – Using standard Linux commands I - PS1, cd, find, less, ls, man, mkdir, passwd, pwd, rmdir - [Worksheet #03 HTML] – Using standard Linux commands II - cat, clear, cp, find, grep, history, less, man, mv, rm, sleep, touch - [Worksheet #04 HTML] – GLOB Patterns and Aliases - bash GLOB patterns, alias, sum - [Worksheet #05 HTML] – I/O Redirection and Pipes - I/O redirection and pipes, date, head, nl, tail, tr, wc ### Optional Bonus VIM Assignment – extra marks - [Assignment #03 HTML] – *Optional* VIM Text Editor Practice - this is an *optional* worksheet for a BONUS assignment using `vim` - Optional Reading: [The VI (VIM) Text Editor] - [Worksheet #06 HTML] – *Optional* VIM Text Editor Practice - this is an *optional* worksheet for a BONUS assignment using `vim` - Optional command-line VIM tutorial: the `vimtutor` program on the CLS. From the Class Notes link on the Course Home Page ================================================= - Review last week. Did you do everything assigned last week? From the Classroom Whiteboard/Chalkboard ======================================== - **Take notes in class!** Your in-class notes would go here. - Searching your shell history using `^R` - Quick review of commands used in worksheets: - copying a directory - review difference between `cp -r dir1 dir2` and `mkdir dir2 ; cp -r dir1 dir2` - Review absolute and relative paths as arguments to commands such as `cd` - I will do spot inspections of your [command lists][List of Commands You Should Know]. Show me that you have a list of each command name and what it does. - Using the `vim` tutorial and text editor - The CentOS Linux you install later this term uses only this editor. - Using the `nano` text editor - This editor is *not* installed on your CentOS Linux machine - Do you know your **Lab** section number? (Hint: not 400 or 410.) - Quick review of commands used in worksheets. - the `sort` command sorts one or more files to standard output - the `uniq` command removes or counts adjacent duplicate lines - the `hostname` command shows your computer’s local name - the `whoami` command shows your userid - the `wc` command has useful options to limit output - the `locate` command finds file names using an existing list - the `cut` and `awk` commands select fields in lines - [using GLOB patterns to match case-insensitive], e.g. `dog`, `doG`, `Dog`, `DOG`, etc. - `$ echo [dD][oO][gG]` - Using `-ls` instead of `-print` with `find`, e.g. `find . -ls` - don’t use alphabetic `[a-z]` GLOB ranges until you understand Internationalization - using numeric ranges is usually safe: `$ echo [0-9]*` - is this directory empty? `ls` vs. `ls -a` - what is the difference between `grep` and `fgrep` ? - Redirection and pipes this week. Midterm Test Dates ------------------ - First Midterm test date was chosen by you to be Wednesday February 12 (Week 6) in your regular class period. - See the survey results (60/97 [responses]): - Second Midterm test date was chosen by you to be Wednesday March 12 (Week 9) in your regular class period. - See the survey results (26/97 [responses][1]): - For full marks, you must read the [Test Instructions] before the test for important directions on how to enter your answers, your lab (not lecture) section number, and the test version number on the question sheet and the mark-sense forms. - There may be more questions on the test than you can answer in the time allowed; answer the ones you know, first. Real Sysadmin Work ================== Attacks on the Course Linux Server ---------------------------------- Here is a command pipeline that does real-time monitoring of who is trying to attack the [Course Linux Server] (may require privileged read permission on the log files): $ fgrep 'refused connect' /var/log/auth.log | awk '{print $NF}' \ | sort | uniq -c | sort -nr | head -n 5 686 (188.165.173.230) 65 (70.35.59.13) 53 (211.161.45.222) 31 (61.160.215.170) 31 (1.93.34.211) The top listed attacker is from France. They tried to log in as several different accounts and the `denyhosts` intrusion protection package quickly blocked that IP address: $ host 188.165.173.230 230.173.165.188.in-addr.arpa domain name pointer isis.cleonet.fr. $ whois 188.165.173.230 [...] address: 59100 Roubaix address: France $ fgrep '188.165.173.230' /var/log/auth.log # (output edited slightly) Jan 19 21:51:18 Invalid user aion from 188.165.173.230 Jan 19 21:51:18 Invalid user asterisk from 188.165.173.230 Jan 19 21:51:18 Invalid user bugzilla from 188.165.173.230 Jan 19 21:51:20 Failed password for invalid user aion from 188.165.173.230 port 49674 ssh2 Jan 19 21:51:20 Failed password for invalid user asterisk from 188.165.173.230 port 50007 ssh2 Jan 19 21:51:21 Failed password for invalid user bugzilla from 188.165.173.230 port 50347 ssh2 Jan 19 21:51:26 Failed password for invalid user bugzilla from 188.165.173.230 port 50683 ssh2 Jan 19 21:51:28 Failed password for invalid user bugzilla from 188.165.173.230 port 51020 ssh2 Jan 19 21:51:28 refused connect from isis.cleonet.fr (188.165.173.230) Jan 19 21:51:31 refused connect from isis.cleonet.fr (188.165.173.230) [...600 repeat lines deleted...] The next most persistent attacker is from California: $ host 70.35.59.13 13.59.35.70.in-addr.arpa domain name pointer 70-35-59-13.static.wiline.com. $ whois 70.35.59.13 [...] City: Foster City StateProv: CA Country: US $ fgrep '70.35.59.13' /var/log/auth.log # (output edited slightly) Jan 9 05:13:10 Failed password for invalid user admin from 70.35.59.13 port 47336 ssh2 Jan 9 05:13:12 Failed password for invalid user admin from 70.35.59.13 port 47336 ssh2 Jan 9 05:13:17 Failed password for invalid user admin from 70.35.59.13 port 47471 ssh2 Jan 9 05:13:19 Failed password for invalid user admin from 70.35.59.13 port 47471 ssh2 Jan 9 05:13:30 Failed password for invalid user admin from 70.35.59.13 port 47635 ssh2 Jan 9 05:13:33 Failed password for invalid user admin from 70.35.59.13 port 47635 ssh2 Jan 9 05:13:55 refused connect from 70-35-59-13.static.wiline.com (70.35.59.13) Jan 9 05:14:12 refused connect from 70-35-59-13.static.wiline.com (70.35.59.13) [...] The third most persistent attacker is from China: $ host 211.161.45.222 222.45.161.211.in-addr.arpa domain name pointer Dns-Slave.bjgwbn.net.cn. $ whois 211.161.45.222 descr: Beijing,China $ fgrep '211.161.45.222' /var/log/auth.log # (output edited slightly) Jan 23 10:03:31 Invalid user szabol from 211.161.45.222 Jan 23 10:03:36 Invalid user szabol from 211.161.45.222 Jan 23 10:03:44 Invalid user szabol from 211.161.45.222 Jan 23 10:03:56 refused connect from 211.161.45.222 (211.161.45.222) Jan 23 10:04:03 refused connect from 211.161.45.222 (211.161.45.222) [...] ![Take Notes in Class] -- | Ian! D. Allen - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Unix Shell I/O Redirection (including Pipes)]: 200_redirection.html [List of Commands You Should Know]: 900_unix_command_list.html [Video Tutorials on Lynda.com]: 910_lynda_index.html [lynda.com]: https://lyceum.algonquincollege.com/Lynda [Unix for Mac OS X Users]: http://wwwlyndacom.rap.ocls.ca/Mac-OS-X-10-6-tutorials/Unix-for-Mac-OS-X-Users/78546-2.html [Test Instructions]: 000_test_instructions.html [224 practice Midterm Test #1 questions]: practicetest1.pdf [The VI (VIM) Text Editor]: 300_vi_text_editor.html [using GLOB patterns to match case-insensitive]: 190_glob_patterns.html#using-to-match-case-insensitive [responses]: data/first_midterm_responses1.png [1]: data/second_midterm_responses2.png [Course Linux Server]: 070_course_linux_server.html [Take Notes in Class]: data/remember.jpg "Take Notes in Class" [Plain Text]: week04notes.txt [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/