% CST8207 Week 04 Notes – redirection, text editors, nano, vim % Ian! D. Allen – – [www.idallen.com] % Fall 2014 - September to December 2014 - Updated Mon Sep 29 09:02:20 EDT 2014 Readings, Assignments, Labs, and ToDo ===================================== - Read (at least) these things (All The Words): - [Week 04 Notes HTML] – this file – **Read All The Words** - [Shell GLOB patterns (wildcard pathname matching)] - [The VI (VIM) Text Editor] – know the basics; the rest is optional - [Unix Shell I/O Redirection (including Pipes)] - [List of Commands You Should Know] - [Video Tutorials on Lynda.com] - Using your [lynda.com] account, watch [Unix for Mac OS X Users] - **3. Working with Files and Directories** - Unix text editors 6m 39s - `vi/vim, nano` - `head, tail` - **6. Directing Input and Output 20m 39s** - Standard input and standard output 1m 24s - Directing output to a file 4m 13s - Appending to a file 2m 44s - Directing input from a file 5m 28s - Piping output to input 4m 40s - Suppressing output 2m 10s - **7. Configuring Your Working Environment 41m 28s** - Setting command aliases 6m 59s - saving aliases in your `.bashrc` file Midterm Test #1 ---------------- - Midterm #1 takes place Friday, October 3 (end of Week 5) in your scheduled lecture hour (not in your lab period). - For full marks, you must read the [Test Instructions] before the test for important directions on how to enter your answers, your lab (not lecture) section number, and the test version number on the question sheet and the mark-sense forms. - There may be more questions on the test than you can answer in the time allowed; answer the ones you know, first. - Here is a complete set of practice questions for the midterm test (PDF): [Midterm #1 Practice Test] The answer key is posted in the [Class Notes]. - Blackboard has some quizzes taken randomly from the practice test. See below. Quizzes: Midterm #1 Quiz ------------------------- This quiz is one of several quizzes in this course. Each midterm and final exam will have an associated quiz. See the course outline for the mark weight of all course quizzes, midterm tests, and exams. The quizzes are open-book, but the midterm tests and final exam are closed-book. This quiz is based on the [Midterm #1 Practice Test] questions that are posted in the [Class Notes]. The quiz is 10 questions long and you see the answers right after you submit the quiz. You can take the quiz as many times as you like. Every time you take the quiz, you get a random set of ten questions from the practice test. You will not see all the practice questions by doing quizzes; to see all the practice questions, you must do all the questions in the actual practice test posted in the Course Notes. This quiz closes just before the Final Exam in this course; quizzes submitted after the Final Exam begins may not count toward your best score. Your mark for this quiz is the average of your five best quiz scores. Examples: - Your best scores: 10 10 10 10 10 10 10 9 8 7 7 4 - Your quiz mark: (10+10+10+10+10)/50 = 100% - Your best scores: 10 10 10 10 9 8 7 7 4 3 - Your quiz mark: (10+10+10+10+9)/50 = 98% - Your best scores: 10 10 10 9 8 7 7 4 3 2 - Your quiz mark: (10+10+10+9+8)/50 = 94% - Your best scores: 10 8 8 8 8 5 4 4 4 - Your quiz mark: (10+8+8+8+8)/50 = 84% - Your best scores: 10 10 8 (only three quizzes submitted) - Your quiz mark: (10+10+8+0+0)/50 = 56% Your score will be taken from the five best scores. The more times you do the quiz, the more likely you are to have a set of excellent best scores. You must have five perfect quiz scores to get a perfect averaged quiz mark. Missing quizzes (fewer than five) count as zeroes. Only the five best scores are averaged. (Blackboard averages all the scores and thus displays your score incorrectly.) This quiz closes just before the Final Exam in this course; quizzes submitted after the Final Exam begins may not count toward your best score. See the “Quizzes” section in the Blackboard left side-bar for your course. Note: Blackboard displays the quiz mark incorrectly, since it cannot calculate “best 5” and instead averages the marks of *all* your quiz attempts. Your quiz mark is actually the average of your five *best* attempts, not all the attempts as shown by Blackboard. Assignments and Lab work this week ---------------------------------- Check the due date for each assignment and put a reminder in your agenda, calendar, and digital assistant. - Read All The Words, Do, and then Submit via Blackboard: - [Assignment #02 HTML] – simple file system commands on the CLS - Coming soon: Assignment #04 HTML – GLOB and redirection - Really do **Read All The Words**. You don’t get a second chance to get it right. ### Worksheets The worksheets are available in four formats: Open Office (ODT), PDF, HTML, and Text. Only the Open Office format allows you “fill in the blanks” in the worksheet. The PDF format looks good but doesn’t allow you to type into the blanks in the worksheet. The HTML format is crude but useful for quick for viewing online. Do **NOT** open the ODT files using any Microsoft products; they will mangle the format and mis-number the questions. Use the free Libre Office or Open Office programs to open these ODT documents. On campus, you can [download Libre Office here]. - [Worksheet #02 HTML] – Using standard Linux commands I - `PS1, cd, find, less, ls, man, mkdir, passwd, pwd, rmdir` - [Worksheet #03 HTML] – Using standard Linux commands II - `cat, clear, cp, find, grep, history, less, man, mv, rm, sleep, touch` - [Worksheet #04 HTML] – GLOB Patterns and Aliases - bash GLOB patterns (wildcards), `alias, sum` - [Worksheet #05 HTML] – I/O Redirection and Pipes - bash I/O redirection (including pipes), `date, head, nl, tail, tr, wc` ### Optional Bonus VIM Assignment – extra marks - [Assignment #03 HTML] – *Optional* VIM Text Editor Practice - this is an *optional* worksheet for a BONUS assignment using `vim` - Optional Reading: [The VI (VIM) Text Editor] - [Worksheet #06 HTML] – *Optional* VIM Text Editor Practice - this is an *optional* worksheet for a BONUS assignment using `vim` - Optional command-line VIM tutorial: the `vimtutor` program on the CLS. From the Class Notes link on the Course Home Page ================================================= - Review last week. Did you do everything assigned last week? From the Classroom Whiteboard/Chalkboard ======================================== - **Take notes in class!** Your in-class notes would go here. - This week: Finish GLOB patterns, start redirection and pipes. - [using GLOB patterns to match case-insensitive], e.g. `dog`, `doG`, `Dog`, `DOG`, etc. - Use `echo` to see what the shell does with a GLOB pattern before you use the GLOB pattern in a shell command line: - `$ echo [dD][oO][gG]` - `$ touch [dD][oO][gG]` - Don’t use alphabetic `[a-z]` GLOB ranges until you understand Internationalization and collating order - Using numeric ranges is usually safe: `$ echo [0-9]*` - I will do spot inspections of your own personal [command lists][List of Commands You Should Know]: - Show me that you have a list of each command name and what it does. - Using the `vim` tutorial and text editor - The CentOS Linux virtual machine you install later this term uses only this editor. You need to know the basics. - Using the `nano` text editor - This editor is *not* installed on your CentOS Linux virtual machine. - Do you know your **Lab** section number? (Hint: not 010 or 020.) - Quick review of commands used in worksheets. - the `sort` command sorts one or more files to standard output - the `uniq` command removes or counts adjacent duplicate lines - the `hostname` command shows your computer’s local name - the `whoami` command shows your userid - the `wc` command has useful options to limit output - the `locate` command finds file names using an existing list - the `cut` and `awk` commands select fields in lines - Using `-ls` instead of `-print` with `find`, e.g. `find . -ls` - is this directory empty? `ls` vs. `ls -a` - what is the difference between `grep` and `fgrep` ? Real Sysadmin Work ================== Attacks on the Course Linux Server ---------------------------------- Here is a command pipeline that does real-time monitoring of who is trying to attack the [Course Linux Server] (may require privileged read permission on the log files). I ran this command last term: $ fgrep 'refused connect' /var/log/auth.log | awk '{print $NF}' \ | sort | uniq -c | sort -nr | head -n 5 686 (188.165.173.230) 65 (70.35.59.13) 53 (211.161.45.222) 31 (61.160.215.170) 31 (1.93.34.211) The top listed attacker is from France. They tried to log in as several different accounts and the `denyhosts` intrusion protection package quickly blocked that IP address: $ host 188.165.173.230 230.173.165.188.in-addr.arpa domain name pointer isis.cleonet.fr. $ whois 188.165.173.230 [...] address: 59100 Roubaix address: France $ fgrep '188.165.173.230' /var/log/auth.log # (output edited slightly) Jan 19 21:51:18 Invalid user aion from 188.165.173.230 Jan 19 21:51:18 Invalid user asterisk from 188.165.173.230 Jan 19 21:51:18 Invalid user bugzilla from 188.165.173.230 Jan 19 21:51:20 Failed password for invalid user aion from 188.165.173.230 port 49674 ssh2 Jan 19 21:51:20 Failed password for invalid user asterisk from 188.165.173.230 port 50007 ssh2 Jan 19 21:51:21 Failed password for invalid user bugzilla from 188.165.173.230 port 50347 ssh2 Jan 19 21:51:26 Failed password for invalid user bugzilla from 188.165.173.230 port 50683 ssh2 Jan 19 21:51:28 Failed password for invalid user bugzilla from 188.165.173.230 port 51020 ssh2 Jan 19 21:51:28 refused connect from isis.cleonet.fr (188.165.173.230) Jan 19 21:51:31 refused connect from isis.cleonet.fr (188.165.173.230) [...600 repeat lines deleted...] The next most persistent attacker is from California: $ host 70.35.59.13 13.59.35.70.in-addr.arpa domain name pointer 70-35-59-13.static.wiline.com. $ whois 70.35.59.13 [...] City: Foster City StateProv: CA Country: US $ fgrep '70.35.59.13' /var/log/auth.log # (output edited slightly) Jan 9 05:13:10 Failed password for invalid user admin from 70.35.59.13 port 47336 ssh2 Jan 9 05:13:12 Failed password for invalid user admin from 70.35.59.13 port 47336 ssh2 Jan 9 05:13:17 Failed password for invalid user admin from 70.35.59.13 port 47471 ssh2 Jan 9 05:13:19 Failed password for invalid user admin from 70.35.59.13 port 47471 ssh2 Jan 9 05:13:30 Failed password for invalid user admin from 70.35.59.13 port 47635 ssh2 Jan 9 05:13:33 Failed password for invalid user admin from 70.35.59.13 port 47635 ssh2 Jan 9 05:13:55 refused connect from 70-35-59-13.static.wiline.com (70.35.59.13) Jan 9 05:14:12 refused connect from 70-35-59-13.static.wiline.com (70.35.59.13) [...] The third most persistent attacker is from China: $ host 211.161.45.222 222.45.161.211.in-addr.arpa domain name pointer Dns-Slave.bjgwbn.net.cn. $ whois 211.161.45.222 descr: Beijing,China $ fgrep '211.161.45.222' /var/log/auth.log # (output edited slightly) Jan 23 10:03:31 Invalid user szabol from 211.161.45.222 Jan 23 10:03:36 Invalid user szabol from 211.161.45.222 Jan 23 10:03:44 Invalid user szabol from 211.161.45.222 Jan 23 10:03:56 refused connect from 211.161.45.222 (211.161.45.222) Jan 23 10:04:03 refused connect from 211.161.45.222 (211.161.45.222) [...] ![Take Notes in Class] - [Remember this for later] -- | Ian! D. Allen - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Shell GLOB patterns (wildcard pathname matching)]: 190_glob_patterns.html [The VI (VIM) Text Editor]: 300_vi_text_editor.html [Unix Shell I/O Redirection (including Pipes)]: 200_redirection.html [List of Commands You Should Know]: 900_unix_command_list.html [Video Tutorials on Lynda.com]: 910_lynda_index.html [lynda.com]: http://algonquincollege.com/onlineresources/mobileStudent/lynda.htm [Unix for Mac OS X Users]: http://www.lynda.com/Mac-OS-X-10-6-tutorials/Unix-for-Mac-OS-X-Users/78546-2.html [Test Instructions]: 000_test_instructions.html [Midterm #1 Practice Test]: practicetest1.pdf [download Libre Office here]: 050_course_introduction.html#install-libreoffice-or-openoffice-into-windows [using GLOB patterns to match case-insensitive]: 190_glob_patterns.html#using-to-match-case-insensitive [Course Linux Server]: 070_course_linux_server.html [Take Notes in Class]: data/remember.jpg "Take Notes in Class" [Remember this for later]: http://somethingofthatilk.com/index.php?id=293 [Plain Text]: week04notes.txt [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/