------------------------ Week 3 Notes for NET2003 ------------------------ -Ian! D. Allen - idallen@idallen.ca Keep up on your readings (Course Outline: average 5 hours/week homework) Remember - knowing how to find out an answer is more important than memorizing the answer. Learn to fish! RTFM! (Read The Fine Manual) 1. Read the Course Announcements (via the Course Home page) 2. Reminder: Midterm Test #1 ends Week #4 (see the Course Home Page). See Notes file termtest1directions.txt for the test format. 3. Why learn scripting and command line? - show /etc/init.d/ntpd /etc/init.d/httpd - show list of processes - grep for ntp - the http server dies every hour during critical period - stay up all night, or - write a script to restart it if it is missing while : ; do if ! pgrep httpd ; then /etc/init.d/httpd restart fi sleep 10 done - tell the machine to do the work! Unix is an O/S designed by programmers for programmers - command-line driven (programmers didn't use the GUI) - things work silently (no confirmation) - messages appear only when things fail - most command names are cryptic abbreviations! - hard to learn but easy to use 4. Review Entering commands - the terminal driver - your terminal is two devices, loosely coupled - control chars (unprintables) syntax: ^H ^? - backspace to erase one char - stty erase - line edit: ^H ^? ^W ^U ^R ^L - ^U - erase entire line - ^W - erase word - ^R - redraw line (in case overwritten) - ^C - interrupt - ^D - EOF end of input - ^L - clear screen (in bash shell, less, more, and vim) See Notes file miscellaneous.txt - learn various ways of getting out of programs - EOF and Interrupting Processes $ sort - note difference between ^C and ^D $ wc - note difference between ^C and ^D $ cat - less difference - Unix line ends are \n characters (not \r or \r\n) $ echo hi | wc -c # prints 3 not 2 because of \n at end Pagination commands - one set of commands does all the pagination - pagination is not built in to each command, as in DOS - searching can be done using / in MORE and LESS (and VIM) Man pages - using the man command, man -k, and apropos (man_page_RTFM.txt) Commands: see Notes file unix_command_list.txt - use "rm -r dir" to remove an entire directory tree and all files - keep a list of commands and their common uses - know how to read man pages (man page syntax meaning) - don't send binary (executable machine code) files to your screen - check the contents with the "file" command first - grep pattern file - crude Data Mining: selecting content by looking for lines - using grep on course notes: grep ssh ~idallen/public_html/teaching/net2003/05w/notes/*.txt - grep -v finds lines that do *not* contain the pattern Notes: file_transfer.txt File transfer between machines home_and_HOME.txt Directories: current, HOME, and /home miscellaneous.txt Miscellaneous Unix Facts - terminal characters, ^C ^D netsubmit.txt Using the netsubmit command shell_prompt.txt Setting the BASH shell prompt startup_files.txt Setting up Startup Files: .profile and .bashrc * shell_basics.txt The Unix Shell * arguments_and_options.txt Options and Arguments on Unix Command Lines The Unix Shell (e.g. "bash" - Bourne Again SHell) - see Notes files: shell_basics.txt, arguments_and_options.txt - shell splits line into separate commands, perhaps joined by pipes - pipes separate commands but join output: grep foo /etc/passwd | wc - you can also use ';' to fully separate commands: sleep 4 ; echo BOO - next, shell splits each separate command into options and arguments - purpose: to find and run commands - with some support for programming (scripts) - Notes: shell_basics.txt - The Unix/Linux Shell - shell is an unprivileged program - standard Linux shell is named "bash"; you can start up multiple copies of bash by typing its name as a command - shell prompts when it reads from your keyboard - shell does not prompt if reading input from a file! - most other Unix programs do *not* prompt when they read your keyboard - Notes: shell_prompt.txt - Setting the BASH shell prompt - shell splits your command line on blanks (white space) - first non-redirection word is the command name - the rest of the arguments are passed to the command - Notes: arguments_and_options.txt - Options and Arguments on Command Lines - shells usually operate on pathnames (files, directories, etc.) - many helpful things to make using pathnames easier - Notes: pathnames.txt - Unix/Linux Pathnames - shell can expand pathname GLOB patterns (similar to DOS wildcard) - shell helps with wildcards (GLOB) - * matches zero or more characters in a pathname component - ? matches a single character (any character, even a space) - [abc] matches a single character from the list abc - GLOB patterns do not match names that start with a period (hidden pathnames), unless you explicitly put the period into the file name - like ls, GLOB patterns do not match hidden names - echo * - does not match hidden files - Notes: glob_patterns.txt - GLOB patterns (wildcard pathname matching) - shell can complete pathnames when you use the key - Notes: shell_basics.txt - The Unix/Linux Shell - shell can expand variables, both local and environment - e.g. $variable as in echo "$SHELL" - as with GLOB, the shell does the variable expansion, not the command! - you can use ; to separate multiple commands: sleep 4 ; echo BOO Important: Most Unix commands that take file names as arguments will read standard input (usually your keyboard) if no file names are given. Unlike the shell, the commands will *not* prompt when reading your keyboard. The shell reads stdin, just like any other program: echo date | bash echo 'echo hi ; date' | bash * pathnames.txt Unix/Linux Pathnames More on Pathnames: see Notes file: pathnames.txt - New: hidden files, ls -a to see them - empty dir ; ls and hidden pathnames (define what is a hidden pathname) - making and removing directories (chapter 4) - rm -r to remove a whole directory (not rmdir!) - show simple directory traversal - absolute path ; relative path (pathnames.txt) - mkdir foo ; cp /etc/passwd foo/bar ; ls mkdir bar ; cp /etc/passwd . ; mv passwd bar ; ls mkdir xxx ; cp /etc/passwd . ; mv passwd xxx/../yyy ; ls Review: cp pwd rm mv Review: dot and dot dot New: cd, mkdir, rmdir, rm -r (*no confirmation!*) 5. New commands and/or command options: grep -v -w sort -n -u -r uniq -c diff -u which whereis slocate w New Class Notes presented: finding_files.txt Searching for and finding files by name glob_patterns.txt GLOB patterns (wildcard pathname matching) Finding Files - see Notes file finding_files.txt The shell can expand file GLOB patterns - see Notes file glob_patterns.txt - similar to DOS wildcard; but, the shell does it, not the command - shell helps with wildcards (GLOB) (see shell_basics.txt) - See Notes: glob_patterns.txt - GLOB patterns (wildcard pathname matching) - Hidden files as (not) seen by ls and glob wildcard * - GLOB patterns do not match hidden files unless you explicitly include the leading period: echo .* - * matches zero or more characters - ? matches a single character (any character, even a space) - [abc] matches a single character from the list abc - GLOB patterns do not match names that start with a period (hidden pathnames), unless you explicitly put the period at the start of the pathname component $ echo * a b c $ echo .* . .. .bashrc $ echo /var/www/html/.* /var/www/html/. /var/www/html/.. /var/www/html/.htaccess $ echo * a* *b *b* a*a ? ?? ??? a? ???* - like ls, GLOB patterns do not match hidden pathnames - you can see hidden names using: ls -a - echo * - does not match hidden pathnames - creating command line arguments using quotes and backslashes - wildcards (GLOB) - mention that, like ls, GLOB patterns do not match hidden names Quoting the shell metacharacters, to stop the shell from helping you - hiding GLOB characters from the shell: - use single or double quotes to hide GLOB characters: $ echo "*** warning ***" - quotes and backslashes hide things from the shell - quotes delimit each argument; they tell the shell where it starts/ends - quotes are not part of the argument - two different strengths of quotes - single quotes hide all characters - double quotes hide everything *except* \" and $variable expansions - only single quotes stop backslash processing and variable expansion - both kinds of quotes stop GLOB expansion and blank splitting - you can also use backslash to hide metacharacters from the shell Reminder: Midterm Test #1 ends Week #4 (see the Course Home Page). - See Notes file termtest1directions.txt for the test format. - Look for a practice test in the course notes.