------------------------- Week 07 Notes for NET2003 ------------------------- -Ian! D. Allen - idallen@idallen.ca Remember - knowing how to find out an answer is more important than memorizing the answer. Learn to fish! RTFM! (Read The Fine Manual) Abbreviation "ALN" = your "Advanced Linux Networking" text by R.W.Smith Subscribe to a network security mailing list, e.g. BUGTRAQ and/or SANS Process listing and control --------------------------- The "ps" command on Linux is a mix of the BSD "ps" command and the incompatible SystemV UNIX "ps" command. From "man ps": This version of ps accepts several kinds of options: 1 UNIX options, which may be grouped and must be preceded by a dash. 2 BSD options, which may be grouped and must not be used with a dash. 3 GNU long options, which are preceded by two dashes. - ps # BSD: some of your processes - ps x # BSD: all of your processes - ps xl # BSD: all your processes, long format - ps xlww # BSD: all your processes, long format, full wide listing - ps ax # BSD: all processes - ps -e # UNIX: all processes - ps -elww # UNIX: all processes, long format, full wide - ps f # BSD: ascii art hierarchical display (forest) - pstree - kill - killall - jobs (interactive shells only) Q: how do you see all your processes? Q: how do you see all processes for all users? Symbolic Links -------------- Symbolic Links - similar to Windows/Mac aliases, but they work even for directories - can symlink to directories (cannot hard link to directories) - can point to nonexistent files or directories - a "dangling symlink" A leading "l" in ls output: $ ls -l /bin/*sh -rwxr-xr-x 1 root root 664084 Apr 21 2006 /bin/bash lrwxrwxrwx 1 root root 21 Aug 22 22:30 /bin/csh -> /etc/alternatives/csh lrwxrwxrwx 1 root root 4 Aug 22 18:42 /bin/rbash -> bash lrwxrwxrwx 1 root root 4 Aug 22 18:42 /bin/sh -> bash -rwxr-xr-x 1 root root 303344 Dec 1 2005 /bin/tcsh $ ln -s nosuchfile bar $ ls -l bar lrwxrwxrwx 1 idallen idallen 10 Feb 9 10:42 bar -> nosuchfile $ cat bar cat: bar: No such file or directory The system start-up script directories /etc/rc?.d/ contain symlinks. Q: How do you create a symlink in /tmp/foo that contains the path "/bin"? Shell Control Structures ------------------------ see Notes: exit_status.txt - Return Code, Exit Status, test, if, and while Shells find and run commands; their control structures do the same! - if statement sometimes mistakenly called "if loop" - while command ; do ... ; done - while test 2 -eq 2 ; do echo same ; done - use "let" to do arithmetic (no $ needed in "let") e.g. let x=x+1 let y="x*(3+y)" # protect GLOB chars and () - loop to count up to 10 x=0 while [ $x -lt 10 ] ; do let x=x+1 ; echo "x = $x" ; done Testing file attributes: - test -e pathname # see if pathname exists - test -s pathname # see if pathname exists and is non-empty - test -r pathname # see if pathname exists and is readable all permission tests: -r -w -x - test -d pathname # see if pathname is a directory other tests: -d -f -L (-h) Testing strings - test "$foo" = "bar" - test -z "$foo" # test to see if $foo is an empty string - test -n "$foo" # test to see if $foo is NOT an empty string Don't confuse test -n "$foo" with test -s "$foo" - one tests the string $foo the other tests the size of the file named by $foo. Q: How do you test to see if a string is empty/not-empty? Q: How do you test to see if a pathname is readable/writable/executable? Q: How do you test to see if a file has a size larger than zero? Q: How do you test to see if a pathname is a symlink? Scripts that violate Structured Programming (one exit per program). - see Notes file deep_nesting.txt - see Notes file exit_status.txt - Return, Exit Status, test, if, and while Software distribution - The tar archive: ---------------------------------------- The Unix version of a "zip" file. Software is often distributed in tar archives. A "tar" archive (sometimes called a "tarball") contains multiple uncompressed files and directories. Tarballs may be compressed *as a whole* using gzip or options to the "tar" command: $ tar cvf tarball.tar *.c # create tarball.tar verbosely $ gzip tarball.tar # compress into tarball.tar.gz $ tar cvzf tarball.tar.gz *.c # do both of the above in one step $ tar tvf tarball.tar # verbose table of contents $ tar tzvf tarball.tar.gz # verbose table of contents $ tar xvf tarball.tar # extract contents $ tar xzvf tarball.tar.gz # extract contents Tarballs will archive entire directories if you give them directories: $ cd # go to my home directory $ tar czf /tmp/homedir.tar.gz . # archive everything into a file $ cd /some/backupdir $ tar xzf /tmp/homedir.tar.gz # extract the whole archive The name of the tar archive can be anything; the suffixes are there simply for human readers to better know what the files contain. Q: Is a tarball an archive of individual compressed files, or a compressed archive of individual files? Internet Protocols ------------------ - many use plain-text-based negotiations that you can snoop easily - Unix always preferred text file formats over binary formats - Unix has logging and many tools for massaging text streams The netcat "nc" TCP/IP "Swiss Army Knife" program: - a "pure" way to connect your keyboard with a remote TCP/IP daemon - similar to "telnet", except it doesn't try to auto-negotiate first - useful for connecting directly to many text-based Internet protocols: - to save the output, run a "script" terminal session HTTP: Hyper Text Transfer Protocol RFC: http://tools.ietf.org/html/rfc2616 see Notes file http_session.txt SMTP: Simple Mail Transfer Protocol RFC: http://tools.ietf.org/html/rfc2821 see Notes file smtp_session.txt POP3: Post Office Protocol Version 3 RFC: http://tools.ietf.org/html/rfc1939 see course text ALN p.263 http://www.electrictoolbox.com/article/networking/pop3-commands/ (ALN: course text "Advanced Linux Networking" by Roderick W. Smith) - try using this on your Algonquin account inmail.algonquincollege.com NNTP: Network News Transport Protocol RFC: http://tools.ietf.org/html/rfc977 see Notes file nntp_session.txt Q: T/F HTTP, SMTP, POP3, and NNTP are text-based Internet protocols. Command-line FTP - File Transfer Protocol - See Notes file: file_transfer.txt - be careful of the difference between binary and text transfers! - also available under Windows in a console (DOS) window - FTP protocol is much harder to debug using nc or telnet, since FTP has two separate ports and streams (control and data). System logging - figuring out where things go wrong - logging needs a system log daemon and kernel log daemon running: - daemon is started via: /etc/init.d/syslog or /etc/init.d/sysklogd - syslog config file: /etc/syslog.conf - the config file assigns types of logging to various file names - usual syslog log file directory: /var/log/* - know how to find out where the logs are kept! Q: What file controls and configures system logging? Q: Where are system log files usually kept? Linux Boot and Start-Up ----------------------- BIOS executes the boot block from the first available disk drive. The boot block redirects to a bootloader program such as LILO or GRUB. LILO/GRUB allow a choice of systems to boot. Choosing a Linux kernel, the Linux kernel loads - loader can pass options to the kernel using a "kernel command line" - a copy is available as /proc/cmdline when the system is running - services can look in this command line for keywords/values - e.g. the word "single" boots Linux in single-user maintenance mode Linux kernel runs Process #1 - "init" - all subsequent processes fork/exec and are descendants of this one init (process #1) runs the "rc" scripts corresponding to the default run level - see the initdefault run level setting in /etc/inittab - the "rc" scripts start up daemons and perform system services - /etc/rc?.d/* or /etc/rc.d/rc?.d/* are symlinks to /etc/init.d/* Q: What is the name of the start-up directory for run level 1? Basic network connectivity, from the hardware in - how much autoconfig will your distribution do? - Knoppix does massive autoconfig at boot time; most servers do none Linux server initialization and start up: ALN Chapter 4 ------------------------------------------------------- The /etc/inittab file lists the run levels defined on your system. Not all levels are always used. The keyword "initdefault" is associated with the default run level for your system. Q: What file lists and sets the run levels? The run levels are not "sequential". Going to run level 3 does not mean going through levels 0,1,2 first. Think of the numbers as just names for the levels. When you change levels, some services will be shut down and others will be started, depending on what is running in the old level and what needs to run in the new level. Unix has two major ways to start servers as the system is booting. Originally (Berkeley Unix) servers were started by editing a large "rc.sysinit" file. System V Unix introduced a directory of individual "rc" start-up files; most Unix systems use this System V "rc" method. System-V style server start-up scripts (Debian, RedHat): See course text ALN Chapter 4 "Starting Servers" p.79-89 - chkconfig and ntsysv are not present under Knoppix - chkconfig is available on Mandriva/RedHat but not Debian/Knoppix - some systems have ksysv (X11 graphical only) Master run level directory: /etc/init.d/ or /etc/rc.d/init.d/ - contain scripts that will start/stop each service, given the appropriate command line argument of "start" or "stop" - init.d is a directory of *possible* services; not all of them may be in use Q: In what directory are the master run-level scripts stored? Run level directories: /etc/rc?.d/ or /etc/rc.d/rc?.d/ - contain symbolic links to scripts in /etc/init.d - numbers in the names determine order of script execution - script names starting with K will be called with "stop" when moving to that run level, e.g. "/etc/init.d/httpd stop" - script names starting with S will be called with "start" when moving to that run level, e.g. "/etc/init.d/httpd start" - the scripts look at the first argument and do the appropriate thing Q: What do the prefixes S and K mean in the symlink directory for a run level? Auto-config style RC scripts may probe hardware ----------------------------------------------- - Knoppix has an excellent (if slow) hardware probing start-up script - auto-config is typically not done for servers - server hardware doesn't change daily; autoprobing would slow down a reboot and possibly make unauthorized changes to configuration files - auto-config must locate drivers for newly discovered hardware - some boot-time hardware detection scripts: kudzu, harddrake, etc. - without auto-config, you have to know what drivers match which hardware - changing hardware means changing config files - drivers load into the running kernel using "insmod", usually called from an easier-to-use wrapper routine named "modprobe" - use command "lsmod" to see drivers currently loaded into the kernel: $ lsmod | grep scsi - use "rmmod" to remove a loaded kernel module (if it isn't busy) - Note: most drivers can be compiled into the base kernel instead of loaded as modules at run time; lsmod will not show these compiled-in drivers! Q: What is the easiest command to use to load a module into the kernel? Q: What command shows modules that have been loaded? Q: T/F The lsmod command shows all modeules, even ones that have been compiled into the kernel. some Knoppix services that might be useful if enabled: - dhcp client - dhcp server (be careful where you do this!) - ssh server (sshstart) [telnet localhost 22] - ntp - smtp server - fetchmail - http - smb Network hardware ---------------- Reference: ALN p.31-51 Most Linux distributions contain auto-configuration programs that detect your network card during installation and create the necessary start-up code to load the correct network drivers for you. You can see your network hardware using these commands for PCI and USB: $ lspci $ lsusb Both have "-v" verbose options. Q: What two commands show hardware devices on your PCI and USB buses? In Linux, Ethernet card network interfaces are given names such as "eth0" and "eth1" at boot time. Modern systems have a "udev" capability that attaches these names to specific network cards without the need for config files. Older systems have alias names set down in /etc/modules.conf or, for older-newer 2.6 kernels, /etc/modprobe.conf (ALN p.32), or, for Debian systems, directory /etc/modprobe.d/. The alias names connect the abstract names "eth0", "eth1", etc. with the actual Ethernet driver module names, e.g. "eth0 tulip", "eth1 3c95x". Linux network driver modules are kept in /lib/modules/*/kernel/drivers/net/ and you can insert these drivers into your kernel (so that it recognizes a particular Ethernet card) using "modprobe" and just the module name, e.g.: # modprobe 3c59x # ALN p.32 says "insmod"; use modprobe To display loaded kernel modules (including drivers for Ethernet cards): # lsmod To see the latest kernel status messages related to loading a module or driver, use the "dmesg" command, and also check the system log files under /var/log/ (usually files "messages" or "syslog" are important here). Once the correct modules are loaded, you can display the known network interfaces using netstat or ifconfig with the "-a" option: # ifconfig -a # all possible interfaces # netstat -ai # all possible interfaces # ifconfig # only active (up) interfaces # netstat -i # only active (up) interfaces Kernel Interface table Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-DRP TX-OVR Flg eth0 1500 0 41880706 0 0 0 59937708 0 0 BMRU eth1 1500 0 96932 0 0 0 9 0 0 BMRU eth2 1500 0 9462750 1 0 0 12525371 0 0 BMRU lo 16436 0 12111590 0 0 0 12111590 0 0 LRU ppp0 1452 0 128366 0 0 0 181827 0 0 MOPRU Q: What command(s) display all possible network interfaces? Many proprietary hardware drivers cannot be distributed with the Linux kernel; you may have to search the manufacturer web site to find them and download them, and you have to match the driver to the specific Linux kernel you are running. You can find out your kernel name using: $ uname -a Linux elm 2.6.15-27-k7 #1 SMP PREEMPT Sat Sep 16 02:35:20 UTC 2006 i686 GNU/Linux Readings -------- ALN Chapter 4 - "Starting Servers" p.79 Read: Using SysV startup scripts Read: Using inetd Skip: the /etc/inetd.conf file format Read: using TCP wrappers Read: Using xinetd Read: Using local startup scripts Skip: Using GUI tools Read: When to use each startup method Read: Summary Q: What is the purpose of the two-digit number that follws S or K in the symlink names in the run-level start-up directories? p.82 Q; How can you arrange that a start-up script is always started when the system enters, say, run level 3? p.83 Q; How can you temporarily (until the next boot) start or stop a server? p.83 Q: How can you set the run level in preparation for the next reboot? p. 87 Q: T/F Run levels 2-5 have standard meanings across all Linux distributions. p.87-88 Q: What command changes the current run level in a running system? p.88 Q: T/F System start-up scripts are standard across all linux distributions. p.100-101 ALN Chapter 10 - p.241 - Maintaining Consistent Time: Time Servers Read: When to run a time server Read: Setting up an NTP server Skip: Using samba to serve time Read: Summary