------------------------ Week 7 Notes for NET2003 ------------------------ -Ian! D. Allen - idallen@idallen.ca Remember - knowing how to find out an answer is more important than memorizing the answer. Learn to fish! RTFM! (Read The Fine Manual) Keep up on your readings (Course Outline: average 5 hours/week homework) Abbreviation "ALN" = your "Advanced Linux Networking" text by R.W.Smith Internet Protocols - many use plain-text-based negotiations that you can snoop easily - Unix always preferred text file formats over binary formats - Unix has logging and many tools for massaging text streams The "nc" TCP/IP "Swiss Army Knife" program - a "pure" way to connect your keyboard with a remote TCP/IP daemon - similar to "telnet", except it doesn't try to auto-negotiate first - useful for connecting directly to many text-based Internet protocols: - to save the output, run a "script" terminal session HTTP: Hyper Text Transfer Protocol see Notes file http_session.txt RFC: http://www.faqs.org/rfcs/rfc2616.html NNTP: Network News Transport Protocol see Notes file nntp_session.txt RFC: http://www.faqs.org/rfcs/rfc977.html SMTP: Simple Mail Transfer Protocol see Notes file smtp_session.txt RFC: http://www.faqs.org/rfcs/rfc2821.html POP3: Post Office Protocol Version 3 see course text ALN p.263 (ALN: course text "Advanced Linux Networking" by Roderick W. Smith) RFC: http://www.faqs.org/rfcs/rfc1939.html - try using this on your Algonquin account inmail.algonquincollege.com Command-line FTP - File Transfer Protocol - See Notes file: file_transfer.txt - be careful of the difference between binary and text transfers! - also available under Windows in a console (DOS) window - FTP protocol is not so easy to debug using nc or telnet System logging - figuring out where things go wrong - logging needs a system log daemon and kernel log daemon running: - daemon is started via: /etc/init.d/syslog or /etc/init.d/sysklogd - syslog config file: /etc/syslog.conf - the config file assigns types of logging to various file names - usual syslog log file directory: /var/log/* - know how to find out where the logs are kept! Symbolic Links - similar to Windows/Mac aliases, but they work even for directories - can symlink to directories (cannot hard link to directories) - can point to nonexistent files or directories Linux Boot and Start-Up ----------------------- BIOS executes the boot block from the first available disk drive. The boot block redirects to a bootloader program such as LILO or GRUB. LILO/GRUB allow a choice of systems to boot. Choosing a Linux kernel, the Linux kernel loads - loader can pass options to the kernel using a "kernel command line" - a copy is available as /proc/cmdline when the system is running - services can look in this command line for keywords/values - e.g. the word "single" boots Linux in single-user maintenance mode Linux kernel runs Process #1 - "init" - all subsequent processes fork/exec and are descendants of this one init (process #1) runs the "rc" scripts corresponding to the default run level - see the initdefault run level setting in /etc/inittab - the "rc" scripts start up daemons and perform system services - /etc/rc?.d/* or /etc/rc.d/rc?.d/* are symlinks to /etc/init.d/* Linux server initialization and start up: ALN Chapter 4 ------------------------------------------------------- The /etc/inittab file lists the run levels defined on your system. Not all levels are always used. The keyword "initdefault" is associated with the default run level for your system. The run levels are not "sequential". Going to run level 3 does not mean going through levels 0,1,2 first. Think of the numbers as just names for the levels. When you change levels, some services will be shut down and others will be started, depending on what is running in the old level and what needs to run in the new level. Unix has two major ways to start servers as the system is booting. Originally (Berkeley Unix) servers were started by editing a large "rc.sysinit" file. System V Unix introduced a directory of individual "rc" start-up files; most Unix systems use this System V "rc" method. System-V style server start-up scripts (Debian, Mandriva, RedHat): See course text ALN Chapter 4 "Starting Servers" p.79-89 - chkconfig and ntsysv are not present under Knoppix - chkconfig is available on Mandriva/RedHat/Debian - some systems have ksysv (X11 graphical only) Master run level directory: /etc/init.d/ or /etc/rc.d/init.d/ - contain scripts that will start/stop each service, given the appropriate command line argument of "start" or "stop" - init.d is a directory of *possible* services; not all of them may be in use Run level directories: /etc/rc?.d/ or /etc/rc.d/rc?.d/ - contain symbolic links to scripts in /etc/init.d - numbers in the names determine order of script execution - script names starting with K will be called with "stop" when moving to that run level, e.g. "/etc/init.d/httpd stop" - script names starting with S will be called with "start" when moving to that run level, e.g. "/etc/init.d/httpd start" - the scripts look at the first argument and do the appropriate thing Auto-config style RC scripts may probe hardware ----------------------------------------------- - Knoppix has an excellent (if slow) hardware probing start-up script - auto-config is typically not done for servers - server hardware doesn't change daily; autoprobing would slow down a reboot and possibly make unauthorized changes to configuration files - auto-config must locate drivers for newly discovered hardware - some boot-time hardware detection scripts: kudzu, harddrake, etc. - without auto-config, you have to know what drivers match which hardware - drivers load into the running kernel using "insmod", usually called from an easier-to-use wrapper routine named "modprobe" - use command "lsmod" to see drivers currently loaded into the kernel: $ lsmod | grep scsi - use "rmmod" to remove a loaded kernel module (if it isn't busy) - Note: most drivers can be compiled into the base kernel instead of loaded as modules at run time; lsmod will not show these compiled-in drivers! Network hardware ---------------- Reference: ALN p.31-51 Most Linux distributions contain auto-configuration programs that detect your network card during installation and create the necessary start-up code to load the correct network drivers for you. In Linux, Ethernet card network interfaces are given names such as "eth0" and "eth1" using alias names in /etc/modules.conf or, for newer 2.6 kernels, /etc/modprobe.conf (ALN p.32). The alias names connect the abstract names "eth0", "eth1", etc. with the actual Ethernet driver module names, e.g. "eth0 tulip", "eth1 3c95x". Linux network driver modules are kept in /lib/modules/*/kernel/drivers/net/ and you can insert these drivers into your kernel (so that it recognizes a particular Ethernet card) using "modprobe" and just the module name, e.g.: # modprobe 3c59x To display loaded kernel modules (including drivers for Ethernet cards): # lsmod To see the latest kernel status messages related to loading a module or driver, use the "dmesg" command, and also check the system log files under /var/log/ (usually files "messages" or "syslog" are important here). Once the correct modules are loaded, you can display the known network interfaces, use: # netstat -ai # all possible interfaces # netstat -i # only active interfaces Most proprietary drivers cannot be distributed with the Linux kernel; you may have to search the manufacturer web site to find them and download them. Four Network Parameter Requirements ----------------------------------- Your machine needs four network parameters to be a good network citizen: 1. a host name (fully qualified with a domain name) 2. an IP address assigned to at least one connected network card 3. a default route to a gateway machine 4. addresses of DNS server(s) to resolve host names The "hostname" command shows and sets the machine host name. ALN p.48 The "ifconfig" command shows and sets IP addresses on interfaces. ALN p.36 A "gateway" machine is a machine on your local network to which packets will be sent if your machine doesn't know where else to send them. Without a gateway, your machine can only communicate with other machines on the local network segment (the local ARP domain). The "arp" command shows the current kernel table listing known MAC addresses on the local network. The "route" command shows you your routing tables, including the "default" route to your gateway machine. ALN p.40-42 You can run your machine without defining any DNS servers, in which case you will have to use IP addresses (not names) for all hosts. The file /etc/resolv.conf ("man resolv.conf") contains definitions of your domain name and DNS servers. ALN p.47 Network configuration - DHCP or static --------------------- With reference to the above four network parameter requirements: 1. DHCP often supplies the host name for your machine. If needed, your host name can be set as a parameter "HOSTNAME" defined in file "/etc/sysconfig/network". 2. Mandriva/RedHat systems keep basic network configuration information (including IP address, unless DHCP is used) for a card named "eth0" in a file named "/etc/sysconfig/network-scripts/ifcfg-eth0". 3. Interface-independent information (including the default gateway used for routing, unless DHCP is used) is kept in "/etc/sysconfig/network". 4. All Unix/Linux machines keep DNS resolver information in "/etc/resolv.conf". (A DHCP client overwrites this file, installing a domain name and set of DNS servers, if DHCP is used.) A. Configuring DHCP networking ------------------------------ For the DHCP client to work, your system must have some DHCP client software package installed. One package used on Mandriva/RedHat systems is "dhcp-client" (the exact version number will vary): # rpm -q dhcp-client dhcp-client-3.0.2-1mdk A Mandriva DHCP client configuration requires you to edit just two files: # cat /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 BOOTPROTO=dhcp ONBOOT=yes # cat /etc/sysconfig/network NETWORKING=yes A DHCP-configured machine usually receives all four of the required network parameters (hostname and domain, IP address, Gateway address, and DNS server info) via the DHCP protocol. The /etc/resolv.conf file is set up automatically (overwritten!) when your client gets its IP address via DHCP. You can use these commands to query the state of your machine (host name, IP address(es), gateway address [default route], and DNS information): # hostname foo.happy.machine.ca # ifconfig eth0 eth0 Link encap:Ethernet HWaddr 01:A0:CC:2C:A8:00 inet addr:192.168.1.2 Bcast:192.168.1.255 Mask:255.255.255.0 [...] # route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 192.168.1.0 * 255.255.255.0 U 0 0 0 eth0 default 192.168.1.1 0.0.0.0 UG 0 0 0 eth0 # ip route 192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.2 default via 192.168.1.1 dev eth0 # cat /etc/resolv.conf search happy.machine.ca domain happy.machine.ca nameserver 192.168.1.3 nameserver 192.168.1.4 You can use the above commands to display the current host name, network, route, and DNS information. Given a properly configured set of ifcfg-eth? files using DHCP, there is no need to manually use "hostname", "ifconfig" or "route" to configure your network address or routing tables in a Mandriva/RedHat system - the start-up script "/etc/init.d/network" (run via symbolic links in the /etc/rc?.d directories) does all that for you. B. Configuring static networking -------------------------------- If you use static addresses instead of DHCP for one or more of your network interfaces, you must set some or all of the required four network configuration parameters yourself (host name, IP address(es), gateway machine, DNS servers). You may not have to set all of the parameters if your host has multiple interfaces and some other interface already sets some of the four required network parameters. If you use only static IP addressing (no DHCP) in the ifcfg-eth* files, you must also provide the Gateway IP (default route) as the "GATEWAY" parameter in /etc/sysconfig/network and the DNS domain and name server info in file /etc/resolv.conf. ("man resolv.conf") A static (non-DHCP) configuration requires you to edit three files (the actual values used in the files below are just examples): # cat /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 BOOTPROTO=static ONBOOT=yes IPADDR=192.168.1.2 NETMASK=255.255.255.0 # cat /etc/sysconfig/network NETWORKING=yes HOSTNAME=my.happy.machine.ca GATEWAY=192.168.1.1 GATEWAYDEV=eth0 # cat /etc/resolv.conf search happy.machine.ca domain happy.machine.ca nameserver 192.168.1.200 nameserver 192.168.1.201 nameserver 10.1.23.1 The ifcfg-eth? files now have a BOOTPROTO of "static" instead of "dhcp". The host name is defined. You set your own gateway machine IP address, e.g. 1.2.3.4, by adding "GATEWAY=1.2.3.4" to file "/etc/sysconfig/network". You can define your domain name and up to three DNS servers manually in /etc/resolv.conf ("man resolv.conf"). The same commands as before display the current host name, network, route, and DNS information: hostname, ifconfig, route (and "ip route"), and "cat /etc/resolv.conf" Linux Networking Booting / Start / Stop / Restart ------------------------------------------------- Reference: ALN Chapter 4 - "Starting Servers" p.79-89 Most Unix/Linux distributions provide an RC "network" start-up script - may be called /etc/init.d/network - script may detect and load special modules (e.g. ipv6) - script may set network kernel options using sysctl - see /etc/sysctl.conf - may use helper scripts such as "ifup" and "ifdown" to start network - see files in /etc/sysconfig/ and /etc/sysconfig/network-scripts/ - each network device has a config file named after it: e.g. /etc/sysconfig/network-scripts/ifcfg-{lo,eth0,eth1} - may start network interface monitoring programs such as ifplugd - see the config file under /etc/ifplugd/ - enabling the network device also adds a route for that device, and possibly a gateway for that network start-up script symlinks in /etc/rc?.d/ or /etc/rc.d/rc?.d/ start "daemons" - many daemon programs listen on well-known TCP and UDP network ports - /etc/services is a plain text list of names of many well-known ports - see also the master list at http://www.iana.org/assignments/port-numbers Mandriva networking is started/stopped using the SysV start-up script named "network" (found under /etc/init.d/): # chkconfig --list network network 0:off 1:off 2:on 3:on 4:on 5:on 6:off # /etc/init.d/network status Configured devices: lo eth0 eth1 eth2 Currently active devices: lo eth0 The "initd.d/network" SysV script is linked symbolically into the run level directories rc?.d. You can also call the script manually: # /etc/init.d/network stop # /etc/init.d/network start Some distributions provide a "service" wrapper script: # service network stop # service network start Using this network start/stop script will affect all your configured network interfaces, including the "loopback" interface, and configure your default gateway. Using the "ifup" and "ifdown" scripts affects only one interface and may not set your default gateway. Use the main network script unless you are doing something unusual. Network-related system start-up scripts --------------------------------------- Once you have your network working, it's time to enable network clients and servers. Here is a list of some possible service scripts started in the Mandriva Mini system after we have installed the updated list of packages described in mandriva_config.txt (your list may differ slightly): # chkconfig --list | grep :on | sort alsa 0:off 1:off 2:on 3:on 4:on 5:on 6:off crond 0:off 1:off 2:on 3:on 4:on 5:on 6:off dm 0:off 1:off 2:off 3:off 4:off 5:on 6:off haldaemon 0:off 1:off 2:off 3:on 4:on 5:on 6:off harddrake 0:off 1:off 2:off 3:on 4:on 5:on 6:off httpd 0:off 1:off 2:off 3:on 4:on 5:on 6:off iptables 0:off 1:off 2:on 3:on 4:on 5:on 6:off keytable 0:off 1:off 2:on 3:on 4:on 5:on 6:off kheader 0:off 1:off 2:on 3:on 4:off 5:on 6:off mDNSResponder 0:off 1:off 2:off 3:on 4:on 5:on 6:off messagebus 0:off 1:off 2:off 3:on 4:on 5:on 6:off named 0:off 1:off 2:off 3:on 4:on 5:on 6:off netfs 0:off 1:off 2:off 3:on 4:on 5:on 6:off network 0:off 1:off 2:on 3:on 4:on 5:on 6:off nfslock 0:off 1:off 2:off 3:on 4:on 5:on 6:off nifd 0:off 1:off 2:off 3:on 4:on 5:on 6:off ntpd 0:off 1:off 2:on 3:on 4:on 5:on 6:off numlock 0:off 1:off 2:off 3:on 4:on 5:on 6:off partmon 0:off 1:off 2:off 3:on 4:on 5:on 6:off portmap 0:off 1:off 2:off 3:on 4:on 5:on 6:off postfix 0:off 1:off 2:on 3:on 4:on 5:on 6:off proftpd 0:off 1:off 2:off 3:on 4:on 5:on 6:off rawdevices 0:off 1:off 2:off 3:on 4:on 5:on 6:off smb 0:off 1:off 2:off 3:on 4:off 5:on 6:off sound 0:off 1:off 2:on 3:on 4:on 5:on 6:off sshd 0:off 1:off 2:on 3:on 4:on 5:on 6:off syslog 0:off 1:off 2:on 3:on 4:on 5:on 6:off udev 0:off 1:off 2:on 3:on 4:on 5:on 6:off xfs 0:off 1:off 2:on 3:on 4:on 5:on 6:off The Mandriva Mini system by default does not run the "inetd" (ALN p.89) or "xinetd" (ALN p.96) super-servers, and by consequence none of the services that these super-servers support. You would have to install the package to gain access to either of the super servers. System Logging -------------- The Mandriva Mini system comes with system and kernel logging enabled via the SysV start-up script named "syslog" (found under /etc/init.d/). Looking at /etc/syslog.conf, we see that most system messages ("*.*") get logged to file /var/log/syslog ("man syslog.conf"): *.*;auth,authpriv.none -/var/log/syslog Private authentication messages are not saved to this file. The leading dash on the file name permits internal buffering of messages by syslog process; messages are not flushed immediately to disk. X11 GUI ------- If installed, the X11 interface with XDMCP display manager "dm" are started only in run level 5. (See ALN Chapter 14.) You can change the default boot run level via the "initdefault" line in /etc/inittab ("man inittab"). Network servers typically don't run a GUI. The "telinit" command can change run levels in a running system ("man telinit"). Changing run levels will affect which services are running, using the start/stop scripts defined in /etc/rc?.d/. Other key network-related services you will find on many servers: httpd - the Apache web server - ALN Chapter 20 named - the Berkeley Internet Name Daemon (BIND) DNS server - ALN Ch. 18 ntpd - Network Time Protocol daemon - ALN Chapter 10 postfix - an SMTP server (Mail Transfer Agent - MTA) - ALN p.507 proftpd - an FTP server - ALN Chapter 21 shorewall - a script to configure iptables firewall rules - ALN Ch. 25 smb - Samba: a Windows-style SMB file share server - ALN Chapter 7 sshd - the Secure Shell SSH server - ALN p.321 xfs - the X Font Server (for X11 fonts) - ALN Chapter 15 xinetd - a multi-server server that can start many others - ALN p.96. Network Diagnostic Tools ------------------------ Please re-read the "Acceptable Use Warning" on the course home page. Note that at Algonquin College many/most network probe ports and protocols (even ping and traceroute) are blocked and will *NOT* work. Sorry! Try them at home instead. These tools are helpful in diagnosing network problems: arp - show/change MAC addresses currently known to this host ethereal - GUI packet sniffer fuser - (-n tcp, -n udp) list processes with open TCP or UDP ports mtr - ping-style traceroute: packet route diagnostic netcat (nc) - Network Swiss Army Knife: connect/listen to ports netstat -natu - list open and listening TCP and UDP connections nmap - Network Mapper - port prober ping - ICMP echo generator/receiver tcpdump - command-line (non-GUI) packet sniffer telnet - TCP connection program: use when netcat is not available traceroute - packet route diagnostic Major service port numbers (often seen in trace output): - port numbers are given names in the file /etc/services - see also the master list at http://www.iana.org/assignments/port-numbers TCP 20 ftp-data TCP 21 ftp (control) TCP 22 SSH TCP 23 telnet TCP 25 SMTP (sending mail only) UDP/TCP 53 domain (DNS) UDP 67-68 DHCP TCP 80 HTTP (WWW) TCP 110 POP3 (receiving mail only) TCP 113 ident (identifying incoming TCP connections) TCP 119 NNTP (Network News) UDP/TCP 123 NTP (Network Time) UDP/TCP 137-139 Microsoft netbios (SMB) (Samba) TCP 443 HTTPS (secure WWW) UDP/TCP 445 Microsoft-DS UDP/TCP 631 Internet Printing Protocol (IPP - CUPS) Individual network servers/daemons (e.g. ssh, http) may have individual start-up scripts, or they may run on demand out of the master "inetd" or "xinetd" super-servers (ALN p.89,96). Example init.d script: http --------------------------- The name of a script in /etc/init.d/ may or may not relate to the published name of the service port! Some distributions name the http service script "apache" or "httpd". (Since many different installed programs may serve the same port, often the scripts are named after the program, not the port name, to avoid naming conflicts.) Use "grep" to find keywords in /etc/init.d/* to locate services on an unfamiliar system. Maintaining the web server on Mandriva (*NOT* on the course linux server): 1) become root on Mandriva using "su" 2) as root on Mandriva, run the appropriate action: # /etc/init.d/httpd stop # /etc/init.d/httpd start # /etc/init.d/httpd restart You can often abbrviate these using the "service" wrapper as: # service httpd stop # service httpd start # service httpd restart 3) test that http is being served locally using a browser and the URL: http://localhost/ 4) (for a faster http probe:) test http using "nc" or "telnet": $ nc -v localhost http $ telnet localhost http (see the HTTP session in the course notes) 5) HTTP server start page update: - the directory that Apache uses for HTTP files is given in /etc/httpd/conf/httpd.conf via the DocumentRoot directive: DocumentRoot "/var/www/html" - change the start page file(s): vim /var/www/html/index.* - add your name to the page title or change the text and reload the page 6) to shut down the httpd service temporarily (only until next reboot): # /etc/init.d/httpd stop 7) To stop it from re-appearing at next boot, adjust the run level directory symbolic links. You can do this manually: # cd /etc # rm rc?.s/*httpd (Be careful not to remove the master copy under init.d/httpd!) You can remove httpd using a helper program (if there is one installed): # chkconfig --del httpd Note that removing the symbolic links does not stop the currently running server. Without the links, changing run levels will have no effect on the server. 8) To reset the symbolic links to the defaults (listed at the beginning of the start-up script), use the helper program: # chkconfig httpd reset You can make custom changes to the symbolic links by hand (rarely done). Review System V start up scripts and tools - helper programs: - chkconfig (run in any terminal window) [not available on Knoppix] - ntsysv (run in any terminal window) [not available on Knoppix] - ksysv (X11 graphical only)