------------------------ Exercise #12 for NET2003 due March 28/30, 2005 ------------------------ -Ian! D. Allen - idallen@idallen.ca Remember - knowing how to find out an answer is more important than memorizing the answer. Learn to fish! RTFM! (Read The Fine Manual) Global weight: 4% of your total mark this term Due dates: 1. in-lab live demo March 28 (1%) 2. electronic deliverables due 23h59 March 30 (3%) The electronic deliverables for this exercise are to be submitted online on the Course Linux Server using the "datsubmit" method described in the exercise description, below. No paper; no email; no FTP. Late-submission date: I will accept without penalty electronic deliverables that are submitted late but before 23h59 on Friday, April 1. After that late-submission date, the electronic deliverables are worth zero marks. In-lab demos cannot be submitted late. Exercises submitted by the *due date* will be marked online and your marks will be sent to you by email after the late-submission date. Exercise Synopsis (Summary Description): Repair a broken SysV Postscript start-up script. Configure push/pull email for your updated Mandrake Mini system. Set up and demo an SSH tunnel from the course linux server to the SMTP port on your Mandrake Mini system. Where to work: Work anywhere that you can install your hard drive caddy and have your Mini system booted with networking at the same time. Exercise Preparation: See notes and readings in: week10notes.txt, week11notes.txt ----------------------------------------------- Exercise Details (on your Mandrake Mini System) ----------------------------------------------- Have you done all the preparation steps? If not, go back and do them. Finish your Notes Readings (see the weekly Notes files). Any questions? See me in a lab or post questions to the Discussion news group (on the top left of the Course Home Page). Most of the work will require that you become the root user using the "su" command. Your prompt will change to include a "#" character, warning you that you have elevated privilege. ----------------------------------- Repairing a broken Postfix start-up ----------------------------------- The Postfix SMTP server start-up for the Mini system has a bug in it. At first, things appear fine, even if you manually stop/start the server: # /etc/init.d/postfix stop Shutting down postfix: [ OK ] # /etc/init.d/postfix start Starting postfix: [ OK ] The postfix control program self-check command passes without errors: # postfix check # (For more information on this postfix control program, see "man postfix".) The log files look clean: # tail /var/log/syslog [...] Mar 24 02:50:01 postfix/postfix-script: stopping the Postfix mail system Mar 24 02:50:01 postfix/master[5751]: terminating on signal 15 Mar 24 02:50:01 postfix: succeeded Mar 24 02:50:02 postfix/postfix-script: starting the Postfix mail system Mar 24 02:50:02 postfix/master[5865]: daemon started -- version 2.1.4 Mar 24 02:50:02 postfix: succeeded (Note: If you are getting a syslog error similar to "postfix: warning: My hostname xxx is not a fully qualified name - set myhostname or mydomain in /etc/postfix/main.cf", add the line "myhostname = test.idallen.ca" [without quotes] to the file /etc/postfix/main.cf and restart Postfix.) However, if you try to connect to the SMTP port to make sure the server is working, you'll see the problem: # nc -v localhost smtp localhost [127.0.0.1] 25 (smtp) open No SMTP response is received from the server - the connection hangs. Interrupt the nc command and now check the main system log file: # tail /var/log/syslog [...] Mar 24 02:52:27 postfix/smtpd[5875]: fatal: open database /etc/postfix/aliases.db: No such file or directory Mar 24 02:52:28 postfix/master[5865]: warning: process /usr/lib/postfix/smtpd pid 5875 exit status 1 Mar 24 02:52:28 postfix/master[5865]: warning: /usr/lib/postfix/smtpd: bad command startup -- throttling The error message says aliases.db is missing. As you know (week11notes, ALN p.508), the aliases.db file should be built from the text file "aliases" using the "postalias" command, and this should happen automatically when the SysV Postfix start-up script is run during system start-up. The "aliases" file is there; the "aliases.db" file is missing, causing SMTP to fail. Why didn't the aliases.db file get created? Looking in the SysV Postfix start-up script, we see that there is a shell "for/do/done" loop in the start() function that supplies file names to the postalias command. We insert a DEBUG statement after the "do" to see what file names are being generated by the loop: echo "DEBUG '$i'" Stopping and re-starting the postfix server produces no DEBUG output - the postalias command is never called to process the "aliases" file. Something is wrong with this loop - no file names are being generated. It turns out that the little Perl script in the "for" loop has an error in the pattern-matching code that looks for file names. Change this line: /^[[:space:]]alias_(database|maps) =/ || next; to this (add an asterisk to make the leading spaces optional): /^[[:space:]]*alias_(database|maps) =/ || next; Now the code works, and the debug statement we added displays the alias file: # /etc/init.d/postfix stop Shutting down postfix: [ OK ] # /etc/init.d/postfix start Starting postfix: DEBUG 'hash:/etc/postfix/aliases' The file aliases.db is now created, and the SMTP server is working: # ls -l /etc/postfix/aliases* -rw-r--r-- 1 root root 2262 Aug 28 2004 /etc/postfix/aliases -rw-r--r-- 1 root root 12288 Mar 24 03:06 /etc/postfix/aliases.db # nc -v localhost smtp localhost [127.0.0.1] 25 (smtp) open 220 foo ESMTP Postfix (2.1.4) (Mandrake Linux) QUIT There is a second error. Looking at the next "for/do/done" loop in the start() function (the one with the comment "refresh other maps"), we see a second place where the Perl script is wrong, in the same type of pattern matching code. Change this line in the second "for/do/done" loop: /^alias_(database|maps) =/ && next; to this (adding a Perl expression that skips over leading spaces): /^[[:space:]]*alias_(database|maps) =/ && next; 1. Make the above two fixes to the Postfix start-up script in your Mandrake Mini system. (Do not include the DEBUG statement.) Copy the fixed start-up script to file "postfix-start12.txt" on the course linux server for later submission. ---------------------------- Postfix (SMTP) configuration ---------------------------- Postfix needs some reconfiguration to be fully functional on our Mini system. Edit the main Postfix configuration file and make these changes: a. Change the Postfix smtpd banner to say "Mandrake Linux XXX" where XXX is replaced by a word or phrase of your own choosing. Use the "postfix reload" command to load the new line into the SMTP server. Your banner might look like this (you choose the word or phrase): # nc -v localhost smtp localhost [127.0.0.1] 25 (smtp) open 220 foo ESMTP Postfix (2.1.4) (Mandrake Linux idallen mail service) QUIT b. Reduce the delay warning time from four hours down to one hour. (This is the length of time that Postfix waits before sending you a warning email telling you that a message has not yet been delivered.) If you look at the list of open TCP ports on your machine using the "netstat -nat" command, you will see this for the SMTP port: # netstat -nat Proto Recv-Q Send-Q Local Address Foreign Address State [...] tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN You will recognize "127.0.0.1" as the localhost interface. Postfix is listening for connections only on the "localhost" interface, not on your network interfaces. Make one more change to the main Postfix configuration file: c. Comment out the inet_interfaces line. (This line is set to "localhost", which restricts SMTP connections to be from the local machine only. We want to receive SMTP from all network interfaces.) You must actually stop and start Postfix for this change in interfaces to take effect - using the "postfix reload" command will not work. Confirm that Postfix is now listening on "all" TCP interfaces: # netstat -nat Proto Recv-Q Send-Q Local Address Foreign Address State [...] tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN Pair up with another student and have them connect using "nc -v" to the SMTP port on your machine. (You must tell them your IP address.) Make sure they can connect to you. 2. Copy the edited main Postfix config file to file "postfix-config12.txt" on the course linux server for later submission. ---------------------- Testing Postfix (SMTP) ---------------------- To test the email system, we need a command-line program that can send email. The Mini system doesn't have one. Fetch and install the "mailx" software package on your Mini system and then send a quick message to your own local userid or to the "root" account on your Mini system (do not try to send email off-machine yet): # date | mail -s "this is to your userid" abcd0001 # date | mail -s "this is to the super-user" root Replace "abcd0001" with the local userid you defined on your Mini system. Check the system log to see that the mail worked (see "delivered to mailbox"): # tail /var/log/syslog [...] Mar 24 05:05:54 postfix/pickup[7858]: 42D7F17957: uid=0 from= Mar 24 05:05:54 postfix/cleanup[8040]: 42D7F17957: message-id=<20050324100554.42D7F17957@ip97-175.ott.istop.com> Mar 24 05:05:54 postfix/qmgr[7859]: 42D7F17957: from=, size=361, nrcpt=1 (queue active) Mar 24 05:05:54 postfix/local[8042]: 42D7F17957: to=, orig_to=, relay=local, delay=0, status=sent (delivered to mailbox) Mar 24 05:05:54 postfix/qmgr[7859]: 42D7F17957: removed Postfix places incoming mail for each userid in a directory defined by a config file parameter named "mail_spool_directory". Look up the value of this parameter in the Postfix config files. In this mail directory you should see the mailbox files for people with waiting email (in "mbox" text format). (Note: Mail to the super-user "root" is delivered to userid "postfix" instead, for safety reasons.) In the mail directory, examine the mailbox file for your chosen email recipient to verify that the "date" email you sent actually arrived. The message in the user's mailbox file will look similar to this: From root@ip97-175.ott.istop.com Thu Mar 24 05:05:54 2005 Return-Path: X-Original-To: root Delivered-To: postfix@ip97-175.ott.istop.com Received: by ip97-175.ott.istop.com (Postfix, from userid 0) id 42D7F17957; Thu, 24 Mar 2005 05:05:54 -0500 (EST) To: root@ip97-175.ott.istop.com Subject: this is a subject Message-Id: <20050324100554.42D7F17957@ip97-175.ott.istop.com> Date: Thu, 24 Mar 2005 05:05:54 -0500 (EST) From: root@ip97-175.ott.istop.com (root) Thu Mar 24 05:05:54 EST 2005 3. Copy your successful mail message from the user's mail file to file "mail12.txt" on the course linux server for later submission. (At login, most Unix systems place a path to your own incoming mail file in the $MAIL environment variable. Mail reading programs use this variable to find your incoming mailbox file.) Note: You now have a fully functioning SMTP mail sending system; however, Algonquin College blocks SMTP port 25 and will not let you use your system to send email to remote addresses (e.g. foo@yahoo.ca) while on campus. The work-around for this is to configure your Postfix system to send through an Algonquin "relay" instead of sending directly. See ALN p.512. --------------------------- Postfix (SMTP) log messages --------------------------- Under Mandrake, the system logging daemon ("man syslogd") is configured to write mail system log information to three mail-specific log files. (System logging configuration was covered in Notes file week08notes.txt.) Look in the config file for the system logging daemon and know how to identify the names of the three mail-specific system log files. --------------- IMAP/POP server --------------- The IMAP and POP servers come in the Mandrake "imap" RPM package file. 4. Save into a temporary file a record of (only) the TCP ports open on your machine (less than 20 lines). Fetch and install the "imap" software package. Check the system log for messages. Note that installing the imap package causes the xinetd super-server to reload and re-read its config files (ALN p.96), which makes xinetd aware of the new services. The imap package implements several "pull" mail servers, including ones for IMAP and POP3. After you install the imap package, xinetd will start listening on some new port(s) on your machine. Use "diff -u" to generate a difference between the old list of open ports (saved above) and the current list of open ports. (The older file should be the first file name on the command line.) Copy the diff output to file "diff-ports12.txt" on the course linux server for later submission. Uninstall the imap package. (See the "rpm" command in Notes file mandrake_config.txt.) Check the system log for messages. Uninstalling imap causes the xinetd super-server to reload, minus the imap services that have been removed. Verify that the opened port(s) are now closed. Use the chkconfig command to show a list of the current xinetd servers. Fetch and install the imap software package again. Use the chkconfig command to confirm the existence of new services under its output titled "xinetd based services". You will note that some new xinetd services (from the imap package) are installed but not enabled. 5. Save into a temporary file the chkconfig list of all your current services (about three dozen lines). Edit the supplementary xinetd config file for the "imap" service and enable it. Tell xinetd to reload its configuration files. Use "diff -u" to generate a difference between the old list of services (saved above) and the current list of services. (The older file should be the first file name on the command line.) Copy the diff output to file "diff-services12.txt" on the course linux server for later submission. The output should show the IMAP port enabled. Confirm that the IMAP port is open by connecting to it: # nc -v localhost 143 localhost [127.0.0.1] 143 (imap) open * OK [CAPABILITY IMAP4REV1 LITERAL+ SASL-IR LOGIN-REFERRALS AUTH=LOGIN] localhost IMAP4rev1 2004.350mdk at Thu, 24 Mar 2005 08:05:17 -0500 (EST) 1 LOGOUT * BYE ip97-175.ott.istop.com IMAP4rev1 server terminating connection 1 OK LOGOUT completed Note that "1 LOGOUT" is the way to exit an IMAP server conversation. Make sure that some account on your Mini system has email waiting in the system email spool directory. (You sent this email earlier.) 6. Start a "script" session log (see Notes file miscellaneous.txt) with the name "script12.txt". Use "nc -v" to connect to the POP3 port on your localhost, login to an account, and retrieve and delete the first email message. (Follow the example in ALN p.263.) Copy the script log file to the course linux server for later submission. ----------------------- Port tunnelling via SSH ----------------------- Make sure you can use SSH to connect to the course linux server from your Mandrake Mini system. Log out. Now, add the following option and argument to the start of your usual SSH command arguments: -R 1234:localhost:110 If you get "Warning: remote port forwarding failed for listen port 1234", log out again and change "1234" to some other random number between 1025 and 65530. Keep trying port numbers until some number works; I will assume 1234 worked in the examples that follow. You have created an encrypted "IP tunnel" between port 1234 on the course linux server and POP3 port 110 on your localhost Mini system. Anyone logged into the course linux server can now connect to port 1234 on the linux server, and the connection will be encrypted and tunneled back over your SSH connection to your Mandrake Mini system localhost port 110. Try it out (while connected to the linux server): [idallen-home1]$ nc -v localhost 1234 # use the number that worked localhost [127.0.0.1] 1234 (?) open +OK POP3 localhost 2004.87mdk server ready QUIT +OK Sayonara That POP3 prompt is from the POP3 server running on port 110 on your Mandrake Mini system. You can tunnel the other way (Mini system to linux server) using -L instead of -R. You can have many -R and -L tunnels at the same time. ----------------------- In-Lab Demo #1 March 28 ----------------------- Create an SSH tunnel between a port (you choose a random port) on the course linux server and the SMTP port on your Mandrake Mini system. Demonstrate that when you connect to the port on the course linux server, you get the customized Postfix SMTP banner of your Mandrake Mini system (the banner you created above under "Postfix (SMTP) configuration"). --------------------- Electronic Submission --------------------- Submit these files for marking: $ datsubmit 12 postfix-start12.txt postfix-config12.txt mail12.txt \ diff-ports12.txt diff-services12.txt script12.txt Always submit all files for marking at the same time.