--------------------------------
SMTP Client Test Script - autotest_smtp.sh
--------------------------------
-Ian! D. Allen - idallen@idallen.ca - www.idallen.com

We will be using a shell script autotest_smtp.sh to execute an automated
test of our SMTP client; so, we don't have to redo all the tests manually
every time we change something in our client and we want to re-test it.

To test SMTP responses, we can "fake" a whole set of responses from
an SMTP server by putting the lines we want the fake server to send
in a text file and using "nc" as the fake SMTP server:

   $ nc -v -l localhost 55555 <file       # Red Hat systems
   $ nc -v -l -p 55555 localhost <file    # Debian systems

If we start the above fake SMTP server and then connect our SMTP client to
port 55555, it will read lines from the given file used as input to "nc",
allowing us to test our client against a variety of SMTP server responses.
(Of course you can also start a fake SMTP server and type lines by hand.)

This auto test script will start a fake SMTP server with input redirected
from various files, and it will run your SMTP client against the fake
server and show you the output.  Anything sent to the fake SMTP server
will be echoed onto the screen with a line number prepended.

For best results using the autotest_smtp.sh script, add a "sleep 1"
in your SMTP client after every write to the server, so that the fake
SMTP server has a chance to echo the line onto the screen before
your client goes on to do the next line.

The autotest_smtp.sh script will pause before running each separate test.
Run this auto test script with input from /dev/null to avoid prompts
between tests.  Remember to capture stderr as well as stdout in the
same output pipe/file.  Run the output through "cat -v" to make extra
CR characters visible (though the script should do most of that already).

Usage (prompting - no saving of output):

    $ ./autotest_smtp.sh

Usage (prompting and displaying tests on screen and into output file):

    $ ./autotest_smtp.sh 2>&1 | cat -v | tee test_out.txt

Usage (run all tests; no prompting; no display on screen; use "tail -f"):

    $ ./autotest_smtp.sh </dev/null 2>&1 | cat -v >test_out.txt

If you don't use "tee", then in a separate window you can run "tail -f
test_out.txt" to see the progress of a test script in writing to the
test_out file.

----- cut here ---------------------
#!/bin/sh -u
#
# Syntax:   $0 (no arguments)
# 
# This script runs tests on an SMTP client in the current directory.
# It creates a fake SMTP server using "nc" that feeds SMTP response
# lines from a file back to the client to see how the client handles
# the responses.  No mail is actually sent.
#
# The script waits for RETURN before every test, if stdin is a terminal.
# (To run all the tests without prompting, redirect stdin from /dev/null )
#
# This only runs tests of the SMTP client by connecting to "localhost".
# Some other testing should verify that the client can connect to other hosts.
#
# If your machine is loaded down, you may find that lines sent from
# client to server don't appear on the screen right after they are
# sent; they collect together and appear after the client exits.
#
# Software License: GNU GPL version 2 or later
# - Ian! D. Allen idallen@idallen.ca

# set standard shell PATH and file creation mask
#
PATH=/bin:/usr/bin ; export PATH
LC_COLLATE=C ; export LC_COLLATE
LANG=C ; export LANG
umask 022

# We expect the client to be tested to be named as follows:
#
CLIENT=./smtpclient.pl            # name of client executable
PORT=55555                        # an unused fake SMTP server port

# The correct command line used to run the test SMTP client:
#
RUNCLIENT=" $CLIENT --to user@domain.com --from sender@domain.com --smtpserver localhost --port $PORT "

# The command line used to run the fake netcat SMTP server:
#
if [ -e /etc/redhat-release -o -e /etc/mandriva-release ] ; then
    RUNSERVER=" nc -l localhost $PORT "     # Red Hat Linux
else
    RUNSERVER=" nc -l -p $PORT localhost "  # Debian/Ubuntu
fi

# A temp file used to build SMTP responses to be handed out by the fake
# SMTP server; a temp file used as the message being sent (used as
# standard input to the SMTP client $CLIENT being tested):
#
smtp=/tmp/$USER.smtp.$$
rm -f $smtp
message=/tmp/$USER.message.$$
rm -f $message

# Remove the temp files on exit, SIGHUP, or SIGINT.
# Also kill any left over "nc" processes run by this $USER.
#
trap 'MyTrap' 0 1 2
MyTrap () {
    rm -f $smtp $message
    killall -u $USER -q nc
    killall -u $USER -q $CLIENT
    echo 1>&2 " "
    echo 1>&2 "*** SCRIPT EXIT $0"
    trap 0  # remove exit trap
    exit
}

# The mail message body below tests to make sure leading periods are escaped:
#
cat <<EOF >$message
From: $USER@algonquincollege.com
Subject: This is a test message subject line

Body Line 1.
.Body Line 2 starting with a period (should be escaped for SMTP).
Body Line 3.
.
..
Body Line 6.
EOF

# This is how we actually run a test.
# - all passed arguments are used as the title of the test ($*)
# - pause and ask to run this test (only ask if stdin is a tty)
# - connect to and flush any existing left-over netcat server (make it exit)
# - start a fake server in background, reading from the SMTP response file
#   - number the output lines of the server to make them easy to see
# - run the SMTP client against the fake netcat server
#   - supply the test message as standard input to the client
# - kill the server in case it didn't exit
#
DoTest () {
    echo " "

    # Connect to and flush any existing left-over server (make it exit).
    #
    nc -w 1 localhost $PORT </dev/null >/dev/null 2>&1
    killall -u $USER -q nc
    wait

    # If stdin is a terminal, prompt to continue with this test.
    # (To run tests without prompting, redirect stdin from /dev/null )
    #
    if [ -t 0 ] ; then
        echo 1>&2 "==> TEST $*"
        echo 1>&2 -n "==> Press RETURN to execute this test ..."
        read junk || exit 1   # exit testing if EOF is seen
        echo 1>&2 " "
    fi

    echo "=============================================================="
    echo "Test $*"
    echo "$(date)"
    echo "=============================================================="

    # Start our fake SMTP server in the background:
    # Number all the server output to make it easier to see on the screen.
    # Use cat -v to make control characters (e.g. ^M) visible.
    # Sleep a bit, to allow time for server to actually get going.
    # Make sure the server started (kill -0).
    #
    if [ ! -s $smtp ] ; then
        echo 1>&2 "$0: WARNING: missing or empty input file '$smtp'"
    fi
    $RUNSERVER <$smtp 2>&1 | cat -v -n &
    serverpid=$!
    sleep 1
    if ! kill -0 $serverpid 2>/dev/null ; then
        echo 1>&2 "$0: Unable to start a new server: $RUNSERVER"
        echo 1>&2 "$0: Is the server already running?"
        pgrep -u $USER -f -l -s 0 nc
        exit 1
    fi
    echo " "

    # Run our SMTP client, with message coming from stdin, against the server.
    #
    $RUNCLIENT <$message
    echo "*** Client exit status is $?"
    echo " "

    # Sleep a bit then kill any hung left-over background nc server.
    # Wait for background server to exit before continuing.
    #
    ( { sleep 5 ; kill $serverpid 2>/dev/null ; } & )
    wait
}

##########################################################################
# Tests of the SMTP client go below this line.
# Each test uses a different set of server response lines.
# Each test must be labelled and referenced in the README.txt test plan.
##########################################################################

echo " "
echo "==============================="
echo "Category A: Successful Messages"
echo "==============================="
echo " "

##########################################################################
# This test is for a plain SMTP session - no continuation lines.
# Expected exit status is EX_OK (0).
#
cat <<EOF >$smtp
220 test greeting
250 test good response to EHLO
250 test good response to MAIL FROM
250 test good response to RCPT TO
354 test good response to DATA
250 test good response to end of DATA
221 test good response to QUIT (closing)
EOF

DoTest "A-1: A simple message (no continuation lines)"

##########################################################################
# This test is for a plain SMTP session with a few continuation lines.
# Expected exit status is EX_OK (0).
#
cat <<EOF >$smtp
220 test greeting
250-test good response to EHLO
250-test SMTPoption1 (continued)
250-test SMTPoption2 (continued)
250-test SMTPoption3 (continued)
250 test SMTPoptionlast (continued)
250 test good response to MAIL FROM
250 test good response to RCPT TO
354 test good response to DATA
250 test good response to end of DATA
221 test good response to QUIT (closing)
EOF

DoTest "A-2: A few continuation lines"

echo " "
echo "========================"
echo "Category B: Minor Errors"
echo "========================"
echo " "

##########################################################################
# This tests a small message length - message will be truncated as given below.
# Expected exit status (from <sysexits.h>) is EX_OK (0).
#
cat <<EOF >$smtp
220 test greeting
250-test good response to EHLO
250-test SMTPoption1 (continued)
250-test SMTPoption2 (continued)
250-SIZE 50
250-test SMTPoption4 (continued)
250 test SMTPoptionlast (continued)
250 test good response to MAIL FROM
250 test good response to RCPT TO
354 test good response to DATA
250 test good response to end of DATA
221 test good response to QUIT (closing)
EOF

DoTest "B-1: A fixed small message length"

echo " "
echo "============================"
echo "Category C: Permanent Errors"
echo "============================"
echo " "

##########################################################################
# This tests a permanent refused greeting.
# Expected exit status is EX_UNAVAILABLE (69).
#
cat <<EOF >$smtp
554 test permanent error refused greeting
EOF

DoTest "C-1: A refused greeting - exit status EX_UNAVAILABLE (69)"


##########################################################################
# This tests a permanent refused EHLO.
# Expected exit status is EX_UNAVAILABLE (69).
#
cat <<EOF >$smtp
220 test greeting
504 test permanent error refused EHLO
EOF

DoTest "C-2: A refused EHLO - exit status EX_UNAVAILABLE (69)"

echo " "
echo "============================"
echo "Category D: Temporary Errors"
echo "============================"
echo " "

##########################################################################
# This tests a temporary refused EHLO.
# Expected exit status is EX_TEMPFAIL (75).
#
cat <<EOF >$smtp
220 test greeting
421 test temporary refused EHLO
EOF

DoTest "D-1: A temporary refused EHLO - exit status EX_TEMPFAIL (75)"

##########################################################################
# This tests an SMTP response line that is shorter than 3 digits.
# Expected exit status is EX_TEMPFAIL (75).
#
cat <<EOF >$smtp
xx
EOF

DoTest "D-2: An SMTP code shorter than 3 - exit status EX_TEMPFAIL (75)"

##########################################################################
# This tests a time-out.  (Make sure the client has small time-outs!)
# Expected exit status is EX_TEMPFAIL (75).
#
cat <<EOF >$smtp
220 test greeting
EOF

DoTest "D-3: time-out after greeting banner - exit status EX_TEMPFAIL (75)"

echo " "
echo "============================"
echo "Category ???: ...."
echo "============================"
echo " "

# more test categories and more tests go here ...

echo " "