-----------------------
Lab #03 for CST8165 due now and October 11, 2006  (*UPDATED Sept 28*)
-----------------------
-Ian! D. Allen - idallen@idallen.ca

Remember - knowing how to find out an answer is more important than
memorizing the answer.  Learn to fish!  RTFM!  (Read The Fine Manual)

* UPDATED September 28
* Given that almost nobody managed to get the assignment in on time
* this week, I'm moving the due date to the week after the midterm test
* and upping the marking value to better reflect the time spent.
* This assignment is now worth 6% and is due September 28 and October 11.
* To earn the extension to October 11, you must submit whatever work you
* have done so far (working or not) by midnight September 28.
*
* Only students submitting their interim work by midnight September 28
* earn an extension.  The interim submission does not have to work.
*
* Students submitting the assignment by the original due dates earn
* bonus marks.  (Well done!)
*
* UPDATED September 28

* UPDATED September 19
* Based on feedback in class (and a large project due this week), I've
* moved the due date a week, upped the mark value, and reduced/changed the
* submission requirements: no /nick processing, no demo; test plan instead.
* UPDATED September 19

Global weight: 6% of your total mark this term.
Work so far (working or not): due by midnight September 28.
Due date: At the start of your lab period on Wednesday October 11.

The on-line deliverables for this exercise are to be submitted on-line in
the Linux Lab T127 using the "cstsubmit" method described in the exercise
description, below.  No paper; no email; no FTP.

Late-submission date: I will accept without penalty exercises that are
submitted late but before 10h00 (10am) Thursday October 12, 2006.
After that late-submission date, the exercise is worth zero marks.

Only students submitting their interim work by midnight September 28
earn an extension.

Exercises submitted by the *due date* will be marked on-line and your
marks will be sent to you by email after the late-submission date.

Exercise Synopsis:

    Upgrade your multi-client/chat-server pair.
    Develop a protocol for client/server communications:
    Handle message typing in both directions.
    Test it to death.

Where to work:
    Submissions must compile and run cleanly in the T127 Linux Lab,
    though you are free to develop and work on them anywhere you like.

    If you develop elsewhere, make sure the code works on the Linux Lab
    machines as well; that's where I test it!

Recall that most any program or C function has a Unix/Linux manual page: RTFM

Code submitted without your added useful comments will not be marked.

Code Quality and Portability
----------------------------

A.  Your C programs must compile without warnings (gcc -Wall must be silent).

B.  Your code must work on any machine, no matter what its byte ordering.
    (Always convert bytes between host and network byte order.)

C.  You must include useful comment blocks ahead of the code you write or
    modify.  Code submitted without your added useful comments will not
    be marked.

D.  Marks are awarded for readability and elegance, not just correctness.  
    If your code can't be read, you're useless in a team project.

Coding client/server
--------------------

1)  Read the new class note "header_files.txt".

2)  Make sure your sendall() function is in its own sendall.c source
    file.  Create a "sendall.h" file for its function prototype, and
    include that header in your client and chat server.  Adjust your
    Makefile accordingly.

For our enhanced chat client/server, we need to ensure that we read
"all" the data from a socket.  We choose to indicate this by sending
single lines of text ending in a newline character.  Our programs must
read until they find this newline; then, they know they have a complete
line of text.  A complete line of text, ending in a newline, becomes
the unit of communication in both directions.

Since we don't have infinite size buffers, you need to decide what
to do if you don't see a newline after reading some amount of data.
For example, your server may decide to use a fixed buffer to read lines
from the client and reject lines that are longer than the buffer.  Or,
it might dynamically add to the buffer ("man realloc"), reading and
growing the buffer to take very long lines from the client.  (But when do
you stop - you can't grow forever.)  In either case, you need to handle
the case where the client sends a line longer than what the server is
prepared to buffer.  What will your server do?

If your server decides to throw away data because the size exceeds the
buffer space allocated, it must have a way to throw away all the data
up to the next newline, at which point it can return to reading and
processing lines of text that do fit in the buffer.  You can't simply
throw away the first buffer worth of data.

Hopefully most of the lines coming from the client will fit into the
server's buffer.

*** Update: The chat server should use the "readall_poor" version of
*** readall().   See notes file "readall_poor.txt"

3)  In the chat client, replace the simple "read" or "recv" used now with
    a "readall" function.  In the chat server, replace "read" or "recv"
    with the "readall_poor" function; use the posted readall_poor code,
    with credit.  Each function tries to fetch full lines ending in
    a newline.  The rest of this description talks about the readall()
    function that you must write for your client.

    The readall() function must loop until it finds the first newline in
    the input (or until EOF, time-out, or error).  Copy the entire line,
    including the newline, into the given buffer:

	ssize_t readall(int fd, char *buf, size_t count);

    You must not end the line with '\0', since you return the count.

    If not all of the line you collect fits in the caller's passed
    buffer, copy as much as fits and save the rest for a later call,
    in which case the ending newline won't be copied until a later call.

    "readall()" should be a drop-in replacement for "read()" or "recv()".
    The only difference is that "readall()" will fetch complete lines (or
    as much of the line as fits in the passed buffer from the caller).

    Your function will signal EOF or error in the same way as "read()"
    when all the data has been read.  Note that, because of buffering,
    readall() itself may encounter an EOF in its input long before it
    signals EOF to the program calling it.  The program calling readall()
    won't see EOF returned until the internal readall() buffer is empty
    and the input source used by readall() is also at EOF.

    You can implement this very inefficiently by reading single bytes
    until you find the newline (don't do that, except for debugging!);
    or, you can do it efficiently by reading a large amount of data
    (e.g. 256 to 8K bytes) into a private buffer and looking for the
    newline in the buffer.  (RTFM "man strchr") Loop, adding to the
    buffer, until you find the newline, then copy as much of that full
    line as you can (including the newline) to the caller.  (Be mindful
    of the size of the buffer passed by the caller!)

    If you can't copy the whole line, including the newline, into the
    caller's buffer, copy as much as you can and remember where you
    left off.  When called again, you will copy to the caller from where
    you left off in your private buffer up to and including the newline
    (or up to the point where the caller's buffer is full again).

    It's up to readall()s caller to decide what to do if there is no
    newline in the data collected by readall().  (What is an efficient way
    to test to see if the collected data ends in a newline?)  The caller
    may decide to throw away this data, because it exceeds the buffer
    size, at which point you have to loop reading and throwing away data
    until you finally find the ending newline.

    If a newline appears in the middle of the data read by readall(),
    you have more than one line in your buffer.  Your readall() will copy
    to its caller as much of the data as it can up to and including the
    first newline, remember how much data it copied, and pick up copying
    where it left off on subsequent calls.  Each call to readline()
    tries to copy out a full line ending in a newline.

    Your readall() must always get more data as needed to ensure that
    it fetches either a full line (ending in newline) or a full buffer
    for the caller.  It must never copy an embedded newline; stop
    copying after the first newline you find and leave the rest for a
    subsequent call.  The function must have a way to store unread data
    between calls and remember where it left off.

    Testing: The buffer handling in this function is critical.  Be brutal.
    Test your code with different private buffer sizes and different
    caller buffer sizes to make sure it handles all cases.  Run your
    program under "valgrind" (RTFM).  Can you handle varying buffer
    sizes correctly?

    For isolating problems, you might wish to have available a dumb,
    inefficient version of readall() that simply reads single bytes
    and copies them (up to and including the first newline) into the
    caller's buffer.  The inefficient version of readall() is simple and
    has no buffer management needed.  For debugging, you can substitute
    it for your efficient readall() if you suspect your efficient version
    is causing errors.

    Put readall() in a separate source file, say "readall.c", with
    function prototype in "readall.h" included by the client.
    Adjust your Makefile accordingly.

    Further discussions of the readall() function used in this assignment
    can be found collected from the mailing list, edited, and stored in
    the notes file named readall.txt:

       http://teaching.idallen.com/cst8165/06f/notes/readall.txt
       http://teaching.idallen.com/cst8165/06f/notes/readall_poor.txt

    I also included some sample pseudocode for one implementation of readall().

    Make sure you use readall_poor() in your chat server.  Don't use
    readall() in the chat server; use it only in the client.

4) Modify the chat server to detect an incoming client line that
    starts with the four characters "/msg" followed by blanks or tabs,
    a dotted-quad address, more blanks or tabs, and message ending in
    a newline, e.g.:
       /msg 1.2.3.4 this is a message to 1.2.3.4 only\n
    Have the server send the text part of the message only to the matching
    client IP.  (RTFM "man inet_aton")

    The keyword "/msg" may be written as upper-case, lower-case, or
    mixed-case and may have any amount of blanks or tabs before or
    after it on the line.  (Any number of blanks or tabs may separate
    the arguments: "Be liberal in what you accept.")

    (RTFM "man tolower", "man strspn", "man strpbrk", "man memchr",
    "man strtok", "man strsep" but read the BUGS section)

    The client message text starts after the *first* blank or tab after
    the dotted-quad - do not strip off any other leading blanks on the
    client message line when you pass it on.  Leave the other leading
    blanks on the text message untouched.

    Do you allow a /msg to be sent to yourself?  Why or why not?  In a
    plain text file "README.txt", document the syntax and rules for
    using your version of the /msg command.  Document what is allowed
    and what errors are possible.

5)  If the server detects that the /msg keyword is not followed by
    a recognized dotted-quad for any known client, send a good error
    message back to the client.

    Question: Do "/msg 1.2.3.4 hi" and "/msg 01.002.0003.00004 hi" both work?
    Why or why not?  What do your users expect?  Document your choices
    in the "README.txt" documentation file.

    Does a syntactically invalid IP address give the same error message
    as a valid but unrecognized address?  Document your choice.

6)  Modify the server to flag the error message lines sent to the client
    so that the client distinguishes error message lines from regular
    chat data lines.

    When the client realizes that this incoming line from the server is
    an error message from the server, the client will:

     - precede the display of the error message on the screen with some
       string indicating to the user that this is not ordinary chat data;
       it's an error message, and

     - write the above text and the text of the error message on standard
       error (stderr or fd 2), not on standard output (stdout or fd 1).

    How can your client know that some data that it gets from the server
    is chat data and some data is error data?  You will have to design
    a client/server message prefix protocol that identifies the type of
    each message sent from server to client.  For example, perhaps the
    first part of the message from server to client indicates if the
    text that follows is chat data or is an error message.

    Choose a prefix that allows for future expansion to other types of
    server/client messages.  Document your chosen message prefix protocol
    in the "README.txt" file.

    What happens if the server sends a message prefix to the client that
    the client doesn't recognize?  Document what should happen in README.txt.

(*UPDATE: /nick requirement deleted from this lab; moved to next lab*)

If you find yourself duplicating code, re-factor your source and create a
(possibly static) function to handle the duplicate code in one place.

Code submitted without your added useful comments will not be marked.

Test Plan (*replaces in-lab demo*)
----------------------------------

Develop a simple writen Test Plan for your client/server and do it.
Partial test plans are work partial marks.

Record the output of applying your Test Plan in file test_out.txt using
the "script" command and submit it with your files for marking.

I recommend using a shell script test_plan.sh to implement an automated
test plan; so, you don't have to redo all the tests manually every time
you change your client/server.  See the notes file testplanscript.txt
for an example of a script that tests a client with various inputs.

I suggest testing with a minimum of three clients, two being on other
(different) machines than the server.

Your README.txt file should document the above client->server and
server->client message protocols and show clearly what both client
and server should do given various valid and invalid test inputs.
Number each section.  Your test plan will refer to the README.txt
document by exact section number and your tests will demonstrate that
your client/server does behave the way your README.txt says it should.

Your test plan needs to demonstrate that your program behaves according
to your README.txt document.  Tests might include:

    *) A regular message broadcast to all clients (showing IP address)
    *) A /msg with a valid IP address sent to the remote client
    *) A /msg with a valid IP address sent to the local client
    *) A /msg with an invalid IP address; client displays error
    *) A /msg with an unrecognized IP address; client displays error
    *) tests of any limits in your program, etc.

Note that your client can read test data lines from standard input; so,
you can build a test script that runs various test lines against your
client/server automatically.  See the notes file testplanscript.txt for
an example of a script that tests a client.

*) Write and execute a test plan that tests the client against the server.
*) Either write a text file test_plan.txt and perform all your indicated tests
   manually at the keyboard (tedious), or
*) write a commented shell script to automatically perform your tests for you.
   (one or the other; not both!)
*) Your plan must refer to the numbered sections in your README.txt file.
*) Your testing output should be recorded using "script test_out.txt".
   - You may want the "-a" option if you run more than one test!
   - Put a header on the script test output file; no other editing required.
*) If you use a shell script, it will be much easier to re-run your
   tests when you modify and upgrade your client/server!
   - a sample testing script is in the notes: testplanscript.txt
   - the script only works if your client handles EOF correctly;
     see notes file:  eof_handling.txt
*) Test both valid and invalid client inputs.
*) If your programs have limits, test the limits.

C Programming Note: The isatty() function tests if a file descriptor is
attached to a terminal.  You can usefully use it to prevent prompts for
input from printing on your screen if input is not coming from a keyboard.
This makes your testing output much cleaner, since it removes all the
unnecessary prompts from the automated testing output:

  if( isatty(0) ) fprintf(stderr,"Enter input:"); /* prompt only if a tty */

Rather than repeat code, you could wrap the above line into a function
Prompt(char *) and call it:  Prompt("Enter input:");

Submission and Marking Scheme
-----------------------------

Submission Standards (see earlier labs for details):

1.  At the top of each and every submitted file, as comments, create an
    Exterior Assignment Submission label.

2.  For material you copy from other sources, credit the author and source.

3.  Submit all your source files for marking as Exercise 03 using a
    *single* cstsubmit command line (always submit all files together):

      $ ~alleni/bin/cstsubmit 03 ...list all your source files here...

    Submit all the files necessary to build your client and chat server,
    your README.txt file and your test_plan.txt or test_plan.sh file.
    Do not submit object files or binary files.

    After running "make clean ; make all" I expect to find your client
    named "client" and your chat server named "selectserver" in the
    current directory.

    Note: Nothing in your code or Makefile can include absolute pathnames;
    in particular, you must not reference your home directory.

To be marked, the files named above must have the exact names given.
Code submitted without your added useful comments will not be marked.

If you aren't sure if you've submitted all the necessary files to compile
your project, change to a new empty directory and use cstsubmit to fetch
your submission back, expand it, and type "make".  It should work.

Marking Scheme
--------------

Internal documentation: 20%
 - see note file programming_style.txt
 - are the comments you add useful in understanding what the code does?
 - no "useless comments" - see programming_style.txt
 - comments are in block form, not excessively interleaved with code
 - sparing use of end-of-line comments

Programming correctness and robustness: 20%
 - all return codes checked
 - all buffer overflows prevented
 - all input presumed hostile and handled safely
 - no dynamic memory leaks (RTFM "man valgrind")
 - no off-by-one errors when copying data

Programming style: 20%
 - "be liberal in what you accept"
 - do the error messages appear on stderr and contain full information,
   including the name of the program issuing them?
   - never say "too many" - print the limit
   - never say "not enough" - tell exactly what was expected
   - no user input, short of child pornography, should be mislabelled "illegal" 
 - is this code easy to read and understand?
 - is this code easy to change and maintain?
 - is this code efficient?  (no unnecessary use of strlen or bzero!)
 - is the indentation consistent (Unix tabs are every 8 - "man expand")
 - no duplicate code (uses functions for common actions, repeated code)
 - "less code is better code"

External documentation in README.txt : 20%
 - Number the sections in this document; refer to them in your Test Plan.
 - Document what should happen for every possible type of good and bad input.
 - How do these two programs communicate?
 - What are the limits, if any?
 - What errors are possible; how are they handled?

Test Plan and Testing output in test_out.txt : 20%
 - Your plan must refer to the numbered sections in your README.txt file.
 - Submit either a written test_plan.txt or an executable test_plan.sh
   (one or the other - not both)
 - Submit the testing script output file test_out.txt

All files submitted must be named correctly and have assignment headers.