----------------------- Lab #03 for CST8165 due March 10, 2008 (Week 9) ----------------------- -Ian! D. Allen - idallen@idallen.ca - www.idallen.com Remember - knowing how to find out an answer is more important than memorizing the answer. Learn to fish! RTFM! (Read The Fine Manual) Global weight: 7% of your total mark this term. Interim submission: Submit what you have done so far in lab on March 3. Due date: before 16h00 Monday, March 10. Interim submissions in Week 8: You will submit whatever progress you have made on this assignment before the end of your weekly lab period in Week 8. The deliverables for this exercise are to be submitted online in the Linux Lab T127 using the "cstsubmit" method described in the exercise description, below. No paper; no email; no FTP. Late-submission date: I will accept without penalty exercises that are submitted late but before 16h00 on Tuesday, March 11. After that late-submission date, the exercise is worth zero marks. Exercises submitted by the *due date* will be marked online and your marks will be sent to you by email after the late-submission date. You will submit whatever progress you have made in-lab on March 3. The program does not have to work. Submit whatever you have done. If you have code that works at the interim submission lab, I am available to pre-test your code and check for bugs. Exercise Synopsis and Objectives: 1. Iteratively design and code a forking two-process TCP/IP echo client. 3. Test it thoroughly. Coding and submission standards: - provide File Headers (Program Headers) using my Assignment Label - provide Function Headers documenting arguments and return values - use Block Comments (see programming_style.txt) - error messages must have the four-part format from programming_style.txt Where to work: Submissions must make, compile, and run cleanly in the T127 Linux Lab, though you are free to work on them anywhere you like. Since this course has no textbook, use the Internet instead: Background tutorial on using client sockets: The Sockets Tutorial: http://www.cs.rpi.edu/academics/courses/fall96/sysprog/sockets/sock.html (alternate: http://www.linuxhowtos.org/C_C++/socket.htm ) Beej also has a line-by-line socket programming tutorial here: http://beej.us/guide/bgnet/ The simple client code is here: http://beej.us/guide/bgnet/output/html/singlepage/bgnet.html#simpleclient Another client: http://linux.omnipotent.net/article.php?article_id=5424 Part I - design the program and test suite ------------------------------------------ Using the PDL/pseudocode given in class, design a two-process forking TCP/IP "echo" client and a test suite to test it. The command line calling sequence looks like this: $ ./client serverIP serverPORT For example: $ ./client localhost 80 $ ./client google.ca http The host name or the port name may optionally be in text (not numeric) form. You will need to use library routines that turn names into numbers. The client creates two processes. One process reads standard input and sends bytes to the remote server. The other process reads from the remote server and sends bytes to standard output. Issue a prompt on standard error if reading from a terminal. (See "man isatty".) EOF from standard input will cause that process to shut down the writing half of the server socket (to signal EOF to the server) and then exit. EOF from the server will cause that process to kill the other process (the one reading standard input) and then exit. The client must be 8-bit clean - able to send and receive unlimited binary data. You should be able to pass a binary file through your client, through a remote echo server, back through your client, with no data loss. We will work on coding this client in stages. Part II updates the existing non-forking, non-looping client. Part III makes it fork and loop. Part II - Coding the basic single-process one-line client --------------------------------------------------------- This Part II brings the reference client.c up to par with good coding practices. It doesn't make the client fork() or loop yet; that comes later. This Part II client only reads and writes one line. Reference: http://www.cs.rpi.edu/academics/courses/fall96/sysprog/sockets/client.c (alternate: http://www.linuxhowtos.org/data/6/client.c ) The client.c code is explained line-by-line in the above web page. 1. Using wget, download the reference TCP/IP client code named "client.c". 2. Add the appropriate target "client" to your Makefile. Make sure you can rebuild the client with all the required CFLAGS, given in previous labs. 3. Just as you did with the server, fix the compilation errors and warnings. 4. Test the sample client. Make sure the sample client can send one line to the server and back: a) Test against a netcat ("nc") server that you start up like this: $ nc -v -l localhost 55555 # server on Red Hat systems $ nc -v -l -q 9 -p 55555 localhost # server on Debian systems In another window, connect your client to the netcat server: $ ./client localhost 55555 Your client should be able to send one line to the netcat server and, if you type it, receive one line back from the netcat server. b) Test your code against the "echo" port on my Course Linux Server located at this address (inside the firewall only): $ ./client 10.50.254.148 7 The TCP echo port (7) is an IETF standard port that echoes back any lines it receives. Most servers have it disabled these days. You may also use your own working echo server for this test. Your client should be able to send one line to the echo server and back to your screen. c) Test your code against your own echo server from Lab 1. Your client should be able to send one line to the echo server and back to your screen. 5. Replace the deprecated bzero() and bcopy() functions in the client. 6. As you did in the server, set a 30-minute alarm() in the client so that forgotten client processes eventually kill themselves. 7. As you did in the server, ignore SIGPIPE in the client. If you don't do this, a write on a closed socket will kill your program. 8. Fix the argument code to accept only *exactly* one non-empty host and one non-empty port argument, not more, not less. Do not run the program if too many or too few arguments are given, or if either argument is the null string. 9. Modify the code to use the GNU error() function for all warnings and errors. Remove the existing error() function from the client. 10. The client.c code intermixes argument parsing and handling with the network code. For example, socket() is called before validating the existence of the argv[1] server name using gethostbyname(). The argv[2] port argument isn't validated at all in the sample code (and atoi() doesn't detect errors!). The code would have better structure if all the argument parsing and basic validation were done before using the arguments, as much as possible. Errors in arguments should result in an error message and a non-zero exit() code before trying to create any sockets. 11. Implement the use of getservbyname() to turn a non-numeric port name (e.g. "http") into an integer. Validate the passed port number before using it. Print an error message and exit non-zero if the port isn't valid or can't be found. 12. Using "#ifdef POSIX_2001" define alternate code that uses the POSIX 2001 functions getaddrinfo(), freeaddrinfo(), and gai_strerror() to turn the host name and port directly into a socket structure suitable for connect(). This is the "modern" way to do this. Set up the getaddrinfo() "hints" structure to select the Internet family and stream sockets. Use getprotobyname() to set the "hints" protocol field to the number for the "tcp" protocol. Check for all errors and handle them. 13. Re-work your code so that the #ifdef POSIX_2001 code and the non-POSIX code both set up the same socket structure for use by connect(). Compile and test both the POSIX and non-POSIX versions of the code by adding and removing -DPOSIX_2001 from the gcc compile line. Make sure you check for errors in both versions of the code and, if necessary, print an error message and exit non-zero. 14. Upgrade the error messages to the standard four-part format documented in Notes file programming_style.txt. Make sure that the error messages in code that uses command line input contain what the user typed as part of the error message. Error messages should echo what the user entered, if what the user entered is relevant to the error. (An error in calling socket() does not depend on anything entered by the user. An error in bind() usually does.) Note: For extra arguments present on the command line, rather than echoing them all to the user in an error message (which would require coding a loop), you may optionally compromise and simply tell the user how many extra incorrect arguments were given, not what they were. 15. Recode the client to remove the need to bzero/memset() the buffers used for read/write I/O. Zeroing the entire buffer is wasteful and not necessary if we know how much of the buffer is in use: a) Since fgets() doesn't handle binary data and doesn't tell you how many characters were read, get rid of fgets() and use a plain "read()" system call to read from standard input (from Unix fd unit 0). (Recall: The server dostuff() function also uses a plain read() syscall to read from a socket.) Check for EOF and errors. b) Since printf() doesn't handle binary data, replace it with a plain write() system call that does. Check for errors. On I/O errors from read() or write(), print useful error messages. Follow the four-part error message format in programming_style.txt You don't need to make the client loop yet; you only need to fix the EOF and error handling for both reading and writing. 16. Fix the prompt. Make sure the prompt for input appears on standard error, not standard output. Prompts should always be sent to standard error so that they don't get redirected into output files. 17. You may need to call fflush() with the correct stdio output descriptor to flush any buffered stdio prompt output before you call read() to read from standard input; otherwise, the prompt for input may not appear. (If you use the syscall write() to output the prompt, this won't be a problem. Buffering only happens using stdio.) 18. Use isatty() ("man 3 isatty") to avoid printing a prompt for input if standard input is not a terminal (is not a keyboard). Prompts should only be printed if a human is required to enter input. 19. On any EOF, whether from standard input or from the server socket, print this debug message on standard error preceded by the program name: DEBUG EOF reading from unit %d where %d is the I/O descriptor that was being read when EOF was detected. (The GNU error() function is good for producing this sort of message.) 20. Use a netcat server to test that your client produces both EOF messages. EOF on the keyboard must produce: ./client: DEBUG EOF reading from unit 0 Killing the netcat server when the client is reading from it must produce: ./client: DEBUG EOF reading from unit 3 21. When the client is done writing the server socket, it needs to signal EOF on the socket to the server. It can't close() the socket, since closing it would prevent the client from reading the server's response from the socket later. Also, in a multi-process forking client, having only one process close the socket would still leave the socket open in the other process and the server would not see EOF until *both* processes closed the socket. The client needs a way to close only the *writing* part of the socket, signalling EOF to the server. (See Notes file eof_handling.txt ) To do this, use the special syscall "shutdown(sockfd,SHUT_WR)". The time to call shutdown() is when the client is done writing to the socket. That happens on EOF or error on standard input (no bytes are available to write to the socket), or after writing the one line to the socket. Use shutdown() to close the writing half of the connected server socket after EOF or error on standard input, or after writing the line to the socket. The shutdown() will cause the server to see EOF (zero bytes read) on the next read from the client. The shutdown() function is explained in Notes file eof_handling.txt 22. Use a TCP echo server to test that your client now produces both EOF messages when it gets EOF on the keyboard: ./client: DEBUG EOF reading from unit 0 ./client: DEBUG EOF reading from unit 3 EOF on the keyboard causes a client socket shutdown() that causes EOF in the TCP echo server that causes a server socket shutdown that causes EOF in the client. 23. Make sure that your server gets EOF from the client call to shutdown() *before* the client exits. Test this! For debugging, you can put a sleep(30) after shutdown(), to make sure that the server sees EOF (and prints a DEBUG message saying it sees EOF) as soon as the shutdown() happens. Test this! 24. Since your client must now always call shutdown() even after an EOF or error, you can't simply exit the client on errors. You must ensure that your program detects and handles an error and then shuts down the socket before exiting. 25. If the client read any bytes from standard input, write those bytes to the open server socket. (Error tests should already be done as specified above.) Make sure you only write the number of bytes read! Don't write the whole buffer! Don't write zero bytes! 26. Writing to a network socket may not write all the bytes. Convert the write() that sends bytes to the server socket to use your sendall() function, just as you did in the server. Check for errors. 27. Fix your Makefile to include the sendall files as dependencies. 28. If the client read any bytes from the server socket, write those bytes onto standard output. Don't write EOF, zero, or negative bytes! 29. Make sure that your client is rebuilt by the Makefile if any of its source files change, including any of your relevant header files. Test this! Part III - Coding the fork()ing, looping client ----------------------------------------------- The Part II client is single-process and only reads and writes one line. This Part III client forks and each process reads and writes multiple lines. 30. Add a fork() to create two separate processes to handle simultaneously reading standard input and reading the server socket. The parent process should have the prompt and all the code that reads standard input and writes the open server socket. The child process should have all the code that reads the open server socket and writes standard output. You don't need to make the client loop yet; you only need to make sure the fork() works and that everything still works for one line of input and one line of output. Test this before you go on! 31. As you did in the parent, set a 30-minute alarm() in the child process so that forgotten child processes eventually kill themselves. 32. At the exit of the parent process, write this debug message on standard error, preceded by the name of the program: DEBUG parent exit The GNU error() function is good for producing this sort of message. 33. Write a similar debug message before the exit of the child process: DEBUG child exit The GNU error() function is good for producing this sort of message. 34. Sleep for 1 second just before either process exits. This will allow output from your program to appear before the program exits and the shell prompt reappears. Adding a 1-second sleep before prompting also makes the output look a bit nicer. (Don't sleep if not prompting!) 35. Test your forking processes. The client still reads and writes only a single line; looping isn't implemented yet: Test #A: Start a netcat server. Start your client and connect to the netcat server. Kill the server. The client child process should say: ./client: DEBUG EOF reading from unit 3 ./client: DEBUG child exit The client parent process should still be reading (one line) from the keyboard. Enter some input and push RETURN a few times. The parent process will get an error on writing to the closed server: ./client: ERROR writing to unit 3 : Broken pipe ./client: DEBUG parent exit If you don't see these messages, perhaps you forgot to ignore SIGPIPE? Test #B: Use a TCP echo server. Start your client. Enter one line of input to the client. The line should go to the echo server and back to the client, appearing on your screen. Both the client parent and child processes will exit with debug messages: ./client: DEBUG parent exit ./client: DEBUG child exit Test #C: Use a TCP echo server. Start your client. Signal EOF from the keyboard. You will see these messages from the client (perhaps not in this order): ./client: DEBUG EOF reading from unit 0 ./client: DEBUG parent exit ./client: DEBUG EOF reading from unit 3 ./client: DEBUG child exit If these tests fail, find out why before you continue! 36. Using the pseudocode developed in class, make each of the two forked processes loop reading and writing until EOF or error: The client parent process reading standard input should stop looping and clean up and exit on EOF or error from standard input, or if writing the server socket fails. The client child process reading the server socket should stop looping and clean up and exit on EOF or error from the server socket, or if writing standard output fails. The client parent process reading standard input and the client child process reading the server socket are largely independent. The exception to the independence of the two processes is this: 37. Before the client child process (reading the server socket) exits for any reason, it should kill (with SIGHUP) the client parent process that is reading standard input. The reason is: if the client child process is exiting and will not be printing anything more from the server, then nothing more should be sent to the server either. The client parent process won't be able to send any more input to the server, and both client processes should exit; so, the child has to kill() the parent that is blocked reading standard input. See "man 2 kill" for a function to send a signal to a process. You can get the pid (process ID) of the current (parent) process before you fork() using getpid(). See: "man 2 getpid" 38. Since your client child process must now always call kill() before it exits, even after server EOF or error, you can't simply exit the child process on errors. You must ensure that your child detects and handles an error and then kill()s the parent process before exiting. 39. Optional Challenge: Write your code so that the parent and child status of the two processes are interchangeable; that is, after fork() it doesn't matter which process is coded to be the child/parent. You will need to add a little bit of logic before the kill() so that kill() always kills the "other" process, no matter whether it is the parent or the child process. (Hint: The process doing the kill() has to kill the child pid if it is coded to be the parent process, or kill the parent pid if it is coded to be the child process.) 40. Test that the kill with SIGHUP works before the client child exits: Start a netcat server. Start your client. Kill the server. The client should say: ./client: DEBUG EOF reading from unit 3 ./client: DEBUG child exit and the shell should report a "Hangup" signal from the client as the child process calls kill() on the parent and exits, and your shell prompt should appear. Both client processes should be gone. 41. Remember: You can't simply exit when you detect EOF or error after forking in either the child or the parent processes. Both the client child and parent have things that must be done before exiting. You must print the error or EOF message, break the loop, and do some cleanup before the process exits. 42. If you want to use sendall() to write to a file descriptor that is not a socket (e.g. your terminal screen), remember that you must ensure that sendall() uses write() and not send(). Your data path in both parent and child must be 8-bit clean; do not use any functions that depend on NUL bytes in the input stream. 43. Notice that your looping parent and child processes are derived from the same pseudocode and thus contain almost exactly the same code. They both loop reading from one fd and writing to another fd. They both check for EOF and errors. The only differences in the code are (1) whether or not to prompt for input and (2) whether to call shutdown() or kill() after the loop finishes, before exiting. 44. Combine most of the parent and child loop code into one single looping function, where you pass in the input fd and the output fd. (Hints: Most of the code for child and parent is the same and can be parametrized to read from one fd and write to another. You only prompt for input if the fd you read from is a terminal; the code that tests for a terminal (isatty()) is harmless if applied to a socket that is not standard input.) 45. Optional Challenge: Use the same code that you already wrote for your echo server that copies data from one file descriptor to another. Your server, your client parent, and your client child, should all use exactly the same function to copy data from one place to another. Move that identical code into its own function and its own file and compile it separately. Part IV - Official Test Results ------------------------------- All "script" sessions below must use my script cover: ~alleni/bin/script Never use the native "script" command. Use my version, above. If you forget how to use script, see the previous labs. Note: Your output file will not contain the full results of your script session until you exit the subshell started by the script command. When you exit the subshell, script will tell you the name of the output file. These are the official tests for handing in. Make sure your script session contains *only* the required commands below and their output - do not flood the script session with VIM sessions or other unnecessary interactions. The script command has an option to append to a script file, if you want to run the testing below in separate sections. 46. Start a ~alleni/bin/script session with output file "testing.txt". Never use the native "script" command. Use my version. 47. Test the Makefile. Run a full clean and recompile test, twice: $ make clean $ make clean # second time should not produce any errors $ make client $ make client # second time should say everything is up to date Use the exact four tests above, in the above order. Make sure you compile code with all these required CFLAGS from previous labs. 48. Verify that your client works using the standard TCP server netcat. Since your client is two processes, test each direction separately: a) Send data (one-way) from client to fake netcat server: Start a fake netcat server and redirect its output into a file. Start and redirect the input of your client from a large file. Compare the input and output to make sure they are identical: $ nc -l localhost 55555 >out & # server on Red Hat systems $ nc -l -q 9 -p 55555 localhost >out & # server on Debian systems $ ./client localhost 55555 out $ cmp /bin/bash out # files should be identical b) Send data (one-way) from fake netcat server to client: Start a fake netcat server and redirect its intput from a large file. Start and redirect the output of your client into a file. Compare the input and output to make sure they are identical: $ nc -v -l localhost 55555 out Connection from 127.0.0.1 port 55555 [tcp/*] accepted <...some prompt for input prints here...>: ./client: DEBUG EOF reading from unit 3 ./client: DEBUG child exit Hangup [1]+ Done nc -v -l localhost 55555 out ./client: DEBUG EOF reading from unit 0 ./client: DEBUG parent exit ./client: DEBUG EOF reading from unit 3 ./client: DEBUG child exit Hangup $ cmp /bin/bash out # should be identical You may also use your own working echo server for this last test. 49. Optional Challenge: Test your client against your own TCP echo server. 50. Exit the script session subshell. Script will tell you the name of your testing.txt script output file. $ exit Script done, file is testing.txt $ less testing.txt You may now examine your saved script session. For full marks, verify that your output file contains *only* the above test commands. No VIM sessions. Not hundreds of lines of DEBUG output. Documentation ------------- 51. In a new file named "README.txt", summarize briefly the results of your testing. For each of the points in this assignment, state whether you succeeded or failed, e.g. you might write this: 1-2. done 3. could not get rid of htonl and htons warnings (harmless) 4-46. done 47. "make clean" still has errors 48-49. all tests passed, script output is accurate for all tests If anything did not work properly, document which tests failed. If everything worked, say so. 52. In the README.txt file, under a heading: Feedback to instructor: A. Was this assignment helpful in achieving the course objectives? B. Tell me how easy/difficult this assignment was. C. How long did it take to complete? 53. Document the code. Did you bring all source files up to programming and comment standards? Are comments relevant? Is indentation consistent? Is the GNU error() function used for all debug, error, and warning output? Reference: Notes file programming_style.txt 54. I haven't asked for any separate User Manual for using the client. Include brief syntax/usage information (including how to run the program) in the comments at the top of the file containing main(). 55. Add assignment headers to the top of all your files and submit the files (see below). Add a header to *every* file you submit. Verify that your testing.txt file contains *only* the above commands. No VIM sessions. Not hundreds of lines of DEBUG output. 56. Submit the files using the exact file names given below. Make sure you submit *all* the files needed to compile your client at the same time! You can't do partial submissions. Submission ---------- Note: The Assignment Label is not a substitute for a proper program file header giving the purpose of this program code. Add a proper file header to all program source code submitted. A. At the top of each and every submitted file, as comments, create an Exterior Assignment Submission label following the directions from last week's lab. Every file must have a label; use comments as needed. B. For material you copy from other sources, credit the author and source, following the directions from last week's lab. C. Submit these text files for marking as Exercise 03 using the following exact and *single* cstsubmit command line: $ ~alleni/bin/cstsubmit 03 Makefile README.txt testing.txt \ client.c sendall.c sendall.h <... other files as needed ...> See last week's lab for details on using cstsubmit. All file names must be spelled *exactly* as given above. Always submit *all* files any time you submit. Incorrect and past-cut-off-time submissions are worth zero marks. If you use any automated test scripts, you may submit those on the same cstsubmit command line. (You can safely submit extra files, as long as you always submit the required files at the same time.) P.S. Did you spell all the assignment label fields and file names correctly?