----------------------- Lab #05 for CST8165 due November 26, 2007 (Week 13) (*NEW DATE*) ----------------------- -Ian! D. Allen - idallen@idallen.ca Remember - knowing how to find out an answer is more important than memorizing the answer. Learn to fish! RTFM! (Read The Fine Manual) Global weight: 8% of your total mark this term. Interim submission: Submit what you have done so far in lab on November 13 Interim submission: Submit what you have done so far in lab on November 20 (*NEW*) Due date: before 23h59 Monday November 26 (Week 13) (*NEW DATE*) Interim submission in Week 11 and Week 12: You will submit whatever progress you have made on this assignment at the end of your weekly lab period in Week 11 and Week 12. (*NEW*) The deliverables for this exercise are to be submitted on-line in the Linux Lab T127 using the "cstsubmit" method described in the exercise description, below. No paper; no email; no FTP. Late-submission date: I will accept without penalty exercises that are submitted late but before 13h00 on Tuesday, November 27. (*NEW DATE*) After that late-submission date, the exercise is worth zero marks. Exercises submitted by the *due date* will be marked on-line and your marks will be sent to you by email after the late-submission date. Exercise Synopsis: Modify an existing multi-client server that uses select() instead of fork(). Test it. This submission is not dependent on any previous labs, though it can use some of the same parsing code and helper functions. Coding and submission standards: - provide File Headers (Program Headers) using my Assignment Label - provide Function Headers documenting arguments and return values - use Block Comments (see programming_style.txt) Where to work: Submissions must make, compile, and run cleanly in the T127 Linux Lab, though you are free to work on them anywhere you like. Recall that most any program or C function has a Unix/Linux manual page: RTFM Code submitted without your added useful comments will not be marked. Code Quality and Portability ---------------------------- A. Your C programs must compile without significant warnings. Some spurious "never be executed" warnings are allowed. B. Your code must work on any machine, no matter what its byte ordering. Always convert bytes between host and network byte order. C. You must include useful comment blocks ahead of the code you write or modify. Code submitted without your added useful comments will not be marked. Credit the sources of any code you are allowed to copy from Internet sources. (Any other copying requires permission.) Reference: Notes file programming_style.txt D. Functions that are not used outside the source file must be declared "static". This prevents global name space pollution, misuse by other modules, and documents to the reader that this function is local to this file. E. Forrbidden Functions: You must not use some functions (e.g. sprintf(), strcpy()) in your code. See the section "Forbidden Functions" in Notes file programming_style.txt for their replacements. F. If you find yourself duplicating code, refactor your source to eliminate the duplication. Perhaps you need to create a (possibly static) function to handle the duplicate code in one place? Move code that is common to both the IF and ELSE clauses to either before or after the statement. Don't write code more than once! G. If you use malloc() or calloc() or friends, your program must pass a "valgrind" test for memory leaks. Try to avoid dynamic memory. Coding a "select" server ------------------------ Class Notes reference: week10notes.txt This lab starts with sample code copied (with credit) from here: http://beej.us/guide/bgnet/output/html/singlepage/bgnet.html 1) Read the section on "6.2. select()--Synchronous I/O Multiplexing" in http://beej.us/guide/bgnet/output/html/singlepage/bgnet.html Note (RTFM) the functions/macros used to manipulate bit vectors in the select() call: FD_ZERO, FD_SET, FD_ISSET, FD_CLR You must know what these macros do. 2) Get the "chat" server from "6.2. select()--Synchronous I/O Multiplexing": http://beej.us/guide/bgnet/output/html/singlepage/bgnet.html#select $ wget http://beej.us/guide/bgnet/examples/selectserver.c Add credit for the source of this code to the source code itself. 3) Integrate the compiling of the selectserver into your Makefile: Update your Makefile to build selectserver from its source. "make selectserver", "make clean", and "make all" should work. The mandatory Makefile CFLAGS for compilation are: CFLAGS = -g -O -Wall -Wextra -Wshadow -Wstrict-prototypes \ -Wmissing-prototypes -Wmissing-declarations \ -Wdeclaration-after-statement -Wmissing-field-initializers \ -Wredundant-decls -Wunreachable-code -fstack-protector-all New (Nov 11, 2007) add to the above: -fstack-protector-all Test the chat server using telnet, netcat, or your own client program. You can have your buddies connect to your server from other machines. $ ./selectserver & [... run one netcat to port 9034 in another window ...] selectserver: new connection from 127.0.0.1 on socket 4 [... run another netcat to port 9034 in another window ...] selectserver: new connection from 127.0.0.1 on socket 5 Clients do not have the input they type echoed back to them. You need at least two clients connected to see any output. 3a) Ignore the SIGPIPE signal in your chat server. Add this line of code near the start of main(): signal(SIGPIPE,SIG_IGN); This will prevent your server from being killed by SIGPIPE (with no error message) if you accidentally write onto an incorrect file descriptor. You will need to make this compile correctly. 3b) Add this one line near the start of main(): alarm(30*60); /* kill program after 30 minutes */ This is in case you forget to kill the program yourself when you leave. To see your background processes on Linux/Unix, use command: ps x 4) The given source code contains nine (9!) levels of indentation and quite rightly says it's "UGLY!". Rework the existing code to cut the indentation levels in half to improve readability. (Hints: See Notes file deep_indentation.txt - use "continue" statements inside loops instead of "else"; create separate functions to handle new connections and clients, etc.) Run "indent -kr -i8" to tidy up. I used some "continue" statements in loops, and two functions, and got my program down to a maximum of four levels of indentation; can you do it?) 5) Modify the chat server to take the listening port as a command-line argument, similar to the way your other server works: $ ./selectserver 55555 & As with your other server, perform some minimal input validation on the port number to make sure it is reasonable. (Re-use your tested code from the previous assignment; don't write new buggy code.) 6) Test the chat server using telnet, netcat, or your own client program. You can have your buddies connect to your server from other machines. $ ./selectserver 55555 & [... run netcat to port 55555 in another window ...] selectserver: new connection from 127.0.0.1 on socket 4 [... run netcat to port 55555 in another window ...] selectserver: new connection from 127.0.0.1 on socket 5 7) Fix all the error messages to correspond to the 4-item checklist from Notes file programming_style.txt. (Copy the messages and use the same error functions you already used in your other server. Don't write new buggy code; re-use the code you already tested.) $ ./selectserver 5 ./selectserver: ERROR on binding to port 5 : Permission denied 8) Remove the hard-coded "selectserver" name from the code and replace it with the actual run-time program name from argv[0]. (You may use one global variable for this. Don't invent your own name; see the non-portable GNU error() function "man 3 error" and the program_invocation_name global variable GNU uses.) 9) Modify the chat server to use your sendall() function on its socket writes. Be careful how you pass arguments to sendall() in a loop: If sendall() encounters an error and fails to write all the requested bytes to a client during one of the loop iterations, it will change the value of the number of bytes passed in (probably to zero), and that could incorrectly affect the number of bytes sent to all subsequent clients in the loop. Make sure this doesn't happen. 10) Make sure your Makefile includes any new source files needed to build the selectserver target. Touching any source or header file should trigger a recompile of the chat server. Test this! 11) Modify the chat server to adjust the value of fdmax downward (if possible) when a client connection terminates and that file descriptor is no longer in use. Make sure you get this right; it isn't a trivial piece of code. (Hint: If clients 4,5,6 connect, fdmax is 6. If clients 4 and 5 disconnect, you cannot decrease fdmax. When client 6 finally disconnects, you can then decrease fdmax to 3.) $ ./selectserver 55555 & [... run netcat in another window ...] ./selectserver: new connection from 127.0.0.1 on socket 4 ./selectserver: socket 4 hung up DEBUG ./selectserver fdmax is now 3 Test this to make sure multiple clients and multiple disconnections all work correctly, no matter which clients disconnect first. 12) Modify the chat server to announce the arrival of a new client to all the other connected clients (if any). (Don't announce the arrival to the client itself.) Include the source IP address (in ASCII dotted-quad format) of the connecting client in the announcement. (See "man inet_ntoa" for a function that will help, and make sure you get your byte ordering correct before you convert.) $ nc -v localhost 55555 localhost [127.0.0.1] 55555 (?) open [... run another netcat in another window ...] ./selectserver: new connection from 127.0.0.1 on socket 5 Note that the chat server already prints this information. All you have to do is send the same information to all connected clients. (Do not duplicate code! Create the information message once and send it both to the screen and to all the clients.) 13) Modify the chat server to announce the departure (EOF or error) of a client to all the other clients (if any). $ nc -v localhost 55555 localhost [127.0.0.1] 55555 (?) open [... run another netcat in another window ...] ./selectserver: new connection from 127.0.0.1 on socket 5 hi [... kill the netcat in the other window ...] ./selectserver: socket 5 hung up Make sure your announcement goes out on both EOF or error; anything that causes the chat server to disconnect the client. The function strerror(errno) may be useful here to find out what went wrong. Don't announce the departure back to the client that has left! Writing on a closed descriptor may cause your program to be killed with a SIGPIPE signal. 14) If your code now has three separate loops that send bytes to connected clients, merge the three loops into a single function. Less code is better code. (You can even make the function a varargs/variadic function, using the models I gave you in Notes file myerror.c.txt.) 15) Modify the chat server to prefix the bytes distributed to all the clients with the IP address of the source of the bytes; so, all messages are always prefixed by their source IP number, e.g. 1.2.3.4: ./selectserver: new connection from 1.2.3.4 on socket 5 1.2.3.4: hello this message comes from me at this address 1.2.3.4: ./selectserver: socket 5 hung up You can do this most easily by sending the IP address to the client before you send the buffer of data to the client. (You could also allocate buffer space and combine the prefix and the data into one buffer and send that; but, then you would need buffer management. If you use malloc() or calloc() or friends, your program must pass a "valgrind" test for memory leaks. Try to avoid dynamic memory.) Make sure you are supplying the IP address of the client that sent the data, not the IP address of the client that connected most recently. To be sure, test this with clients on different machines (with different IP addresses). You might find getpeername(2) useful: http://beej.us/guide/bgnet/output/html/singlepage/bgnet.html#getpeername http://beej.us/guide/bgnet/output/html/singlepage/bgnet.html#getpeernameman Challenge: System calls are expensive. If you use the getpeername() syscall, write a cover function for it that caches (in static storage) the last IP address it returned; so, you don't have to do the syscall if you are looking up the same socket fd that you looked up last time. (This will be a great time-saver in loops that send the same data to many clients.) Be sure to invalidate the cache if that socket fd is closed; otherwise, your cache will return the wrong value if a new client connects with the same fd but from a different IP. 16) Change the chat server status messages about client connections to print on stderr instead of using printf() to stdout. Nothing should be using printf() in the chat server. (Note: stdout is buffered and output will not appear unless a newline is present. stderr is not buffered - anything written there appears right away.) 17) Add the port number of the remote socket to the "new connection" message printed by the chat server: ./selectserver: new connection from 127.0.0.1 port 34141 on socket 4 Make sure you get the remote port bytes in the right order before printing them. You can find out which TCP ports netcat and your selectserver are using via "netstat" or "ss": $ netstat -ntp | grep -w -e selectserver -e nc tcp 0 0 127.0.0.1:55555 127.0.0.1:34141 ESTABLISHED10565/selectserver tcp 0 0 127.0.0.1:34141 127.0.0.1:55555 ESTABLISHED10758/nc $ ss -na | grep 55555 LISTEN 0 0 *:55555 *:* ESTAB 0 0 127.0.0.1:55555 127.0.0.1:34141 ESTAB 0 0 127.0.0.1:34141 127.0.0.1:55555 (You will see only one line of output if your selectserver and netcat are running on different machines.) Verify that the port number shown by netstat matches the port number printed by your chat server. Documentation ------------- 18) Document the code using the guidelines in Notes file programming_style.txt. Bring all source files up to programming and comment standards, even the code you did not write yourself. Add file headers, function headers and block comments to the code (all of it, even the parts you did not write!). Are all the comments relevant? Remove useless comments, e.g. "// listen", "// bind", etc. Document the purpose of all variables when they are declared. Is indentation consistent ("indent -kr -i8")? Review the "Code Quality and Portability" items at the start of this assignment. 19) I haven't asked for any separate User Manual for using the program. Include brief syntax/usage information (including how to run the program) in the comments at the top of the file containing main(). 20) Code submitted without your added useful comments will not be marked. Official Testing ---------------- You are responsible for testing your own program, and demonstrating conclusively that it works. All "script" sessions below must use my script cover: ~alleni/bin/script Never use the native "script" command. Use my version. Note: Your output file will not contain the full results of your script session until you exit the subshell started by the script command. When you exit the subshell, script will tell you the name of the output file. These are the official tests for handing in. Make sure your script sessions contain *only* the required commands below and their output - do not flood the script session with VIM sessions or other unneccessary interactions. Remember that you can throw away unwanted DEBUG output by shell redirection of standard output and/or standard error to /dev/null. The script command has an option to append to a script file, if you want to run the testing below in separate sections. When you are done, you will have four script sessions to hand in (see below): A. testing.txt: for the Makefile and server status and DEBUG messages B. localhost.txt: for a connection from the local host (first machine) C. test1.txt: for a connection from a second different machine D. test2.txt: for a connection from a third different machine 21) (A) Start a ~alleni/bin/script session with output file "testing.txt". Never use the native "script" command. Use my version. 22) (A) "testing.txt": Test the Makefile. Run a full clean and recompile test: $ ~alleni/bin/script testing.txt $ make clean $ make clean # second time should not produce any errors $ make selectserver $ make selectserver # second time should say everything is up to date Use the exact four tests above, in the above order. Make sure you rebuild with all the required CFLAGS given earlier. 23) (A) "testing.txt": Start your server in the foreground (not background): $ ./selectserver 55555 Note: If your server produces many hundreds of lines of debugging output, you must turn that off (or redirect server output to /dev/null) before running this test, so that you don't flood the script session with hundreds of lines of unnecessary DEBUG output. A few lines of DEBUG output are expected as clients connect and disconnect. 24) (B) "localhost.txt": Open a new shell window and start a new "script" session with output file "localhost.txt". Run netcat to your server: $ ~alleni/bin/script localhost.txt $ nc -v localhost 55555 This is your (B) first client connection. 25) (C) "test1.txt": Open a new shell window and start a new "script" session with output file "test1.txt". Login to a (second) different machine in the Linux lab. Run netcat to your server (replace XX and YY by the right numbers): $ ~alleni/bin/script test1.txt $ ssh wt127-YY $ nc -v wt127-XX 55555 This is your (C) second client connection. 26) (D) "test2.txt": Open a new shell window and start a new "script" session with output file "test2.txt". Login to a (third) different machine in the Linux lab. Run netcat to your server (replace XX and ZZ by the right number): $ ~alleni/bin/script test2.txt $ ssh wt127-ZZ $ nc -v wt127-XX 55555 This is your (D) third client connection. 27) At this point verify that you have four script sessions going. A. testing.txt: for the server status and DEBUG messages B. localhost.txt: for a connection from the local host (first machine) C. test1.txt: for a connection from a second different machine D. test2.txt: for a connection from a third different machine 28) Into netcat on the (B) localhost.txt machine, type one line: "this line comes from the (B) localhost test client" Spelling counts - I will be looking for this exact line. 29) Into netcat on the (C) test1.txt machine, type one line: "this line comes from the (C) test1 test client" Spelling counts - I will be looking for this exact line. 30) Into netcat on the (D) test2.txt machine, type one line: "this line comes from the (D) test2 test client" Spelling counts - I will be looking for this exact line. 31) Now interrupt (ctrl-C) the netcat processes on (B) localhost.txt and (C) test1.txt, in that order. Restart them in the reverse order. (Start the netcat on (C) test1.txt first, then do the one on (B) localhost.txt second.) 32) Repeat the typing in steps 28, 29, 30. Spelling counts - I will be looking for the exact lines. 33) Interrupt (ctrl-C) the netcat process on (D) test2.txt and (C) test1.txt, in that order. Restart them: Start the netcat on (D) test2.txt first, then do the one on (C) test1.txt. 34) Repeat the typing in steps 28, 29, 30. Spelling counts - I will be looking for the exact lines. 35) Interrupt (ctrl-C) all the netcat processess in the order (B), (C), (D). Restart them in the order (D), (C), (B). 36) Repeat the typing in steps 28, 29, 30. Spelling counts - I will be looking for the exact lines. 37) Interrupt (ctrl-C) your chat server in session (A). All your netcat processes (B), (C), (D) should exit. Exit from your two remote login sessions. Exit all four script sessions, creating four non-empty script files: A. testing.txt: for the server status and DEBUG messages B. localhost.txt: for a connection from the local host (first machine) C. test1.txt: for a connection from a second different machine D. test2.txt: for a connection from a third different machine 38) In a new file named "README.txt", summarize briefly the results of doing each of the points in this assignment. For each of the 38 points in this assignment, state whether you succeeded or failed that step, e.g. you might write this about each of the 38 steps: 1-9. done successfully 10. Makefile does not build correctly 11-16. done successfully 17. port number not implemented yet 18-21. done successfully 22-37. all official tests passed (except no port numbers) 38. done 39) In the README.txt file, under a heading: Feedback to instructor: Tell me how easy/difficult this assignment was. How long did it take to complete? 40) Add assignment headers to the top of all your files and submit the files (see below). Add a header to *every* file you submit, including each script session. Verify that your script files contain *only* the above testing commands. No VIM sessions. Not hundreds of lines of DEBUG output. Submission ---------- Note: The Assignment Label is not a substitute for a proper program file header giving the purpose of this program code. Add a proper file header to all program source code submitted. A. At the top of each and every submitted file, as comments, create an Exterior Assignment Submission label following the directions from last week's lab. Add this header to *every* file you submit. B. For material you copy from other sources, credit the author and source, following the directions from last week's lab. C. Submit these text files for marking as Exercise 05 using the following exact and *single* cstsubmit command line: $ ~alleni/bin/cstsubmit 05 Makefile README.txt \ testing.txt localhost.txt test1.txt test2.txt \ selectserver.c myerror.c myerror.h sendall.c sendall.h See last week's lab for details on using cstsubmit. All file names must be spelled *exactly* as given above. Incorrect submissions are worth zero marks. If you use any automated test scripts, you may submit those on the same command line. (You can safely submit extra files, as long as you always submit the required files at the same time.) P.S. Did you spell all the assignment label fields and file names correctly?