------------------------- Week 03 Notes for CST8165 ------------------------- -Ian! D. Allen - idallen@idallen.ca Remember - knowing how to find out an answer is more important than memorizing the answer. Learn to fish! RTFM! (Read The Fine Manual) Client/Server Programming ------------------------- Review: low-level Unix system calls: - socket,bind,listen,accept,read/recv,write/send,close - see last week's notes Q: what IP address is this (as a dotted quad): int ipaddr = -1; ? Q: what are the basic inputs and outputs of the Unix syscalls: socket,bind,listen,accept,read/recv,write/send,close Unix read/recv and write/send system call return values: - for low-level I/O syscalls such as read() and write() that return an integer: - a return of less than zero means an error - the error reason is put in errno; use perror() to print it - a return of zero means EOF when reading - no more reading can be done after EOF is seen - a return of zero means nothing was written when writing - this is not an error: you may need to loop to write everything - see the sendall() function below under "writing to network sockets" - a return of > 0 means you did read or write (some) of the data - you may not have read or written *all* of the data! - see the sendall() function below under "writing to network sockets" - EOF is not an error - never call perror() after seeing EOF Note that a *successful* system call may or may not change errno: - see "man 3 errno" - errno is only set for sure after a system call *fails* - errno is *undefined* after a successful syscall - Thus, you cannot test errno to know if a system call failed - Thus, the perror() function is only usable on the most recent syscall. - If you execute other syscalls (e.g. using printf()), they will overwrite errno and you will lose the preceding syscall error. - The following perror() is incorrect, since printf() may overwrite errno: n = write(...); /* the syscall we want to check */ printf("%d bytes read\n", n); /* another successful syscall */ if ( n < 0 ) perror("write failed"); /* WRONG CHECK OF ERRNO */ - Read the NOTES section of "man errno" for how to save/restore errno across a call to another system call, e.g. printf() Q: Why can't I use a printf() before calling perror? * functions that take multiple arguments (e.g. printf) Do you know how to write va_list (variadic, varargs) functions? I highly recommend converting myerror() to varargs, and also creating a varargs myperror() that prints but does not exit (or modify myerror() not to exit and add exit() calls where needed). The two functions can share the same varargs base function vmyperror(fmt,ap) that uses a va_alist argument, in the manner of vfprintf(). You might find such functions useful for printing error messages; since, they avoid the need for buffers and snprintf(). Details: http://www.gnu.org/software/libc/manual/html_node/Variadic-Functions.html See the Notes file: myerror.c.txt Pedantic Coding --------------- We used htons(portno) but not htonl(INADDR_ANY) - why? - Look at: http://beej.us/guide/bgnet/output/htmlsingle/bgnet.html#bind and find the paragraph starting "If you are into noticing little things, you might have seen that I didn't put INADDR_ANY into Network Byte Order! Naughty me.". Read the fix; fix your own code. - If you don't fix this, then when you later use some other value than INADDR_ANY here, your code will break. The code is wrong; fix it now! Q: Why isn't the short int AF_INET put into network byte order? my_addr.sin_family = AF_INET; // host byte order serv_addr.sin_addr.s_addr = htonl(INADDR_ANY); // long, network byte order my_addr.sin_port = htons(MYPORT); // short, network byte order - "man bind" refers us to "man 7 ip" which contains these lines: sa_family_t sin_family; /* address family: AF_INET */ u_int32_t s_addr; /* address in network byte order */ u_int16_t sin_port; /* port in network byte order */ "Note that the address and the port are always stored in network byte order. In particular, this means that you need to call htons(3) on the number that is assigned to a port. All address/port manipulation functions in the standard library work in network byte order." - the sin_family is never sent over the network; it doesn't have to be in network byte order Q: Why doesn't the sin_family = AF_INET need to use htonl() or htons()? See "man bind" for the correct cast to use on the second argument to bind() and connect(): "The only purpose of this structure is to cast the structure pointer passed in my_addr in order to avoid compiler warnings." Do you mean AF_INET or PF_INET ? - from "man socket" "The manifest constants used under 4.x BSD for protocol families are PF_UNIX, PF_INET, etc., while AF_UNIX etc. are used for address fami- lies. However, already the BSD man page promises: "The protocol family generally is the same as the address family", and subsequent standards use AF_* everywhere." Q: T/F PF_INET and AF_INET are effectively the same thing everywhere The truth about keyboard ^D and end-of-file (EOF) Typing the character ^D at your keyboard does not actually signal EOF to a process. What ^D does is act somewhat like pushing the RETURN key, in that whatever characters have been typed since the last RETURN are sent to the receiving process. Unlike pushing the RETURN key, ^D does not send a newline - it simply flushes the characters. If there are no characters to flush, e.g. ^D is typed right after starting your process or right after pushing RETURN, then the ^D sends zero characters to the process. When a process reads zero characters, it interprets that to mean EOF. If you type a few characters on your keyboard and then type ^D instead of RETURN, the ^D sends those few characters to the process without a newline on the end. If you then immediately type a second ^D, the second ^D sends zero characters, and the process interprets that read of zero characters as EOF. The ^D means "send now", and if you send zero bytes, that's interpreted as EOF. Details on socket programming ----------------------------- Reference: http://beej.us/guide/bgnet/output/htmlsingle/bgnet.html Q: Which port numbers can only be used by the super-user on Unix/Linux? Do all operating systems restrict these low-numbered ports? http://www.freesoft.org/CIE/RFC/1122/87.htm (Note the above URL has the Well Known Port range wrong!) What is the IANA name for this port range? Q: Under what circumstances can one omit a call to bind() a socket? What happens when you connect() using such an unbound socket? Give an example of an application that does this. Ref: http://beej.us/guide/bgnet/output/htmlsingle/bgnet.html#bind Q: What happens if you forget to call bind() before you call listen() in a server? Does your server program fail to start up? Can clients connect to such a server? (Why/how, or why not?) Q: True/False - after a call to accept() you have *two different* open socket file descriptors. Q: True/False - if you close the socket descriptor that is the return value from an accept(), you also close the original socket() descriptor (and vice-versa - they are the same descriptor). Q: You want a server to accept only a single client. True/False - after the accept() call, you can close the original socket file descriptor and use only the one returned by the accept() call. Writing to Network sockets -------------------------- Writes to sockets can be incomplete! The system may not write all the bytes you asked. You need to loop to send the remaining bytes. - http://beej.us/guide/bgnet/output/htmlsingle/bgnet.html#sendall - can "*len" be replaced by "len" in this function? - BUG: what is returned by sendall() if *len <= 0 ? - why can't sendall() just return the number of bytes written? - if you want to use sendall() to write to a file descriptor that is not a socket, you must ensure that sendall() uses write() and not send(). Q: T/F writes to network sockets may only write some of the requested bytes Q: T/F the sendall() function may write some bytes but still return -1 indicating an error Q: why does the sendall() function need both a return value and a pass-by-reference number of bytes written? Q: under what circumstances will sendall() indicate a positive number of bytes written but still return -1 indicating failure? Q: T/F the opposite of x > 0 is x < 0 ? Q: T/F if(x>0) is the same as if(!(x<0)) ? Note: Reading from network sockets can also be incomplete. We'll talk about handling that later. Clients and Servers ------------------- Diagram: http://community.borland.com/article/0,1410,26022,00.html Q: In one column list in flow-chart form the Unix system calls made to set up a TCP/IP server that loops accepting clients, forking children that each read one packet, write one packet, and exit. In a parallel column list the system calls used in a TCP/IP client that sends one packet and receives one packet then exits. Connect the two columnts with arrows, showing the relationship of the system calls and the direction of data travel. Google for more TCP/IP client server socket examples: - http://www.perl.com/doc/manual/html/pod/perlipc.html - http://beej.us/guide/bgnet/ Coding ------ I reviewed the work to be done in Lab #3: See Notes: programming_style.txt * Server modifications: - the conversion of server2.c to a looping echo server: - I showed PDL for server2.c and the revised PDL for the converted server * Client modifications: Reference: http://www.cs.rpi.edu/courses/sysprog/sockets/sock.html The client.c code is explained line-by-line in the above web page. - reorganize the command line argument parsing in front of the socket code - add a check for a valid port number that is within range - replace the deprecated bzero() and bcopy() functions - I discussed the use of connect() in client.c - fix the error message to say what host and port failed - fix the prompt - detect errors and EOF when reading standard input - use shutdown() to half-close the socket when finished writing to the server See Notes: eof_handling.txt - more updates to do, see Notes: lab03.txt * Error messages should only show information from the command line if that information is relevant to the error: - errors from bind() and connect() can depend on command line arguments - the error messages must include the user's supplied arguments - errors from socket() and listen() have nothing to do with the command line - the error messages do not need to show command line arguments * Writing a looping process that reads one fd and writes to another fd: - know before you code: what are the terminating conditions for the loop? - for clients/servers you need to terminate on these three conditions: - when reading the fd: (1) break loop on errors, and (2) break loop on EOF - when writing the fd: (3) break loop on errors - start coding a loop with "while(1)" - don't worry about putting any conditions in the while loop test at the top until you're done the loop. Perhaps the loop will be cleaner if each of the terminating conditions uses "break" in the body of the loop and you keep "while(1)" at the top? /* this loop has three terminating conditions, as given above */ WHILE 1 READ some data into a fixed-size buffer IF read error THEN print error message and break loop /* (1) */ IF end-of-file THEN print EOF message and break loop /* (2) */ ASSERT( numread > 0 ) /* see below for meaning of assert() */ WRITE from buffer the data that was read IF write error THEN print error message and break loop /* (3) */ END WHILE The "WRITE from buffer" should use the sendall() function, to make sure all the bytes are written. Only write the number of bytes that were read; don't write the whole buffer! Q: Write the PDL for any process that wishes to read data from one place and write it to another place. - what are the three terminating conditions for the loop? Q: How many of the above loops are coded in an echo server (server2.c)? Q: How many of the above loops are coded in a client (client.c) that reads from your keyboard and writes to a server, and reads from the server and writes to your screen? * Checking for internal consistency using the assert() macro: http://www.cs.utah.edu/dept/old/texinfo/glibc-manual-0.02/library_28.html "When you're writing a program, it's often a good idea to put in checks at strategic places for "impossible" errors or violations of basic assumptions. These checks are helpful in debugging problems due to misunderstandings between different parts of the program." - mentions the assert() macro that will abort your program, printing the file and line number where the abort happened - use assert() ("man assert") to find bugs in your program - e.g. assert( numread > 0 ); Q: how does the assert() macro work? References to Notes files (required reading): ------------------------- programming_style.txt eof_handling.txt Optional (but useful): myerror.c.txt