----------------------- Lab #08 for NET2003 due March 26, 2007 (* NEW DATE *) ----------------------- -Ian! D. Allen - idallen@idallen.ca Remember - knowing how to find out an answer is more important than memorizing the answer. Learn to fish! RTFM! (Read The Fine Manual) Global weight: 4% of your total mark this term. Due date: before 10h00 Monday March 26, 2007 (* NEW DATE *) The deliverables for this lab exercise are to be submitted online on the Course Linux Server using the "netsubmit" method described in the lab exercise description, below. No paper; no email; no FTP. Late-submission date: I will accept without penalty lab exercises that are submitted late but before 12h00 (noon) on Wednesday, March 28. (*NEW*) After that late-submission date, the lab exercise is worth zero marks. Lab exercises submitted by the *due date* will be marked online and your marks will be sent to you by email after the late-submission date. Lab Synopsis ------------ Create UML machines in a router chain with three networks. Set up routing for all four machines so every machine can ping every other machine. Set up routing so that all the UMLs can ping the VNS (and vice-versa). Add Network Address Translation so that all the UMLs have masqueraded Internet access. (The initials "ALN" refer to your required course textbook - Advanced Linux Networking.) Resources and Hints ------------------- The UML machines come with "tcpdump". To dump packets on eth1, without DNS lookups and without time stamps, use: # tcpdump -t -n -i eth1 The man page lists other useful options such as "-e" to show the MAC adddresses in the packets. You can diagnose 99% of network problems by knowing (1) what is my IP address, (2) what is my network mask, (3) what is the address of my gateway machine. The tcpdump command will help you see packets entering/leaving each interface card. The ifconfig command defaults your netmask and broadcast to the old *obsolete* Class of the IP address you type. Always specify *both* the netmask *and* the broadcast address explicitly when you use ifconfig: # ifconfig 1.2.3.1 netmask 255.255.255.128 broadcast 1.2.3.127 The ifconfig command only understands netmask, not /24 prefix notation. The ipcalc command (RTFM) is useful for turning nework prefixes into masks. You can also install ipv4calc by using "apt-cache search ipv4calc" to find the correct package name to give to apt-get. You may use the old "route" syntax or the new "ip route" syntax to set up routes (ALN p.46). These are equivalent: # route add -net 172.20.0.0 netmask 255.255.0.0 gw 172.21.1.1 # ip route add 172.20.0.0/16 via 172.21.1.1 Create a shell script named uml.sh (located on your virtual disk, so you don't lose it!) to set up each UML machine; so, if you have to reboot you can simply re-run the script to get the machine configured. If you don't use a script, and the system reboots, you'll have to start everything over from scratch. You can create one script with all the configuration lines in it and have the script select the correct configuration lines for a machine using an IF or CASE statement, e.g. hostname=$( hostname ) case "$hostname" in red*) echo "Do this for Red only" ifconfig eth1 10.1.1.1 netmask 255.255.255.0 broadcast 10.1.1.255 # ...more lines for Red go here... ;; green*) echo "Do this for Green only" ifconfig eth1 10.1.1.2 netmask 255.255.255.0 broadcast 10.1.1.255 # ...more lines for Green go here... ;; yellow*) echo "Do this for Yellow only" # ...enter config info for Yellow here... ;; magenta*) echo "Do this for Magenta only" # ...enter config info for Magenta here... ;; *) echo 1>&2 "$0: Error - unknown machine name '$hostname'" exit 1 ;; esac One uml.sh script is easier to maintain and keep consistent than four. You can run this script to configure each machine after you boot each machine. Where do you put this script so that all the UMLs can use it? On your virtual disk, mounted to all the UML machines. See the next section: Part 0 - Re-aiming /host to /mnt/hdb1 ------------------------------------- When you start a UML machine, the UML machine mounts on /host whatever directory /host points to on the base VNS machine. Every UML then shares the same /host with the VNS and with all the other UML machines. This allows the VNS and UMLs to all share a directory; unfortunately, the default is for the VNS /host to be a symlink that points to /home/root, a directory that resides in Knoppix memory. We want /host to point to our virtual disk at /mnt/hdb1/ before we start any UML machines. The VNS system boots up with the /host symlink pointing to /home/root. You can use the VNS UserModeLinux sub-menu "/host" to change the /host symlink to point to /mnt/hdb1; or, you can simply remove the symlink and recreate it yourself, before you start up any UML machines. (Do this in your VNS setup.sh script.) After you fix the symlink, you will see: # ls -l /host lrwxrwxrwx 1 root root 10 Mar 10 12:34 /host -> /mnt/hdb1 Now, when you start a UML, it will have your /mnt/hdb1 virtual disk mounted as its /host directory for sharing files. Any edits you do on your virtual disk are immediately visible to the VNS and all the running UML machines. 0) You should put this recreation of the VNS /host symlink into your VNS setup.sh script, after the mounting of /mnt/hdb1, so that every time you set up your VNS it points /host to your virtual disk, allowing your virtual disk to be shared by all your UML machines. Part I - Basic Routing ---------------------- 1) Go back and read the Resources and Hints sections you just skipped over. In particular, re-aim your VNS /host symlink to point to your mounted virtual disk on /mnt/hdb1 so that you can run your UML configuration script uml.sh from there. 2) All UML machines start with one hub attached to eth1 and a second attached to eth2. Start a third hub, either using the UML menu "ManageHubs", or by running "umlHub" from the command line of the VNS base system. This third hub will automatically be attached to all eth3 interfaces in all UMLs. (If you start a fourth hub, it is attached to eth4 in all UMLs, etc.) 3) Create UMLs Red, Green, Yellow, and Magenta, either from the UML menu or by running "uml Name &" in the background. Arrange the UML machine windows so that you can access each one conveniently. 3b) Make sure you can see the third hub on eth3 in all the UML machines: # netstat -ia | grep eth ...shows eth0, eth1, eth2, eth3 # ifconfig -a | grep eth ...shows eth0, eth1, eth2, eth3 Create a UML configuration script named "uml.sh" to do the following configuration of the UML machines once you have them booted. Read the Resources and Hints, above. In the steps below, some things should be added to your UML script and some things are command lines that you should type after you run your script to make sure the script works. Command lines (lines that should not be put into the script, except perhaps as debugging) are prefixed with a shell prompt, i.e. "#". Don't put these command lines in the script. 3c) In the new uml.sh script, we want to make sure this script only executes on a UML machine. (We don't want the script to run on the base VNS machine, where it might damage the existing network.) The device /dev/ubd0 only exists in a UML; make use of that: Run "df" on the pathname /dev/ubd0 - if this fails, print a good error message (this is not a UML machine) and exit the script with a non-zero exit status. Throw away the standard output and error output of "df"; we only care about the exit status. 4) Have uml.sh remove the 10-second delay loop from the current UML machine (unless you like waiting while it reboots). (See last lab.) If you find your UML machines hang on boot, put back the delay. 4b) Have uml.sh flush any existing network routing from the "main" routing table before we add any new routes for a particular UML: # for Red, flush old routes before using ifconfig or adding new ones ip route flush table main [...] ifconfig eth1 [...] ip route replace default [...] After you do the flush, "route" and "ip route" will show no routes defined. This flush should be done for each of the UML machines in your script *before* you call ifconfig or add any new routes. Do *not* change or flush routes for UML machines that are not defined in your script; leave unknown machines unchanged. 4c) Have your UML script check for the existence of string "eth3" in the output of "netstat -ia". If the string isn't found, print a good error message (tell the user to start the missing hub) and exit the UML script. (Optional: You could also examine the shell script /usr/local/bin/umlHub and copy the code that starts the third hub.) 5) Using ifconfig, put Red and Green on the net 10.1.1.0/24 via eth1: - have uml.sh give Red IP address 10.1.1.1 - have uml.sh give Green IP address 10.1.1.2 - using ifconfig automatically creates routes for interfaces - test it: make sure you can ping both ways 6) Using ifconfig, put Green and Yellow on the net 10.2.2.0/24 via eth2: - have uml.sh give Green IP address 10.2.2.3 - have uml.sh give Yellow IP address 10.2.2.4 - using ifconfig automatically creates routes for interfaces - test it: make sure you can ping both ways 7) Using ifconfig, put Yellow and Magenta on the net 10.3.3.0/24 via eth3: - have uml.sh give Yellow IP address 10.3.3.5 - have uml.sh give Magenta IP address 10.3.3.6 - using ifconfig automatically creates routes for interfaces - test it: make sure you can ping both ways Note that the act of bringing up an interface with an IP address using ifconfig automatically generates a route for that network interface. Note that Green and Yellow will each be on two networks and have two IP addresses each. Hub1 handles the Red/Green net via eth1. Hub2 handles the Green/Yellow net via eth2. Hub3 handles the Yellow/Magenta net via eth3. 7b) Make sure that you don't bring up interfaces that are not being used! Add lines to your uml.sh script to explicitly set *all* interfaces down before you bring any of them up again. You must configure as down even interfaces that you will bring up; since, the kernel will not add routing for an interface unless the interface was brought up by the configuration. Set *all* interfaces as down, e.g.: # for Magenta, only eth3 should be up ifconfig eth0 down ifconfig eth1 down ifconfig eth2 down ifconfig eth3 down ifconfig eth3 10.3.3.6 ...etc... If you don't do this, your routing will get very confused as packets sneak across using the other hubs. Set your interfaces down in each machine before you apply a new configuration to them. Do *not* change the status of any interfaces for UML machines that are not defined in your uml.sh script; leave unknown machines unchanged. 8) Make sure you have flushed existing routes before running ifconfig or adding new routes (see "ip route flush" above). Have uml.sh set the default route for Red to be Green. (ALN p.44) Have uml.sh set the default route for Green to be Yellow. (ALN p.44) Have uml.sh set the default route for Yellow to be Magenta. (ALN p.44) The text (ALN p.44) is wrong about the IP 0.0.0.0 being "precisely the same" as the keyword "default" for the "route" or "ip route" commands, at least for Debian. On Debian, "route 0.0.0.0" or "ip route 0.0.0.0" are the same as "route -host 0.0.0.0" or "ip route 0.0.0.0/32" - a route to a single machine with address 0.0.0.0. Using the keyword, "route default" or "ip route default" are the same as "route -net 0.0.0.0" or "ip route 0.0.0.0/0" - a route to the network 0.0.0.0/0 (a proper "default" route). Use the keyword "default" to be safe. (ALN p.44) If a gateway machine has two IP addresses (e.g. Green, Yellow), how do you know which is the right address to use for the default route (gateway)? Pick the gateway address that is on the *local* network. A gateway machine must be reachable on the local network. 8b) Verify that "ip route" shows a "default" route listed for the machines you set, above. If you see a route labelled 0.0.0.0 in the output of "ip route", then you got the route prefix or netmask wrong when you created the default route. Delete the mistake and add the correct default route with the right prefix or netmask. (ALN p.44 "On rare occasions, you must...") You should see this: # ip route [...] default via ?.?.?.? dev eth? # route [...] default ?.?.?.? 0.0.0.0 UG 0 0 0 eth? If you see 0.0.0.0 instead of "default", or "UGH", or a Genmask of 255.255.255.255, then you have created a "host" route (a route to a single machine), not a default route to a network of machines. Make sure you create a route to a network. (See the previous Step comments about "default".) 9) Have uml.sh enable routing/forwarding on all machines. (ALN p.44) Without this, packets will not cross between interface cards. Make sure your UML script executes this for all machines. 9b) Verify that your script enabled routing between cards in every UML: # hostname red # repeat and verify for all UMLs # cat /proc/sys/net/ipv4/ip_forward 1 10) "Martian" packets are packets that arrive with a source address that doesn't match the routing information for that interface, i.e., a reply to a martian packet would not work. We want to log this. Have uml.sh enable reporting of "martian" packets on all UML machine interfacess: echo 1 >/proc/sys/net/ipv4/conf/all/log_martians This will assist you in knowing that you haven't got routing correct. Make sure your uml.sh script executes this for all machines. 10b) Verify that your script enabled martian reporting in every UML: # hostname red # repeat and verify for all UMLs # cat /proc/sys/net/ipv4/conf/all/log_martians 1 11) Have uml.sh add explicit return routes so that every machine can ping every other machine (e.g. so that Red can ping Magenta and vice-versa) as follows: (see ALN p.46) a) On Yellow, add a route to Red/Green's network via gateway Green. - make sure Red, Green, and Yellow can all ping each other b) On Magenta, add a route to Red/Green's network via gateway Yellow. On Magenta, add a route to Green/Yellow's network via gateway Yellow. - make sure all four machines can now ping each other If a machine has two IP addresses, and is on two networks, how do you know which is the right network address and right gateway address to use? Always choose the address that is on the *local* network and on the *near* side of a router. 12) Have uml.sh set up an /etc/hosts file on each UML machine using this script fragment: # define some IP address variables for UML ether cards 1, 2, and 3 red1=10.1.1.1 # eth1 green1=10.1.1.2 # eth1 green2=10.2.2.3 # eth2 yellow2=10.2.2.4 # eth2 yellow3=10.3.3.5 # eth3 magenta3=10.3.3.6 # eth3 # define host names for IP addresses (e.g. for ping and tcpdump) cat >/etc/hosts </etc/hosts output file. The "EOF" must be the only word on the line, at the left margin, with no spaces before or after it. Shell variables are expanded as the lines are copied to the output file. Look at /etc/hosts after you run the script. 13) Make sure you can now ping by name, e.g. "ping red1", "ping yellow3" 13b) You can use the above variable names in your script, instead of the raw IP addresses, if you think it makes your script more readable. Part II - routing between the UMLs and the VNS ---------------------------------------------- The "Summary" at the end of this section gives a summary of the modifications that need to be made to the VNS and UML set-up scripts to enable routing between the UMLs and the base VNS. We will be experimenting with and diagnosing mis-routed packets in this lab. Linux has features for deleting packets with unusual characteristics, and we must turn off those features to be able to see how the packets flow. 14) The "rp_filter" check will silently discard any packet coming from an interface where a reply packet would not return via that same interface. Have your VNS and UML scripts disable silent filtering of misrouted packets on all VNS and UML machines: echo 0 >/proc/sys/net/ipv4/conf/all/rp_filter This will assist you in knowing that you haven't got routing correct. Make sure your UML script executes this for all machines. Also add it to your VNS set-up script setup.sh. Reference: /usr/src/linux/Documentation/networking/ip-sysctl.txt Reference: http://tools.ietf.org/html/rfc1812#section-5.3.8 15) Run your UML script and verify that your UML script disabled filtering in every UML: # hostname red # repeat and verify for all UMLs # cat /proc/sys/net/ipv4/conf/all/rp_filter 0 16) Run your VNS setup.sh script and verify that your VNS script disabled filtering in the VNS: # hostname vns # cat /proc/sys/net/ipv4/conf/all/rp_filter 0 17) Have your VNS and UML scripts verify that filtering is disabled: - put the output of "cat /proc/sys/net/ipv4/conf/all/rp_filter" into a shell variable - test that the value of the variable is zero - print a good error message and exit the script if necessary 18) As you did with the UMLs, enable "log_martian" and "ip_forward" for your VNS machine. Add this enabling code to your VNS setup.sh script and re-run the script in the base VNS. 19) Verify that "log_martian" and "ip_forward" are enabled in your VNS: # hostname vns # cat /proc/sys/net/ipv4/conf/all/log_martians 1 # cat /proc/sys/net/ipv4/ip_forward 1 20) Have your VNS setup.sh script verify that "log_martian" and "ip_forward" are enabled using the same method as described above. Your UML machines still can't reach the Internet; because, among other reasons, the final router in the chain, Magenta, doesn't have a default route that leads there. Packets to non-local networks follow the chain of default gateways until they hit Magenta, at which point they get lost: 21) On Red, run "traceroute -n" to some external address and verify that the route breaks at Magenta: # hostname red # traceroute -n 72.18.159.15 traceroute to 72.18.159.15 (72.18.159.15), 30 hops max, 38 byte packets 1 10.1.1.2 0.997 ms 0.697 ms 2.869 ms 2 10.2.2.4 1.230 ms 1.070 ms 1.048 ms 3 10.3.3.6 1.757 ms !N 2.110 ms !N 1.548 ms !N Note the "!N" flags on the last line, indicating "no route to network" from that machine. You can also allow name lookups on the trace: # hostname red # traceroute 72.18.159.15 traceroute to 72.18.159.15 (72.18.159.15), 30 hops max, 38 byte packets 1 green1 (10.1.1.2) 1.145 ms 0.925 ms 0.684 ms 2 yellow2 (10.2.2.4) 42.162 ms 1.646 ms 1.074 ms 3 magenta3 (10.3.3.6) 5.683 ms !N 1.765 ms !N 9.345 ms !N Let's fix the default route for Magenta to lead to our VNS base machine, which has access to the Internet. Every UML machine may be connected to the base VNS by bringing up preconfigured network interface eth0 in the UML. (No need for an address - the eth0 interface is preconfigured in each UML.) The UML end of the network appears as eth0; the base VNS end of the connection appears as one of the "tap" devices in the VNS, one "tap" for each running UML. Since we have four running UMLs, our VNS base has four tap interfaces active: # hostname vns # netstat -ia | grep tap tap1 1500 0 0 0 0 0 0 0 5 0 BMRU # this leads to Red tap2 1500 0 0 0 0 0 0 0 5 0 BMRU # this leads to Green tap3 1500 0 0 0 0 0 0 0 5 0 BMRU # this leads to Yellow tap5 1500 0 0 0 0 0 0 0 5 0 BMRU # this leads to Magenta Every tap device in the base VNS has an IP address and a route installed: # hostname vns # ip route | grep tap5 192.168.245.0/24 dev tap5 proto kernel scope link src 192.168.245.1 Using the above route, any packets in the base VNS destined for 192.168.245.0/24 will go out via tap5 to Magenta. Let's connect the Magenta end of the network to the base VNS using Magenta's preconfigured eth0: 22) Have your uml.sh script bring up interface eth0 on Magenta only, using ifconfig. Don't assign eth0 an address or mask - it already has one: 192.168.245.2. Your VNS base machine is at the other end of that network using interface tap5 with address 192.168.245.1 Note that the act of bringing up an interface with an IP address using ifconfig automatically generates a route for that network interface. 23) On Magenta, verify that eth0 has address 192.168.245.2. Make sure Magenta can ping your VNS base machine using the IP address of tap5, connected to the other end of eth0 (tap5 is at 192.168.245.1). Make sure your VNS base machine can ping Magenta at the IP address of Magenta's eth0 interface (eth0 is at 192.168.245.2). 24) On your base VNS machine, run tcpdump on tap5. On Magenta run the above ping of VNS interface 192.168.245.1. The VNS tcpdump output will be similar to this in response to the pings from Magenta: # hostname vns # tcpdump -t -l -n -i tap5 listening on tap5, link-type EN10MB (Ethernet), capture size 96 bytes IP 192.168.245.2 > 192.168.245.1: icmp 64: echo request seq 1 IP 192.168.245.1 > 192.168.245.2: icmp 64: echo reply seq 1 IP 192.168.245.2 > 192.168.245.1: icmp 64: echo request seq 2 IP 192.168.245.1 > 192.168.245.2: icmp 64: echo reply seq 2 Note that even though Magenta has three network interfaces, it chooses 192.168.245.2 as the source address to send the ping requests to the VNS. When the VNS replies to the ping requests, the replies naturally have a destination address of 192.168.245.2 and the VNS route for tap5 leads the replies back to Magenta. 25) Leave the tcpdump running on the base VNS. Kill the ping from Magenta and start it up on Yellow instead. Ping the same tap5 VNS address you used from Magenta: 192.168.245.1. The tcpdump now shows this: # hostname vns # tcpdump -t -l -n -i tap5 listening on tap5, link-type EN10MB (Ethernet), capture size 96 bytes IP 10.3.3.5 > 192.168.245.1: icmp 64: echo request seq 1 IP 10.3.3.5 > 192.168.245.1: icmp 64: echo request seq 2 IP 10.3.3.5 > 192.168.245.1: icmp 64: echo request seq 3 IP 10.3.3.5 > 192.168.245.1: icmp 64: echo request seq 4 We see the pings coming from Yellow3 into the VNS base via tap5; but, the echo replies don't go back out tap5 again. Why not? Let's find out if the ICMP echo replies are going somewhere else: 26) Leaving the pings running from Yellow, kill the tcpdump on tap5 in the base VNS and restart tcpdump on eth0 on the base VNS. You should see something like this: # hostname vns # tcpdump -t -l -n -i eth0 listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes IP 192.168.245.1 > 10.3.3.5: icmp 64: echo reply seq 1 IP 192.168.245.1 > 10.3.3.5: icmp 64: echo reply seq 2 IP 192.168.245.1 > 10.3.3.5: icmp 64: echo reply seq 3 IP 192.168.245.1 > 10.3.3.5: icmp 64: echo reply seq 4 Here are the lost ICMP echo replies, on eth0! The VNS is sending the replies out the default route eth0 instead of back out tap5. Of course this makes sense, given the source address of the incoming ICMP echo requests - 10.3.3.5 (Yellow3). When the VNS creates an echo reply to destination address 10.3.3.5 the VNS routing table has no specific route to that network; so, the packet heads for the default route for the VNS base system and so it goes out eth0, not tap5. This illustrates a key point in network routing that is easy to forget - just because a packet came in via one interface doesn't mean that the reply packet must leave by that interface (even though we want it to!). A reply packet is routed independently of the source packet that triggered it. You have to get the routing right to make sure that reply packets return to the interfaces from which the source packets came. Many routers (and the Linux kernel) will silently drop incoming packets if replies to those packets would not return by the same interface - this is a security measure called "avoiding source spoofing". We have already turned off this silent drop feature by disabling rp_filter. To fix our broken routing on the VNS base, we could add an explicit route to Yellow's network so that replies to packets coming from Yellow go out via Magenta (don't do this): # ip route add 10.3.3.0/24 via 192.168.245.2 # Yellow via Magenta But, this only works for Yellow's network. It doesn't work for machines such as Green and Red. We could add routes for them, too (don't do this): # ip route add 10.2.2.0/24 via 192.168.245.2 # Green via Magenta # ip route add 10.1.1.0/24 via 192.168.245.2 # Red via Magenta But, this won't work if we add more networks behind Magenta. Instead of adding three separate routes to Yellow, Green, and Red, we choose to "aggregate" the 10.0, 10.1, 10.2, and 10.3 networks using a single network 10/14 and return any packets destined for any machine on 10/14 via Magenta (*NOTE: this section was changed from using 10/8 to 10/14 on March 22; because, Algonquin college uses many 10/8 addresses): 27) On the VNS base, instead of adding three routes, add just one route that sends everything destined for 10/14 via Magenta's eth0 interface at 192.168.245.2. Hint: If you use "ip route replace" instead of "ip route add", you won't get a "RTNETLINK answers: File exists" error if you run this command more than once. 27b) Ideally, we should put this 10/14 route into a script so that it happens automatically. We can't put it into the VNS setup.sh script, since the VNS setup.sh script should run *before* we create any UML machines, and thus before the interface to Magenta even exists. Create a small startuml.sh script that uses the "uml" command to start the four UML machines (red, green, yellow, magenta), each in the background, and then sets the correct 10/14 route. You can run this startuml.sh script after running your VNS setup.sh script. (Note that you can safely run startuml.sh more than once - starting a uml machine that is already started does nothing.) The script needs to wait for the tap5 device to be created before setting the route: uml red & uml green & uml yellow & uml magenta & # tap5 is the interface to Magenta; wait for it while ! ifconfig tap5 >/dev/null 2>&1 ; do sleep 1 done ip route replace [...] You will either need to omit setting the PATH in this script (so that the script uses the inherited PATH); or, you will need to add the missing directories to the PATH statement that contain commands such as "uml", "xterm", and "xhost". The commands that locate where these other commands are found were covered in Week 6. 27c) At the beginning of startuml.sh, put in a test and exit to make sure that this script is *not* run on a UML system. (This is the opposite test to the one you added to the start of uml.sh.) 28) Confirm that all four UML machines can now ping 192.168.245.1, and that the VNS base can ping any address on all four UML machines. Let's see if we can also ping one of the other tap interfaces in the base VNS from our UML machines: 29) On each of Red, Green, Yellow, and Magenta, try pinging the tap1 interface in the base VNS (IP address 192.168.241.1). Confirm that Red, Green, and Yellow all return "Destination net Unreachable", and Magenta says "connect: Network is unreachable". A tcpdump run on tap5 on the base VNS will show no activity during these pings. No packets are being sent into the VNS through the tap5 connection from Magenta. What is the problem? The big clue is that even Magenta doesn't know how to send to IP address 192.168.241.1. We never set a default route for Magenta. We want Magenta to forward all packets to the base VNS. Fix it: 30) In your UML uml.sh script, set the default route for Magenta to be via the IP address of tap5 on the base VNS. Run the script. 31) Confirm that all four UML machines can now ping the IP address of the tap1 interface in the base VNS (IP address 192.168.241.1). 32) Confirm that all four UML machines can now ping the IP address of the eth0 interface (allocated by DHCP) in the base VNS. Everything looks good. All the UML machines can reach all the interfaces in the base VNS, and vice-versa. * Summary - script modifications for Part II * Here is a summary of what you need in your set-up scripts to do to make routing work between the UMLs and the base VNS: A) The VNS setup.sh script needs to fix the /host symlink, disable rp_filter, enable log_martians, and enable ip_forward. (Make sure you put this code in the correct place in your setup.sh script so that it *always* executes, even if your script exits early.) B) The UML uml.sh script section for Magenta needs to bring up eth0 (do not set an IP address - it's preconfigured) and set a default route via the IP address of the tap5 interface in the base VNS. C) The little startuml.sh script (which you will only run in the base VNS) needs to start the four UML machines and then route the whole 10/14 network via Magenta's eth0 IP address (which is at the other end of the tap5 interface on the base VNS). Part III - routing between the UMLs and the Internet ---------------------------------------------------- The "Summary" at the end of this section gives a summary of the modifications that need to be made to the VNS to enable masqueraded routing between the UMLs and the Internet. Let's work on getting the UML machines connected to the Internet. 33) Verify that the base VNS can ping some external IP address, e.g. perhaps google.ca (66.102.7.104, 64.233.187.104, 64.233.161.104) or (at Algonquin) the Course Linux Server (10.50.254.148). Make sure this ping works before proceeding! 34) Try pinging the same address you used above, from any UML machine, e.g try pinging from Magenta. It doesn't work - no replies. 35) Run tcpdump on tap5 (in the base VNS) and verify that the ping echo requests from Magenta are coming into the base VNS via tap5; but, nothing is returning. Kill the tcpdump when you have verified this. 36) Run tcpdump on the base VNS eth0 and verify that the ping echo requests from Magenta are leaving via eth0; but, nothing is coming back: # hostname vns # tcpdump -t -l -n -i eth0 listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes IP 192.168.245.2 > 66.102.7.104: icmp 64: echo request seq 12 IP 192.168.245.2 > 66.102.7.104: icmp 64: echo request seq 13 IP 192.168.245.2 > 66.102.7.104: icmp 64: echo request seq 14 IP 192.168.245.2 > 66.102.7.104: icmp 64: echo request seq 15 Why aren't the replies coming back? First, answer the question: where are the packets going? The ping echo requests leave by eth0, as tcpdump shows; but, what is their first stop on the way? The external destination address 66.102.7.104 isn't on any known locally attached network in the base VNS; so, the VNS kernel sends the packet to the default gateway machine for the base VNS - the gateway machine lies out eth0. That gateway machine might forward the packet to its own gateway; however, eventually, the packet will hit a router that refuses to forward it. The RFC1918 private IP source address of Magenta at 192.168.245.2 isn't routable on the open Internet. Even if the packet were to be forwarded all the way to its destination, the echo reply packet could never find its way back; because, there are no routes on the Internet to return it. The Internet doesn't know how to get back to Magenta or any of the other UML addresses. (Remember - just because a packet arrived from some interface doesn't mean that the reply packet will leave by that same interface; reply packets follow routes independently of the source packets that triggered them.) We can't "fix" this routing problem by adding a few reverse routes in the same way that we fixed the problem with the base VNS not returning packets to the UMLs. We can't add reverse routes to all the Internet routers between us and our destination just for our little UML machines and their private addresses. We can't modify existing Internet routes. In short, we cannot allow any private-IP source address packets to go out eth0 on the base VNS; because, the replies will never return. The only packets that can go out eth0 and that will return are packets that carry the IP source address of the base VNS eth0 interface itself. On the base VNS, when we ping an external address, the kernel gets to choose the source address for the packets. The source address of the ping packets leaving on eth0 is logically set by the base VNS kernel to be the address of eth0, and so replies work correctly. Our gateway machine on eth0 knows how to get reply packets back to the base VNS if the base VNS uses its eth0 IP address as the source address of packets it sends; the base VNS and the gateway machine are on the same eth0 network. Packets coming into the base VNS from any of the UMLs already have private source IP addresses set; the base VNS kernel doesn't get to set or change them. Those private addresses don't work when they go out via eth0 to our gateway machine. The base VNS forwards the UML packet with its private IP source address to the default gateway out eth0, but the default gateway has no route information that will direct reply packets back to the base VNS for those private UML source IP address. Remember - just because a UML packet arrived from some interface into our base VNS gateway machine doesn't mean that the reply packet will leave by that same interface; reply packets follow routes independently of the source packets that triggered them, and our base VNS gateway machine has no routing information to return UML packets back to the base VNS. * Source Network Address Translation - ALN p.677-682 * Suppose we could have the base VNS kernel translate the source addresses of packets coming from the UML machines and going out eth0. In every outgoing packet, the kernel could save and replace the UML source address with the IP address of the eth0 interface, so that reply packets to that source address would be able to return to the eth0 address. The base VNS kernel would replace the unknown UML source IP address with its own known eth0 IP address, so that replies would return correctly. When the reply packets did return to the base VNS eth0 address, the kernel would undo the translation by replacing its own eth0 destination address with the original saved UML source address, and letting the reply packets return to the UML that sent the origininal packets. The packets would follow the base VNS route back through Magenta to the originating UML machine (because the base VNS has routing information that lets it route packets to the UMLs via Magenta). This saving and replacement is a form of "Source Network Address Translation" (SNAT) or "IP Masquerading". It allows one external IP address to be shared and used for outgoing packets from one or more internal IP addresses. (ALN p.45, 677-682) The Linux kernel permits two different types of source network address translation ("man iptables"): SNAT - Source Network Address Translation - suitable for interfaces that have a static IP address - translation persists even when interface is taken down and up MASQUERADE - suitable for interfaces that have a dynamic (DHCP) IP address - translation is removed when interface goes down Since our base VNS eth0 IP address is dyamic (DHCP), we select the MASQUERADE flavour. Here's how to enable it (ALN p.681): 37) To enable masquerading of source addresses on the dynamic (DHCP) eth0 interface in your base VNS system, add these iptables commands to your VNS setup.sh script to flush, zero, and then create a MASQUERADE entry: iptables -t nat -F POSTROUTING iptables -t nat -Z POSTROUTING iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE This masquerades any packets (including UML packets!) going out eth0 to have the IP address of the base VNS eth0 network. 38) Run your setup.sh script and then verify the new iptables settings: # hostname vns # iptables -t nat -v -L POSTROUTING Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 MASQUERADE all -- any eth0 anywhere anywhere 39) Verify that all your UML machines can now ping the external address. Leave the ping running on, say, Red, and do some dumps on the VNS: 40) Run tcpdump on the base VNS tap5 interface and see the packets coming in from the UML with the original Red source address, and coming in from the Internet and returning to the UML Red address: # hostname vns # tcpdump -t -l -n -i tap5 listening on tap5, link-type EN10MB (Ethernet), capture size 96 bytes IP 10.1.1.1 > 66.102.7.104: icmp 64: echo request seq 1 IP 66.102.7.104 > 10.1.1.1: icmp 64: echo reply seq 1 IP 10.1.1.1 > 66.102.7.104: icmp 64: echo request seq 2 IP 66.102.7.104 > 10.1.1.1: icmp 64: echo reply seq 2 41) Run tcpdump on the base VNS eth0 interface and see the packets going out with the original Red source address replaced by the IP address of eth0 (your IP address will differ from the dump shown here), and returing from the Internet to the eth0 IP address: # hostname vns # tcpdump -t -l -n -i eth0 listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes IP 192.168.1.100 > 66.102.7.104: icmp 64: echo request seq 1 IP 66.102.7.104 > 192.168.1.100: icmp 64: echo reply seq 1 IP 192.168.1.100 > 66.102.7.104: icmp 64: echo request seq 2 IP 66.102.7.104 > 192.168.1.100: icmp 64: echo reply seq 2 The original unmodified UML addresses are used for traffic on tap5. The kernel NAT translates these to the eth0 IP address for use on eth0. * Summary - script modifications for Part III * A) The VNS setup.sh script needs to flush, zero, and add a NAT rule on eth0 using "iptables" (ALN p.681). (Make sure you put this code in the correct place in your setup.sh script so that it *always* executes, even if your script exits early.) Part IV - fix for new Daylight Savings Time ------------------------------------------- You will note that the VNS and UML machines are outdated and don't have the correct Daylight Savings Time configured. 42) Copy the file /etc/localtime from the Course Linux Server to your base VNS virtual disk (e.g. copy to /host/localtime). This file sets the rules for translating kernel UTC (GMT) time to local time. 43) In both your VNS and UML set-up scripts, remove /etc/localtime and replace it with a symbolic link to the correct localtime file you just copied to your VNS virtual disk (/host/localtime). Verify the correct time using "date". Part V - final test ------------------- This is how I will test your uml.sh, startuml.sh, and setup.sh scripts: 44) Halt and purge all your UML machines using either the GUI UserModeLinux sub-menu or the command "umlPurge". Make sure nothing is left under /host/virtuals/ and that all the machines are gone from under /host/.uml/. Reboot your VNS base machine. Then do: 1. Mount your virtual disk. 2. Start the third hub. 3. Run your VNS setup.sh script. 4. Run your startuml.sh script. 5. Log in to the four machines and run the uml.sh script in each of them. Does everything work? Can you ping from every UML to external addresses? Can you ping from the base VNS to every UML? 45) Put your assignment label on uml.sh, startuml.sh, and setup.sh and prepare them for submission on the Course Linux Server. Submission ---------- Submission Standards: See Lab #1 for details. A. Make sure all files contain an Exterior Assignment Submission label. For full marks, where possible, lines that you type must be shorter than 80 columns. B. Submit your files for marking as Lab 08 using the following *single* netsubmit command line exactly as given here: $ netsubmit 08 uml.sh setup.sh startuml.sh Always submit *all* three files at the same time for every submission. Files submitted under the wrong names are worth zero marks. P.S. Did you spell all the assignment label fields and file names correctly?