------------------------- Week 14 Notes for CST8165 ------------------------- - Ian! D. Allen - idallen@idallen.ca - www.idallen.com Remember - knowing how to find out an answer is more important than memorizing the answer. Learn to fish! RTFM! (Read The Fine Manual) Keep up on your readings (Course Outline: average 4 hours/week homework) [************************************************************] [************************************************************] [*** Students should be taking their own notes in class ***] [*** and updating them with my published summaries. ***] [************************************************************] [************************************************************] =============================================================================== Protocols - Telnet ------------------ Started with rfc0097 (Feb 1971!) - too many revisions to list! http://tools.ietf.org/html/rfc854 (1983, 15 pages) http://tools.ietf.org/html/rfc855 (options: 1983, 3 pages) http://tools.ietf.org/html/rfc856 (binary: 1983) ...etc... http://tools.ietf.org/html/rfc1097 (subliminal: April 1 1989) ...etc... http://tools.ietf.org/html/rfc4248 (URI: 2005) Q: T/F, never believe anything you read in an April 1 RFC document. Telnet URI (RFC submitted in 2005): http://tools.ietf.org/html/rfc4248 - e.g. telnet://:@:/ "Few implementations handle the user name and password very well, if at all." Q: Give the full URI for a "telnet" connection. - "The purpose of the TELNET Protocol is to provide a fairly general, bi-directional, eight-bit byte oriented communications facility. - "the symmetry of the TELNET model requires that there is an NVT at each end of the TELNET connection" p.6 - "symmetry is an operating principle rather than an ironclad rule." p.4 Q: What is the purpose of the TELNET protocol? - a line-oriented "Network Virtual Terminal" with option negotiation - options start with Interpret As Command (IAC) byte - 255 0xFF) - IAC must therefore be doubled when sent as part of data stream - this IAC option negotiation may confuse some applications - "The code set is seven-bit USASCII in an eight-bit field" (p.4) - which is why netcat ("nc") is better for TCP/IP debugging Q: How does the TELNET protocol signal that an option is coming? Q: How does the TELNET protocol transmit the IAC byte if it appears as part of the data stream? - TELNET command "Go Ahead" (GA) for old 2741 lockable keyboards (!) p.5 - some things don't map well to the data byte stream: - Interrupt Process (IP) (this is mandatory if local system supports it) - Abort Output (AO) (optional) - Are You There (AYT) (optional) - Erase previous Character (EC) (optional) - Erase current Line (EL) (optional) How do you send an Interrupt or Abort command to a remote terminal? - don't want to add the command to the end of the queued data stream - don't want flow control to hold up commands such as IP, AO, AYT - ANS: use "out-of-band" socket data that goes "around" the main data stream: - TCP provides an "Urgent notification" packet that bypasses socket queues - "Synch signal consists of a TCP Urgent notification, coupled with the TELNET command DATA MARK (DM) [in the data stream]..." p.9 - SYNCH causes client to throw away data and messages (except for IP,AO,AYT) until it finds the DM you put at the end of the data stream - the SYNCH mechanism discards all data (not TELNET commands) between the sender of the Synch and its recipient Q: How do you send an Interrupt or Abort command to a remote terminal? The TELNET NVT "printer": - 95 USASCII graphics (codes 32 through 126) - a very few control characters - to send 8-bit data, you have to escape the IAC 255 byte by doubling it - though TELNET has an option to pass raw data, netcat is better since it doesn't need to escape the IAC byte Telnet options (many, many following RFCs): http://tools.ietf.org/html/rfc854 Telnet protocol was eventually adopted for the FTP control stream (port 21). Q: How does FTP use the TELNET protocol? ============================================================================= Domain Name System - DNS ------------------------ http://tools.ietf.org/html/rfc1034 (concepts; Nov 1987; 55 pages, index at end) http://tools.ietf.org/html/rfc1035 (implementation; Nov 1987) http://www.dns.net/dnsrd/rfc/rfc1035/rfc1035.html (annotated with pictures) Additional: http://tools.ietf.org/html/rfc920 (Initial Set of Top Level Domains; October 1984) http://tools.ietf.org/html/rfc4343 (case sensitivity; January 2006) http://tools.ietf.org/html/rfc4033 (DNS security; March 2005) Annotated: http://www.dns.net/dnsrd/rfc/ - DNS is for turning names into IP addresses, vice-versa, and more - originally done using a big HOSTS.TXT file "Host name to address mappings were maintained by the Network Information Center (NIC) in a single file (HOSTS.TXT) which was FTPed by all hosts [RFC-952, RFC-953]. The total network bandwidth consumed in distributing a new version by this scheme is proportional to the square of the number of hosts in the network, and even when multiple levels of FTP are used, the outgoing FTP load on the NIC host is considerable. Explosive growth in the number of hosts didn't bode well for the future. - http://tools.ietf.org/html/rfc1034 - file /etc/hosts still used for local (non-DNS) IP/host name map on Unix - other Unix config options determine whether local file check comes before or after checking the DNS - still have file LMHOSTS on Windows? Q: What is the purpose of the Domain Name System? Q: Give any four (of many) specific functions that can be performed by the Domain Name System (DNS)? Q: In what file do Unix/Linux systems keep local non-DNS names? Assumptions about usage (RFC1034 Section 2.3 p.3) 1. size proportional to number of hosts, then number of users 2. most of the data changes slowly, some isolated parts may change quickly 3. administrative divisions and boundaries have their own name servers 4. availability of local "trusted" name servers to do external referrals 5. access is more important than timely updates or consistency Q: Give three (of five) assumptions made when DNS was designed. Query styles: - iterative query: NS refers client to another NS (blocked at Algonquin?) - the client has to query each new NS - recursive query: NS does lookup for client (must be used at Algonquin) - the client just waits for the answer Q: Describe and differentiate between the two types of DNS queries. Three major DNS components ("Elements") (RFC1034 2.4 p.6): 1. Domain Name Space and Resource records - tree-structured name space and data 2. Name Servers - hold complete information about a subset and may cache more 3. Resolvers are programs or libraries that query Name Servers - "directly accessible to user programs; hence no protocol is necessary" - this is the user-visible part Notes: - the resolver may have its own cache - Unix/Linux resolvers start with /etc/resolv.conf Q: T/F, the DNS name space is flat. Q: What is the function of the "name server" part of a DNS (not the "resolver")? Q: What is the function of the "resolver" part of a DNS (not the "name server")? Q: On Unix/Linux, what file is used by the resolver library to find name servers to query? Rules for DNS names (RFC1034) - case-insensitive (but case is preserved) p.7 - domain name components are *separated* by dots - "absolute" names end in the ROOT - a zero-length domain: idallen.ca. - "relative" names don't end in ROOT - no trailing dot: idallen.ca - longest domain is 255 characters (plus dots, which separate components) - subdomains are fully contained within domains (3.1): - For example, A.B.C.D is a subdomain of B.C.D, C.D, D, and " " (the root). Q: T/F, DNS records are case-sensitive. Q: T/F, DNS records are converted to lower-case. Q: Describe and differentiate between a DNS "absolute" and "relative" name. Q: T/F, the dots in a DNS name are included in the 255 character name limit. - reverse DNS (ptr) lookups map into otherwise unused "IN-ADDR.ARPA." e.g. to reverse-look-up 1.2.3.4 you search for ptr in 4.3.2.1.in-addr-arpa. - some DNS software will automatically do the ptr look up for you: $ host 72.18.159.15 15.159.18.72.in-addr.arpa domain name pointer server320.tchmachines.com. $ host -t ptr 15.159.18.72.in-addr.arpa. 15.159.18.72.in-addr.arpa domain name pointer server320.tchmachines.com. - some software will not do the ptr look up for you: e.g. "dig" does not - you have to be explicit with the domain and type: $ dig 72.18.159.15 # fails - NXDOMAIN $ dig 15.159.18.72.in-addr.arpa. ptr # works $ dig -x 72.18.159.15 # also works Q: What is a "reverse-DNS lookup"? Q: How does a resolver look up the PTR record for IP address 1.2.3.4? Q: Give a Unix command that will turn a host name into an IP address. Q: Give a Unix command that will turn an IP address into a host name. - name syntax is given in RFC1034 section 3.5 p.11: "The labels must follow the rules for ARPANET host names. They must start with a letter, end with a letter or digit, and have as interior characters only letters, digits, and hyphen. There are also some restrictions on the length. Labels must be 63 characters or less. - but in 2006 we have many violations: 3com.com, etc. Resource Records (RRs): Type, Class, TTL, RData ----------------------------------------------- http://www.dns.net/dnsrd/rr.html - resource records (RRs) of various types are stored in Name Servers - most common look-up is for A (address) records - a "forward DNS look-up" - a "reverse DNS look-up" turns an IP into a domain name via PTR records RR: owner type class TTL RDATA Owner: the domain, e.g. idallen.ca Many Types: (see http://www.dns.net/dnsrd/rr.html ) - A - CNAME - HINFO - MX - NS - PTR - SOA - TXT (see rfc1035) - SRV (not in rfc1034) - AAAA (not in rfc1034) - A6 (not in rfc1034) Know these record types: Q: What data is contained in a DNS type "A" record? Q: What data is contained in a DNS type "MX" record? Q: What data is contained in a DNS type "NS" record? Q: What data is contained in a DNS type "PTR" record? Classes of records; - IN (Internet system) - CH (CHAOS system) - only IN is important in our Internet TTL: time to live of this record (expiry and time-out) RData - various types of data, depending on Type (see p.13) Querying DNS for a specific record type using "host" or "dig": $ host -t a idallen.ca. idallen.ca has address 72.18.159.15 $ host -t mx idallen.ca. idallen.ca mail is handled by 0 idallen.ca. $ host -t ns idallen.ca. idallen.ca name server ns2.totalchoicehosting.com. idallen.ca name server ns1.totalchoicehosting.com. # dig idallen.ca mx [...] ;; ANSWER SECTION: idallen.ca. 224 IN MX 0 idallen.ca. Q: use a Unix command to query a DNS server for A, MX, NS, PTR records Watching it work ---------------- Need to use a "resolver" library. On Unix/Linux, it starts here: $ cat /etc/resolv.conf search somedomain.ca nameserver 0.0.0.0 nameserver 192.168.0.1 nameserver 192.168.0.2 $ host idallen.ca. idallen.ca has address 72.18.159.15 # "A" record idallen.ca mail is handled by 0 idallen.ca. # "MX" record $ dig idallen.ca. ; <<>> DiG 9.3.2 <<>> idallen.ca. ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11389 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;idallen.ca. IN A ;; ANSWER SECTION: idallen.ca. 14373 IN A 72.18.159.15 ;; Query time: 1 msec ;; SERVER: 205.211.30.21#53(205.211.30.21) ;; WHEN: Tue Nov 28 09:38:30 2006 ;; MSG SIZE rcvd: 44 Queries for PTR records are handled automatically by "host": $ host 72.18.159.15 15.159.18.72.in-addr.arpa domain name pointer server320.tchmachines.com. $ host -t a 15.159.18.72.in-addr.arpa 15.159.18.72.in-addr.arpa has no A record $ host -t ptr 15.159.18.72.in-addr.arpa 15.159.18.72.in-addr.arpa domain name pointer server320.tchmachines.com. Tracing a query --------------- Here is a trace of an iterative lookup for the A record for "www.idallen.ca.": - name www.idallen.ca. is: - the name "www" - in subdomain .idallen - in subdomain .ca - in the ROOT domain (".") Steps: 1. Locate the IP addresses of the ROOT name servers (the NS records) (this info may be compiled in, or kept cached in a local file) 2. Query some root name server for the .ca domain NS name server IP addrs. 3. Query some .ca name server for the .idallen domain NS name server addrs. 4. Query some .idallen name server for the A record IP address of "www". Q: Given the domain www.idallen.ca, list the steps of an iterative DNS query that would resolve this domain to its IP address. An example of a command that can do an iterative query: $ dig +trace idallen.ca. (may not work at Algonquin due to blocking) ; <<>> DiG 9.3.1 <<>> +trace www.idallen.ca. ;; global options: printcmd *** 1. locate addresses of root NS *** . 3600000 IN NS F.ROOT-SERVERS.NET. . 3600000 IN NS G.ROOT-SERVERS.NET. . 3600000 IN NS H.ROOT-SERVERS.NET. . 3600000 IN NS I.ROOT-SERVERS.NET. . 3600000 IN NS J.ROOT-SERVERS.NET. . 3600000 IN NS K.ROOT-SERVERS.NET. . 3600000 IN NS L.ROOT-SERVERS.NET. . 3600000 IN NS M.ROOT-SERVERS.NET. . 3600000 IN NS A.ROOT-SERVERS.NET. . 3600000 IN NS B.ROOT-SERVERS.NET. . 3600000 IN NS C.ROOT-SERVERS.NET. . 3600000 IN NS D.ROOT-SERVERS.NET. . 3600000 IN NS E.ROOT-SERVERS.NET. ;; Received 436 bytes from 127.0.0.1#53(127.0.0.1) in 1 ms *** 2. locate addresses of .ca NS *** ca. 172800 IN NS CA04.CIRA.ca. ca. 172800 IN NS CA05.CIRA.ca. ca. 172800 IN NS CA06.CIRA.ca. ca. 172800 IN NS NS-EXT.ISC.ORG. ca. 172800 IN NS CA01.CIRA.ca. ca. 172800 IN NS CA02.CIRA.ca. ;; Received 284 bytes from 192.112.36.4#53(G.ROOT-SERVERS.NET) in 43 ms *** 3. locate addresses of .idallen NS *** idallen.ca. 86400 IN NS ns2.totalchoicehosting.com. idallen.ca. 86400 IN NS ns1.totalchoicehosting.com. ;; Received 90 bytes from 192.228.28.9#53(CA04.CIRA.ca) in 80 ms *** 4. look up A record for name "www" *** www.idallen.ca. 14400 IN CNAME idallen.ca. idallen.ca. 14400 IN A 72.18.159.15 idallen.ca. 86400 IN NS ns2.totalchoicehosting.com. idallen.ca. 86400 IN NS ns1.totalchoicehosting.com. ;; Received 136 bytes from 65.254.32.122#53(ns2.totalchoicehosting.com) in 43 ms Since most DNS traffic is UDP, it is optimized to fit in one single UDP packet. (Full zone transfers will use TCP.) Only 13 ROOT name servers exist because only 13 resource records fit in a single UDP packet. Q: Why aren't there more than 13 ROOT name servers? Q: T/F, most DNS traffic uses UDP. Configuring Name Servers ------------------------ How do we get the address of the root name servers "."? Unix/Linux keeps a copy in a local file. The BIND name server "named" also has a copy compiled in (the compiled-in copy may be outdated). Q: How does a Unix/Linux system know the addresses of the ROOT name servers, to start an iterative DNS query? Unix/Linux file name /var/named/named.ca (unreadable in Linux lab) - Use "dig @A.ROOT-SERVERS.NET . ns" to update this file if it's outdated. - but not at Algonquin (blocked) Unix/Linux DNS server package name is "BIND" - Berkeley Internet Name Daemon - actual program name is "named" - see /etc/named.conf for the location of the "." domain "hints" Q: What is the package name of the common Unix DNS server? Q: What is the name server program name in that package? You only need to find one working ROOT server, at which point you can use it to find the current addresses of the rest. Below is the config file for "named" from Linux Fedora Core 5. Note the "type hint" file named.ca containing the ROOT name server info. --------------------------------------------------------------------------- // // named.conf for Red Hat caching-nameserver // options { directory "/var/named"; dump-file "/var/named/data/cache_dump.db"; statistics-file "/var/named/data/named_stats.txt"; /* * If there is a firewall between you and nameservers you want * to talk to, you might need to uncomment the query-source * directive below. Previous versions of BIND always asked * questions using port 53, but BIND 8.1 uses an unprivileged * port by default. */ // query-source address * port 53; }; // // a caching only nameserver config // controls { inet 127.0.0.1 allow { localhost; } keys { rndckey; }; }; zone "." IN { type hint; file "named.ca"; }; zone "localdomain" IN { type master; file "localdomain.zone"; allow-update { none; }; }; zone "localhost" IN { type master; file "localhost.zone"; allow-update { none; }; }; zone "0.0.127.in-addr.arpa" IN { type master; file "named.local"; allow-update { none; }; }; zone "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa" IN { type master; file "named.ip6.local"; allow-update { none; }; }; zone "255.in-addr.arpa" IN { type master; file "named.broadcast"; allow-update { none; }; }; zone "0.in-addr.arpa" IN { type master; file "named.zero"; allow-update { none; }; }; --------------------------------------------------------------------------- Finding domain owners --------------------- $ whois idallen.ca. (may not work at Algonquin due to blocking) Status: EXIST Registrar: SIBERNAME INTERNET AND SOFTWARE TECHNOLOGIES INC. Registrar-no: 108 Registrant-no: 445525 Domaine-no: 445525 Subdomain: idallen.ca Renewal-Date: 2007/05/29 Date-Approved: 2002/05/29 Date-Modified: 2006/03/30 Organization: Ian D. Allen Description: Admin-Name: Ian! D. Allen Admin-Title: Admin-Postal: idallen.ca 22 Oak Street Ottawa ON K1R 6S9 Canada Admin-Phone: +1-613-235-6216 Admin-Fax: Admin-Mailbox: idallen@idallen.ca Tech-Name: Ian! D. Allen Tech-Title: Tech-Postal: idallen.ca 22 Oak Street Ottawa ON K1R 6S9 Canada Tech-Phone: +1-613-235-6216 Tech-Fax: Tech-Mailbox: idallen@idallen.ca NS1-Hostname: ns1.totalchoicehosting.com NS2-Hostname: ns2.totalchoicehosting.com Q: What Unix/Linux command can find the owner/registrar of a domain name? Resources http://directory.google.com/Top/Computers/Internet/Protocols/DNS/ http://www.root-servers.org/ - Ottawa has a copy of "F" Probes and Tools http://www.dnsreport.com/ http://www.dnsstuff.com/ Software http://www.isc.org/index.pl?/sw/bind/ http://www.dns.net/dnsrd/ Problems http://en.wikipedia.org/wiki/Site_Finder (3 weeks in September 2003) http://en.wikipedia.org/wiki/DNS_cache_poisoning ============================================================================= State of IPV6 and DNS --------------------- http://tools.ietf.org/html/rfc3363 "Working group consensus as perceived by the chairs of the DNSEXT and NGTRANS working groups is that: a) AAAA records are preferable at the moment for production deployment of IPv6, and b) that A6 records have interesting properties that need to be better understood before deployment. c) It is not known if the benefits of A6 outweigh the costs and risks. "Thus, we are forced to conclude that indiscriminate use of long A6 chains is likely to lead to increased user frustration." Q: Which DNS record type is currently preferred for IPV6 addresses? Q: How many bits are in an IPV6 address? ============================================================================= File Transfer Protocol - FTP ---------------------------- http://tools.ietf.org/html/rfc114 (Apr 1971, 17 pages, no index) http://tools.ietf.org/html/rfc454 (Feb 1973, 35 pages, no index) http://tools.ietf.org/html/rfc686 (May 1975 "leaving well enough alone") http://tools.ietf.org/html/rfc691 (May 1975 "one more try on the FTP") http://tools.ietf.org/html/rfc765 (Jun 1980, 70 pages, no index!) http://tools.ietf.org/html/rfc959 (Oct 1985, 69 pages, no index!) - another text-based protcol (you can run it with "nc") - commands are not case-sensitive - but it's tricky to handle the parallel data connection at the same time... "The objectives of FTP are 1) to promote sharing of files (computer programs and/or data), 2) to encourage indirect or implicit (via programs) use of remote computers, 3) to shield a user from variations in file storage systems among hosts, and 4) to transfer data reliably and efficiently. FTP, though usable directly by a user at a terminal, is designed mainly for use by programs. Q: Give two (of four) objectives of the FTP protocol. Q: T/F, FTP was designed for humans to use directly (e.g. via netcat). 2.3. THE FTP MODEL (p.8) - nice ASCII art diagram PI - protocol interpreter DTP - data transfer process - two types of FTP: Active FTP and Passive FTP (see below) http://en.wikipedia.org/wiki/File_Transfer_Protocol "FTP runs exclusively over TCP. The FTP server always listens on port 21 for incoming connections from FTP clients. A connection to this port from the FTP Client forms the control stream on which commands are passed to the FTP server from the FTP client and on occasion from the FTP server to the FTP client. For the actual file transfer to take place, a different connection is required which is called the data stream. Depending on the transfer mode, the process of setting up the data stream is different. 1. In active mode, the FTP client opens a random port (> 1023), sends the FTP server the random port number on which it is listening over the control stream and waits for a connection from the FTP server. When the FTP server initiates the data connection to the FTP client it binds the source port to port 20 on the FTP server. 2. In passive mode [the client first tells the server to use passive mode using the FTP "PASV" command, then] the FTP Server opens a random port (> 1023), sends the FTP client the port on which it is listening over the control stream and waits for a connection from the FTP client. In this case the FTP client binds the source port of the connection to a random port greater than 1023. Summary: - Active FTP: client opens a port and tells the server to connect to it. - client control connects to server; server data connects back to client - server binds outgoing data stream to privileged data port 20 - client needs to open a port in the firewall to let in the server - Passive FTP: client tells server to use passive mode using "PASV" command; server opens a port and tells the client to connect to it. - client control connects to server; client data connects to server - both ends of data stream use unprivileged port numbers - no incoming port needs to be opened in a firewall Q: Describe how FTP works (including port numbers), and including a description of the difference between FTP "active" and "passive" modes. http://slacksite.com/other/ftp.html "One of the most commonly seen questions when dealing with firewalls and other Internet connectivity issues is the difference between active and passive FTP and how best to support either or both of them. Hopefully the following text will help to clear up some of the confusion over how to support FTP in a firewalled environment. - The diagram on this site is wrong: no [4] is sent from client - FTP uses the TELNET protocol on the control connection Q: What protocol does FTP use for its control connection? - control channel can extend across multiple servers (Fig. 2 p.9) http://en.wikipedia.org/wiki/FXP Q: T/F, the FTP control channel can chain across multiple servers. Q: T/F, an FTP client can arrange to transfer data from one remote server to another without routing this data through the client's connection. - FTP servers can be told to open ports on *any* machine! http://en.wikipedia.org/wiki/FTP_bounce_attack http://www.cert.org/tech_tips/ftp_port_attacks.html "To conform with the FTP protocol, the PORT command has the originating machine specify an arbitrary destination machine and port for the data connection. However, this behavior also means that an attacker can open a connection to a port of the attacker's choosing on a machine that may not be the originating client. Making this connection to an arbitrary machine for unauthorized purposes is the FTP bounce attack. Q: What command in the FTP protocol set permits an "FTP bounce attack", and how is that command used to do the attack? - The FTP protocol commands only have one argument! - a rename requires RNFR followed by RNTO (silly) Q: How do you implement a "rename" using the FTP command set? - server responses are 3-digit codes followed by message text - client "should wait" for server reply before issuing next command (5.4 p.49) Q: T/F, FTP is a pipelining/streaming protocol - you can send many commands without waiting for server responses in between. - multi-line server responses use the SMTP '-' trick, e.g. 220-message for the first line, and code followed by space (e.g. 220 end) for the last line of the multi-line response - but, unlike SMTP, the intervening lines after the first status line may look like anything (e.g. output from "STAT")! - if a server response generates something that looks like it might prematurely end the response, the line must be prefaced with "some neutral character" (e.g. a space) Q: How does the FTP server produce a multi-line response to a client? - FTP reply codes are structured (Section 4.2 p.36) 1yz Positive Preliminary reply 2yz Positive Completion reply 3yz Positive Intermediate reply 4yz Transient Negative Completion reply 5yz Permanent Negative Completion reply x0z Syntax x1z Information x2z Connections x3z Authentication and accounting x4z Unspecified x5z File system Q: T/F, FTP 3-digit response codes have assigned meanings for both the first and second digit. [do not memorize these codes!] - command response sequences (5.4 p.49) - state diagrams (6 p.54) - see RETR, p.51 125, 150 (110) 226, 250 425, 426, 451, 551, 552 Using netcat on the control connection -------------------------------------- Because FTP has a separate data connection, you would need two netcat sessions going to transfer a file, and you would have to get the port numbers right. The FTP control connection is essentially the same as Telnet - it uses the same protocol. Q: Why does FTP need two parallel netcat sessions to run, when SMTP, POP3, HTTP, and TELNET only need one netcat? Note: The FTP protocol STAT command does the same as LIST; but, it receives the server info over the control connection instead of over the data connection; this makes it useful on a (single) netcat connection. Some basic directory listing is possible with STAT, even without a data connection: Q: What is the difference between the FTP "STAT" and "LIST" commands? Q: T/F, since the FTP "LIST" returns data via the data connection, there is no way to use FTP to see a directory listing of a remote site without opening a data connection first. Q: T/F, the FTP "STAT" command works even if you have a firewall that prevents FTP data channels from being set up Using "STAT" on the control connection: $ nc -v ftp.idallen.ca 21 ftp.idallen.ca [72.18.159.15] 21 (ftp) open 220---------- Welcome to Pure-FTPd [TLS] ---------- 220-You are user number 3 of 50 allowed. 220-Local time is now 20:46. Server port: 21. 220-This is a private system - No anonymous login 220-IPv6 connections are also welcome on this server. 220 You will be disconnected after 15 minutes of inactivity. ---> HELP 214-The following SITE commands are recognized ALIAS CHMOD IDLE 214 Pure-FTPd - http://pureftpd.org/ ---> USER idallen 331 User idallen OK. Password required ---> PASS XXXX 230-User idallen has group access to: idallen 230 OK. Current restricted directory is / ---> STAT .twmrc 213-STAT -rw-r--r-- 1 32096 idallen 14706 Jun 21 2002 .twmrc 213 End. ---> NOOP 200 Zzz... ---> QUIT 221-Goodbye. You uploaded 0 and downloaded 0 kbytes. 221 Logout. or 421 Timeout - try typing a little faster next time Active FTP session with server responses ---------------------------------------- Section 7, p.59 Many FTP commands have options that let you see the server/client dialog on the control connection. Below is a sample "Active" FTP session. Note the encoding of the 16-bit PORT into two 8-bit integers. The high byte is sent first, followed by the low byte, - e.g. 164,39 means 164*256 + 39 = 39424 Q: An FTP server sends you this PORT string: 1,10 What PORT number is that? (1*256 + 10 = 266) Q: What is wrong with this FTP PORT string: 1,300 $ ftp -vd ftp.idallen.ca Connected to ftp.idallen.ca. 220---------- Welcome to Pure-FTPd [TLS] ---------- 220-You are user number 1 of 50 allowed. 220-Local time is now 23:13. Server port: 21. 220-This is a private system - No anonymous login 220-IPv6 connections are also welcome on this server. 220 You will be disconnected after 15 minutes of inactivity. Name (ftp.idallen.ca:idallen): idallen ---> USER idallen 331 User idallen OK. Password required Password: ---> PASS XXXX 230-User idallen has group access to: idallen 230 OK. Current restricted directory is / ---> SYST 215 UNIX Type: L8 Remote system type is UNIX. Using binary mode to transfer files. ftp> ls i.sh ---> PORT 192,168,1,250,164,39 # 164*256 + 39 = 39424 200 PORT command successful ---> LIST i.sh 150 Connecting to port 39424 -rwxr-xr-x 1 32096 idallena 1737 Oct 19 2005 i.sh 226-Options: -a -l 226 1 matches total ftp> get i.sh local: i.sh remote: i.sh ---> TYPE I 200 TYPE is now 8-bit binary ---> PORT 192,168,1,250,172,40 # 172*256 + 40 = 44072 200 PORT command successful ---> RETR i.sh 150 Connecting to port 44072 226-File successfully transferred 226 0.132 seconds (measured here), 12.86 Kbytes per second 1737 bytes received in 0.15 secs (11.5 kB/s) ftp> quit ---> QUIT 221-Goodbye. You uploaded 0 and downloaded 2 kbytes. 221 Logout. In an Active FTP session, the client sends the IP address and port to the server and the server connects the data stream to that address and port. (Usually the address and PORT are on the client machine; but, they could be on any machine.) Q: My FTP client wants to set up an Active session with a remote server and have the server connect to IP address 1.2.3.4 and port 513. What FTP command does the client send to the server? (PORT 1,2,3,4,2,1) Passive FTP session with server responses ----------------------------------------- Same as above, but the client tells the server to use PASV mode where the server then tells the client what port to connect to: $ ftp -vd -p ftp.idallen.ca Connected to ftp.idallen.ca. 220---------- Welcome to Pure-FTPd [TLS] ---------- 220-You are user number 1 of 50 allowed. 220-Local time is now 23:24. Server port: 21. 220-This is a private system - No anonymous login 220-IPv6 connections are also welcome on this server. 220 You will be disconnected after 15 minutes of inactivity. Name (ftp.idallen.ca:idallen): idallen ---> USER idallen 331 User idallen OK. Password required Password: ---> PASS XXXX 230-User idallen has group access to: idallen 230 OK. Current restricted directory is / ---> SYST 215 UNIX Type: L8 Remote system type is UNIX. Using binary mode to transfer files. ftp> get i.sh local: i.sh remote: i.sh ---> TYPE I 200 TYPE is now 8-bit binary ---> PASV 227 Entering Passive Mode (72,18,159,15,164,151) # 164*256 + 151 = 42135 ---> RETR i.sh 150 Accepted data connection 226-File successfully transferred 226 0.000 seconds (measured here), 24.03 Mbytes per second 1737 bytes received in 0.02 secs (85.7 kB/s) ftp> ls i.sh ---> TYPE A 200 TYPE is now ASCII ---> PASV 227 Entering Passive Mode (72,18,159,15,218,170) # 218*256 + 170 = 55978 ---> LIST i.sh 150 Accepted data connection -rwxr-xr-x 1 32096 idallen 1737 Oct 19 2005 i.sh 226-Options: -a -l 226 1 matches total ftp> quit ---> QUIT 221-Goodbye. You uploaded 0 and downloaded 2 kbytes. 221 Logout. Q: My FTP client wants to set up a Passive session with a remote server. What command to the server triggers a Passive session? Q: What response does an FTP server give to a PASV command, and what do the numeric arguments in the response mean? Q: A server responds to PASV from a client with (1,2,3,4,2,3). What IP address will the client connect to? (1.2.3.4) What port will the client connect to at that IP address? (515) =============================================================================== The following section on NAT and FTP-through-NAT is not on the final exam, except perhaps as bonus question material. Aside: Network Address Translation (NAT) ---------------------------------------- http://tools.ietf.org/html/rfc2663 (Aug 1999, 30 pages, "informational") http://tools.ietf.org/html/rfc3022 (Jan 2001, 16 pages, "informational") "Basic NAT and NAPT are two variations of traditional NAT, in that translation in Basic NAT is limited to IP addresses alone, whereas translation in NAPT is extended to include IP address and Transport identifier (such as TCP/UDP port or ICMP query ID). NAT and NAPT allow machines on a private, internal network to send their packets through a NAT router. Network Address Translation modifies the *source* address (and with NAPT, possibly the source port) of TCP or UDP packets leaving the router for the Internet, and it modifies the *destination* address (and possibly the destination port) of reply packets returning to the router from the Internet. The saved source address/port is placed into the destination address/port of the reply packets when they return. Example of NAT: Consider a Client on an internal network (e.g. 192.168.0.0/24) connecting through a single Router to a remote Server. The Router has the only public Internet IP address (e.g. given by your ISP). There are four IP addresses involved in the connection, A,B,C,D: A. client - internal network - e.g. 192.168.0.2 B. router - internal network interface: e.g. 192.168.0.1 C. router - external network interface: e.g. 206.47.37.39 D. server - public Internet address - e.g. 72.18.159.15 INTERNAL CLIENT NAT ROUTER INTERNET SERVER --------------- ------------- --------------- 1. a client src=A generates a packet destined for server dst=D [src=A,dst=D] --> 2. the packet [A,D] heads to the router B interface on its way to the Internet --> [src=A,dst=D] 3. the router appplies NAT to src=A, saving and replacing src=A with src=C [src=C,dst=D] --> 4. the new packet [C,D] exits C and continues to the server D via the Internet --> [src=C,dst=D] 5. the packet arrives at D; server generates a reply packet from src=D to dst=C <-- [src=D,dst=C] 6. the reply packet [D,C] returns to the router C interface via the Internet [src=D,dst=C] <-- 7. the router appplies un-NAT to dst=C, replacing dst=C with saved dst=A <-- [src=D,dst=A] 8. the reply packet [D,A] exits B and continues to the internal client dst=A [src=D,dst=A] <-- 1. Client A creates a packet destined for server D with [src=A, dst=D] 2. Client routing table says to send packet via router gateway B 3. Packet arrives at router internal interface B on the way to D: - Router applies NAT, removing and remembering the internal src address A and changing the src address A to be its own external interface address C in the IP packet header: [A,D] becomes [C,D] - Router saves the src address A in a table for later use (Every NAT connection needs its own separate saved address!) 4. Packet leaves router external interface C with new [src=C, dst=D] Packet traverses Internet and arrives at server dst D. 5. Server constructs a reply packet with [src=D, dst=C] This reply packet is destined for the router's public Internet address. 6. Reply packet traverses Internet and arrives at router interface C: 7. Router recognizes that this packet is a reply to the earlier packet for which the router saved the src address A. The router takes the saved src address A out of its table and puts it back into the dst address of the reply packet: [D,C] becomes [D,A] 8. Reply packet leaves router internal interface B with new [src=D, dst=A] Reply packet arrives at internal client interface A. Round trip complete. - Neither the client nor the server have any idea that the packets were changed by the router. - The server thinks it is talking directly to the router (not the client). - The server has no idea that the *actual* source of the packets is the client behind the router. - The *data* portion of packets is untouched by plain NAT/NAPT. Only the TCP/IP headers are altered by the router, not the data. - TCP checksums also have to be fixed! (pseudo-header) - IP packet fragments cannot be translated (missing header info) - in "NAPT" (with Port Translation), the router also saves and changes the TCP/UDP port numbers on packets leaving the router for the Internet, and the router replaces the saved port numbers (as well as replacing the saved IP addresses) as the reply packets return - NAT provides some security against unexpected incoming TCP/UDP packets. - your private net is hidden behind the router - mappings from Internet to private net cannot be triggered from the Internet side of the router; the internal net client must initiate the first packet through the NAT process in the router - NAT can also map ICMP packets to/from an external network Q: How does NAT work? Describe (with pictures) the changes made to a packet as it leaves a client, traverses a NAT-ing router to a server, and then the reply packet returns from the server to the router to the client. Show the source and destination addresses at each interface. Q: T/F, a router's NAT translation table can be initialized either by a packet going from internal net to external, or by a packet going from external net to internal http://tools.ietf.org/html/rfc2663 (considerations, Aug 1999) "NAT devices are application unaware in that the translations are limited to IP/TCP/UDP/ICMP headers and ICMP error messages only. NAT devices do not change the payload of the packets, as payloads tend to be application specific. Q: T/F, a basic NAT device changes IP addresses only in the IP header, not in the data portion of a packet. "NAT devices (without the inclusion of Application Level Gateways [ALGs]) do not examine or modify transport payload. For this reason, NAT devices are transparent to applications in many cases. There are two areas, however, where NAT devices often cause difficulties: 1) when an application payload includes an IP address, and 2) when end-to-end security is needed. Q: T/F, if an application protocol puts a local IP address into the data portion of a packet, a plain NAT router (no ALG) won't modify it; so, the protocol may not work (because the local IP address won't be translated to work on the Internet) On designing new protocols: http://tools.ietf.org/html/rfc3235 (Jan 2002, 13 pages, "informational") "This document discusses those things that application designers might wish to consider when designing new protocols. While many common Internet applications will operate cleanly in the presence of Network Address Translators, others suffer from a variety of problems when crossing these devices. Q: Why is it a Very Bad Idea for a protcol to put IP information in the data portion of a packet (as FTP does with its PORT command)? Active FTP through NAT using a smart router ------------------------------------------- http://tools.ietf.org/html/rfc2616 - section 7.1 p.24 "7.1. FTP support" Normally (without an Application Level Gateway [ALG]), NAT doesn't change anything in the *data* in each packet. For active FTP, which sends a local FTP client IP address to the remote server in the data part of the PORT command packet, plain NAT isn't clever enough to fix the IP address hidden in the FTP PORT command packet's data. The server gets the local client IP address from the PORT command and is unable to use it to contact the client; because, the IP address in the data packet is the client's *internal* network address, not the Internet-visible address of its NAT router. To fix this, better routers have "application specific gateways [ALGs]" built in that reach into the *data portion* of the FTP PORT packet and modify it, replacing the client internal address in the PORT command with the public external address of the router (and fixing all the TCP/IP checksums and TCP window sizes as well!). - note that this sort of ALG intervention requires unencrypted data packets! http://tools.ietf.org/html/rfc2616 "[The] "PORT" command and "PASV" response in FTP control session payload identify the IP address and TCP port that must be used for the data session it supports. The arguments to the PORT command and PASV response are an IP address and a TCP port in ASCII. An FTP ALG is required to monitor and update the FTP control session payload so that information contained in the payload is relevant to end nodes. The ALG must also update NAT with appropriate data session tuples and session orientation so that NAT could set up state information for the FTP data sessions. "Because the address and TCP port are encoded in ASCII, this may result in a change in the size of packet. For instance, 10,18,177,42,64,87 is 18 ASCII characters, whereas 193,45,228,137,64,87 is 20 ASCII characters. If the new size is same as the previous, only the TCP checksum needs adjustment as a result of change of data. If the new size is less than or greater than the previous, TCP sequence numbers must also be changed to reflect the change in length of FTP control data portion. A special table may be used by the ALG to correct the TCP sequence and acknowledge numbers. The sequence number and acknowledgement correction will need to be performed on all future packet of the connection. Q: T/F, an ALG in a router may need to change the size of an FTP packet to fully implement NAT. Example of NAT through a router with an FTP ALG: Below are two ethereal TCP stream dumps. The first dump shows the TCP stream as seen on the local net side of the router (the side connected to the FTP client). The second dump shows the same TCP stream as seen on the external Internet side of the router. Note that the ALG in the router has changed (and remembered!) the IP address in the PORT command. The ALG in a smart router has to notice the FTP traffic and reach into the data portion of the control stream TCP packet to replace the actual PORT IP address 192.168.1.250 with the Internet address 206.47.37.39 (and then fix all the TCP window and checksum information!). Dump 1: This TCP stream trace is taken on the client side of the router. Traffic leaving host 192.168.1.250 heading to router (from ethereal): 220---------- Welcome to Pure-FTPd [TLS] ---------- 220-You are user number 1 of 50 allowed. 220-Local time is now 23:34. Server port: 21. 220-This is a private system - No anonymous login 220-IPv6 connections are also welcome on this server. 220 You will be disconnected after 15 minutes of inactivity. ---> USER idallena 331 User idallena OK. Password required ---> PASS XXXX 230-User idallena has group access to: idallena 230 OK. Current restricted directory is / ---> SYST 215 UNIX Type: L8 ---> PORT 192,168,1,250,181,11 # 181*256 + 11 = 46347 200 PORT command successful ---> LIST i.sh 150 Connecting to port 46347 226-Options: -a -l 226 1 matches total ---> TYPE I 200 TYPE is now 8-bit binary ---> PORT 192,168,1,250,237,226 # 237*256 + 11 = 60898 200 PORT command successful ---> RETR i.sh 150 Connecting to port 60898 226-File successfully transferred 226 0.000 seconds (measured here), 21.22 Mbytes per second ---> QUIT 221-Goodbye. You uploaded 0 and downloaded 2 kbytes. 221 Logout. Dump 2: The same TCP stream traffic as above, seen on the other side of the router. Traffic leaving the router 206.47.37.39 heading to Internet (from ethereal): 220---------- Welcome to Pure-FTPd [TLS] ---------- 220-You are user number 1 of 50 allowed. 220-Local time is now 23:34. Server port: 21. 220-This is a private system - No anonymous login 220-IPv6 connections are also welcome on this server. 220 You will be disconnected after 15 minutes of inactivity. ---> USER idallena 331 User idallena OK. Password required ---> PASS XXXX 230-User idallena has group access to: idallena 230 OK. Current restricted directory is / ---> SYST 215 UNIX Type: L8 ---> PORT 206,47,37,39,181,11 # note new IP addr from the ALG 200 PORT command successful ---> LIST i.sh 150 Connecting to port 46347 226-Options: -a -l 226 1 matches total ---> TYPE I 200 TYPE is now 8-bit binary ---> PORT 206,47,37,39,237,226 # note new IP addr from the ALG 200 PORT command successful ---> RETR i.sh 150 Connecting to port 60898 226-File successfully transferred 226 0.000 seconds (measured here), 21.22 Mbytes per second ---> QUIT 221-Goodbye. You uploaded 0 and downloaded 2 kbytes. 221 Logout. ============================================================================= //EOF - end of notes CST8165 Winter 2007 ============================================================================= http://www.pengalert.com/