Week 4 Notes for DAT2330 - Ian Allen Supplement to Text, outlining the material done in class and lab. Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM![*] ([*] Read The Fine Manual) ------------------------ Review and repetition... ------------------------ Redirection: into files and through pipes Variables and Quoting More fun with pipes. "Print the fifth directory from your $PATH environment variable." Iterative solution: $ echo $PATH $ echo $PATH | tr ':' '\n' $ echo $PATH | tr ':' '\n' | head -5 $ echo $PATH | tr ':' '\n' | head -5 | tail -1 "Print the second-to-last directory from your $PATH environment variable." $ echo $PATH | tr ':' '\n' | tail -2 | head -1 "Sort the elements in the PATH variable in ascending order." $ echo $PATH | tr ':' '\n' | sort $ echo $PATH | tr ':' '\n' | sort | tr '\n' ':' "Keep only the first five elements of the PATH." $ echo $PATH | tr ':' '\n' | head -5 | tr '\n' ':' "Keep only the first and last elements of the PATH" $ echo $PATH | tr ':' '\n' | sed -n -e '1p;$p' | tr '\n' ':' "How many unique shells are in the /etc/passwd file?" $ cat /etc/passwd | head $ cat /etc/passwd | awk -F: '{print $7}' | head $ cat /etc/passwd | awk -F: '{print $7}' | sort | head $ cat /etc/passwd | awk -F: '{print $7}' | sort | uniq OR $ awk -F: '{print $7}' /etc/passwd | sort | uniq OR $ awk -F: '{print $7}' /etc/passwd | sort -u Note that many Unix commands can act as filters (no file names - read from standard input and write to standard output), or as commands that process file names given on the command line. If the command does read file names, it is more efficient to let it open the file than to use "cat" to feed the file in on standard input. (There is less data copying needed!) Let commands open their own files. "Now, count the number of each kind of shell in /etc/passwd." $ awk -F: '{print $7}' /etc/passwd | sort | uniq -c "Count the number of each kind of shell in /etc/passwd and display the results sorted in descending numeric order." $ awk -F: '{print $7}' /etc/passwd | sort | uniq -c | sort -nr Understanding "sort": Explain the difference in output of these two pipelines: $ list="1 11 2 22 3 33 4 44 3 33 2 22 1 11" $ echo "$list" | tr ' ' '\n' | sort $ echo "$list" | tr ' ' '\n' | sort -n Aliases: Converting lower-case to upper-case: If your boss is an experienced IBM programmer, she's used to reading text in all UPPER CASE. This alias is for her: $ alias who="who | tr 'a-z' 'A-Z'" $ who Are the single quotes required? You can use a similar command to convert a lower-case file of IBM MVS JCL into upper-case. EXPERIMENT: Why doesn't this work? $ alias upper="tr 'a-z' 'A-Z'" $ upper myfile What is stored in "myfile" after this command is run? Using the above alias, the following command line doesn't work either: $ upper myfile >new Why not? (RTFM) Archeology in binary format files (non-text files): Sometimes you want to dig out just the printable strings from a binary file containing program code and other unpritable characters. $ strings /bin/bsh | more $ strings a.out | more $ strings . | more The shell is a programming language: while : ; do echo hi | write abcd0001 sleep 1 done ":" is a Bourne shell command that does nothing except return a true status. ------------------------------------------ Unix/Linux File System - (the weird parts) ------------------------------------------ Read the text (Chapter 4) for the non-weird parts. All the *diagrams* showing file systems and links in the text are wrong and range from confusing to seriously misleading. Here's the truth. Names are stored in directories, not with the things to which the names refer. The text makes the error of putting the names on the things that are being named. That is misleading and the cause of many misunderstandings about Unix files and directories. Here is the correct diagram for Figure 4.4 on page 70. Note that the names for things are one level above the things to which the names actually refer: +------+ | home | This box is the ROOT directory. It has the name HOME in it. +------+ The *directory* HOME is not here; only the *name* is here. | | +------+ | alex | This box is the HOME directory. It has the name ALEX in it. +------+ The name "HOME" isn't here; it's up in the ROOT directory. | | +----------------------------------+ This box is the ALEX directory. | names | temp | literature | demo | It has many names in it. Note: +----------------------------------+ the dir name ALEX isn't kept here. | | | | The name is up in the HOME directory. | | | | *----* *----* | *----* These are disk nodes containing the |data| |data| | |data| data for each of the files named in *----* *----* | *----* the directory ALEX. No names are here! +-------+ | promo | This box is the LITERATURE directory. +-------+ It has the name PROMO in it. | | *----* This is a disk node (inode) containing the |data| data for the file named PROMO. The name PROMO *----* is one level above, in the LITERATURE directory. To follow a path such as /home/alex/literature/promo just walk the tree. Start with the ROOT directory (which doesn't have a name, since nothing refers to it) and look for the first pathname component inside. Let's trace this pathname: Look in the ROOT directory for the *name* of the first pathname component: HOME. We find it. Follow the link in the ROOT directory that leads from the *name* HOME to the *directory* that is HOME. (Note how the names are separate from the things they name. The actual *directory* is nowhere near its name.) In the *directory* that is HOME, look for the *name* ALEX. We find it. Follow the link in the HOME directory that leads from the *name* ALEX to the *directory* that is ALEX. (Again, the *name* ALEX is separate from the *directory* ALEX.) In the *directory* that is ALEX, look for the *name* LITERATURE. We find it. Follow the link in the ALEX directory that leads from the *name* LITERATURE to the *directory* that is LITERATURE. (Again, the name is separate from the directory itself.) In the *directory* that is LITERATURE, look for the *name* PROMO. We find it. Follow the link in the LITERATURE directory that leads from the *name* PROMO to the *file* that is PROMO. (Again, the name is separate from the thing it names, so the name is not part of the file.) You now have the disk node (inode) that is your data. If the file has appropriate permissions, you can read it or write it. "What permissions do I need on a file to delete it from a directory?" None! You don't even have to own the file. You need *write* permissions on the *directory* that contains the *name* of the file, to remove the *name* of the file from the *directory*. (When all links to a file are gone, the file is reclaimed by the system.) Removing a file is a directory operation that deletes a name (and a link); it has absolutely nothing to do with the permissions or owner on whatever the link happens to point to. If you can write the directory that contains the name, you can delete the name from the directory. "What if I want to delete a sub-directory from a directory?" You can only delete from a directory a name that points to a sub-directory if that other sub-directory is *empty*. If you don't have permission to empty out that sub-directory, you won't be able to delete its name from its parent directory. Scenario: Alex creates a directory foo with write permissions for everyone. Ian creates a sub-directory foo/bar with permissions only for Ian, and then creates a file foo/bar/haha. It is now impossible for Alex to delete the foo directory, because Alex cannot empty out and remove the sub-directory foo/bar, because only Ian can remove foo/bar/haha (because only Ian can write foo/bar). Directories Rule (p.81): "X" permissions on a directory mean you can pass "through" the directory to access a thing if you already know the name of the thing you want to access. ("Search permission.") "R" permissions on a directory mean you can see the names in the directory. (The output on the bottom of page 81 is wrong. You can't get a long listing of the files in a directory [ls -l] unless you have X permissions on the directory. Why?) In other words: If you don't have R permission, you can't see the names inside a directory. If you happen to know some names, and you have X permission, you can go *through* the directory to get to the things pointed to by the names that you know. If you don't have X permission, then you can't *get to* the files, even if you have R permission and can see their names. Names are separate from content; you may be able to see the names without being able to pass through the directory to go get the content. "I'm logged in as idallen over in /home/idallen. What permissions do I need to read ../alex/demo?" This is a *relative* pathname. It starts with the current directory and goes up from there. You need X permission on the current directory "." ("." is /home/idallen) to let you pass through using name ".." to your parent directory. In this example, your parent directory would be "/home". ".." is an entry in your current directory, and so you must have X perms on the current directory to use ".." to go up one level. You need X permission on your parent directory to pass through it using name "alex" to directory named "alex". (Remember - names are separate from the things they name!) You need X permission on directory "alex" to pass through using name "demo" to the actual file inode for "demo". You need R permissions on the file inode to read the data. "I'm logged in as idallen over in /home/idallen. What permissions do I need to read /home/alex/demo?" This is an *absolute* pathname. It ignores the current directory. Nothing about the current directory is needed. The path search starts from the ROOT directory. The following sequence is identical, no matter what directory is your current directory: You need X permission on the ROOT directory to pass through it using name "home" to directory named "home". (Remember - names are separate from the things they name!) You need X permission on the "home" directory to pass through it using name "alex" to directory named "alex". (Remember - names are separate from the things they name!) You need X permission on directory "alex" to pass through using name "demo" to the actual file inode for "demo". You need R permissions on the file inode to read the data. "The *name* of a file is separate from the actual file data. What information *is* kept with the file data?" Almost everything else, except the name, is part of the inode that holds the file data: owner, group, access and modify times, permissions, size, etc. None of this appears in the directory; it is all kept in the file inode along with pointers to the disk blocks that contain the actual file data. No matter what name you use to find a file's inode on disk, the file has the same owner, group, permissions, etc. because one copy of all that information is kept *with the file data*, not in the directory. Explain this sequence (removing X permission on a directory): $ mkdir /tmp/idallen $ cd /tmp/idallen $ touch a b c d $ ls a b c d $ chmod -x . $ ls ls: .: Permission denied $ ls .. ls: ..: Permission denied $ ls a b c d ls: a: Permission denied ls: b: Permission denied ls: c: Permission denied ls: d: Permission denied $ ls /tmp/idallen a b c d $ ls -l /tmp/idallen ls: /tmp/idallen/a: Permission denied ls: /tmp/idallen/b: Permission denied ls: /tmp/idallen/c: Permission denied ls: /tmp/idallen/d: Permission denied $ ls /tmp/idallen/a ls: /tmp/idallen/a: Permission denied $ ls -l /tmp/idallen/a ls: /tmp/idallen/a: Permission denied $ chmod +x . chmod: .: Permission denied $ chmod +x /tmp/idallen $ ls a b c d Note how "ls -l /tmp/idallen" can find the *names* in the directory (because it can read the directory); but, it can't go *through* the directory to actually look at what the names point to! So it can't tell you anything about what the names actually *are*. And now study this one (removing R permission on a directory): $ mkdir /tmp/idallen $ cd /tmp/idallen $ touch a b c d $ ls a b c d $ chmod -r . $ ls ls: .: Permission denied $ ls .. file1 file2 file3 idallen $ ls a b c d a b c d $ ls /tmp/idallen ls: /tmp/idallen: Permission denied $ ls -l /tmp/idallen ls: /tmp/idallen: Permission denied $ ls /tmp/idallen/a /tmp/idallen/a $ ls -l /tmp/idallen/a -rw-r--r-- 1 idallen users 0 Jan 28 13:43 /tmp/idallen/a $ chmod +r . $ ls a b c d Without read permission on the directory, we can't find out what names are in the directory. But, if we already know some of the names in the directory we can go through the directory to get the details about what the names point to. --------------------- A Study in File Links --------------------- What is the "link count" field displayed by the "ls -l" command? What causes a file's link count to increment? What happens when a file's link count becomes zero (text p.84)? On ACADAIX, us the "-i" option to "ls" on this set of files: $ ls -li /bin/*sh Note the inode numbers (text p.84) of each name in this directory. Which names are really the same file? Sort the output to make the numbers that are the same come together and be easier to see: $ ls -li /bin/*sh | sort Look at the entire /bin directory and note which names in this directory are actually pointers to the same file: $ ls -li /bin | sort | more Where is the name of the directory "bin" stored? --------------------- Links and Directories --------------------- - What command and options are needed to see the access permissions and link count of a directory, instead of the *contents* of a directory? (text, p.81) - When you are inside a directory, what is the name you use to refer to the directory itself? - How many links does a brand new, empty directory have? Why isn't it just one link, as it is for a new file? - Why does creating a sub-directory in a directory cause the directory's link count to increase by one for every sub-directory created? - Why doesn't the link count of the directory increase when you create files in the directory? --------------------- Building a CGI Script --------------------- A CGI script is a dynamically-generated web page. The content of the page is produced not from a static text file, but rather from the output of commands running on the web server. The content of the web page changes as the output of the commands change. 1) Create a public_html directory in your HOME directory. 2) Put these four lines of text into the file public_html/who.cgi: The lines must start at the left margin. No indenting! (If you haven't learned any VI yet, use the method on the top of page 97 to create the who.cgi text file.) #!/bin/sh echo "Content-Type: text/plain" echo "" who Display the file to verify that it looks EXACTLY as above. 3) Turn on execute permissions (for you, group, and other) on the new who.cgi file. What command do you use? What arguments? Next, test the executable file at the shell prompt first: $ cd public_html $ ./who.cgi | more Verify that the first line of output is "Content-Type: text/plain". The second line must be blank. The rest of the output should be the output of "who". Fix any errors you find. 4) Enter a URL in this form into Netscape: http://acadaix/~abcd0001 Replace abcd0001 with your acadaix userid. If you get a permission denied error, add "x" permissions to your HOME directory to allow others to search it. Make sure the public_html directory has both "r" *and* "x" permissions for everyone. You should now see the name of your who.cgi script. Click on it to make the script execute. If you get a permission denied error, fix the permissions to allow others to read and execute the file. 5) Modify the end of the script to include any other Unix commands you might know, e.g. date, echo, ls, write, etc. Test the script from the shell prompt first, then finally try it from Netscape.