For full marks, follow these task directions below exactly as written. READ ALL THE WORDS to work effectively and not waste your time.
Your instructor will also mark on the due date the work you do in your account on the CLS. Leave all your work on the CLS and do not modify it. Do not delete any assignment work from the CLS until after the course is over.
Set Up – The Base Directory on the CLSIndex
You must keep a list of command names used each week and write down what each command does, as described in the [List of Commands You Should Know]. Without that list to remind you what command names to use, you will find assignments very difficult.
Do a Remote Login to the Course Linux Server (CLS) from any existing computer, using the host name appropriate for whether you are on-campus or off-campus. All work in this assignment must be done on the CLS.
Base Directory: Make the CLS directory named ~/CST8207-16F/Assignments/assignment12, in which you will create the files and scripts resulting from the following tasks. (You do not have to create any directories that you have already created in a previous assignment.) Spelling and capitalization must be exactly as shown:
check
- Create the
check symbolic link needed to run the Checking Program, as described in the section Checking Program below.
This assignment12 directory is called the Base Directory for most pathnames in this assignment. Store your files and answers in this Base Directory, not in your HOME directory or anywhere else.
Use the symbolic link to run the Checking Program to verify your work so far.
Checking only one of your scriptsIndex
Normally the Checking Program checks all the scripts. This can be slow if you are only interested in the check output for one script that you are working on. You can check just one or more individual scripts by giving the script names as arguments to the Checking Program:
$ ./check acol1.sh # only check this script
$ ./check acol1.sh acolNF.sh # only check these two scripts
Do not submit for marking the output of checking only a few scripts!
awk wrapper script: extracting a column of inputIndex
Recall in lecture that we used used the awk program to extract the first space-delimited column of an input stream. We will develop a script named acol (Awk COLumn) that extracts any column of input.
Review Properties of all Scripts, above.
You need to understand Shell Scripts and Control Structures to do these scripts.
The first four scripts have no error checking code. Only the final acol.sh script needs to have full error checking.
Create a one-line script named acol1.sh that uses awk to read its standard input and extract the first column, exactly as used in the lecture notes.
Make sure all the examples below work before you run the checking program! Examples:
$ echo a b c | ./acol1.sh
a
$ date
Sun Dec 4 02:11:41 EST 2016
$ date | ./acol1.sh
Sun
$ last | ./acol1.sh
idallen
idallen
kelleyt
donnelr
[...etc...]
Add comments to Document Your Script.
Check your work so far using the checking program symlink.
Create a one-line script named acol2.sh that uses awk to read its standard input and extract the second column of input.
Make sure all the examples below work before you run the checking program! Examples:
$ echo a b c | ./acol2.sh
b
$ date
Sun Dec 4 02:11:41 EST 2016
$ date | ./acol2.sh
Dec
$ last | ./acol2.sh
pts/9
pts/14
pts/50
pts/50
[...etc...]
Add comments to Document Your Script.
Check your work so far using the checking program symlink.
Create a one-line script named acolNF.sh that uses awk to read its standard input and extract the last (NF) column of input.
Make sure all the examples below work before you run the checking program! Examples:
$ echo a b c | ./acolNF.sh
c
$ echo a b d e f g h i j | ./acolNF.sh
j
$ date
Sun Dec 4 02:11:41 EST 2016
$ date | ./acolNF.sh
2016
$ last | ./acolNF.sh
(00:31)
(02:11)
(02:02)
(00:01)
[...etc...]
Add comments to Document Your Script.
Check your work so far using the checking program symlink.
It should be clear that having a separate script for every possible number of columns is not a good thing. Let’s write one script that takes as its only argument the column number we want awk to print.
Create a one-line script named acolnew.sh that uses awk to read its standard input and extract the column of input given as an argument on the command line.
This new one-line script is a very small modification of your previous one-line scripts; you only need to change about a half-dozen characters in the one line to make it work.
Make sure all the examples below work before you run the checking program! Examples:
$ echo a b c d e f | ./acolnew.sh 1
a
$ echo a b c d e f | ./acolnew.sh 2
b
$ echo a b c d e f | ./acolnew.sh NF
f
The script takes a single argument that is either a number (e.g. 1, 2, etc.) or the string NF and substitutes that argument directly into the awk command line inside the script as the column to print, instead of using the previous hard-coded column number 1, 2, or NF that you used in the three previous scripts.
Hint: Instead of hard-coding the column number in the awk command line, as you did in the above three previous scripts, use the first script argument variable instead of the hard-coded number. You will need to adjust the Quoting around the awk command arguments to allow the shell argument variable to be expanded by the shell but still keep hidden from the shell the $ used by awk to select the column number. You cannot use the awk -v option here.
The new one-line script is a simple modification of your previous one-line script: it is still a one-line script; it does not yet do any input checking or argument processing. If you don’t supply an argument, or you pass something that awk doesn’t understand, you will get an error from the shell or from awk – this is expected:
# examples of errors when script or awk gets bad input:
$ echo a b c d e f | ./acolnew.sh
./acolnew.sh: 8: ./acolnew.sh: 1: parameter not set
$ echo a b c d e f | ./acolnew.sh ''
awk: { print $ }
awk: ^ syntax error
$ echo a b c d e f | ./acolnew.sh ' '
awk: { print $ }
awk: ^ syntax error
$ echo a b c d e f | ./acolnew.sh @
awk: { print $@ }
awk: ^ invalid char '@' in expression
If you supply more than one argument, the script won’t detect the error and will simply ignore the extra arguments – this is okay for this prototype script:
$ echo a b c | ./acolnew.sh NF these arguments after NF are ignored
c
This one-line script does not validate its input, so it gives cryptic errors if the argument is incorrect and it does not warn you if you give too many arguments. This is not a production-quality script. We will fix it in the next step.
Add comments to Document Your Script.
Check your work so far using the checking program symlink.
Copy acolnew.sh to acol.sh and add full input validation to the new script.
The new script must check to make sure it has exactly one input argument and do validation of that one argument before using it with awk to print the given column number. The only valid input allowed must be an integer from 1 to 231-1, or the string NF.
You do not need to change the awk line in your script at all. You only need to add input validation to the script, before calling awk. The input validation will cause the script to exit if the input is bad, so that awk doesn’t process bad input.
Your script should enforce the “only one argument” requirement. Print both a Good Error Message and Usage Message on stderr and exit with a bad error status if the number of arguments is not exactly one.
Also print an error and Usage message and exit if the one argument is equal to the empty string (''), or if it is not all-digits or NF.
Also print an error and Usage message and exit if a numeric argument contains more than 15 digits. (The built-in test command can’t test very large integers!) Hint: You can do this check with a GLOB pattern.
Also print an error and Usage message and exit if a numeric argument is zero or greater than or equal to 231. (The awk program can’t handle huge column numbers: awk: fatal: attempt to access field -2147483648)
Make sure all the examples below work before you run the checking program! Examples:
$ echo a b c | ./acol.sh
...error message about wrong number of arguments prints here...
...usage message prints here...
$ echo $?
1
$ echo a b c | ./acol.sh 1 2 3 a b c >out
...error message about wrong number of arguments prints here...
...usage message prints here...
$ echo $?
1
$ echo a b c | ./acol.sh ''
...error message about empty argument prints here...
...usage message prints here...
$ echo $?
1
$ echo a b c | ./acol.sh ' '
...error message about non-digit, non-NF argument prints here...
...usage message prints here...
$ echo $?
1
$ echo a b c | ./acol.sh '*'
...error message about non-digit, non-NF argument prints here...
...usage message prints here...
$ echo $?
1
$ echo a b c | ./acol.sh 0000000000123456 # 16 digits
...error message about field number being more than 15 digits...
...usage message prints here...
$ echo $?
1
$ echo a b c | ./acol.sh 2147483648 # 2**31
...error message about field number being >= 2**31...
...usage message prints here...
$ echo $?
1
$ echo a b c | ./acol.sh 0
...error message about field number being zero...
...usage message prints here...
$ echo $?
1
$ echo a b c | ./acol.sh 1
a
$ echo $?
0
$ echo a b c | ./acol.sh NF
c
$ echo $?
0
$ echo a b c | ./acol.sh 2147483647 # 2**31 - 1
$ echo $?
0
$ echo a b c | ./acol.sh 000000000000002 # 15 digits
b
$ echo $?
0
Add comments to Document Your Script.
Check your work so far using the checking program symlink.
Extra Work: The awk program interprets numbers with leading zeroes as octal (base 8), which produces unexpected results:
$ seq -s ' ' 1 10
1 2 3 4 5 6 7 8 9 10
$ seq -s ' ' 1 10 | ./acol.sh 10
10
$ seq -s ' ' 1 10 | ./acol.sh 010
8
Find a way to remove leading zeroes before the number gets passed to awk.
acol symbolic linkIndex
You need to understand Symbolic Links, Shell Variables, and Search Path to do this task.
Make sure you have a personal bin/ directory in your account. (You created this directory in a previous assignment.)
Make sure your personal bin/ directory is added to the end of your search path every time you log in. (You did this in a previous assignment.)
Use a relative symlink to link the acol.sh script into your personal bin/ directory under the name acol. Execute the acol program using this name in the bin/ directory to make sure the relative symlink works.
Make sure you can type acol into the shell and have your shell find and run your script from in your bin/ directory using your search path.
Hints: Please Read All The Words in the sentence that begins with the words “Use a” above, especially the word that begins with the letter “r”.
Check your work so far using the Checking Program symlink.
Use your acol script whenever you need to see just one column of data. Take your acol script with you to your next job!
$ fgrep 'refused connect' /var/log/auth.log | acol NF | sort | uniq -c | sort -nr | head -n 3
71064 (116.31.116.28)
17217 (116.31.116.24)
9271 (116.31.116.23)
$ last | acol 1 | sort | uniq -c | sort -nr | head -n 5
192 idallen
150 user1234
121 user2345
117 user3456
110 user4567
$ last | acol 3 | sort | uniq -c | sort -nr | head -n 5
131 cpef0f249a3b983-
122 147.194.28.242
117 23-91-139-86.cpe
94 cpebc4dfb967133-
89 otwaon1056w-lp1
When you are doneIndex
That is all the tasks you need to do.
Read your CLS Linux EMail and remove any messages that may be waiting. See Reading EMail for help.
Check your work a final time using the Checking Program below and save the standard output of that program into a file as described below. Submit that file (and only that one file) to Blackboard following the directions below.
When you are done, log out of the CLS before you close your laptop or close the PuTTY window, by using the shell exit command:
$ exit
