!**************************************************** ! Comments on DAT2330 lecture notes and slides by IB. !**************************************************** ! IB's comments have leading !. )---------- introslide.txt -- Tue 2001-May-29 16:15:07 ================================== Summary Slide for O/S Introduction ================================== (Based on an original by Harold Smith) 9. What is an Operating System? a. 'Software intended to aid in the preparation and execution of application programs' (Harold) OR b. 'Manages the Hardware' (Ian) 10. OS Evolution - 1 Users -> Application Program -> *** OS *** -> Hardware ! That isn't evolution--it's the current state of layering. 11. OS Components a. Boot Facility (IPL) b. Shell (Command Processor, GUI, JCL) c. Task Management d. Memory Management (Virtual Memory) e. I/O Ctrl System (Data Mgmt, Device Drivers) f. Security System g. Networking Facilities h. Utilities (System Tools, Text Editor etc.) i. HELP facilities 12. OS Terms a. Kernel (nucleus) - that portion of the OS that is running at all times. ! What does "running" mean? ! The statement isn't right, for any plausible value of "running". b. Batch vs Interactive Processing -periodic transaction processing vs as occurring' c. Spooling - printer management ! "Spooling" includes more than printing--in the JES context, it includes ! input stream management: inline data and JCL queuing. d. Batch Files (macros, WSH, shell scripts, "procs") e. Multitasking vs Multiprogramming ! There's no broad agreement that those terms contrast, so whatever is ! intended here is artificial--expand "MVT". 13. OS Selection Criteria RASSIM: Reliability, Availability, Serviceability, Scalability, Integrity, Manageability a. Performance (Throughput vs Response Time) b. Availability (Reliability, Serviceability) c. Manageability, Scalability d. AP's/SDK's available (Watch for Vapourware) e. User Friendliness i. Ease of learning vs Ease of use ii. Tailorability f. Security g. Cost (SW, HW, install, support) 14. Major OS's a. DOS/Win 3.1 - Win95/98 b. Win/NT - Win2000 c. UNIX (Ultrix, AIX, Solaris, HP-UX, Linux) ! List of Unices seems incomplete without mention of SysV and BSD. d. MVS - OS/390 ! Current name is "zOS". Not sure of spelling. e. Open VMS (VAX/VMS) ! "OpenVMS" is written as one word; the "Open" is silent. ! The "VAX/VMS" name is very obsolete--when VMS was ported to the Alpha ! architecture, the architecture-specific names were "OpenVMS/VAX" and ! "OpenVMS/Alpha". f. VM/XA, VSE ! VSE has nothing do to with VM, so it's misleading to lump them together. g. OS/2 h. MacOS 15. DOS/Windows a. DOS (81-128K XT) + Win3.1(90)->Win95->Win98->Win2K->WinME->Win?? Windows 98: Suggested "Improvements" over Win 95 i. improved administration tools ii. support for new hardware iii. more efficient file system iv. automated updates via Web browser v. Win Scripting Host (VBS, JS) vi. integrated IE, Active Desktop interface b. Strengths i. AP's available (Installed base) - more than 100M copies sold so far - more than>80% of PC market, and share growing (PCMag) ii. GUI, Multitasking iii. 9 out of 10 (home) users satisfied c. Weaknesses i. single user ii. Task/memory mgmt problems iii. stability, security iv. GUI! v. Constant O/S upgrading and obsolescence - Integrating and maintaining 32-bit OS's "top concern" - survey of 200 IT mgrs at Fortune 1,000 companies. (Datamation) vi. Active Desktop will mean extensive training vii. Windows Update: one of the most compelling features of Win98 - one of the least suited for corporate use: (PCWeek) 16. WinNT / Windows 2000 a. Arch i. 32 bit, UNIX-like; but: 12MB for OS! ii. Intended for high-end workstations and servers b. Strengths i. 32 bit performance, 3D graphics support ii. Networking (NT server) iii. Secure Inet access ! Misleading to use "secure" with WinNT--slashdot, etc. iv. Win95 GUI (in NT 4.0) c. Weaknesses i. lack of RASSIM (vs. UNIX, MVS) iii. See Datamation article: aunixserver.txt iv. GUI (weak batch processing tools) 17. UNIX a. Arch - designed for MT program devel't, kernel/shell b. Strengths i. Cross vendor support ("Open Arch") ii. Efficient, elegant' kernel, MT iii. File System iv. Powerful cmd interface c. Weaknesses i. Open Architecture! (incompatible versions) ii. Cryptic cmd interface (X Windows GUI may help a bit) iii. Lack of RASSIM 6. UNIX Demand? a. Internet! b. Inet, WWW, TCP/IP, all Unix-based c. >60% of Inet servers are Unix today d. >80% of Inet firewalls are Unix e. 1.5 jobs for every available consultant @ $40-60/hr -SVN f. Linux Interest! - NASA using for new space station - MS mentions Linux as only OS named when explaining competition 18. MVS a. Arch - Multiple Virtual Storages, VIO, JES b. Strengths - "85% of existing mainframe apps will stay right where they are to take advantage of mainframe security, performance, and reliability" (Gartner Group) - The mainframe is a natural for e-commerce because that's where the data typically resides. (Datamation) - RevCan, Statcan, CSIS, Bank of Can etc. are MVS shops - no signs of extinction - shipments of System/390 growing steadily since 1994 i. batch job scheduling and control ii. big' (eg 16TB per address space) iii. security (RACF), availability (99.9999%) iv. system management facilities c. Weaknesses i. interactive facilities added later (non GUI) ii. complexity (necessary?), JCL interface 19. The OS of the Future a. Mainframes i. will be with us for the foreseeable future due to... (a) size of investment ('legacy applications') (b) RASSIM' needs of 'mission critical apps' ii. MVS/UNIX, OS/390 b. Servers i. Netware vs WinNT vs OS/390 vs OS/2 ii. UNIX (std version? POSIX? Linux?) c. Desktop i. Win95/8 vs WinNT/2000 vs UNIX (MacOS, OS/2?) ii. NC - WinCE? W2KTrmServs? JavaOS? TalOS? Rhapsody? (1) Linux! )---------- intro.txt -- Wed 2001-May-30 00:00:00 ====================================== Notes on Assessing an Operating System ====================================== (Based on material given in lectures by Harold Smith) Topic 1 - Introduction to Operating Systems - primary focus is detailed technical knowledge of MVS and Unix - College introduces Windows first, so you may think it's the best - don't be blinded by popularity - look at the requirements - you must know, in general, what an O/S is - your clients may ask you for O/S advice - switching O/S requires re-purchasing applications, or rewriting them What is an O/S? - a program (software or firmware) - early hardware had no O/S - punch cards directly into memory ! This misleadingly conflates data content with data entry method. ! There is no reason other than performance and operational nuisance why ! an OS can't be loaded from punch cards (or from paper tape, or even ! from front-panel data-entry switches). ! It is true that early hardware had no OS, but that fact did not result ! from availability or unavailability of peripherals. - had to have your own I/O package to read disk - each hardware change caused programs to be rewritten - O/S took over management of hardware - changing hardware means changing O/S (drivers) ! "Changing" is ambiguous--I first read that as "replacing the whole of ! one's hardware requires replacing the OS". This must be intended to mean ! "adding architecturally new hardware options requires adding device ! drivers to support them". ! But that is contradicted by the next line--"changing" *does* mean ! "replacing". ! So the claim is wrong. One can entirely replace mainframe hardware with ! better-sized mainframe hardware and continue running the same OS. With ! a multi-architecture OS such as VMS or Unix, one can change not only the ! hardware but the architecture and continue running the same OS. - changing O/S means changing applications ! Taking "changing" to again mean "replacing", this claim is wrong. ! Well-written applications in a non-architecture-dependent language should ! (and sometimes even do) port painlessly. - O/S was intended to insulate application from hardware - system programmers handled the O/S; but, now, everyone has to handle the O/S - the O/S is no longer invisible - everyone has to become a Win95 system programmer! - Unix philosophy is better-suited, can hide complexity - "Manages the Hardware" on behalf of all running software - some say: launches other programs (applications) - shells also do part of that - develop applications, debug - software intended to aid in the preparation and execution of applications progrms ! "System software"? Applications *are* software, so that is circular. - Win95 has many things that should be applications built-in to O/S - too many things built into the O/S - MS forgot the purpose of an O/S - can't get rid of built-in things - those who do not learn from the mistakes of the past are doomed to repeat them, e.g. antitrust (IBM) O/S Components - Boot Facility (IPL) - some method to get loaded into memory from disk/CD/tape/cards - "Bootstrap" process, boot deck of cards - BIOS chip separate from Processor - it finds first block of next boot - Shell (Command Line Processor, GUI, JCL) - to tell the O/S what we want it to do - Task Management - allocate processor time - Memory Management - allocate memory (virtual memory - all modern O/S) - very different between O/S! - Win95 task manager crashes a lot - BSOD! - bells and whistles matter more than stability - program size not limited to RAM (virtual memory; swap file) - I/O control system (DOS term) - Data management (MVS) - device drivers - Security System (not Windows) - protect the O/S from you - protect you from other users - protect O/S and you from Internet - Networking Facilities - Utilities (system tools, text editor, etc.) - should O/S include media players or applications? - Help Facilities - GUI in Windows; text-based in Unix - win help is user level only - unix help is deeper (maybe too deep: for programmers!) ! Defect is that Unix help is reference material; assumes that reader ! already knows concepts and needs memory jog. That's the exceptionally good ! parts of it. Much of it just bites, being vague, ambiguous, inconsistent, ! vacuous. I consider all--man, info, HOWTO. Someone who knows how things ! work should begin writing documentation. I promise to stop editorialising. O/S Terms - Kernel (nucleus) - that portion of the OS that is resident (running at all times) - huge for windows due to large O/S ! I criticised "running" previously. "Resident" is also wrong, since ! OSes have dynamically loadable kernel components--chunks of an OS kernel ! may be unresident and unrun for arbitrarily long time intervals. - can tailor kernel to include only relevant services - Batch vs. Interactive processing: periodic vs. real-time - win users never experience batch - MVS is a superb batch processing O/S - mainframes - Win has no batch; Unix has some - JCL is hard to learn because it has to support Batch processing ! JCL is hard to learn because it is poorly designed, with little regard ! for human factors. There is no reason for a batch processing command ! language to be any harder to learn than an interactive command language. ! And, e.g. VMS, the batch command language may be essentially identical ! to the interactive command language--if you learn one, you know both. - many things that you don't have to consider in interactivity - e.g. banking statements, tax forms - e.g. college services - schedule "right away" - transcripts at end of semester - paycheque every 2nd week - a lot of batch work; batch is really different - Spooling - print management - Simultaneous Peripheral Operations On Line - applications were able to share the printer - your application queues print jobs to disk - MVS has complex commands to control spooling - MVS can reprint any page from the spool queue, no need to re-run - Batch files (DOS); macros; WSH; shell scripts; "procs" - Windows Scripting Host (Win2000) - you must master this skill so that you can repeat tasks! - MultiTasking - more than one program loaded; only one is active - supports only one user of the machine ! This is an idiosyncratic use of "multitasking"--zOS, VM, VMS, Unix are ! all commonly described as "multitasking". - MultiProgramming - more than one loaded *and* more than one active - support many users of the same machine (keyboards, screens) O/S Selection Criteria - you cannot answer "which is the "best" O/S"; be more specific - Performance - throughput (batch) vs. Response Time (interactive) - RASSIM: (reliability, availablity, servicablity, scalability, integrity, manageability) - Reliability - how often does it crash? (unplanned downtime) ! Reliability also includes data integrity. ! Data integrity is more important than any other of the listed points-- ! if you don't have data integrity, you don't have anything. ! David Parnas talks at useful length ("the Parnas papers") about reliability ! and/vs trustworth. - Availability - how long is it available, even without crashes? - does it need to be shut down for maintenance? to add software? to be backed up? to replace a disk? to replace memory? - Servicability - how easy is it to maintain and fix? - Scalability - can you keep same O/S as needs increase? - Integrity - security, protection - Manageability - tools to support many users, many disks, etc. - User Friendly - new users vs. experts have different requirements! - tailorable / customizable - fast, efficient, courteous, self-service ! "Courteous"? - Costs - pay vendor, upgrading costs, training costs, maintenance costs - TCO: total cost of ownership - software, hardware, people, installation, support - "Linux is only free if your time is worth nothing" - what Applications are available - what Software Development Kits? - vapourware - promises of nonexistent software - "off the shelf" vs. "in house" software (not outhouse software!) - hardware support (drivers) - technical support - Security Why not replace mainframes with networks of PC's? - you can pull a wagon with a thousand chickens or one good horse - inter-machine overhead becomes significant with networks of machines - BUT: can't get rid of the chickens (users want the friendly interfaces) - BUT: hard to control so many machines - picture of room-size mainframe - silo tape drive - page printer with auto-envelope stuffing - JCL: job control language Major O/S analyses - critical to choose O/S correctly, since applications may not move - may need to train people on new software - installation time to install new software - can I find replacements for all my software that runs on new O/S? - what about my own software (not off-the-shelf) - may take years to convert - rewrite custom applications - think carefully about future of organization Quick review of several O/S: - DOS/Win3.1 (128K, 256K machines of the day became 640K) - Win95 - August 24, 1995; unstable due to old DOS 16-bit code - O/S should protect itself from bad applications! - Win98 (an upgrade) - Win2K - Professional and Server (based on NT, not 95/98) - "Millenium" follow-on to Win98, based on 98 - MS hopes everyone goes from 98 to W2K/Pro; Millenium is to hedge bets - W2K: better tuning, more tools, more knobs; training needed? - better technical depth to help topics (more help) - WinNT: (NotThere, NotTelling); NT5 has become W2K/Server - good for inexperienced LAN administrators - Unix (Ultrix, AIX, Solaris, HP-UX, Linuxes) - MVS - OS/390 (IBM - popular, expanding) - Open VMS (VAX/VMS) (losing share, dying off) [not same as MVS] - OS/2 - IBM attempt to create a windowing O/S to compete with Win3.1 - great O/S; deserves better market share; stable; secure - IBM blew the marketing - likely on the way out? - MacOS - only on Apple Hardware; came first; much more stable ! Before what? Than what? - better design, more efficient; MS stole ideas - now has a Unix kernel (BSD) in Mac OS-X Details on DOS/Windows, Unix, and MVS: DOS/Windows - DOS(81) - Win3.1(90) - Win95 - Win98 - W2K/Pro - Strengths - 100M copies, 80% of market and growing - MS: 9/10 satisfied users - massive amounts of applications - MS gave away SDK's to people to develop programs - large installed base - GUI good for beginners - multi-tasking (under Windows) - Weaknesses - single-user (can't share with another user) - serious RASSIM weaknesses (at least up to W2K) - GUI is an impediment to experienced users (e.g. changing swap file size to prevent it from eating whole drive; full swap file means you can't save data and you can't quit programs to free up memory! No way to create a shortcut to this dialog box.) Experts want direct access to system services. - top concern of 200 IT managers at Fortune 1000 companies is integrating and maintaining 32 bit O/S - integration with other O/S difficult - employers aren't converting to Win98 - improved admin tools? no - support for new hardware? get it for 95 - better file system? get it for 95 - automatic update? bad for corporate control - Corporations say: I'm waiting for 2000 (more stable) - Win95 has no macro facility! - there are: DOS batch file, MVS procedure, UNIX script - Win98 has a Win Scripting Host (VBS, JS) - integrated IE, Active Desktop (different GUI - single click) - ZDnet: you may want to stay with other O/S - WinNT/W2K(server) - designed for 32 bit machines, UNIX-like internals, 12MB for the O/S - 10 times larger than UNIX kernel - Intended for high-end workstations and servers Strengths - 32 bit performance, 3D graphics - Networking (NT/W2K Server) - no experience required to operate - secure Internet access out of the box with inexperienced sysadmin - Win95 GUI (in NT4.0 and later) - you can learn NT: Win95 GUI with Unix-like internals ! I dunno what to make of "Unix-like". I've heard people trying to hype NT ! claim that NT is "VMS-like", intending an implication that NT achieved ! acceptable security. The basis for that claim appears to be only that ! Microsoft hired a couple of ex-DECies and put them to work on NT. So, ! what is "Unix-like"? Weaknesses - RASSIM if compared with experienced sysadmin on UNIX/MVS - GUI still a barrier to experienced users UNIX - 1.5 jobs for every consultant here in Silicon Valley North ($40-$60/hr) - high pressure, pager-tied, evenings, weekends, no family life - architecture designed for program development, not desktops - not designed by a marketing department; two programmers - multi-programming (to share hardware) - designed with a small kernel, most things external to that - can change shells that interact with kernel Strengths - open architecture - started out freely shared - started out open source (now Linux only) ! Unix started out with restrictive licence--the open-source Unix movement ! was late coming, late arriving. Few commercial users had access to source. ! Open-source is not Linux only--see the *BSDs. Linux is the most hyped. - Internet - early development on Unix - 60% of Internet servers are Unix (performance, stability, security) - 80% firewalls are Unix - many vendors have a version - humour is in the O/S; strange things you don't see in MS - Unix taught to a generation of university students (free source) - spread and implemented on many types of hardware - runs on just about anything you can program - commercialization of Unix source meant source became secret - Linus Torvalds wrote an open-source Unix that cannot be made secret again - will show you the copy-left document that mandates open source - therefore, I can change hardware and keep Unix for my applications - efficiency and elegance (showing off programming skill) - e.g. putting command in if statement instead of testing $? - better use of hardware; better performance - elegance is expensive if you have to pay for it - much of Unix programming is done for the love of it - File system - performance, security - Linux lets you mount other O/S (for the love of it) - MS does not mount anything else (for the cash) - Powerful Command line interface - but the command names are very terse and not memorable - not easy to learn; but, easy to use - are GUI's better? the modern way? - cmd line is easier to use - e.g. find all files containing a string in MS vs. Unix - MS gives you file names, not the text containing the string - Unix gets you the lines ! That's a quirk of the utility program design, unrelated to the (de)merits ! of cmd line vs GUI or Unix vs Microsoft. - scripting possible - GUI available to ease transition for new users - Linux interest (the only one MS thinks is a competitor) - employers now taking note Weaknesses - Open architecture - but each vendor modified things slightly - too many flavours of Unix; but, only one version of Windows - hard to standardize an O/S whose philosophy is independence! - many Unix GUI's available too! - cryptic command interface - RASSIM (with respect to MVS) MVS - Multiple Virtual Storage - very large, IBM only, mainframes ! Are there really no clones left standing? - around since 1960's - handles a large percentage of world's computing - runs large mainframes (the best ones!) - only on IBM System/390 hardware (millions of dollars, secure rooms) Strengths - VIO - virtual I/O - can pull entire database into memory, kicking out everyone! ! "Kicking out" makes VIO sound a very bad thing. - JES - job entry subsystem - manage MVS jobs by priority, change parameters, etc. - e.g. you print to Windows and you can't change ordering or copies - reprint a single page from the spool queue - don't want to reprint paycheques! - can't restart some applications that run for hours - need an O/S that has good control of output - batch job scheduling and control - MVS lets you schedule programs based on *ending* time - deadline priority scheduling - how much memory can be assigned to an application? - big: 16TB in each memory address space! - windows limits programs to 4GB, but O/S takes 2GB, leaving 2GB - still 2GB in Win2000! - hundreds/thousands of address spaces! - best security in the industry - RACF Resource Access Control Facility - best commercial security - available: 99.9999% - minutes per year - system management - central control of systems all across the country - mainframe is a natural for e-commerce because that's where the data typically resides - MVS shops: Revenue Canada, Stats Canada, CSIS, Bank of Canada, Metropolitain Life - don't sell your book! - number of installed MVS systems is increasing - it's difficult to learn! but it's worth it - will MVS be used in future? 85% of existing mainframe apps will stay right where they are to take advantage of mainframe security, performance, and reliability. - Gartner Group Weaknesses - built as batch; not originally interactive; interactivity added later - mainframes are batch - e.g. once a week, once a month ! Mainframes run the biggest interactive systems, e.g. bank teller systems ! and airline reservation systems, which have to support tens of thousands of ! concurrent interactive users. ATM withdrawal - interactive Statement produced monthly - batch Yearly tax report - batch - non-GUI - complex (?) JCL Job Control Language - designed in 1960's; not user-friendly - could you control the same things using anything simpler? - need more info about how things run to implement deadline scheduling - necessary complexity to support features - MVS is too big to run here - we have Montreal dial-up account demos (Gricks) (no money for students) - MVS is primarily batch, so JCL is built into a batch of several dozen commands and submitted - use Notepad to write the JCL ! It'd seem more natural to use one of the MVS terminal systems to prepare ! batch jobs for MVS. Sure, you can use anything. I can't deny that someone, ! somewhere, may at some time have used notepad. The OS of the Future? - picking an O/S is a "bet your company" decision - don't pick one that is going to go away - must handle growth - predict what the O/S will be (you do this in your research project) - Harold Smith predicts: - Mainframes - predicted disappearing mainframes (eat your editorial) - size of investment in "legacy" applications - need RASSIM in mission-critical applications - MVS or Unix will run them - prediction: OS/390 (which can now run Unix inside!) - Servers - contenders: Netware(Novell), WinNT, OS/390, OS/2, Unix - Netware has better directory services - Unix (standards? POSIX? Linux?) - good performance; but, complex to administer - too many versions? - prediction: WinNT still easier to install and run on LAN - prediction: UNIX will run on high-performance hardware - Desktop - Win95/98 today; W2K will take over - Unix is not friendly enough; too many GUI - MacOS, OS/2 just haven't made it - prediction: Network Computer / Thin Client - PC will disappear completely - too much intelligence on desktop - centralize the computing; control the software environment - no local data, no way to install software on your own - no floppy, no CDROM; no viruses - no need for everyone to become a system administrator - WinCE, W2K terminal servers, JavaOS, TalOS, Rhapsody, Linux? )---------- acadaix_email.txt -- Fri 2001-Sep- 7 04:33:39 ===================================== Sending and Receiving mail on ACADAIX ===================================== -IAN! idallen@ncf.ca ACADAIX student accounts contain a file named ".forward" whose contents forward all your email to the main Algonqin email server. You normally won't ever receive any email on ACADAIX because of this. If you want to bypass this forwarding temporarily, you can preface an ACADAIX userid with a backslash (two backslashes, if you're typing into a Unix shell). For example, to send mail to "abcd0001" on ACADAIX: $ mail \\abcd0001 Subject: this message will not be forwarded Hi! Userid abcd0001 can read this using the ACADAIX mail command. ^D $ If you want to send mail to ACADAIX from another machine, the backslash trick still works; but, you must specify the full ACADAIX host name: $ mail \\abcd0001@acadaix.algonquincollege.com Subject: this message will not be forwarded Hi! Userid abcd0001 can read this using the ACADAIX mail command. ^D $ If you are typing an email address directly into a mail program (such as Netscape Messenger, Pine, Eudora, etc.), you don't need to double the backslash. One is sufficient: \abcd0001@acadaix.algonquincollege.com Only use double backslashes if typing at the Unix Shell. If you want to permanently disable the ACADAIX forwarding of email to your main Algonquin account, you can remove or rename the .forward file in your ACADAIX $HOME directory. You can also put two addresses into the file, so that your email goes to two places, e.g. \alleni alleni@algonquincollege.com The above contents of my .forward file means that email sent to me on ACADAIX will go both into my real ACADAIX mailbox as well as having a copy of the email sent to my main College mailbox. Note the use of the backslash on the local address to prevent further forwarding. ===================================== Reading your College Email on ACADAIX ===================================== The "fetchmail" program available under Linux has not been compiled and installed on ACADAIX, so you can't use it to get a copy of your main Algonquin email for reading on ACADAIX. (Since the source code to this program is available, you could download it and try to get it to compile yourself, if you're adventurous!) What you can do (from anywhere) is use TELNET to look at individual messages on the POPmail server, by typing the POPmail commands directly into the server: $ telnet inmail.algonquincollege.com pop3 Trying 205.211.30.2... Connected to algmail.algonquincollege.com. Escape character is '^]'. +OK algmail POP3 Server (Version 1.020i) ready. USER alleni +OK please send PASS command PASS mypassword +OK 2 messages ready for alleni in /var/mail/alleni STAT +OK 2 1378 LIST +OK 2 messages; msg# and size (in octets) for undeleted messages: 1 669 2 719 . RETR 1 ...message 1 prints here... RETR 2 ...message 2 prints here... QUIT )---------- chapter1.txt -- Fri 2001-Sep- 7 04:56:36 -------------------------------- DAT2330 - Unix - Prof. Ian Allen -------------------------------- Here are questions you should be able to answer. You will likely find questions similar to these on tests and exams. Chapter 1 - Introduction to Linux. Omit material related to the GUI and X Windows. (Learning the GUI is easy to pick up on your own; we won't cover it this term.) - What is the difference between a generic operating system and a proprietary one? - Why can't companies use Linux code in their own proprietary operating systems? - What is a portable operating system? - What has enabled Linux to be a highly portable operating system? - List some important ways in which a Unix-like operating system differs from Windows 95. - What does the book say is the purpose of a shell? What does your instructor say? ("To find and run commands!") - What does "redirection" of input or output mean? - A Unix "pipe" is a form of simultaneous output and input redirection. What does it do? - As programmers and software developers, what features does Linux offer you? )---------- chapter3.txt -- Fri 2001-Sep- 7 04:56:36 -------------------------------- DAT2330 - Unix - Prof. Ian Allen -------------------------------- Here are questions you should be able to answer. You will likely find questions similar to these on tests and exams. Chapter 3 - Introduction to the Utilities. Omit material related to the "lpr" command and printing. Omit material related to the missing "pine" email program. On ACADAIX, read the output of "man mail" to learn about the IBM version of the traditional UCB Mail program. Floppix also uses a version of the UCB mail program. (This mail program was written in the early 1980's.) - Does the ACADAIX "tar" command have the built-in "gunzip" decompress feature mentioned in the text? Does the Floppix "tar" command? How did you find out? - Chapter Review Questions: On ACADAIX or Linux: 1, 2, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15 Start keeping a summary sheet of Unix commands that you have learned. You will find this very helpful on tests! In Chapter 3 you learned a little about all of these command names: cp, diff, file, grep, head, mv, sort, tail, uniq, compress, gunzip, gzip, zcat, tar, apropos, whereis, which, finger, w, who, mesg talk, write, date, echo (This list came from summary pages 63-64 in the text). NOTE: ITS prevents anyone from using the "finger" or "talk" commands over the Internet while at the College. I've been trying for months to get them to change their policy without success. )---------- README.txt -- Fri 2001-Dec-28 02:57:54 ---------------------------------------- README - Index to this DAT2330 directory ---------------------------------------- Ian! D. Allen - idallen@ncf.ca This directory contains a set of notes for DAT2330 Unix/Linux/MVS. File names of the form "week1.txt" are study notes and homework for each week. Each weekly file includes a list of Floppix Labs to do and Chapters to read. File names of the form "chapter1.txt" are study guides and review questions for each of the Linux text chapters. Each file outlines what is important in the chapter and gives you some review questions based on the chapter. )---------- how_to_print.txt -- Fri 2001-Dec-28 02:59:42 -------------------------------- DAT2330 - Unix - Prof. Ian Allen -------------------------------- ========================= How To Print From ACADAIX ========================= How to get text from a file you have written and prepared on ACADAIX (or Floppix) to the printer on another machine. (You must do this because ACADAIX doesn't have a printer to which you have access.) First, go to some machine that can print on a printer, e.g. to one of the Windows machines at the College, or perhaps your home machine. ----------------------- Fetching Method 1 - FTP ----------------------- Summary (details are below): Start up FTP in a DOS window and connect to ACADIX; fetch the file to the local machine; exit FTP and open the local copy of the file in Winodws WRITE and print it locally using a Courier font. Details: Start a DOS window: - Open the Windows START menu. - Choose RUN - Enter the word "command" in the RUN box to start a DOS text window. Change to a directory in which you would like to store a copy of the file you are going to fetch from ACADAIX. You might want to keep a copy on your N: drive, or on a floppy disk (A:). Change to that directory (and maybe to that disk) first: C:\> A: A:\> Start FTP to acadaix and login to FTP with your usual ACADAIX Name and Password: A:\> ftp acadaix.algonquincollege.com Connected to acadaix.algonquinc.on.ca User: abcd0001 Password: 230 User abcd0001 logged in. ftp> (If you are at the College, you don't need to enter the domain name.) Use the "GET" command of FTP to fetch your print file, e.g. ftp> get 040983746wugga ftp: 1234 bytes received in 0.234Seconds ftp> quit A:\> dir 040983746wugga You've got your file. Proceed to the "Printing" step below. ---------------------------------- Fetching Method 2 - TELNET Logging ---------------------------------- Summary (details are below): Login to ACADAIX and turn on TELNET logging; "cat" the file to the screen; turn off logging and exit TELNET; open the local log file in Winodws WRITE and print it locally using a Courier font. Details: Login to ACADAIX using your usual userid and password. Use the mouse to select the TELNET menu that starts logging to a file. (Remember where you put the file!) Use the "cat" command to cat the file to the screen (and thus into the TELNET log): $ cat 040983746wugga (Note: Do *NOT* use commands that paginate output one screen at a time, since they will mess up the output log. You want the output to appear in one long stream in the log, uninterrupted. Use "cat".) Turn off the TELNET log. Exit from TELNET. You've got your output; it's listed in the log file you created. Proceed to the "Printing" step below. ---------------------------------- Printing the Text File or Log File ---------------------------------- Start up a text editor that can print files, e.g. Windows Write, and open the text file or log file containing your output: A:\> write 040983746wugga ...a window starts up with your text in it... Now print your file from inside "write". Adjust the font size so that the output fits across the paper and does not break up long lines. (Do not use Windows Notepad - the lines come out too long.) Warning: Most Unix output looks best when printed using a fixed-width font such as Courier. Do *NOT* use a proportional font such as Times or Helvetica! Use Courier (or Courier New) only. )---------- resetTerminal.txt -- Fri 2001-Dec-28 03:03:34 =================================================== What to do if your terminal goes into graphics mode =================================================== -IAN! idallen@ncf.ca If you send a binary file to your terminal, it may switch to the graphics character set. In that character set, lower-case letters appear as graphics and line drawing characters. What you type into the keyboard isn't affected; only the display on the screen is scrambled. You can still type commands to Unix. To get back to normal, non-graphic display mode on Linux, type: setterm -reset (This command will not display properly; but, you can type it anyway.) The setterm command does not exist on ACADAIX. You can log out of ACADAIX, reset your Linux console using setterm, and then log back in. Or, you might try executing the ACADAIX shell script /thome/alleni/reset to see if that sets your character set back to normal. )---------- floppix_boot.txt -- Fri 2001-Dec-28 03:04:00 ------------------------------------------- Typical Answers for a Floppix Boot Sequence ------------------------------------------- When answering the questions in the examples below, use your own name, choose your own password, and use your own Algonquin email address. Don't type in "Chris Abcdefghi", or "somepass", or "abcd0001", used in the examples here. Use your own name and userid information! ======= Floppix boot: [My Editorial comments are in square brackets] Enter your name: Chris Abcdefghi Enter your initials (up to 5 letters): cba [don't use upper case] Enter a password (it will not appear on the screen): somepass Re-enter password: somepass Network Configuration 1. standalone 2. fixed IP 3. dhcp 4. dial-up Select option [1, 2, 3, 4]: 3 [at home use either 1 or 4] [ NOTE: Make sure DHCP works and you get "Netork configuration complete". If it says "standalone", you have no networking! ] Mail configuration 1. configure for practice only 2. configure for practice plus real email forwarded to an SMTP server Select option [1 or 2]: 2 Enter your complete email address () > abcd0001@algonquincollege.com [<==use your own userid here] Enter outgoing (smtp) mail server name () > outmail.algonquincollege.com Enter pop3 mail server name () > inmail.algonquincollege.com Network Printer Print server IP address (or [enter] to cancel): [just push return - no printer] ---- Your username is: cba ---- login: cba Password: somepass $ date Tue Feb 8 17:55:02 EST 2000 $ whoami cba $ su Password: somepass # whoami root # exit $ whoami cba Try this (use your own userid): $ mail -v abcd0001@algonquincollege.com Subject: test email This is a test email with verbose turned on. ^D Cc: ...see what prints here as the mail goes out... To bring a copy of all your Algonquin email into Floppix: $ fetchmail -v ....password: [enter your Algonquin email password] ...watch a copy of all the email come in... To read mail on Floppix: $ mail & ? ...list of help commands for mail... Another way (quick and dirty): $ echo $MAIL /usr/spool/mail/cba $ less $MAIL ...see all your messages in one big text file... ...(to get exit from "less" type the letter: q) To reboot gracefully into Windows: $ su Password: somepass # reboot )---------- aunixserver.txt -- Fri 2001-Dec-28 03:05:08 ----------------------------- A UNIX Server Is NO Mainframe ----------------------------- Source: Datamation magazine Copied from: http://www.datamation.com/servr/AUNIXServer.html (The above link is no longer working...) As enterprise servers, UNIX systems still fall short of mainframes in some areas. Measuring the gaps will help you put the right apps on the right platform. By David Simpson The mainframe mantra used to be RAS, for reliability, availability, and serviceability. Those are the areas where Big Iron held distinct advantages over smaller hardware platforms. More recently, mainframers have lengthened the mantra to something like RASSIM, adding scalability, integrity, and manageability to the mix. Client/server aficionados, however, are willing to trade some of those benefits for the flexibility of open systems. They think Big Iron belongs on the scrap heap. These IS managers have relied on UNIX servers as their dominant mainframe alternative, despite some drawbacks. UNIX enterprise servers fall short of mainframes when it comes to availability, manageability (including security), and possibly scalability. An increasing number of IS managers are now choosing Windows NT for their server solution (see Users Are Adding Windows and NT to Their Enterprise Client/Server Mix, December 1, 1995). But, all of the potential shortcomings of UNIX used for enterprise servers apply equally well--or more so--to NT. UNIX vendors are narrowing the gaps. But, in the meantime, understanding the differences between UNIX and MVS environments will help you decide which applications to leave on the corporate mainframe and which to off-load to distributed systems. AVAILABILITY UNIX vendors are using clustering to achieve more availability (see Can't Tolerate Server Downtime? Cluster 'Em! August 15, 1995). Mainframe vendors generally claim almost 100% availability (including scheduled and unscheduled downtime). IBM, for example, claims 99.9999% availability for its Parallel Sysplex architectures, which averages out to a few minutes per year. No UNIX system can claim as much, but adding RAID devices, clustering interconnects, and software can bring a UNIX system close to 99.99% availability. Even so, downtime for UNIX platforms can amount to as much as an hour or more per year. For many applications, that's not a big difference, but in 7x24 shops it can be significant. Studies have shown that downtime can cost on average $1,400 per minute and, in mission-critical 7x24 operations, downtime costs can soar much higher (see "Downtime Costs"). As a result, many IS managers simply opt to keep applications that require very high availability on their mainframes. Analysts advise IS managers intent on moving mission-critical apps over to open systems to take a close look at how their particular UNIX vendor tackles the availability issue. The key, they say, is to ignore reliability claims based on downtime due to hardware failure, which is not the most frequent cause of downtime (see table, " Downtime Causes"). Rather, focus on application availability by factoring in scheduled downtime. Scheduled downtime for software and hardware upgrades and maintenance accounts for up to 25% of all downtime, say analysts. Here, mainframes have the advantage. Via "concurrent maintenance," a mainframe can be shut down without bringing down the entire computing operation. Even with clustered UNIX systems, you may have to shut down systems for the duration of the downtime--or at least suffer noticeable performance degradation. UNIX vendors, however, are working on decreasing scheduled downtime by adding to their clustering schemes features such as rolling software upgrades (which enable software to be upgraded without shutting down the system) and journaled file systems (which allow faster reboots). Although you should consider a variety of factors, in particular don't overlook the switchover times (failover) of clusters. In best-case scenarios, failover will take from 30 seconds to five minutes in clusters with expensive high-speed interconnects. If your applications can't tolerate that much downtime, it may be best to keep them on the mainframe. Also bear in mind that actual switchover times depend on a number of factors, including the size of the database, how quickly the database (as opposed to the hardware) can recover, and the number of transactions that have to be recovered. To boost UNIX clusters up to near-mainframe availability, you'll have to add expensive equipment, including fully redundant components and high-speed, often proprietary, interconnects. Costs for this added equipment can add up. Eventually, it may erase the price advantages of an "open" system platform. Network downtime is another factor that affects availability. Here, again, the gap between clustered UNIX systems and mainframes is narrowing, but IS managers with experience in both environments say that network downtime is often a more significant factor than either system hardware or software failures. Mainframe networks approach the uptime of their connected systems, but in large UNIX-TCP/IP LANs (sometimes referred to as "spray-and-pray" by mainframe bigots), the network becomes an Achilles' heel. According to Charlie Burns, an analyst with the Gartner Group consulting firm in Stamford, Conn., large UNIX LANs often drop to 70% availability. The functionality built into MVS, along with the sophistication of its network-monitoring and management tools, makes it easier to detect and recover from network-based errors. "UNIX-based management tools are becoming better, but they're still rudimentary compared to mainframe management tools," says Burns. Scaling Up Scalability MANAGEABILITY Manageability is another often-cited mainframe advantage. And in fact, the high cost of system management in distributed environments is one of the largest contributors to the unexpectedly steep costs of client/server computing. In a Gartner Group survey, IS managers said system management is the second biggest challenge they face in moving to client/server (overall complexity is first). IS managers point to vendors such as Boole & Babbage (Command/Post), Computer Associates (CA-Unicenter), and IBM (SystemView) as having done a decent job of bringing the functionality and ease of use of their mainframe management products into the open system arena. But, management is still a big problem, especially for a far-flung distribution of clients and servers. Gartner Group studies show that the complexity of managing distributed client/server environments rises geometrically as components are added. In other words, as you double the number of managed devices, you quadruple the complexity--and, to a lesser degree, costs--of managing those devices. Again, the problem is related to the network: While mainframe environments are hierarchical, UNIX networks are more peer to peer, which is inherently more difficult to manage. The system management functions IS managers complain most about in client/server systems are performance monitoring and management, problem resolution, software distribution, job scheduling, capacity planning, and storage management. "For systems management in UNIX environments, we're turning the clocks back to the '70s, although the management software vendors are catching up pretty rapidly," says Dave Dubnick, manager of technology operations at CCH, a Lake Success, N.Y., software and service provider for the legal and tax markets. He says that capacity planning tools for UNIX environments are particularly weak compared with the tools available in the mainframe world. Dubnick advises buying management tools from as few vendors as possible to reduce complexity and the potential of noninteroperability between packages. CCH uses CA-Unicenter for almost all of its system management tasks. One reason why UNIX management tools suffer in comparison to mainframes is that vendors have to write for so many different versions of UNIX. "Systems management tools in the UNIX world are too generalized because they have to run on so many hardware platforms, UNIX variants, and networks. Generalization breeds compromise," says Richard Finkelstein, president of Performance Computing, a consulting firm in Chicago. The bottom-line advice on client/server system management? Don't expect the level of top-down management you had in the MVS world and be sure to factor in rising management costs and additional personnel as you budget for your migration to client/server. SECURITY Within the realm of system management, lack of security in UNIX environments is another big concern among IS managers. Security issues range from the operating system to third-party packages and network vulnerability. Gartner Group's Burns says, "Mainframe prices are dropping fast enough [approximately 30% per year] that a lot of companies are rethinking their planned move off of mainframes because it's just not worth the risks." At the layered software level, there are no products in the UNIX arena that rival the security of mainframe mainstays like IBM's RACF or CA's TopSecret and ACF2. However, some near-equivalent packages are starting to emerge. The UNIX package that comes closest to RACF-level security, according to Gartner Group analyst William Malick, is Memco's SeOS, which is resold by Platinum Software. And CCH's Dubnick thinks the security modules in CA-Unicenter are almost equivalent to ACF2 and TopSecret. A variety of other vendors supply beefed-up security packages for client/server environments, including Bull, Dynasoft, 4th Dimension Software, IBM, ICL, and Raxco's Axent subsidiary. Robert Melford, a Mission Viejo, Calif.-based consultant who specializes in security, says that "you can make UNIX basically as secure as an MVS-RACF environment. It's just a lot more work for systems administrators." Third-party security packages, says Melford, may not actually boost the security of UNIX, but they make it easier to administer, with features such as enhanced audit capabilities and improved account management. Although the lack of security in client/server environments is a problem, bear in mind analysts' estimates that typically only 10 to 20% of corporate data requires restricted access. Then again, sacred cows make the best hamburgers. // )---------- week1.txt -- Fri 2001-Dec-28 03:40:00 Week 1 Notes for DAT2330 - Ian Allen Supplement to Text, outlining the material done in class and lab. Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM![*] ([*] Read The Fine Manual) Make sure you can log in to the ACADAIX Unix machine. If not, go to the ITS Help Desk to have your account reset. Buy *both* of the the course texts before they are all gone, and before the bookstore sends the extra copies back to the publisher! Learn how to use the news software. See the link on the course home page. Post a message in the news group algonquinc.test and then cancel it. Read the course announcements frequently, especially before tests. You will find a link to the announcements new group outlined in blue near the top of the home page for this course: http://www.algonquincollege.com/~alleni/dat2330/02w/ Look under the "Notes" button on the course web page for these study notes for the Linux textbook: chapter1.txt (Introduction) Download Floppix and make two (TWO!) separate copies on diskettes. (Floppy disks wear out - you will need two copies of each diskette.) Complete these Floppix Labs from http://floppix.ccai.com/ Floppix Lab: 1 (Geting Started) Floppix Lab: 2 (Running Commands) You can't boot Floppix from an MSDOS window running under Windows9x. You must shut down Windows and restart in "pure" MSDOS mode. Most Algonquin labs will not let you boot anything other than Windows. We apologize for this - is is a new department policy. You can safely run Floppix at home; however, you must choose "standalone" instead of "dhcp" for your networking. =============== Floppix Failure =============== Watch the Floppix boot process for errors! If, during the Floppix boot process, you see anything that looks like "not enough memory" or "failed" or "cannot fork" or "CRC error" or "kernel panic" or "error", your Floppix did not boot correctly. Try booting from a different set of Floppix diskettes. If the problem persists, try different hardware (change to another computer). If your DHCP networking (option 3) did not give you an IP address, you will find that network commands such as "telnet acadaix" will say "no such host name". Become the super-user (see the Floppix lab #27 on "superuser") and retry the networking: $ su Password: # /etc/init.d/network 1.standalone - choose this option if you are unsure 2.fixed IP (experts only) 3.dhcp (server-assigned IP address) 4.dialup connection (limited support - note: a winmodem is NOT a modem) Choose DHCP (#3) again. If it says "Network configuration complete", you now have an IP address and things should work correctly. If it still gives you a "standalone" configuration, the network is not connected and you cannot use network commands such as "telnet" or "ftp". ==================== Rebooting to Windows ==================== To reboot the machine while it is running Floppix (e.g. to reboot into Windows again), you have three choices: 1) become Super User (root) and type: shutdown -r now 2) use the standard CTRL-ALT-DEL key sequence 3) push the RESET button (Floppix ONLY!) Never push the RESET button on a real Linux or Unix system - you will damage the file system. Floppix is RAM-based; it doesn't use a disk. All files created or modified while running Floppix will be lost upon reboot. (Floppix runs entirely in RAM - nothing is changed on disk.) =========================== TELNET to Unix from Windows =========================== I don't recommend using the Windows TELNET clients - they display the screen poorly and students often forget to set up their windows correctly, which may lead to people corrupting files that they edit incorrectly. If you use TELNET under Windows, you must configure your Windows TELNET client to connect correctly to a Unix system. See the course web page named "Using Telnet". What three things must you do when you use TELNET to Unix from Windows? 1) drag TELNET window to full size (it will not expand further) 2) terminal type: vt100 3) lines: 24 From the College, you can telnet to the host name: acadaix From the Internet, telnet to the name: acadaix.algonquincollege.com ============================== EOF and Interrupting Processes ============================== To interrupt a process that is running on your terminal, use the Interrupt Character, often CTRL-C (^C). (You can program a different character; often DEL is used.) Your EOF character, signalling end of input, is usually CTRL-D (^D). (You can program a different character; but, it is almost never done.) Interrupting a process usually terminates the process. Whatever the process was doing is left incomplete and unfinished. (Files will be incomplete.) Sending the EOF character tells the process that you are finished typing at your terminal; but, it does not interrupt or terminate the process. The process will finish whatever it is doing. ==================== Basic Command Syntax ==================== Many Unix commands need both a VERB (what to do) and an OBJECT (what to do it on). The following incorrect attempts at Unix commands are wrong: $ /etc/passwd (missing VERB; what are you trying to DO?) $ cat (missing OBJECT; catenate WHAT file?) Remember to tell Unix both what you want to do and to what object you wish to do it. ===================================== Commands for a multi-user Unix system ===================================== These commands are useful for looking at or interacting with other people on the same machine (e.g. on ACADAIX): - who - w - finger - write - talk Both talk and finger take user@machine names, allowing interaction with people on other machines (if they permit it, and if the network permits it). Note: The ITS department may be blocking finger and talk from working over the Internet. To prevent users from writing on your screen, use: mesg n To exit from write, type your EOF character at the beginning of a line. To exit from talk, type your Interrupt Character. Remember: all these programs have manual pages! "man write" ====================== Notes on GNU and Linux ====================== GNU - Gnu's Not Unix - GNU is a Free Software Foundation (FSF) project - rewrote Unix in the public domain (the way it started out) - chief architect: Richard Stallman (original author of EMACS) Linux "distribution" == Linux Kernel + GNU Utilities - The GNU kernel hasn't progressed far (named HURD) - So, we use the Linux kernel with all the Unix-compatible GNU utility software )---------- week3.txt -- Fri 2002-Jan-11 00:26:00 Week 3 Notes for DAT2330 - Ian Allen Supplement to Text, outlining the material done in class and lab. Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM![*] ([*] Read The Fine Manual) Look under the "Notes" button on the course web page for these study notes for the Linux textbook: chapter4.txt (the Linux File System) Complete these Floppix Labs on http://floppix.ccai.com/ Floppix Lab 3: logical organization of the Linux file system Floppix Lab 4: navigating the file system Floppix Lab 5: directory listings Floppix Lab 6: directory commands Floppix Lab 7: file commands ------------------------------------------ Unix/Linux File System - (the weird parts) ------------------------------------------ Read the text (Chapter 4) for the non-weird parts. All the *diagrams* showing file systems and links in the text are wrong and range from confusing to seriously misleading. Here's the truth. Names are stored in directories, not with the things to which the names refer. The text makes the error of putting the names on the things that are being named. That is misleading and the cause of many misunderstandings about Unix/Linux files and directories. When you look at a Unix pathname, consider that the slashes separate names of pathname components. All the components to the left of the rightmost slash must be directories, including the "empty" directory name to the left of the leftmost slash. For example: /home/alex/names In the above example, there are three slashes and therefore four pathname components. The "empty" name in front of the first slash is the name of the ROOT directory. The ROOT directory doesn't have a name. (Some books get around this by calling the ROOT directory "slash" or "/". That is wrong. ROOT doesn't have a name.) Inside the ROOT directory is the name of the "home" directory. Here is the correct diagram for Figure 4.4 on page 70. Note that the names for things are one level *above* the things to which the names actually refer: +------+ | home | This box is the ROOT directory. It has the name "home" in it. +------+ The *directory* "home" is not here; only the *name* is here. | The ROOT directory itself does not have a name! | +------+ | alex | This box is the "home" directory. It has the name "alex" in it. +------+ The name "home" isn't here; it's up in the ROOT directory. | | +----------------------------------+ This box is the "alex" directory. | names | temp | literature | demo | It has many names in it. Note: +----------------------------------+ the dir name "alex" isn't kept here. | | | | The name is up in the "home" directory. | | | | *----* *----* | *----* These are disk nodes containing the |data| |data| | |data| data for each of the files named in *----* *----* | *----* the directory "alex". No names are here! +-------+ | promo | This box is the "literature" directory. +-------+ It has the name "promo" in it. | | *----* This is a disk node (inode) containing the |data| data for the file named "promo". The name "promo" *----* is one level above, in the "literature" directory. To follow a path such as /home/alex/literature/promo just walk the tree. Start with the ROOT directory (which doesn't have a name, since it does not appear in any parent directory) and look for the first pathname component ("home") inside. Let's trace the pathname: Look in the ROOT directory for the *name* of the first pathname component: "home". We find the name "home" inside the ROOT directory. Follow the link in the ROOT directory that leads from the *name* "home" to the actual *directory* that is "home". (Note how the names are separate from the things they name. The actual *directory* is nowhere near its name.) In the *directory* that has the name "home", look for the *name* "alex". We find it. Follow the link in the "home" directory that leads from the *name* "alex" to the actual *directory* that is ALEX. (Again, the *name* "alex" is separate from the *directory* "alex".) In the *directory* that has the name "alex", look for the *name* "literature". We find it. Follow the link in the "alex" directory that leads from the *name* "literature" to the *directory* that is "literature". (Again, the *name* "literature" is separate from the *directory* "literature".) In the *directory* that is "literature", look for the *name* "promo". We find it. Follow the link in the "literature" directory that leads from the *name* "promo" to the *file* that is "promo". (Again, the name is separate from the thing it names, so the name is not part of the data that makes up the file.) You now have the disk node (inode) that is your file data. If the file has appropriate permissions, you can read it or write it. "What permissions do I need on a file to delete it from a directory?" None! You don't even have to own the file. You need *write* and *search* permissions on the *directory* that contains the *name* of the file, to remove the *name* of the file from the *directory*. (When all links to a file are gone, the file is reclaimed by the system.) Removing a file is a directory operation that deletes a name (a name is a link); it has absolutely nothing to do with the permissions or owner on whatever the link happens to point to. If you can write and search the directory that contains the file name ("wx" permissions on the directory), you can delete the file name (the link) from the directory. "What if I want to delete a sub-directory from a directory?" You need "wx" permissions to remove the sub-directory name from the directory, just as if the name pointed to a file. However: You can only delete from a directory a name that points to a sub-directory if that other sub-directory is *empty*. If you don't have permission to empty out that sub-directory, you won't be able to delete its name from its parent directory, even if you have permissions on the parent directory. Scenario: Alex creates, in his home directory, a sub-directory "foo" with write permissions for everyone. Ian creates a sub-sub-directory "foo/bar" with permissions only for Ian, and then Ian creates a file "foo/bar/haha". It is now impossible for Alex to delete the foo sub-directory from his own home directory, because Alex cannot empty out and remove the sub-sub-directory foo/bar, because only Ian has permissions to remove the file name foo/bar/haha from the foo/bar sub-sub-directory. (Only Ian can write the sub-sub-directory named foo/bar.) Alex is stuck with the foo sub-directory in his account. Alex can rename it (e.g. mv foo .junk); but, he can't delete foo until Ian makes it empty. Only empty directories may be deleted. Directories Rule (p.81): "X" permissions on a directory mean you can pass "through" the directory to access a thing if you already know the name of the thing you want to access. ("Search permission.") "R" permissions on a directory mean you can see the names in the directory. The output on the bottom of page 81 is wrong. You can't get a *long* listing of the files in a directory ("ls -l") unless you have X permissions to pass through the directory to find out what kind of nodes the names in the directory represent. You can get a short listing of just the *names* in a directory without needing X permissions. In other words: If you don't have R permission, you can't see the names inside a directory. If you happen to know some names, and you have X permission, you can go *through* the directory to get to the things pointed to by the names that you know. If you don't have X permission, then you can't *get to* the files, even if you have R permission and can see their names. Names are separate from content; you may be able to see the names without being able to pass through the directory to go get the content. "I'm logged in as idallen over in /home/idallen. What permissions do I need to read ../alex/demo?" This is a *relative* pathname. It starts with the current directory and goes up from there. You need X permission on the current directory "." ("." is /home/idallen) to let you pass through using name ".." to your parent directory. In this example, your parent directory would be "/home". ".." is a relative pathname, meaning it starts with an entry in your current directory, and so you must have X perms on the current directory to use the name ".." in the current directory to go up one level. You need X permission on your parent directory ("home") to pass through it using name "alex" to the inode that is the actual directory named "alex". (Remember - names are separate from the things they name!) You need X permission on directory "alex" to pass through it using name "demo" to the actual file inode for the name "demo". You need R permissions on the "demo" file inode to read the data. "I'm logged in as idallen over in /home/idallen. What permissions do I need to read /home/alex/demo?" This is an *absolute* pathname. It ignores the current directory. Nothing about the current directory is needed. The path search starts from the ROOT directory. The following sequence is identical, no matter what directory is your current directory: You need X permission on the ROOT directory to pass through it using name "home" to the inode of the directory named "home". (Remember - names are separate from the things they name!) You need X permission on the inode of the "home" directory to pass through it using name "alex" to inode of the directory named "alex". (Remember - names are separate from the things they name!) You need X permission on inode of the directory "alex" to pass through it using name "demo" to the actual file inode for "demo". You need R permissions on the "demo" file inode to read the data. "The *name* of a file is separate from the actual file data. What information *is* kept with the file data?" Everything else, except the name, is part of the inode that holds the file data. This means: owner, group, access and modify times, permissions, size, etc. None of this information appears in the directory; it is all kept in the file inode along with the pointers to the disk blocks that contain the actual file data. No matter what name you use to find a file's inode on disk, the file has the same owner, group, permissions, etc. because one copy of all that information is kept in the inode *with the file data*, not in the directory. The only things you will find in a Unix directory are names and inode numbers - everything else is in the inode of the file. Explain this sequence (removing X permission on a directory): $ mkdir /tmp/idallen $ cd /tmp/idallen $ touch a b c d $ ls a b c d $ chmod -x . $ ls ls: .: Permission denied $ ls .. ls: ..: Permission denied $ ls a b c d ls: a: Permission denied ls: b: Permission denied ls: c: Permission denied ls: d: Permission denied $ ls /tmp/idallen a b c d $ ls -l /tmp/idallen ls: /tmp/idallen/a: Permission denied ls: /tmp/idallen/b: Permission denied ls: /tmp/idallen/c: Permission denied ls: /tmp/idallen/d: Permission denied $ ls /tmp/idallen/a ls: /tmp/idallen/a: Permission denied $ ls -l /tmp/idallen/a ls: /tmp/idallen/a: Permission denied $ chmod +x . chmod: .: Permission denied $ chmod +x /tmp/idallen $ ls a b c d Note how "ls -l /tmp/idallen" can find the *names* in the directory (because it can read the directory); but, it can't go *through* the directory to actually look at what the names point to! So it can't tell you anything about what the names actually *are*. An "ls -l" long listing needs to know what the names *are* - it requires more permissions that a plain "ls" that only needs to know the names, not what they are. And now study this one (removing R permission on a directory): $ mkdir /tmp/idallen $ cd /tmp/idallen $ touch a b c d $ ls a b c d $ chmod -r . $ ls ls: .: Permission denied $ ls .. file1 file2 file3 idallen $ ls a b c d a b c d $ ls /tmp/idallen ls: /tmp/idallen: Permission denied $ ls -l /tmp/idallen ls: /tmp/idallen: Permission denied $ ls /tmp/idallen/a /tmp/idallen/a $ ls -l /tmp/idallen/a -rw-r--r-- 1 idallen users 0 Jan 28 13:43 /tmp/idallen/a $ chmod +r . $ ls a b c d Without read permission on the directory, we can't find out what names are in the directory. But, if we already know some of the names in the directory we can go through the directory to get the details about the inodes that the the names point to, because we still have X (search) permission. --------------------- A Study in File Links --------------------- What is the "link count" field displayed by the "ls -l" command? What causes a file's link count to increment? What happens when a file's link count becomes zero (text p.84)? On ACADAIX, use the "-i" option to "ls" on this set of files: $ ls -li /usr/bin/*sh Note the inode numbers (text p.84) of each name in this directory. Which names are really the same file? (The inode numbers will tell you!) Sort the output to make the inode numbers that are the same come together and be easier to see: $ ls -li /usr/bin/*sh | sort Look at the entire /usr/bin directory and note which names in this directory are actually pointers (hard links) to the same file: $ ls -li /usr/bin | sort | more Where is the name of the file "zsh" stored? Where is the name of the directory "bin" stored? Where is the name of the directory "usr" stored? --------------------- Links and Directories --------------------- - What command and options are needed to see the access permissions and link count of a directory, instead of the *contents* of a directory? (text, p.81) - When you are inside a directory, what is the name you use to refer to the directory itself? (This name works inside any directory.) - How many links does a brand new, empty directory have? Why isn't it just one link, as it is for a new file? (In other words, why does a new file have one link and a new directory have more than that?) - Why does creating a sub-directory in a directory cause the directory's link count to increase by one for every sub-directory created? - Why doesn't the link count of the directory increase when you create files in the directory? - Give the Unix command and its output that shows the inode number and owners of the following directories: a) your HOME directory b) the /bin directory c) the root directory Note: Show only one line of output for each single directory; do not show the contents of the directory. Use a command (and options) that will show only the directory itself, not its contents. )---------- chapter4.txt -- Fri 2002-Jan-11 00:33:00 -------------------------------- DAT2330 - Unix - Prof. Ian Allen -------------------------------- Here are questions you should be able to answer. You will likely find questions similar to these on tests and exams. Omit the optional material related to Symbolic Links in this chapter. Chapter 4 - The Unix/Linux File System. - The file system hierarchy diagrams in this chapter are misleading. See the weekly lecture notes for a correct sample diagram. (Names in directories are separated from the things they name!) - It's up to every individual program on Unix to decide if file name extensions have any meaning; there is no system-wide support for one standard. For example, the C compiler has a convention that recognizes *.o and *.a as being object files and libraries, but the "more" command is unaware of this convention and happily displays these binary files as if they were plain text files. There is no universal concept that a file with a particular extension is associated with a particular application. - If you telnet twice to a Unix machine (either ACADAIX or Floppix), and you change the working directory of the shell in one telnet session, does the working directory for the shell in the other sesion change too? - If you change the working directory of your shell, then you start up a new copy of the shell by typing, say, "ksh" or "bash", does the new copy of the shell have the changed working directory or the original working directory? - What is the difference between the command lines "ls" and "ls ."? (If you don't supply an argument to the "ls" command, on what pathname does it operate?) - True or False: The "mkdir" command creates a new directory, and then makes the new directory your current working directory. - What option to the "ls" command will change its output to enable you to distinguish file names from directory names? Create a shell command alias for "ls" that uses this option. (To create an alias, see the Linux text index entries for "alias".) - What is the difference between pathname "foo" and "./foo" and "./././foo"? - What is the difference between pathname "/bin/ksh" and "/bin/./ksh" and "/bin/./././././ksh"? - Explain this output (pay close attention to the glob patterns used): $ echo /* | wc 1 56 471 $ echo /*/* | wc 1 5356 81164 What do you expect would be the shell expansion of "/*/*/*"? (Don't type this on ACADAIX - it's a very expensive computation! Try it on Floppix only.) Would the pathname /bin/ksh be included in the expansion of "/*/*/*"? (Don't type this on ACADAIX - it's a very expensive computation!) - Explain this ACADAIX output (pay close attention to the glob patterns used): $ cd / $ echo ./* | wc 1 57 521 $ echo */. | wc 1 38 310 Why is the number of words different? What is the difference between the shell glob patterns "./*" and "*/."? - Use the "ls" command to explore the list of "standard" directories given on page 75-76 and find out which ones are not present on ACADAIX. - Students who don't take the time to understand how relative pathnames work (especially "..") often type absolute path names to every file they want to access. Why does this slow them down, relative to students who use relative pathnames? What kind of Unix user are you choosing to be? - What Unix command renames files or directories? - What permissions do you need on a file or directory to be able to rename it? - The text says that directories cannot be copied with "cp". This is true of old versions of "cp". New versions, usually found only on Linux, can duplicate a directory and all of the files and directories inside it. The man page for cp will tell you how. - What is the difference in behaviour between a directory that only has X permissions and a directory that only has R permissions? (See the weekly notes for examples of this.) - What permissions are needed on a file to enable you to remove the file name from a directory? (Trick question!) - Try an exercise similar to the one the bottom of page 81: Create a sub-directory. Create a file in that sub-directory. Remove X permission from the directory. (Use the ls command to verify that you have done this successfully.) Try to use "ls -l" on the sub-directory. In the text, doing this shows the contents of the sub-directory and the details about the files contained therein. What does it do for you? - What information does the "-i" option to "ls" add to the output? - Execute these commands on ACADAIX and explain the output: $ ls -li /usr/bin/*sh $ ls -li /usr/bin/*sh | sort How does adding the sort make decoding what you see easier? - If you own a file or directory, but you give me permission to write to it, can I use the "chmod" command to change its permissions? - True or False: The first time you create a file, you create a real file. Subsequent hard links to the file, made using the "ln" command, are only pointers to the file and aren't as strong as the original file name. Removing the original file name will destroy the file. Removing one of the links only affects that one link. - True or False: The "ln" command takes two arguments, so the maximum number of hard links a file can have is two. - True or False: To make a hard link to file "foo" that is named "bar", file "foo" must exist. - True or False: You can make a hard link to a directory. - By using a positive number (e.g. +5) as an argument to the "tail" command, you can skip over the first N-1 lines of a file or of standard input. Try: who | tail +5 Now read Review Question #6 and write a Unix pipeline that answers part (a). The pipeline must count the number of file systems on the local machine. (You will need at least three commands in the pipeline.) Does the same pipeline work on ACADAIX and on Linux? - Chapter 4 non-Advanced Review Questions: On ACADAIX or Linux: 1 - 11 (all) - Chapter 4 Advanced Review Questions: On ACADAIX or Linux: 12, 13, 14, 16 Unix commands studied (summary p.88): cd, chmod, ln, mkdir, pwd, rmdir )---------- test1-unix.txt -- Fri 2002-Jan-11 00:38:00 ===================================== What to Study for the First Unix Test ===================================== -IAN! idallen@ncf.ca The First Test (Unix) takes place in the last hour of your Lab period in Week 4 of the course. (See the Course Outline.) The test is 50 minutes long. It is worth 5% of your term mark. It covers Chapters 1-3 in the Linux text. It also covers the web page "Notes" files week1.txt and week2.txt and the chapter notes files and Floppix labs mentioned in those weekly files. What you must bring to the test: 1. You will need your Floppix diskettes (and your backup copies). You must know how to boot Floppix with DHCP networking enabled. 2. You will need a pencil (only a pencil) to write in answers to multiple-choice questions on a mark-sense sheet. You must use a pencil. 3. Your name, EMail, and web password must be registered with me at least 24 hours before the test begins, using the registration form on the course web page. If you are not registered, you won't have a userid on the Linux test machine and won't be able to log in to that test machine to answer questions. You will answer questions on Floppix, on ACADAIX, and on a Linux test machine. You must know how to use telnet on Floppix to telnet to ACADAIX and to the Linux test machine. No Windows access is permitted during the test. The address of the Linux test machine will be given out in the announcements news group a few days before the test begins. Make sure you can login to the machine using the web password you used to register with me. Knowing how to switch among the three Floppix consoles will save you time, since you can have one console logged into ACADAIX and another logged in to the Linux test machine at the same time. The test is open-book, open-note, and open-terminal. You may use any online or written resources available to you; but, you may not consult with or communicate with anyone except the instructor in any way. Answers will be written down on paper. There is no electronic logging or uploading required in this first Unix test. All questions will be multiple-choice. Some questions will require you to execute commands on Floppix, on ACADAIX, and on the Linux test machine. You will not have enough time to try every multiple-choice question on the computer or to look up all the terms in the textbook. The test will appear to be "too long" if you try to type in all the questions to see the output. Thinking about the correct answer will be faster. Most of the questions can be answered quickly from the knowledge you have acquired in lectures and by doing the weekly readings and homework material. Your Floppix lab Exercises and Questions & Answers are very important. )---------- arguments_and_options.txt -- Sat 2002-Jan-12 05:59:52 =========================================== Options and Arguments on Unix Command Lines =========================================== -IAN! idallen@freenet.carleton.ca Someone asked: >Could you explain to me the difference between an option and an argument? The usage depends on whether you are looking at a command line from the point of view of the Unix shell or from the point of view of the command that the shell is finding and running for you. For example, you might type this command line into a Unix shell: $ ls -l -i -s a b c The shell knows nothing about how the "ls" command works. The shell only knows that you want to find and run the "ls" command (the first string on the command line) and you are providing six other strings as "arguments" to that "ls" command: -l -i -s a b c. So the correct answer here, from the point of view of the shell, is "the shell is giving the "ls" command six arguments from the command line". Each "argument" is separated by blanks from its neighbour; there are six of them. However, from the point of view of the "ls" command receiving the six "arguments", three of those arguments are interpreted by the "ls" command as options; because, three of the arguments start with dashes and dashes are interpreted by the "ls" command as options. So the correct answer here, from the point of view of the "ls" command, is "the ls command is interpreting the six command line arguments as three option arguments and three pathname arguments". This is often simplified to say "the ls command is receiving three options and three arguments". So, from the point of view of the shell, there are six "arguments" to the "ls" command. From the point of view of the "ls" command, three of those arguments (the ones starting with dashes) are going to be interpreted as options that change how "ls" operates. To summarize: $ ls -l -i -s a b c Shell: six arguments on the command line to "ls" ls: three options and three arguments received from the command line Look at this similar command line: $ echo -l -i -s a b c The above command line is identical to the "ls" command line example above, except that it uses "echo" as the command name to be found by the shell. In this "echo" example, the number of arguments counted from the point of view of the shell does not change. The command name doesn't matter to the shell - there are still six blank-separated arguments on the command line that are being passed to whatever command you are typing, whether it be "ls" or "echo" or anything else. The shell still parses six arguments. However, from the point of view of the "echo" command, there are *no* options in this argument list; because, the "echo" command does not treat the strings -l or -i or -s as options! None of those strings is a valid option to the "echo" command. All six strings are treated as "arguments" by the "echo" command; all six strings are echoed to your terminal. To summarize: $ echo -l -i -s a b c Shell: six arguments on the command line to "echo" echo: six arguments received from the command line (no options) Whether a token (a string) on the command line is interpreted by a Unix command as an option or a plain argument depends on the command doing the interpretation. In this course, we will look at command lines from both the point of view of the shell and the point of view of the command being found and run by the shell. I will ask you "how many arguments is the shell passing to the "ls" command in the above example" and you will answer "six arguments". I will ask you "how many of the arguments seen by "ls" are options in the above example" and you will answer "three". I will ask you "how many arguments is the shell passing to the "echo" command in the above example" and you will answer "six arguments". I will ask you "how many of the arguments seen by "echo" are options in the above example" and you will answer "none". You asked about the syntax given in the man page for the Unix "who" command: > usage: who [OPTION] . . . [FILE | ARG1 ARG2] > > It is using both the words "option" and "arg". Am I right in > assuming that an option is more like a flag to attach to a command > to get more specific results? But then, so is an argument. Yes, an option is a flag (or "switch" in DOS terms). It changes *how* the command operates. An "argument", from the point of view of the command, is a thing on which the command operates, such as a pathname, userid, string, etc. For example: $ ls a b c From the point of view of the "ls" command, the above command line has three pathname arguments. Now, add another string to the comand line: $ ls -i a b c From the point of view of the "ls" command, the new example still has three pathname arguments. There are still three pathnames that will be shown to you by "ls". There is now one option argument that changes the output of "ls" to include inode numbers. In short, from the point of view of the Unix shell, because it knows nothing about how a command functions internally, everything on the command line is called "an argument". From the point of view of the command that the shell finds and runs, some of those command line arguments may be interpreted as options - it depends on the command. --------------------------------- Additional material (background for C programmers): In C programming (the language of Unix), the main() routine in a program such as "ls" receives all the command line arguments as follows: main(char **argv, int argc){ You may recognize this as an array of pointers to strings and a count of how many "arguments" there are. So, at the programming interface, all the words that follow the command name on the command line are treated as "arguments" in this array. This is the same point of view as the shell - all the command line arguments are called "arguments". At this point, at the beginning of main() inside the "ls" command, we haven't analysed the strings to see which ones might be options. For example, you might type this into a Unix shell: $ ls -l -i -s a b c and the list of strings (char **argv) received by the "ls" command would contain the command name plus six string "arguments": -l -i -s a b c. The code for "ls" will have to loop through the "char **argv" list and identify which arguments are options (starting with a dash) and which are plain pathname arguments. Different Unix commands have different rules for interpreting which command line arguments are options and which are not. A leading dash is *almost* always an option argument and not a plain argument. (The "echo" command is one of the common exceptions!) To a Unix shell, all the tokens on a command line are "arguments". To a particular command, some might be option arguments. )---------- chapter5.txt -- Thu 2002-Jan-17 02:42:00 -------------------------------- DAT2330 - Unix - Prof. Ian Allen -------------------------------- Here are questions you should be able to answer. You will likely find questions similar to these on tests and exams. Chapter 5 - The Shell. Omit material that uses the line printer commands, e.g. "lpr". - True or False: These two command lines are equivalent: $ ls -l -i -d $ ls -lid - True or False: Every Unix command that takes several option letters lets you group them into a single argument starting with one hyphen. - When processing a shell command line, which software handles the ^U (CONTROL-U) line kill function? - the shell or the operating system terminal driver - the application program (e.g. "cat" or "sort") on the command line - In terms of parsing the command line, what is a "word" or "token"? - How does the shell differ in its treatment of command names that contain slashes vs. command names without slashes? Explain the differences: $ ls $ /bin/ls $ who $ /usr/bin/who - True or False: To find commands to run, the shell always looks in every system directory on the machine. - True or False: You cannot change the places where the SHELL looks to find commands; the locations are built into the Unix system. - Will the "echo" command still work if you set your PATH variable to be empty? $ PATH='' $ echo "Does this work?" Explain the results. Now try "ls" and "date". Now try "/bin/ls" and "/bin/date". Your shell won't work very well now, with PATH set to be empty. How can you get a fresh shell with a correct default PATH? - True or False: The shell issues error messages about unrecognized options to commands, e.g. "who -z". - True or False: To the shell, a command is simply an executable file of the same name as the command name. - True or False: Standard input of every command on a command line is always your keyboard. Standard output is always your terminal screen. - Find the Unix pathname of the terminal with which you are connected to Unix (text p.95). Echo something and redirect the output into this pathname. Where does the output appear? Copy a short file onto this pathname using the "cp" command. Where does the output appear? $ cp /etc/group /dev/pts/4 - Study the differences between text figures 5-3, 5-5, and 5-8. Is it possible to redirect both the standard input *AND* the standard output of a Unix command at the same time? Give an example, if this is possible. - Go to the web site for the author and this textbook and find the correction to be applied to Figure 5-9. (The web site is mentioned in the intoductory pages.) - Study the box on page 99 and understand what the shell is doing when you redirect output! - What does "noclobber" mean? - What prompt does the shell use to remind you that you are operating as the Super User on a Unix system? (You can safely become the super-user on your Floppix system - see the "superuser" Lab on www.floppix.com.) - On page 101 the text says that a pipe is identical to redirecting the output of one command into a file, then running a second command and feeding it the file as standard input. While this is a true statement for pipes used in DOS, it isn't true under Unix. Unix pipes start passing data before the first command has finished; unlike DOS, you don't have to generate the full output of the first command before any output starts appearing on the input of the second command. Unix starts passing output as soon as it is available: $ cat -n -u | tr a-z A-Z abcd 1 ABCD hello 2 HELLO ^D $ Note how each line passes all the way through the pipe as it is typed. (What do the two options to "cat" mean on ACADAIX?) - What does it mean to use a command as a "filter"? Can any Unix command be used as a filter? - What happens if you give the pathname to your terminal as the file name argument to the Unix "tee" command? Why does this happen? $ tty /dev/pts/34 $ echo "Testing testing" | tee /dev/pts/34 Explain the output. - To put a sequence of shell commands into the background, enclose the sequence in parentheses before appending the ampersand: bash$ ( sleep 5 ; date ) & [1] 50070 bash$ ( sleep 5 ; echo "Hello there" | write abcd0005 ) & [2] 50085 - Understand the use of these wildcard (glob) characters: * ? [] - What is the difference in output of these two commands? $ echo */. $ echo ./* - True or False: If a Bourne-style shell pathname expansion (globbing, or wildcard matching) fails to match any files, the expansion is passed unchanged as an argument to the underlying command: $ mkdir empty $ cd empty $ echo * [...what output do you see here...] $ touch b $ echo a* b* [...what output do you see here...] - True or False: The shell does pathname expansion before calling the utility (command) in a command line. The utility sees the *result* of the expansion, not the wildcard (glob) characters that generated it. $ touch a b c d $ echo * - True or False: the echo command receives one argument "*" and expands it to be the strings "a" "b" "c" "d" and echoes them on the screen - What is the difference in output of these two "tr" commands? (Study how the shell expands pathnames in these command lines!) $ rm -rf empty $ mkdir empty $ cd empty $ touch a b c A B C $ ls A B C a b c $ head /etc/passwd | tr [a-z] [A-Z] [...what output do you see here...] $ head /etc/passwd | tr "[a-z]" "[A-Z]" [...what output do you see here...] Why is the output different? What is happening? Chapter non-Advanced Review Questions: On ACADAIX or Linux: 1 - 9 (all) Chapter Advanced Review Questions: On ACADAIX or Linux: 10, 11, 12, 13, 14, 15, 16 Unix commands studied (summary p.111): tty, tr, tee, bg, fg, jobs )---------- week4.txt -- Thu 2002-Jan-17 03:29:00 Week 4 Notes for DAT2330 - Ian Allen Supplement to Text, outlining the material done in class and lab. Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM![*] ([*] Read The Fine Manual) Look under the "Notes" button on the course web page for these study notes for the Linux textbook: chapter5.txt (the Shell) Complete these Floppix Labs on http://floppix.ccai.com/ Floppix Lab 14: redirection Floppix Lab 15: pipes and filters Floppix Lab 18: the search path Floppix Lab 20: configuring the bash shell Floppix Lab 27: superuser Note: An early version of bash is also available on ACADAIX. (The default shell on ACADAIX is the Korn shell - "ksh".) -------------- The Unix Shell - Chapter 5 -------------- What is a shell for? To find and run programs (also called commands or utilities)! (It also does programming kinds of things; but, all that is usually to aid in the finding and running of programs.) Most (but not all) commands take what as arguments? Pathnames, either file names or directory names. The shell has features to make matching pathnames easier. (Not *all* programs take pathnames as arguments!) How does the shell help run commands? Shells look for commands in various places, using a list of directories stored in the $PATH environment variable. ! It's the PATH environment variable. ! (You were pedantitally correct above on the point that the name ! "/" does not appear in any directory.) Shells provide aliases and variables to save typing the same things (commands or pathnames) over and over. Shells provide wildcards (glob patterns) to generate lists of pathnames as arguments for commands. Shells provide ways of recalling and editing the last commands you enter, to save retyping them. Shells provide ways of completing command and file names, to save typing. Where are wildcard characters (glob patterns) expanded? The shell does the wildcard (glob) expansion, NOT the commands. The wildcards are expanded before the shell looks for the command! Note how different commands treat the same set of arguments differently: $ echo * - args are interpreted as a list of text words $ ls -l * - args are interpreted as a list of pathnames (files or dirs) $ cat * - args are interpreted as a list of file names to open $ mail * - args are interpreted as a list of userids to email The shell does not know anything about the type of command being used when it prepares the argument list for a command. It is quite possible to prepare a list of arguments that don't make sense for the command being run, e.g. $ sleep * - this probably makes no sense Define: whitespace, arguments, wildcard, prompt ------------- Nested Shells ------------- Your default login shell on ACADAIX is the Korn shell (ksh). Your default login shell on Linux (including Floppix) is the Bourne-Again shell (bash). You can call up other shells by name: sh, ksh, bash, csh, tcsh (Not all shells may be installed on your system.) Each new shell is another Unix process. To get a list of your current processes, use: ps To exit from a shell: exit When you exit from your login shell, you log out from Unix. Another way to exit a shell is to type your EOF character. (Your EOF character is usualy CONTROL-D.) Some shells can be told to ignore EOF. Remember: all Unix programs (should) have manual pages! $ man sh $ man ksh $ man bash $ man csh $ man tcsh The shells sh, ksh, and bash (the "Bourne" shells) all have a common ancestry. They are all derived from the original Bourne shell "sh", and the programming features of these shells (if statements, for loops, etc.) all look and work the same way. The shells csh and tcsh (the "C" shells) are similar. Their syntax for programming is not the same as the Bourne shells. We do not cover the C shell syntax in this course; but, you can read about it in your Linux textbook. ------------------ Output Redirection ------------------ Output redirection diverts (redirects) output that would normally appear on the screen to some other place, either into the input of another command (a pipe) or into a file. This normal output on your screen is called the "standard output" ("stdout") of the command. Output redirection of stdout into files: $ echo hello - stdout goes to terminal $ echo hello >file - erase file; send stdout to file $ echo hello >>file - append stdout to end of file Shells don't care where on or in the command line you do the redirection. All these command lines do exactly the same thing to stdout: $ echo hi there mom >file $ echo hi there >file mom $ echo hi >file there mom $ echo >file hi there mom $ >file echo hi there mom Like wildcarding (called "globbing" on Unix), shells handle redirection before they go looking for the command to run. The command doesn't see any part of the redirection syntax. The redirection is done by the shell, then the redirection information is removed from the command line before the command is called. Redirection is never counted as arguments to a command. Examples: $ echo hello there - shell calls "echo" with two arguments ==> echo(hello,there) - "echo" echoes two arguments - output appears in default location (standard output is your screen) $ echo hello there >file - shell creates "file" and diverts standard output into it - shell calls "echo" with two arguments ==> echo(hello,there) (note NO CHANGE in arguments to "echo" from the previous example) - "echo" echoes two arguments - standard output is captured in output "file", NOT on your screen $ >file echo hello there - this is identical to the above example - standard output is captured in output "file", NOT on your screen - you can put the redirection anywhere in the command line! Unix Big Redirection Mistake #1: Do not do this kind of redirection: $ cat * >z - shell creates "z" and redirects all future standard output into it - shell expands wildcards; wildcard "*" includes file "z" that was just created by the shell (Note: Bourne shells will do the wildcard before the file creation; C Shells do the file creation first.) - shell finds and calls cat command with all file names as arguments ==> e.g. cat(a,b,c,d,e,file1,file2,...etc...,z) - cat command processes each argument, opening each file and sending the output into file "z" - when cat opens file "z", it ends up reading from the top of file "z" and writing to the bottom of file "z" at the same time! - Result: an infinite loop that fills up the disk drive as "z" gets bigger and bigger Fix #1: Use a hidden file name $ cat * >.z - uses a hidden file name not matched by the shell "*" wildcard - the cat command is not given ".z" as an argument, so no loop occurs Fix #2 (two ways): Use a file in some other directory $ cat * >../z $ cat * >/tmp/z - redirect output into a file that is not in the current directory so that it is not read by the cat command and no loop occurs Unix Big Redirection Mistake #2 Do not do this kind of redirection: $ cat a b >a - shell truncates file "a" and redirects command output into it - original contents of "a" are lost - truncated - GONE! - shell finds and calls cat command with two file name arguments ==> i.e. cat(a,b) - cat command processes contents of file "a" (now an empty file) - cat command processes contents of file "b" - output has been redirected by the shell to appear in file "a" - Result: file "a" gets a copy of "b"; original contents of "a" are lost Fix #1: Append to a $ cat b >>a - double-redirect syntax appends file "b" safely to the end of "a" Fix #2: Use a Temporary Third File $ cat a b >c # mv c a - the third file safely receives the output of "a" and "b" ------------------ Input Redirection: ------------------ Most Unix commands read input from files, if file names are given on the command line, and from standard input ("stdin") if no file names are given. (Not *all* commands read from standard input. Examples of common commands that never read from standard input: cp, mv, date, who, etc.) If a command reads from standard input, you can tell the shell to use input redirection to change from where the command reads: $ cat food - reads from file "food" $ cat - reads from stdin (keyboard) $ cat a >b >c >d >e - the "date" output goes into file "e"; the other files are created by the shell but are empty because only the final redirection wins bash$ date >out | wc 0 0 0 - the "date" output goes into file "out"; nothing goes into the pipe Some shells (including the "C" shells) will try to warn you about silly shell redirection mistakes: csh% date a >b >c Ambiguous output redirect. csh% date >a | wc Ambiguous output redirect. The C shells tell you that you can't redirect stdin or stdout to/from more than one place at the same time. Bourne shells do not tell you - they simply ignore the "extra" redirections and do only one of each. ----------- Redirection ----------- A command line to convert lower-case to upper-case from the "who" command: $ who | tr 'a-z' 'A-Z' Shell question: Are the single quotes required around the two arguments? (Are there any special characters in the arguments that need protection?) You can use a similar command to convert a lower-case file of IBM MVS JCL into upper-case. (You will do this in the MVS section of the course.) EXPERIMENT: Why doesn't this convert the file "myfile" to upper-case? $ tr 'a-z' 'A-Z' myfile Why is the file "myfile" empty after this command is run? What about the following command lines - what is in "myfile" when the command finishes? $ cat myfile $ sort myfile $ head myfile Given the above, why is "myfile" not empty in the following case? $ wc myfile The following command line doesn't work because the programmer doesn't understand the "tr" command syntax: $ tr 'a-z' 'A-Z' myfile >new Why does this generate an error message from "tr"? (The "tr" command is unusual in its handling of command line pathnames. RTFM) The following command line redirection is faulty; however, it sometimes works for small files: $ cat foo bar | tr 'a' 'b' | grep "lala" | sort | head >foo There is a critical race between the first "cat" command trying to read the data out of "foo" before the shell truncates it to zero when launching the "head" command at the end of the pipeline. Depending on the system load and the size of the file, "cat" may or may not get out all the data before the "foo" file is truncated or altered by the shell in the redirection at the end of the pipeline. ! Does one process' EOF modification affect another process' view of a ! file? I think not, at least sometimes (and I do have in mind tail -f): ! I think that (with some file systems? with some kernels?), a process ! sees the open-time EOF regardless of later changes, meaning the example ! should see either the full original content, or 0 bytes, but not part. ! I think this because I think I have seen something other than tail -f ! fail to find data that were added to the end of a file during execution; ! thus I suspect that tail -f is doing stat/close/open or similar. Don't depend on long pipelines saving you from bad redirection! Never redirect output into a file that is being used as input in the same command or anywhere in the command pipeline. ------- Aliases ------- Watch out for "helpful" system admin that define aliases for your shells when you log in. (This is especially true on ACADAIX!) The aliases may mislead you about how Unix commands actually work. (For example, the "rm" command does *not* prompt you for confirmation. On some systems, "rm" is an alias for "rm -i", which *does* prompt.) To avoid pre-defined aliases, start up a fresh copy of the shell: $ alias [...many ACADAIX aliases print here...] $ bash bash$ alias [...no more aliases here...] To define your own aliases, look up "aliases" in the Linux Text index. You must put your aliases in a file to have them saved between sessions. ------------------------------------- Studies in Quoting Special Characters ------------------------------------- Understand how the shell handles quotes and blanks: $ echo hi there hi there $ echo "hi there" hi there $ echo 'hi there' hi there Explain the above three outputs. How many arguments are passed to the "echo" command in each case? The "touch" command creates empty files by name. Try this: $ touch "a b" $ ls a b $ rm a b Explain the error message that is output by the above "rm" command. How many arguments are passed to the "touch" and "rm" commands? Here are some more things to try, and to understand. $ echo "'hello'" $ echo '"hello"' $ touch a b c d $ echo ' * ' $ echo '" * "' $ echo '"' * '"' $ echo '"'" * "'"' $ echo ' * ' * " * " You must be able to predict the output of each of the above command lines without having to type them in to try them. -------------------------------------- More shell wildcards: Character Ranges -------------------------------------- As shown in your textbook, the shell can do globbing (wildcard expansion) on ranges of characters. Try these shell patterns in your home directory on ACADAIX: $ echo ../[a-c]* $ echo ../*[3-5] $ echo ../[r-t]*[13579] How would you count the number of pathnames generated by the shell? What would be different if you changed "echo" to "ls"? -------------------------------------------- Understanding the different types of "sort": -------------------------------------------- Explain the difference in output of these two "sort" pipelines: $ list="1 11 2 22 3 33 4 44 3 33 2 22 1 11" $ echo "$list" | tr ' ' '\n' | sort $ echo "$list" | tr ' ' '\n' | sort -n (The translate command "tr" is turning blanks into newlines so that the numbers appear on separate lines on input to sort.) Why is the sort output different in these two examples? ------------------------------------------------------------------- Using commands and pipes to "mine" and extract data from the system ------------------------------------------------------------------- Problem: "Print the fifth directory from your $PATH environment variable." Iterative solution built up slowly using simple commands: $ echo "$PATH" $ echo "$PATH" | tr ':' '\n' $ echo "$PATH" | tr ':' '\n' | head -5 $ echo "$PATH" | tr ':' '\n' | head -5 | tail -1 Problem: "Print the second-to-last directory from your $PATH environment variable." $ echo "$PATH" | tr ':' '\n' | tail -2 | head -1 Problem: "Sort the elements in the PATH variable in ascending order." $ echo "$PATH" $ echo "$PATH" | tr ':' '\n' $ echo "$PATH" | tr ':' '\n' | sort $ echo "$PATH" | tr ':' '\n' | sort | tr '\n' ':' Problem: "Keep only the first five elements of the PATH." $ echo "$PATH" | tr ':' '\n' | head -5 | tr '\n' ':' Problem: "How many unique shells are in the /etc/passwd file?" Build up the solution iteratively, starting with simple commands. The shell is the seventh colon-delimited field in the passwd file. Either the "awk" or "cut" commands can pick out a field from a file. We will use "cut" to pick out the 7th field delimited by a colon. Because the /etc/passwd file on ACADAIX is huge (and the output on our screen would be huge), we will start with the first 10 lines of the passwd file until we know we have the correct command line, then we will use the whole passwd file. $ head /etc/passwd $ head /etc/passwd | cut -d : -f 7 $ head /etc/passwd | cut -d : -f 7 | sort $ head /etc/passwd | cut -d : -f 7 | sort | uniq We have the correct command line. Now do the whole file: $ cat /etc/passwd | cut -d : -f 7 | sort | uniq - OR - $ cut -d : -f 7 /etc/passwd | sort | uniq - OR - $ cut -d : -f 7 /etc/passwd | sort -u Does this pipeline (the reverse of the above) give the same output? $ sort -u /etc/passwd | cut -d : -f 7 ------- Filters ------- Note that many Unix commands can act as filters - reading from stdin and writing to stdout. With no file names on the command line, the comands read from standard input and write to standard output. (You can redirect both.) If file names are given on the command line, the commands usually ignore standard input and only operate on the file names. $ grep "/bin/sh" /etc/passwd | sort | head -5 The "sort" and "head" commands are acting as filters. Each command is reading from stdin and writing to stdout. The "grep" command is not a filter - it is reading from the supplied argument pathname, not from stdin. If a command does read from file names supplied on the command line, it is more efficient to let it open its own files than to use "cat" to open the files and feed the data to the command on standard input. (There is less data copying done!) Advice: Let commands open their own files; don't feed them with "cat". Do this: $ head /etc/passwd $ sort /etc/passwd Do not do this (wasteful of processes and I/O): $ cat /etc/passwd | head # DO NOT DO THIS - INEFFICIENT $ cat /etc/passwd | sort # DO NOT DO THIS - INEFFICIENT Problem: "Now, count the number of each kind of shell in /etc/passwd." $ cut -d : -f 7 /etc/passwd | sort | uniq -c Problem: "Count the number of each kind of shell in /etc/passwd and display the results sorted in descending numeric order." $ cut -d : -f 7 /etc/passwd | sort | uniq -c | sort -nr Problem: "Count the number of each kind of shell in /etc/passwd and display the top two results sorted in descending numeric order." $ cut -d : -f 7 /etc/passwd | sort | uniq -c | sort -nr | head -2 )---------- practiceCh1-5.txt -- Thu 2002-Jan-17 03:37:00 -------------------------------------------- Practice Unix/Linux Questions - Chapters 1-5 -------------------------------------------- -IAN! idallen@ncf.ca -- You are in your home directory. Create Unix pipelines (combinations of commands) to perform the following actions. Do not use any change directory commands; use relative or absolute pathnames to place your output in the correct directories. Minimize your typing - use the shortest pathnames you can. 1. Create a file in the current directory named "out" that contains the last 15 lines of the Unix password file, sorted in reverse. 2. Create a file in the /tmp directory named "out" that contains only line 15 from the Unix passwd file. (Hint: use "tail -1" as part of the pipeline.) -- On Floppix, you are in the "/home" directory (owned by root). Without changing directories, create a file named "recent" in *your* home directory that contains the names of the five most recently modified files in the "/bin" directory. (Hint: the -t option to "ls" may be useful. See the man page.) (1) Answer the question using absolute paths for the input and output file names. (2) Now use the shortest possible relative paths for both the input and output file names. (3) Produce a long listing of the 15 oldest files, instead of the newest files. (Hint: the -r option to "ls" may be useful.) -- Which program issues the error message in each case below? $ date >/etc/passwd permission denied $ cp foo /etc/passwd permission denied $ cat no-such-file cannot open (or no such file) $ sort bar $ sort >bar foo $ bar $ tail d e f g $ wc a b c g -- On ACADAIX, what is the output on your screen of each of these command lines, and what is in the file "foo" when the command finishes? $ cat foo >foo $ cat foo $ sort foo >foo $ sort foo $ more foo >foo $ more foo $ grep "pattern" foo >foo $ grep "pattern" foo $ head foo >foo $ head foo $ tail foo >foo $ tail foo $ tr a b foo Hint: The answer is the same for all the command lines. Some versions of the "cat" command will warn you if you use an output file that is the same as any of the input files. The version on ACADAIX does not warn you. -- Why is the file "myfile" not empty after this command sequence? $ date >myfile ; wc myfile -- What is the difference between the outputs of the two "wc" commands? $ date >foo $ wc foo $ wc foo $ ln foo one $ ln foo two $ ln foo three -- Does this work to give a file a link count of four? $ date >foo $ ln foo one $ ln one two $ ln two three -- Does this work to give a file a link count of four? $ date >foo $ ln foo one $ ln one two $ ln one three -- Does this work to give a file a link count of four? $ date >foo $ ln foo new $ ln foo new $ ln foo new -- Create a file, then do some commands to give it a link count of five. (Give the file four other names, for a total of five names.) Now, remove all (ALL) permissions from the above file. How many Unix commands are necessary to remove all permissions from a file that has five names? After removing all permissions, try to copy one of the above file names to another file. Which program issues the error message, and why? -- Compare these two command sequences and explain how they differ, if they do differ: $ date >a $ date >a $ ln a b $ ln a b $ ln a c $ ln b c $ ln a d $ ln c d -- What information is stored with a file name in a Unix directory? What information is stored in the inode of the file? What command shows the permissions of the CURRENT DIRECTORY ONLY? (Do not show the *contents* of the current directory!) What command shows the inode numbers associated with names in a directory? What permissions do you need on a directory to see the names it contains? What permissions do you need to find out the owner of a file in the current directory? What permissions to you need to find out if the current directory contains an entry named "secret"? What permissions do you need on a file to make a link to it? What permissions do you need on a file to remove its name from a directory? What permissions do you need on a sub-directory to remove its name from a directory? What must be true about the contents of the sub-directory? What (minimal) permissions do you absolutely need on a directory to remove a name from the directory? -- What does this do? $ mkdir my dir -- What does this do? $ mkdir "my dir" Is it the same as this?: mkdir my dir -- How do you append to a file, so that you can put the output of several commands into the same file without losing what was there before? -- What is wrong with this command to mail the contents of existing file abcd.txt to user alleni? $ cat abcd.txt > mail alleni@algonquincollege.com What error message will you see? What does the above command actually do? *Exactly* how many arguments are passed to the "cat" command? Rewrite the above command to work, having the "mail" command read the file text from standard input and removing the unnecessary "cat". -- What is wrong with this command for copying abcd.txt into file foo? $ cat abcd.txt | foo As seen by the Unix shell, list the exact command names and arguments for all the commands being used in the above pipeline. Give the correct command line. -- What is wrong with the following command for copying file foo to file bar? What will be the output on the screen? What will be in file "bar"? $ cp foo >bar (Hint: How many arguments are passed to cp? How many are required? RTFM) -- What is wrong with the following command for creating file bar? What will be the output on the screen? What will be in file "bar"? $ echo "Some Text" | cp bar (Hint: How many arguments are passed to cp? How many are required? RTFM) -- What is wrong with the following command for copying file foo to file bar? What will be the output on the screen? What will be in file "bar"? $ cat foo | cp >bar (Hint: How many arguments are passed to cp? How many are required? RTFM) -- What is the output of this command? $ echo "Hello There" | mv foo bar What does the "mv" command do with information coming in on standard input (after the pipe symbol) (RTFM)? -- The following command line actually works (in the Bourne-style shells), but the programmer who wrote it didn't really understand how piping operates, given the two commands involved: $ tr a b bar | mv bar foo What is the actual output (on standard output) of the part of the command line before (to the left of) the pipe? What does the "mv" command do with information coming in on standard input (after the pipe symbol) (RTFM)? Some shells will not permit the above command line: cshell% tr a b bar | mv bar foo Ambiguous output redirect. -- Rewrite the following command line (from page 172 of the textbook) to get rid of the useless "cat" command: kudos% cat memos.new | rsh bravo diff - memos/memo.draft Let programs open their own files, or let the shell do it. Don't use "cat" unnecessarily. )---------- week2.txt -- Sun 2002-Jan-20 15:09:00 Week 2 Notes for DAT2330 - Ian Allen Supplement to Text, outlining the material done in class and lab. Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM![*] ([*] Read The Fine Manual) Read the course announcements frequently, especially before tests. Look under the "Notes" button on the course web page for these study notes for the Linux textbook: chapter2.txt (Getting Started) chapter3.txt (Utilities) Complete these Floppix Labs from http://floppix.ccai.com/ Floppix Lab 13: Who's on? Floppix Lab 22: e-mail Floppix Lab 23: telnet Floppix Lab 24: ftp Floppix Lab 25: links [the www browser] Floppix Lab 27: Super User Floppix Lab 12: grep =========================== Notes on FTP command syntax =========================== The syntax of FTP commands is not the same as the syntax of Unix commands. This FTP command doesn't do what you think it does: ftp> ls -l filename output to local-file: filename? If you answer "yes" to this prompt, you will copy the output of "ls -l" into the file "filename" in your current directory, erasing what was there before. This is probably not what you want. The FTP command names resemble Unix command names; but, they are *not* Unix commands. The syntax is different. Be careful. (Also, don't type FTP commands into Unix and expect that they will work, e.g. "put filename".) ========================== Unscrambling your Terminal ========================== How to unscramble a terminal emulator that is in graphics characters set mode, where see many special and line-drawing characters instead of your typed text. (This might happen after you accidentally use "cat" to send a non-text file to your terminal screen, e.g. "cat /bin/ls".) To Fix (you may not be able to read what you are typing!): - On a Linux system (including Floppix) type: setterm -reset - On ACADAIX type: reset - On either type: echo ^V^O (that's CTRL-V CTRL-O) The above should switch your terminal emulator back to its normal character set. Practice this now, in case it happens to you on a test! ======================= Getting Out of Programs ======================= Getting out of Unix programs; or, getting help: Try various things such as: help ? quit Q exit x ^D (CTRL-D) ^C (CTRL-C) :q! (used in VI to exit without saving) logout bye ESC (the ESC key) . (a period) ^\ (CTRL-backslash) One of the above usually works. Sometimes you can use ^Z (CTRL-Z) to "stop" the process temporarily, and then type "kill %%" to kill it. (Remember to kill it, or it will sit there forever.) )---------- chapter7.txt -- Sun 2002-Jan-20 16:10:00 -------------------------------- DAT2330 - Unix - Prof. Ian Allen -------------------------------- Here are questions you should be able to answer. You will likely find questions similar to these on tests and exams. Skip over these sections of the text: Linux Text: "rwho" (p.173) Linux Text: "NIS" (p.176) Linux Text: "NFS" (p.177) Linux Text: "Network Services / Daemons" (p.178) Linux Text: "Internet Services" (p.179-197) Skip over the above sections of the text. Review the "Internet Basics" Web page (under the course home page). Chapter 7 - Networking and the Internet Learning to use network commands at Algonquin College is difficult due to the restrictions that ITS places on network traffic. Most network use is forbidden; only a few select services are allowed into or out of each machine. You may or may not be able to use the "talk", "rcp", "rsh", "rlogin", "ftp", or "finger" commands between different machines here at Algonquin College. If the command just hangs or says "connection refused", ITS is probably blocking you from using that service here at Algonquin. You may be able to use the service from another Internet Service Provider at your home. You can sometimes try to use these commands back into the machine you are on, pretending that the local machine is itself a remote computer on the Internet. At the time this was written, the "ftp" command is also blocked for use at Algonquin by Algonquin's DNS misconfiguration. Many FTP sites will refuse to let you connect to them from any Algonquin labs. Try using ftp from your ACADAIX account instead. - What kind of network (broadband, token ring, point-to-point) is connected to the network card on your lab computer? Which type of network is commonly used by a dial-up modem? - Do network routers contain a fixed or dynamic map of the Internet? Linux Text: "Host Addresses" (p.161) Hosts on the Internet are identified by "IP Numbers", e.g. 205.211.41.98 is currently the world-assigned IP number for ACADAIX. You can use the IP number to reach ACADAIX from anywhere on the Internet, even when the name ACADAIX is not recognized. Most people use names for computers; but, the computers want and use only the IP addresses. The most popular way of turning a name into a IP address is via the Domain Name Service (DNS). (The old /etc/hosts file method is almost always augmented by a DNS installation.) All programs that allow you to use names for computers turn the names into IP addresses before they actually use them on the Internet. Most Internet programs that accept host names also accept host IP numbers. For example, you can telnet directly to the IP address 205.211.41.98 (the IP number for the ACADAIX machine), and you can send email to alleni@[205.211.41.98]. (The square brackets are necessary only for an email address - you will have to escape these special characters in your shell command line to protect them from the Unix shell.) The IP numbers can change frequently as the machines move from network to network; the names change less often. The machine ACADAIX changed IP addresses in December 2001 - the DNS system was updated and now the name "acadaix" leads you to its new IP address. Programs using the name "acadaix" still work, even though the machine has moved. Not all computers have public names for their IP addresses. Some computers only have IP addresses. Only the numbers are necessary to connect a machine to the Internet - the name is optional. Some services on the Internet have names that are not associated with actual IP numbers. Often, these names are aliases for other names that do have IP numbers. - Find out from your instructor the IP number of the Linux test computer and "finger" it, using the example syntax given in the textbook chapter. (Note: ITS usually blocks finger traffic - this may not work unless you are actually logged into the Linux test computer itself.) - You can use "rlogin" to connect to Unix systems on the Internet. Here at Algonquin, you can connect from acadaix from acadaix or from the ! Typo. Linux test machine to itself. For example: $ rlogin acadaix Password: Academic unix (AIX) server ...etc... $ who am i alleni pts/0 May 27 20:26 (acadaix.algonqui) $ exit Connection closed $ (Note: ITS usually blocks rlogin/rsh/rcp traffic - this may not work unless you are actually logged into the computer itself.) - You can use telnet to connect to Internet hosts. Try telnet to "telnet.ncf.carleton.ca" and login as "guest". $ telnet telnet.ncf.carleton.ca ...etc... The Library of Congress example in the text will also work. Linux Text: "Trusted Hosts" (p.168) Use of "rsh" and "rcp" is severely hampered by ITS policies at Algonquin. Here's what you can do. On ACADAIX, create a file named ".rhosts" in your HOME directory, with the single host name "acadaix" in it. Then, while on ACADAIX, you will be able to do "rsh acadaix date" without giving your password. If you use the host name "localhost" in the file, you can use "rsh localhost date" instead. ("localhost" is always a name for the local computer.) On the Linux test machine, you can do the same experiment using the single host name "idallen-firewall" or "localhost". You probably can't use rsh between machines at Algonquin due to restrictive ITS policies. You can't use rsh or rcp at all on Floppix - it doesn't have the rsh command or rshd service installed. Linux Text: "Using RCP and FTP" (p.169) Yes, "rcp" works on ACADAIX, if you have a .rhosts file that permits your trusted host name: $ hostname acadaix $ date >foo $ rcp foo acadaix:bar rshd: 0826-813 Permission is denied. $ echo acadaix >>.rhosts $ rcp foo acadaix:bar $ ls -l foo bar -rw-r--r-- 1 alleni 29 May 27 20:50 bar -rw-r--r-- 1 alleni 29 May 27 20:50 foo $ diff foo bar $ rcp foo acadaix:foo rcp: /thome/alleni/foo and /thome/alleni/foo refer to the same file (not copied). These r-commands (rcp, rsh, rlogin) are useful on a private, trusted network of Unix machines; but, they are not safe to use over the Internet because of domain name security issues. Both the r-commands and FTP have been replaced by "scp" and "ssh" (Secure Shell) programs that use encryption and public key infrastructure. (We don't teach ssh in this course.) Do not use the r-commands or FTP (except for anonymous ftp) on the Internet. You can try the text FTP example (to tsx-11.mit.edu). The archive is a bit larger than it was in 1997! To get a long listing (including sizes), use "ls -l" instead of just "ls". Rememer to select "binary" mode to transfer GZIP files via FTP. Binary mode is appropriate for 99% of the Unix content you will ever download to a Unix machine. Linux Text: "Using RSH" (p.172) Yes, "rsh" works from ACADAIX to ACADAIX or from the Linux test machine to itself, if you have a .rhosts file that permits it. ITS usually blocks rsh from working between machines at the College. ITS DNS errors at Algonquin may prevent rsh from working between ACADAIX and the Linux test machine. Linux Text: "Ping" (p.172) You can usually ping most anything that has an Internet name. For security reasons, some sites do not permit PING packets to enter. (The FreeNet has ping packets disabled by Carleton U.) Host names to ping (from Algonquin): dns netsrv acadaix algnet outmail inmail localhost 127.0.0.1 205.211.47.1 cpu1808.adsl.bellglobal.com google.com tsx-11.mit.edu hotmail.com (doesn't work - ping is blocked) microsoft.com (doesn't work - ping is blocked) netscape.com (doesn't work - ping is blocked) freenet.carleton.ca (doesn't work - ping is blocked) Which host has the longest round-trip time? Another useful command (not mentioned in the text) is "traceroute". It is available on many Linux distributions, and on ACADAIX: $ traceroute google.com $ traceroute -n -m 50 www.eomw.net (this site is in Africa!) This command prints the actual route taken by the IP packets as they travel through the Internet, through each gateway and router along the way. The trace will be somewhat faster if you turn off DNS lookups ("-n") on all the names that are displayed. Some sites do not permit traceroute packets to identify their routers - these sites will print as asterisks. Linux Text: "RWHO" (p.173) Nothing at Algonquin is running the rwho service; rwho will report nothing. At Algonquin, ITS blocks most of the other methods of seeing who is logged in to a remote computer. If you are logged in to ACADAIX, you can try the Sun style "rusers" command instead: rusers localhost Use an option to select the long listing format of "rusers". (RTFM) Linux Text: "Domain Name Service" (p.175) In the news, you might have heard about some new "top-level" domains being added to the DNS in 2001. What are their names? Which end (right or left) of a DNS domain address is the "top" end (the end that changes least frequently)? Which end (right or left) of an Internet numeric IP address is the "top" end (the end that changes least frequently)? Skip over these sections of the text: Linux Text: "rwho" (p.173) Linux Text: "NIS" (p.176) Linux Text: "NFS" (p.177) Linux Text: "Network Services / Daemons" (p.178) Linux Text: "Internet Services" (p.179-197) Skip over the above sections of the text. Review the "Internet Basics" Web page on the course home page. Chapter 7 non-Advanced Review Questions: 1,2,5,6,7,8 Unix commands studied: hostname, finger, talk, rlogin, rcp, rsh, telnet, ftp, ping )---------- chapter8.txt -- Sun 2002-Jan-20 16:14:00 -------------------------------- DAT2330 - Unix - Prof. Ian Allen -------------------------------- Here are questions you should be able to answer. You will likely find questions similar to these on tests and exams. Chapter 8 - The VI Editor. Note: The ACADAIX and Floppix versions of VI are not as advanced as the VI described under Linux. You may find that some of the features described in the text only work on a full installation of Linux (e.g. the "Getting Help" commands). The Linux test machine has a full installation of VI (VIM). ACADAIX has an old version of "vim" (Vi iMproved) installed, in which the ":help" command does work. Omit the "optional" material in this chapter on first reading. You can get by in VI with knowing how to read in a file, delete and append single characters and lines, and write the file back out. Advice: The more you learn about VI, the faster you will edit files and the better your mark will be on tests and exams. VI skill is essential to you as a Unix programmer. Almost every letter, upper and lower case, is a VI command. The more you learn, the faster you can edit. While you may begin by learning to delete a word using ten "x" commands, eventually you should evolve into knowing how to use the single "dw" command. If you are using "vi" (not "vim") on ACADAIX, create a file named .exrc (or .vimrc) in your HOME directory containing this command (see the text, p.229): set showmode That will enable the IBM standard version of VI to remind you when you are in INPUT MODE. (Some versions of VI (VIM) turn this on by default.) Some people also like to have "set number" in their .exrc. The corresponding file for VIM is ".vimrc" and some people like to have "syntax off" in it. -------- WARNING! -------- Make sure your terminal is correctly configured before you enter VI! (See the "Using Telnet" web page off the course home page.) If you use telnet from under Windows to connect to a Unix machine, make sure you have NO SCROLL BARS VISIBLE! Set the correct number of lines and correct terminal type. (This won't be a problem for people working on the Floppix console, since the Floppix console isn't a telnet window.) On ACADAIX, the "resize" command is useful to set the number of lines correctly after you telnet into the machine: $ resize This is outlined in detail in the "Using Telnet" web page. -------- TUTORIAL -------- Complete this short VIM tutorial on the Linux test machine by copying the tutorial file into your directory and editing it: $ cd $ cp /usr/share/vim/tutor/tutor tutor $ vim tutor Follow the instructions in the tutorial. --------------------------------------- Absolute bare minimum basic VI commands --------------------------------------- ESC a i x dd :w file :q! The above commands, plus the arrow keys, will perform most any edit on any file you have. All the other things in VI simply make the editing faster and get you better marks in less time. Some of the other VI commands make the editing much, much faster. [Compare "dd" (delete a whole line) with typing 80 "x" commands!] ------------------------ What to learn Next in VI ------------------------ Other useful things to know in VI, in vague order of importance: u . p P r o O 5dd 10x h j k l 0 ^ $ H L M A I U cc s f F !! :x :$r file :set showmode :set autoindent :set wrapmargin=5 :set number :syntax off To improve your skill, learn a new command every week from Chapter 8. ---------------- Review Questions ---------------- - Chapter 8 non-Advanced Review Questions: On ACADAIX or Linux: 1 - 10 (all) - Chapter 8 Advanced Review Questions: none (but read the paragraph Advice, above) )---------- chapter2.txt -- Mon 2002-Jan-21 15:48:20 -------------------------------- DAT2330 - Unix - Prof. Ian Allen -------------------------------- Here are questions you should be able to answer. You will likely find questions similar to these on tests and exams. Chapter 2 - Getting Started. Omit material related to the GUI and X Windows (e.g. xman, tkman). Omit material related to printing (e.g. lpr). Omit material related to the "joe" editor (p.32-36). Omit material related to the "info" utility. (There is an "info" utility on ACADAIX, but it is written by IBM. Floppix doesn't include the Linux "info" command.) Substitute the file /etc/passwd for the missing file /etc/termcap (text p.37) when practicing on ACADAIX. - "The best way to learn is by doing." Which chapters of the text are suitable for "learning by doing"? - True or False: Making a mistake as an ordinary user while learning Unix or Linux may damage the Unix system and crash it. - True or False: Making a mistake as the user "root" while learning Unix or Linux may damage the Unix system and crash it. - What are the answers to the relevant questions on page 20? You may find the output of the stty command helpful in determining the default erase, line kill, and word erase (werase) characters. Try this: $ stty all - Read the first three-line paragraph under the "Notes" heading on page 867. The default Korn shell (ksh) on ACADAIX also prevents some actions of the stty command from working. - Test each of the erase, word erase, and line kill characters. Do they all work as expected? Start up the C Shell "csh". Test the three characters again. Any differences? Start up the Bourne shell "sh". Any differences? (Type "exit" to exit a shell.) - What is the difference between a command (or utility) and a command line? - True or False: To avoid a syntax error you must type in the same prompt character as used by the shell at the beginning of all your Unix command lines, as illustrated in the textbook by examples such as: $ vi memo.1204 This example shows that you must type the dollar sign in front of all commands that you type into this shell. - True or False: You can end command lines in Unix either by using the TAB key, the RETURN key, or clicking the mouse. - True or False: The first prompt you see after logging in to a Unix system is almost always a text editor, such as "joe" or "vi". - True or False: The Unix password command has 6 letters. - How many virtual consoles are available on Algonquin's ACADAIX Unix system? - Floppix users: How many virtual consoles are configured into Floppix? - List three ways you can "back up" while typing a command, to erase characters that you have typed. Given the three shells "ksh", "csh", and "sh" on ACADAIX, which shells permit all three characters to function as expected? - How do you interrupt/abort a command that is in progress? - ACADAIX does not have the open source "less" command that is available on Linux to paginate output. What similar command should you use instead to paginate output to your screen one page at a time? - What is the name of the command that displays system documentation? - What command line do you type to get the system documentation that is related to the command that gives you the system documentation? - If you say "man ksh" on ACADAIX and then type the single letter "h" at the "(14%)" prompt at the bottom of the first screen, what happens? What program are you talking to when you type the "h"? (Hint: If this were Linux, it would be the "less" command doing the pagination. ACADAIX does not have "less". What program does it use instead?) - In which of the nine traditional sections of the Unix manual do you find manual pages for user programs (also called "utilities" or "commands")? (see your text, p.30) Note: The ACADAIX man command has its own special brand of "man" command that doesn't give pretty results. If possible use the "man" command on a real Linux system that has all the man pages online. - In which section of the manual do you find file formats (e.g. the format of the Unix /etc/passwd file)? (text, p.30) - What is the difference between the behaviour of "cat" and "ls"? NOTE: The file /etc/termcap (text p.37) is not present on ACADAIX. You will find the huge Unix password file named /etc/passwd to be a good substitute for practicing the pagination commands. Note the spelling of the file name. - The IBM proprietary version of its pagination command has some added features that make it resemble the description of "less" given in the text. Name one such feature. - What error message is produced from this command line on ACADAIX: $ cat "screen door" Is this message more or less clear than the similar error message shown for "cat" in the box on page 36 in your text? - Explain the differences in the error messages for the following two command lines (note the subtle difference): $ cat "screen door" $ cat screen door - What does "white space" mean? - Why should you avoid shell "special characters" in file names, e.g. "*"? - What does "quoting" or "escaping" a shell special character mean? - Chapter Review Questions: On ACADAIX or Linux: 1, 2, 3, 7, 8, 9, 11 NOTE: IBM has their own proprietary format for Unix man pages, unlike ! "has its" or "have their". any other vendor. (The man pages are derived from an IBM proprietary document format.) For "real" Linux man pages, such as you would find on most any other Unix system except IBM AIX, telnet to a real Linux server or see the Linux Documentation links on the Web: http://www.linux.com/links/Documentation/ In Chapter 2, you learned a little about these Unix command names: passwd, man, ls, cat, less, more, rm You learned what these special characters do: CONTROL-D CONTROL-H CONTROL-L (bash shell only) CONTROL-U CONTROL-W CONTROL-C Most importantly, you learned that the "man" command can be a big help in finding out what commands do and what commands are available. )---------- quotes.txt -- Tue 2002-Jan-22 00:32:29 ===================================== Unix/Linux Shell Command Line Quoting ===================================== -IAN! idallen@freenet.carleton.ca Ref: Chapter 2, p.37-38 Chapter 10, very bottom of p.320; most of p.321 http://www.washington.edu/computing/training/125/quote.html http://unix.about.com/library/glossary/bldef-quoting.htm http://www.grymoire.com/Unix/Quote.html ====================================== Which programs treat quotes specially? ====================================== Quotes, both double and single, are treated as special characters when fed as input to the Unix shells, usually by typing them on shell command lines or putting them inside shell scripts. Quotes are not special characters when fed as input to most other programs such as "cat" or "head": $ echo "It's a nice day" It's a nice day. <= note how shell processes quoted string $ cat "It's a nice day" <= typed input "It's a nice day" <= unchanged program output on stdout ^D $ head -1 "It's a nice day" <= typed input "It's a nice day" <= unchanged program output on stdout $ The shell treats quotes specially when they are used on a command line or in a shell script. Most other programs do not treat quotes specially. =============== Why use quotes? =============== The shells use quotes to protect other special characters from expansion by the shell. Backslashes also protect special characters from the shell; but, sometimes they are far less convenient to use. For example, one could protect all the special characters in a line of text with backslashes: $ echo This\ line\ is\ a\ single\ \"argument\"\ to\ the\ \"echo\"\ command. This line is a single "argument" to the "echo" command. Alternately, one could simply use single quotes to do the same thing: $ echo 'This line is a single "argument" to the "echo" command.' This line is a single "argument" to the "echo" command. Most agree that the use of quotes is easier than using a lot of backslashes. The quotes are used by the shells to locate a string of characters to protect. The quotes themselves are not part of the string; the quotes are removed once the string of characters is collected. This next echo command line has six string arguments: $ echo "a b" c 'd e' f "g h" "i j" a b c d e f g h i j ^^^^ ^ ^^^^ ^ ^^^^ ^^^^ 1 2 3 4 5 6 Blanks are special characters to the shell, unless they are hidden from ! Use "protected (from expansion)" consistently. the shell by quoting. The shell splits the command line into arguments on any un-quoted blanks. Blanks that are quoted are not seen by the ! Quoted blanks are seen; they aren't processed as special. Their ! "end the current token" function is bypassed. shell; they become part of the argument string passed to the command being invoked. Unquoted blanks, special to the shell, simply separate arguments. You may type one blank to separate the arguments or a hundred blanks; it makes no difference to the shell or to the arguments: $ echo a b c a b c $ echo a b c a b c The three string arguments passed to the echo command in the above two cases are identical. There are three arguments, separated by blanks. ================ Quote Processing ================ Quotes are processed by the Unix shells from left-to-right. Let's examine how this works: $ echo 'one"two"three'four"five"six"seven'eight'nine"ten one"two"threefourfivesixseven'eight'nineten The first quote found by the shell, starting from the left, is a single quote. The shell collects all the characters up to the next single quote. All these characters are protected from further analysis by the shell. The words one, two, and three are all inside this first single-quoted string. The double quotes, inside single quotes, have no special meaning to the shell. They are simply part of the string. The single quotes themselves, used to delimit the string, are not part of the string. The word four is outside of any quotes. Right after four we start a double-quoted string that extends until the next double-quote character. The word five is inside this double-quoted string. The double quotes themselves, used to delimit the string, are not part of the string. The word six is outside of any quotes. Right after six we start a double-quoted string that extends until the next double-quote character. The words seven, eight, and nine are all inside this first double-quoted string. The single quotes, inside double quotes, have no special meaning to the shell. They are simply part of the string. The double quotes themselves, used to delimit the string, are not part of the string. The word ten is outside of any quotes. The given command line argument to "echo" was built up of text that contains three quoted strings. Each quoted string connected immediately to another non-blank character, and all the blanks are inside quoted strings were protected from the shell. Thus, the "echo" command is ! Bad syntax; probably partly updated from description of a previous ! version of the example. handed only *one* command line argument. (Command line arguments are separated by blanks. There were no unquoted blanks used.) The fact that this one argument was made up of several pieces on the command line, including both quoted and unquoted parts, is invisible to the echo command. The shell did the work; the echo command gets its one argument. The quotes seen by the shell are *not* part of the argument. The quotes delimit the string; they are not part of the string. ======================== Single vs. Double Quotes ======================== Single quotes are "stronger" than double quotes. Nothing is special inside single quotes; the shell treats all the characters, no matter what they are, as part of the string being collected and it does not expand any of them inside single quotes. Inside double quotes, the shell still sees and expands some special characters. The most important character expanded by the shell even inside double quotes is the dollar sign that signals the start of a shell variable: $ echo '$SHELL' $SHELL <= single quotes prevent expansion $ echo "$SHELL" /bin/bash <= double quotes permit expansion We will talk more about shell variables in a future lecture. )---------- test2-unix.txt -- Sun 2002-Feb- 3 01:25:00 ====================================== What to Study for the Second Unix Test ====================================== -IAN! idallen@ncf.ca The Second Test (Unix) takes place in the last part of your Lab period in Week 6 of the course. (See the Course Outline.) The test is 90 minutes long. It is worth 10% of your term mark. The test covers Chapters 1-5 in the Linux text, with emphasis on the last two. The test also covers web page "Notes" files for Weeks 1-4 and the Chapter notes files and Floppix labs mentioned in those weekly files (Chapters 1-5). What you must bring to the test: 1. You will need your Floppix diskettes (and your backup copies). You must know how to boot Floppix with DHCP networking enabled. 2. You will need a pencil (only a pencil) to write in answers to multiple-choice questions on a mark-sense sheet. You must use pencil. 3. Your name, EMail, and web password must be registered with me at least 24 hours before the test begins, using the registration form on the course web page. If you are not registered, you won't have a userid on the Linux test machine and won't be able to log in to that test machine to answer questions. You will work on Floppix, on ACADAIX, and on a Linux test machine. You must know how to use telnet on Floppix to telnet to ACADAIX and to the Linux test machine. No Windows access is permitted during the test. The address of the Linux test machine has been given out in the announcements news group. Make sure you can login to the machine. Knowing how to switch among the three Floppix consoles will save you time, since you can have one console logged into ACADAIX and another logged in to the Linux test machine at the same time. The test is open-book, open-note, and open-terminal. You may use any online or written resources available to you; but, you may not consult with or communicate with anyone except the instructor in any way during the test period. Multiple-choice answers will be written down on mark sense forms. Some questions will require you to execute commands on any of the three machines and save the results into files. You may be asked to download and upload files using FTP between any of the three machines. You will not have enough time to try every multiple-choice question on the computer or to look up all the terms in the textbook. You must know your material before the test. The test will appear to be "too long" if you try to type in all the questions to see the output. Thinking about the correct answer will be faster. Most of the multiple-choice questions can be answered quickly, without a computer, from the knowledge you have acquired in lectures and by doing the weekly readings and homework material. Your Floppix lab Exercises and Questions & Answers are very important. Strategy for getting maximum marks: Answer the questions you know, first. Come back later to the questions for which you require a computer. For more strategy on writing Unix tests see: test_preparation.txt )---------- test2-practice1.txt -- Sun 2002-Feb- 3 01:39:46 Practice Practice Practice DAT2330 - Ian Allen - idallen@ncf.ca These are the kinds of questions you might be asked on a practice Unix test. This is not the real test. Part A: Unix work. You will need to be running Floppix to do this test. To log in to the Linux Machine, use telnet with your Algonquin userid (abcd0000). Windows telnet will not be permitted during the test. The IP address of the Linux Test Machine is: Unless otherwise indicated, input files are on the Linux Test Machine. Unless otherwise indicated, all output files must be placed in your HOME directory on the Linux Test Machine (not on ACADAIX; not on Floppix). Marks will be deducted (up to 100%) for files incorrectly placed and/or incorrectly spelled. Marks: 1 Put the single word "hello" into a (new) file named hello.txt under a (new) directory named "mydir" in your HOME directory. Marks: 1 Place a copy of the file /etc/motd into the file named firewall_motd.txt. Marks: 2 Translate all the blanks in file /etc/motd into dollar signs and put the translated output in file dollar_motd.txt. Marks: 2 Create a directory named mysecret.dir that only has read and execute permissions for you and write permissions for your group. (It should have no other permissions.) Marks: 2 Put this sentence into a file named sentence.txt: It isn't hard to lose a $2 coin. Use any method you know to create this file. Marks: 2 Put a simple list of all the names (just the names; but, including all the hidden names) contained in the /etc/skel directory on the Linux Machine into the file named skel.txt. Marks: 2 Show the permissions, owner, and date modified for all the non-hidden names in the /etc directory that end in the letter b. (Don't show any hidden names.) Put the output in the file named etc_permissions.txt. Marks: 2 Find all the lines in the file /etc/termcap that contain the adjacent digits 123. Put the output in the file 123_termcap.txt. Marks: 3 Place a copy of the file /etc/motd from your Floppix system into the file named floppix_motd.txt in your HOME directory on the Linux Machine. Marks: 3 Create a directory named "super secret" in your HOME directory on the Linux machine. (This directory name contains a blank.) Remove all permissions for everyone from the directory. Marks: 3 Create a file named myfile and a directory named "my*" in your HOME directory on the Linux machine. (The directory name contains an asterisk - the name is three characters long.) Marks: 3 File /thome/alleni/dat2330/tt/saturn1.gz on machine ACADAIX is compressed. Find a way to uncompress and read the contents. Do what the file says. Marks: 4 Create a directory named "foo.dir". Under foo.dir, create directory "bar.dir". Under bar.dir, create a file named "one.txt". Under foo.dir, create a second directory "bleen.dir". Create a second name for the one.txt file under directory bleen.dir. The second name should be "two.txt". Both names should lead to the same data. Create a third name for the file, "three.txt", under the foo.dir directory. All three names should lead to the same data. Marks: 4 Put a reverse-sorted copy of your Floppix file /etc/hosts into the file named floppix_sorted_hosts.txt in your HOME directory on the Linux Machine. Marks: 4 Transfer a copy of the file /etc/motd from the machine acadaix into the file named acadaix_motd.txt in your HOME directory on the Linux Machine with all the asterisk characters changed to dollar signs. ____________________________________________________________ Part B - Multiple Choice What permissions do you need on a file to be able to remove its name from a directory? What permissions do you need on a directory to be able to access the data in a file under the directory? How many hidden directory names are in the /var directory on the Linux Machine? How many hidden directory names are in the /etc directory on ACADAIX? How many hidden directory names are in the /home directory on your Floppix system? How can you increase the link count of a directory that you own? Can you make a hard link to a directory? If file "foo" contains nine lines, each of which is the number of the line in the file, what is the output of this command: sort foo foo | head -4" If file "foo" contains twenty lines, each of which is the number of the line in the file, what is the output of this command (as written): sort foo head If file "foo" contains nine lines, each of which is the number of the line in the file, what is in file foo after this command: head -1 foo >foo If file "foo" contains nine lines, each of which is the number of the line in the file, what is in file foo after this command: ls foo >foo If file "foo" contains nine lines, each of which is the number of the line in the file, what is in file foo after this command: wc foo >foo What is in file "foo" after this command executes? who | echo hi | tee foo >bar If file "foo" contains ten lines, and file "bar" contains twenty lines, then how many lines are in file "bleen" after this command: cat foo bar >bar ; cat foo bar >bleen How do you display names in the current directory that are exactly two digits (not two letters)? How do you remove a name containing a special character from a directory? What permissions on my directory "foo" let me see the names in the directory but do not let me access any of the files? )---------- test2-practice2.txt -- Sun 2002-Feb- 3 01:41:15 You will find previous versions of the second test (Unix) online in the web pages for previous terms of this course. Consult my online web pages for these past terms. -IAN! idallen@ncf.ca )---------- argv.txt -- Sun 2002-Feb- 3 01:48:28 #!/bin/sh -u # This is a Unix shell script. # $0 [args...] # Purpose: Count and display each argument to this shell script. # It also displays the command name, which is often referred to as # "Argument Zero" ($0) but is never counted as an argument. # -IAN! idallen@ncf.ca June 2001 # Set the search path for the shell to be the standard places. # Set the umask to be non-restrictive and friendly to others. # export PATH=/bin:/usr/bin umask 022 # The $0 variable contains the name of this script, sometimes called # "argument zero". It is not an argument to the script itself. # The script name is not counted as an "argument" to the script; # only arguments 1 and up are counted as arguments to the script. # echo "Argument 0 is [$0]" # Now display all of the command line arguments (if any). # These are the real arguments to this script. # let count=0 for arg do let count=count+1 echo "Argument $count is [$arg]" done )---------- week5.txt -- Sun 2002-Feb- 3 02:24:08 Week 5 Notes for DAT2330 - Ian Allen Supplement to Text, outlining the material done in class and lab. Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM![*] ([*] Read The Fine Manual) Look under the "Notes" button on the course web page for these study notes for the Linux textbook: chapter8.txt (The VI text editor) chapter7.txt (Networking and Internet) Complete these Floppix Labs on http://floppix.ccai.com/ Floppix Lab 8: introduction to vi Floppix Lab: 22 (e-mail) [Review] Floppix Lab: 23 (telnet) [Review] Floppix Lab: 24 (ftp) [Review] Floppix Lab: 25 (links [www]) [Review] Complete this short VIM tutorial on the Linux test machine by copying the tutorial file into your directory and editing it: $ cd $ cp /usr/share/vim/tutor/tutor tutor $ vim tutor Follow the instructions in the tutorial. Note: An early version of "vim" (Vi iMproved) is also available on ACADAIX. You can get help by typing ":help" after you start the editor. The first help screen tells you how to exit the help screen. You may also study one or more of the VI tutorials available under the "Unix Resources" button on the course home page. All your scripts, and your JCL in the MVS part of the course, will be entered using this VI (or VIM) text editor. Learning to use it well will speed up your work and get you a better mark. Learning to use VI (or VIM) is not an intellectual exercise. You have to use it and get used to it through practice. I will cover only the basics of VI in class - it's up to you to do the practice. The more you learn about the editor, the faster you will be able to edit and the better your marks will be on tests. )---------- pathnames.txt -- Sun 2002-Feb- 3 22:44:37 ==================== Unix/Linux Pathnames ==================== -IAN! idallen@freenet.carleton.ca Ref: Chapter 4 Floppix Lab 3: (logical organization of the Linux filesystem) Floppix Lab 4: (navigating the filesystem) Given this directory structure and current directory: $ pwd /tmp/idallen $ ls -l drwxr-xr-x 2 idallen idallen 4096 Feb 3 20:33 dir1 -rw-r--r-- 1 idallen idallen 0 Feb 3 20:33 file1 All the pathnames below give the same output, because all these pathnames refer to the same (current) directory: $ ls $ ls . $ ls ./. $ ls ././././././. $ ls dir1/.. $ ls ./dir1/.. $ ls dir1/../. $ ls dir1/../././././. $ ls ../idallen $ ls ./../idallen/./././. $ ls ../../tmp/idallen $ ls ../idallen/dir1/.. $ ls ../idallen/dir1/.././././. $ ls dir1/../../idallen $ ls ./dir1/../../idallen/. $ ls /tmp/idallen $ ls /tmp/idallen/././. $ ls /tmp/idallen/dir1/.. $ ls /tmp/../tmp/idallen/../idallen/dir1/.. $ ls /././tmp/./././../tmp/./././idallen/./././../././idallen/./dir1/./.. All the pathnames below give the same output, because all these pathnames refer to the same sub-directory "dir1": $ ls dir1 $ ls dir1/. $ ls ./dir1 $ ls ./dir1/. $ ls ././././dir1/././././. $ ls ../idallen/dir1 $ ls ../idallen/dir1/../dir1/. $ ls ../../tmp/idallen/dir1/../../idallen/dir1/. $ ls /tmp/idallen/dir1 $ ls /././tmp/./././idallen/./././dir1/./. All the pathnames below give the same output, because all these pathnames refer to the root directory: $ ls ../.. $ ls ../../../../.. $ ls dir1/../../.. $ ls dir1/../dir1/../../.. $ ls ../idallen/../.. $ ls ../../tmp/.. $ ls / $ ls /. $ ls /./././. $ ls /tmp/.. $ ls /tmp/idallen/../.. $ ls /tmp/idallen/dir1/../../.. These patterns match all non-hidden names in the /tmp/idallen directory: $ echo * $ echo ./* $ echo dir1/../* $ echo ../idallen/* $ echo ../../tmp/idallen/* $ echo /tmp/idallen/* $ echo /tmp/idallen/dir1/../* These patterns match all non-hidden names in the /tmp directory: $ echo ../* $ echo ./../* $ echo .././* $ echo /tmp/* $ echo /tmp/./* $ echo /././././tmp/./././* These patterns match all non-hidden names in the root directory: $ echo ../../* $ echo ../../../../../../* $ echo /* $ echo /tmp/../* $ echo /tmp/idallen/../../* $ echo /tmp/idallen/dir1/../../../* )---------- week6.txt -- Thu 2002-Feb- 7 21:39:26 Week 6 Notes for DAT2330 - Ian Allen Supplement to Text, outlining the material done in class and lab. Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM![*] ([*] Read The Fine Manual) Look under the "Notes" button on the course web page for these study notes for the Linux textbook: chapter10.txt (The BASH Shell [not the whole chapter!]) Complete these Floppix Labs on http://floppix.ccai.com/ Floppix Lab 26: BASH Scripts Floppix Lab 21: Process Management Note: An early version of "bash" is also available on ACADAIX. I recommend using it rather than the native ksh shell. -------------------------------------- Order of Shell Command Line processing -------------------------------------- The order in which the shell applies various processes (word splitting, pathname expansion, variable substitution, etc.) to each command line is partially outlined on page 358, except that it is wrong. Items 2 through 6 (tilde, parameter, variable, command, and arithmetic) happen *at the same time*, not one after the other. You cannot have a command substitution inside a variable or parameter, or the other way around. The full details of how the shell does command line processing for all possible circumstances would take several pages of documentation, and you don't need to know it all unless you're going to do serious shell script programming. Here's just a taste of the complexity (not to be memorized), from the man page for bash: EXPANSION Expansion is performed on the command line after it has been split into words. There are seven kinds of expansion performed: brace expansion, tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, word splitting, and pathname expansion. The order of expansions is: brace expansion, tilde expan sion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion. On systems that can support it, there is an additional expansion available: process substitution. Only brace expansion, word splitting, and pathname expan sion can change the number of words of the expansion; other expansions expand a single word to a single word. The only exceptions to this are the expansions of "$@" and "${name[@]}" as explained above (see PARAMETERS). [...] Word Splitting The shell scans the results of parameter expansion, com mand substitution, and arithmetic expansion that did not occur within double quotes for word splitting. Don't try to memorize the above paragraphs. For the purposes of this course, you need to know the processing order for the following features we have studied: 1. Initial word splitting (whitespace, semicolons, pipes, etc.) 2. Identification and removal of I/O redirection words 3. Tilde, parameter, and variable expansion of words 4. Further word splitting (not when quoted) 5. Pathname expansion (globbing, wildcards) (not when quoted) 6. Removal of quoting 7. Passing of the arguments to the command Note: The text says parameter expansion happens *before* variable expansion; this is wrong. They happen at the same time. You can't have a parameter that expands to contain a variable and then have the variable also expand, or vice-versa. Note that pathname expansion happens last, so that glob (wildcard) characters hidden inside parameters and variables will be expanded (often unintentionally). Many novice shell programmers forget this. Also note that if a glob pattern matches a filename with a blank, the blank is not treated as a special character (it doesn't generate multiple file names) because word splitting happens *before* globbing. A globbed filename containing one or more blanks is still treated as a single argument by the shell, because the shell already did the word splitting. The same is *not* true for blanks inside unquoted variables, which do cause multiple arguments, which is why you *must* put all your variables inside double-quoted strings to protect them from glob expansion! After the shell is finished processing the command line, it identifies the first word (first token) on the command line as a command name, looks for it in the search path $PATH (only if the word doesn't contain a slash), and then runs that command (with its arguments) and waits for it to finish. (An "&" at the end of the line will cause the command to be run "in the background"; the shell will not wait for it to finish before prompting you for another command.) Exercise: Correct the Summary in Chapter 10, p.358 so that it says that items 2 through 6 (tilde, parameter, variable, command, and arithmetic) happen at the *same* time. Exercise: Put the following actions in their proper order: - shell expands $-variables and ~-variables - shell expands pathname globs (wildcards, e.g. *) - shell looks for the command name in $PATH and runs it - shell identifies and removes redirection - shell splits the command line into words - shell word-splits unquoted variable expansions Use the above order information to answer the following questions, in this order (pay careful attention to the use of single and double quotes): $ mkdir empty $ cd empty $ ls -a (Q1) What output appears here? $ x="foo*" $ echo $x (Q2) What output prints here? $ echo "$x" (Q3) What output appears here? $ echo '$x' (Q4) What output appears here? $ touch foobar $ ls (Q5) What output appears here? $ echo $x (Q6) What output prints here? $ echo "$x" (Q7) What output appears here? $ echo '$x' (Q8) What output appears here? $ echo hi >$x $ ls (Q9) What output appears here? $ echo there >"$x" $ ls (Q10) What output appears here? $ echo mom >'$x' $ ls (Q11) What output appears here? $ rm $x $ ls (Q12) What output appears here? Explain: $ mkdir empty $ cd empty $ x='a b c d' $ set | grep 'x=' x=a b c d $ touch $x $ ls | wc 4 4 8 # why are there four file names? What names? $ rm * $ touch "$x" $ ls | wc 1 4 8 # why is it only one file name now? What name? $ rm * $ touch '$x' $ ls | wc 1 1 3 # what is the name now? Review: Order of shell processing of command line. Explain: $ mkdir empty $ cd empty $ touch aa ab ac ad ae af $ x='a* b* c* d*' $ echo $x aa ab ac ad ae af b* c* d* $ echo "$x" a* b* c* d* $ echo '$x' $x Review: Order of shell processing of command line. Explain: $ x='date >out' $ echo $x date >out $ $x Invalid character in date/time specification. Why isn't the output going into file "out"? ! I'd have said "because the output is to stderr, and stderr has not been ! redirected so remains the tube", but there's been no mention of stderr. $ sort $x sort: can't open date sort: can't open >out $ ls $x date not found >out not found Why isn't the output going into file "out"? $ sort "$x" sort: can't open date >out $ ls "$x" date >out not found $ sort '$x' sort: can't open $x $ ls '$x' $x not found Review: Order of shell processing of command line. Explain: $ touch a b '>out' $ ls >out a b $ echo * >out a b $ echo >out a b $ ls >out a b out $ cat '>out' $ cat out a b Why is there output in the file named "out" but none in the file named ">out"? Review: Order of shell processing of command line. What is the output of this command sequence: $ foo='bar haven' $ echo foo $foo "$foo" '$foo' bar NOTE: blanks will be counted in the answer Here is an alias using a pipe to convert lower-case to upper-case: If your boss is an experienced IBM programmer, she's used to reading text in all UPPER CASE. This alias is for her (Review: Chapter 3, p.101; Chapter10, p.309): $ alias who="who | tr '[a-z]' '[A-Z]'" $ who Are the quotes required around the two arguments to "tr"? (Are there any shell special characters in those two arguments?) You can use a similar command to convert a lower-case file of IBM MVS JCL into upper-case. EXPERIMENT: Why doesn't this convert the file "myfile" to upper-case? $ alias upper="tr '[a-z]' '[A-Z]'" $ upper myfile Why is the file "myfile" empty after this command is run? Using the above alias, the following command line doesn't work either: $ upper myfile >new Why does this generate an error message from "tr"? (RTFM; p.101) )---------- chapter10.txt -- Thu 2002-Feb- 7 21:44:00 -------------------------------- DAT2330 - Unix - Prof. Ian Allen -------------------------------- Here are questions you should be able to answer. You will likely find questions similar to these on tests and exams. Chapter 10 - The Bourne Again Shell (excerpts) Note: For much of the material, the Korn Shell (ksh), Bourne Shell (sh), and Bash Shell (bash) behave identically. Note: For practice, type in all the scripts and command lines! You can't practice debugging the scripts simply by reading them. Practice writing them! Note: Concentrate on the text sections related to material covered in lectures, assignments, and in these questions. Skip over all the "Optional" sections in this chapter. Also skip over these sections: - "Directory Stack Manipulation" p.313 - "Removing Variables" p.322 - "The readonly builtin" p.322 - "The declare builtin" p.324-325 - the table "Special Characters / Display in Prompt" p.331 - "PS2" p.331 - "CDPATH" p.332 - "The shift builtin" p.334 - "The set builtin" p.335-337 - "History" and "Alias" p.340-350 - "Brace Expansion" p.351 - "Arithmetic Expansion" p.354 - "Word Splitting" p.356 - "Command Line Editing" p.359-362 Skip over the above sections. - Which successful shell has been used in all versions of Unix and Linux? - What is a shell script? - True or False: Shell scripts cannot have their input or output redirected; only Unix command output can be redirected. - What *minimal* permissions are needed on the file for a shell to *read* a file containing a shell script, i.e.: $ sh script - What *minimal* permissions are needed for the Unix kernel to *execute* a file containing a shell script, i.e.: $ ./script The Unix kernel will try to execute any file that has execute permissions. Will the Shell (not the kernel) be able to execute (read) a script that has execute permissions but not read permissions? (Try it!) - p.303 - Do not expect the current directory to be in your search PATH. The system administrator has ill-advisedly included it in PATHs set for students on ACADAIX. It is *not* included under most modern versions of Unix/Linux. Read p.304 on how to execute correctly a shell script that is in the current directory. Always use "./" in front of your scripts. Read the top of p.330 for an explanation of the security risk of having the current directory in your search PATH. - Warning: I often create test questions that create scripts that have the same names as existing (but often obscure) Unix commands! If you fail to follow the syntax given at the top of p.304 you will not be able to run your shell script. - Note: The book makes two errors throughout this chapter: 1) The book fails to use "#!/bin/sh -u" to start scripts 2) The book fails to use ./scriptname to run scripts In this course (and in the real world), you must not make these two mistakes. Go through your book now and fix all the script examples! - What does the semicolon special character mean to the shell? - How would you generate the following output sentence on your screen using the "echo" command: I read Chapter 2; so, it's easy to escape "special" characters. - True or False: If the command to the left of a semicolon fails (with a bad return status, or "not found"), the shell will not execute the command to the right of the semicolon. (Hint: Try it.) - Note how an ampersand ("&") puts an entire pipeline into the background: $ sort /etc/passwd | tail -15 | grep '^zoo' & It puts the whole command line into a "job" in the background, not just the last command in the pipeline. - True or False: When printing to the terminal, output on standard error looks different (visually) than output to standard output. - True or False: These two command lines generate the same output: $ ls >out 2>&1 nosuchfile $ ls 2>&1 >out nosuchfile (Hint: Try them. Read p.309) - True or False: The syntax for redirecting the output of an echo command to appear on standard error (instead of on standard output) is: echo 1>&2 "This is an error on standard error" - True or False: The following three commands are identical: 1>&2 echo "This is an error on standard error" echo 1>&2 "This is an error on standard error" echo "This is an error on standard error" 1>&2 - Rewrite the last sentence on p.310 to say: The jobs builtin lists all the background jobs of the current shell. - True or False: The "jobs" command shows all background jobs, including background jobs running in other shells. - True or False: Background jobs can read from your terminal. - What is the name and process number of the very first, "root" process on Unix? (It has process number one.) ! Saying that it "has process number one" reveals that you're fishing for ! "init", but (speculating; didn't read the source) I doubt that that's ! correct. Trivially, "top" shows "init"'s PPID as "0"; I suspect a ! skeletal process compiled into the kernel (and process table). What does ! the scheduler do when it's ready to dispatch a process and no process is ! ready to run? My perception/speculation here is severely colored by other ! (non-unix) systems (well, VMS) that have a compiled-in dummy process that ! never does I/O and is always ready to soak CPU. - The option "-f" gives you the nicest output from the "ps" command, especially on ACADAIX. Try both "ps -l" and "ps -f". See? - True or False: A process can have more than one parent process (PPID). - True or False: A directory can have more than one parent directory. - True or False: A process can have more than one child process. - True or False: A directory can have more than one subdirectory. - What is your shell doing after it has found a command and started it running for you as a child process (in the foreground)? - The "cd" command is a built-in command to the shell; the shell does not look for it in your PATH, nor does the shell fork and start a separate child process to execute it. Why doesn't the shell start up a separate process to do the "cd" command? - Are all the shell variables you set and change in your shell automatically exported to all of your child processes and shells? - True or False: The Unix shells will read commands from a (single) file given as a command line argument. If there is no file argument, the shells read from standard input (as do most Unix commands). When a shell reads from a keyboard, it issues a prompt first. - Erase the sentence at the top of page 317 that talks about scripts running "more slowly". They do not. "bash script" and "./script" are equally good; one requires only read permission, the other requires both read and execute permission. - p.317 - In this course (and in the real world), you *must* put the name of the script interpreter as the first line of your scripts. If you don't do this, the system may later choose the wrong shell to run your script. Always put the interpreter in your scripts. Always start Bourne-type shell scripts with: #!/bin/sh -u (The -u checks for undefined variables. Use it!) The Shell /bin/sh is available on every Unix system on the planet. Only Linux systems have Shell /bin/bash installed "out of the box" - it doesn't exist on many commercial Unix systems. For maximum portability, use "/bin/sh -u" to start all your shell scripts. Note: The book makes two errors thought this chapter: ! "throughout". Redundant--you said it 100ish lines ago. 1) The book fails to use "#!/bin/sh -u" to start scripts 2) The book fails to use ./scriptname to run scripts In this course (and in the real world), you must not make these two mistakes. Go through your book now and fix all the script examples! - If the first line of an executable file is "#!/bin/cat", what will the output be when you execute the file? (Hint: Try it! Copy /etc/motd into a file, add "#!/bin/cat" as the first line, make the file executable, and then execute it. What do you see?) - If the first line of an executable file is "#!/bin/sort", what will the output be when you execute the file? (Hint: Try it!) - If the first line of an executable file is "#!/usr/bin/head -2", what will the output be when you execute the file? (Hint: Try it!) - If the first line of an executable file is "#!/usr/bin/tail -2", what will the output be when you execute the file? (Hint: Try it!) - If the first line of an executable file is "#!bin/nosuch", what will the output be when you execute the file? (Hint: Try it!) Remember this error message! You may see it again. Remember it! - Put temporary echo commands into your files .bashrc and .bash_profile in your HOME directory. Log out and log back in. Which echo commands executed? Now start another bash shell (from the command line). Which echo commands executed? Now, remove the echo commands. - p.318 The last script line at the bottom of the page is missing a blank before the "-f" test. It should read: if [ -f ~/.bashrc ]; then source ~/.bashrc; fi The "if" statement is explained in Chapter 11. - What conventions do Unix Shell programmers use when using UPPER and lower case letters in variable names? - What is a "positional parameter"? Is it the same as "a command line argument"? - Is the variable "$#" a valid name? What about "$$" and "$?"? - True or False: When assigning to a variable, you must both precede and follow the equals sign ("=") with at least one space. - True or False: putting a variable inside double quotes prevents the variable from being expanded by the shell. - True or False: putting a variable inside single quotes prevents the variable from being expanded by the shell. - What command is most often used to display the values of variables on the screen? (These example files, and the text, use this command a lot!) - Fix the syntax of the following variable assignment: $ myvar=You owe me $2 CDN $ echo "$myvar" Fix the above so that the exact text appears on the screen. - How would you append the string " dog" to the (unknown) contents of an existing variable named "foo"? For example: $ foo=my $ [your appending shell command goes here] $ echo "$foo" my dog You don't know what $foo contains before you append to it. You must use a shell assignment to append to the existing value of $foo. - Why is it critical to put double quotes around variables when you use them? (Study the glob expansions on top of p.322 and p.326!) It is essential that you quote all your variables when you use them in shell scripts. Many scripts malfunction because novice shell programmers do not do this. I will always test your scripts with shell glob special characters as input, to see if you have remembered to quote all your variables. - When would you "export" a shell variable? - Can a child process change the value of a variable in its parent process? Can a child process change the current working directory of its parent process? (same answers!) - Note that if you use the "-u" option to the shell to catch undefined variables, you won't be able to run the "subtest" script on p.323 as written (because "subtest" uses the value of an undefined variable, which usually indicates an error in your script). Remove the "-u" option for *this script only*. - Is the "read" command run by the shell as a built-in command, or is it run in a separate process? Why? - True or False: the "read" command reads lines from standard input. If reading from the keyboard, it reads until you type your EOF character (usually ^D - CONTROL-D). - Experiment with Command Substitution. You can put any Unix shell command line or pipeline into a Command Substitution, and the output will be placed into the current command line by the shell, for use by some other command (as if the output were contained in a shell variable): $ echo "There are $(who | wc -l) users online" $ echo "There are $(ls | wc -l) non-hidden names in this directory" $ echo "The file /etc/motd contains $(wc -l " $ PS1="$PWD >" $ PS1=`$PWD >` $ PS1='$PWD >' Test your answer by using "cd" to change directories. Does your prompt change, too? It should! - The "." (dot) or "source" command instructs the current shell to open the given file (you need read permission) and to read commands from that file and execute them directly. This differs from running a script in a subshell or separate process; because, it is the *current* shell that is executing the commands and all variable assignments will change variables in the current shell. - True or False: If I put the single command "exit" into an executable shell script named "myexit" and run it ("./myexit"), my current shell will exit (and I may be logged out). - True or False: If I put the single command "exit" into a readable file "myexit" and source it (". myexit" or "source myexit"), my current shell will exit (and I may be logged out). - True or False: The first file name argument to the "." (dot) or "source" command must have execute permissions so that the shell can read the file and process the commands that it contains. - What is the name of the positional parameter (numeric variable name) that contains the name of the shell script being run, inside the script being run? (p.333) - Study well the short section on "Command-line arguments" (p.333) Note! Fix the sample scripts given in the book to quote all uses of shell variables! You *must* always double-quote your variables to prevent Shell wildcard (glob) expansion! - True or False: The $# variable begins a comment line. - Use the argv.sh script (given above) to learn the difference between the $* and $@ variables. Write and run this small "numargs" script: $ cat numargs #!/bin/sh -u # This script shows how $* and $@ are different. export PATH=/bin:/usr/bin echo "First using $*" ./argv.sh "$*" echo "Next using $@" ./argv.sh "$@" Study these outputs: $ ./numargs a b c d $ ./numargs a\ b c\ d $ ./numargs "a b" "c d" $ ./numargs 'a b' 'c d' - True or False: The $* variable expands to be all the files in the current directory. - True or False: The $@ variable expands to be your email address. - Put the current process id ($$) into your PS1 prompt and export it. Now start up a sub-shell (another copy of bash). The prompt should change to be the new PID of the new shell. Exit that shell. The prompt should indicate the original shell again. - Predict the output of the following: $ echo $$ 1234 $ cat myscript #!/bin/sh -u # This script echoes the process ID of the shell running it. echo The PID is $$ $ ./myscript The PID is 2345 $ . myscript The PID is [what is the output] - What is the "Exit Status" of a command? (See the textbook index.) - What shell variable contains the exit status of the most recently executed command? - Do you need to quote the variables $$, $#, and $? when using them in shell scripts, to prevent wildcard (glob) expansion? Why or why not? - What is one of the first things a shell does to a command line, to generate "tokens" or "words"? - Predict the output of these commands: $ x='$$' $ echo $x $ echo "$x" $ x="$$" $ echo $x $ echo "$x" - Note: The text is wrong about the order of expansion on p.358. Items 2 through 6 (tilde, parameter, variable, command, and arithmetic) happen *at the same time*, not one after the other. You cannot have a command substitution inside a variable or inside a parameter; it doesn't work. - What are the meanings and uses of the following shell variables: HOME MAIL PATH PS1 SHELL $0 $1, $2, $3, etc. $* $@ $# $$ $? - Chapter 10 Chapter Review questions: 1,2,3,4,5,6,10,13,16 Review question 11 is wrong. Parameter expansion happens *at the same time* as variable expansion (but before pathmame expansion). )---------- umask.txt -- Mon 2002-Feb-11 15:56:17 ===================== Umask and Permissions ===================== -IAN! idallen@ncf.ca Every process on Unix (including every shell process) has its own "umask". The default permissions for newly created directories is 777 (rwxrwxrwx) masked by the permission bits set in the umask. (See below for an explanation of Unix numeric permissions "777".) The default permissions for newly created files is 666 (rw-rw-rw-) masked by the permission bits set in the umask. Every bit set in the umask *takes away* that permission from the default permissions for newly created files and directories created by that process. Every process has its own umask; the umask is inherited by child processes. The shell command "umask 022" sets to 022 (----w--w-) the permissions to be removed (masked) from the default permissions, for files and directories created by the shell (and by commands run from that shell). It removes write permission for group and other from newly created directories and files. A new directory would have permissions 777 (rwxrwxrwx) masked by 022 (----w--w-) resulting in 755 (rwxr-wr-x) permissions. A new file would have permissions 666 (rw-rw-rw-) masked by 022 (----w--w-) resulting in 644 (rw-r--r--) permissions. Look for "umask" in some of the following pages for more examples: http://www.ucolick.org/~ksa/manual/level2.html#umask http://www.ebone.at/books/programmers/sonstiges/oreillybookself/unix/upt/ch22_02.htm http://www.cs.arizona.edu/computer.help/policy/DIGITAL_unix/AA-PS2HD-TET1_html/uc6.html#s _umask http://www.acm.uiuc.edu/workshops/security/umask.html http://www.inficad.com/~thurmunit/azunix/lectures/08.shtml http://www.cis.rit.edu/class/simg211/unixintro/Access_Permissions.html http://www.uvm.edu/~hag/wcreate/644.html --------------------------------------- Note on "022"-style numeric permissions --------------------------------------- Unix permissions for user, group, and other have traditionally been expressed using a set of three octal digits, where each digit represents the octal number you get by expressing the three "rwx" permissions in binary form. Convert the on permission bits in "rwx" into binary, then convert the binary number to an octal digit. Examples: octal 7 = binary 111 = rwx octal 6 = binary 110 = rw- octal 5 = binary 101 = r-x octal 4 = binary 100 = r-- octal 3 = binary 011 = -wx octal 2 = binary 010 = -w- octal 1 = binary 001 = --x octal 0 = binary 000 = --- Thus "chmod 741" means "set the mode to 741 (rwxr----x)". That is 7 (rwx) for owner, 4 (r--) for group, and 1 (--x) for others. Shell command "umask 027" means "mask (remove) permissions ----w-rwx" (--- for owner, -w- for group, rwx for others). A new directory created under this umask (e.g. by mkdir) would have permissions 777 masked by 027 = 750 (rwxr-x---). A new file created under this umask (e.g. by output redirection) would have permissions 666 masked by 027 = 640 (rw-r-----). )---------- rotatePuzzle.txt -- Tue 2002-Feb-12 17:04:42 =========== Unix Puzzle =========== -IAN! idallen@ncf.ca Based on Chapters 1-10 in the Linux text. On the machine ACADAIX there is a compressed text file named /thome/alleni/dat2330/tt/neptune/saturn_rot13.txt.gz that has had (only) the lower-case letters encrypted (translated). The file contents tell you (in UPPER CASE) how the file was encrypted. Uncompress and read the file to learn about the encryption used. After you understand how the file was encrypted, unencrypt the lower-case letters in the file (reverse the translation) and read what it says. Write an executable shell script named rot13.sh that will perform the encryption described in the above file. The script should read and process the file given as the script's first command line argument. (No error checking is required to verify that the command line argument exists or is a file.) Output should be to standard output. Test it on the saturn_rot13.txt file. What if you apply the rot13.sh encryption to a file twice? What is the output? What if you apply it three times? Four times? Five? )---------- CGIscript.txt -- Tue 2002-Feb-12 17:18:35 ============================= Building a CGI Script on Unix ============================= -IAN! idallen@ncf.ca A CGI script is a dynamically-generated web page. The content of the page is produced not from a static text file, but rather from the output of commands running on the web server. The content of the web page changes as the output of the commands change. You can build a dynamic web page using Unix commands, without knowing any CGI scripting. Here's how. 1) Create a public_html directory in your HOME directory on ACADAIX. Make sure it has read and execute permissions for everyone. Make sure your ACADAIX HOME directory has execute permissions for everyone. 2) Put these next lines of text into the file public_html/who.cgi. The lines must start at the left margin. No indenting! No blank line! (If you haven't learned any VI yet, start now!) #!/bin/sh # This is a CGI script. export PATH=/bin:/usr/bin umask 022 echo "Content-Type: text/plain" echo "" who Display and word-count the file to verify that it looks EXACTLY as above. $ wc public_html/who.cgi 7 17 114 public_html/who.cgi You can also checksum the file to make sure it is the same as mine: $ sum public_html/who.cgi 61714 1 public_html/who.cgi 3) Turn on execute permissions (for you, group, and other) on the new who.cgi file. Next, test the executable file at the shell prompt first: $ cd public_html $ ./who.cgi | more Verify that the first line of output is "Content-Type: text/plain". The second line must be blank. The rest of the output should be the output of "who". Fix any errors you find. 4) Enter a URL in this form into Netscape: http://acadaix/~abcd0001 Replace abcd0001 with your acadaix userid. If you get a permission denied error, add "x" permissions to your HOME directory to allow others to search it. Make sure the public_html directory has both "r" *and* "x" permissions for everyone. You should now see the name of your who.cgi script. Click on it to make the script execute. If you get a permission denied error, fix the permissions to allow others to read and execute the file. 5) Modify the end of the script to include any other Unix commands you might know, e.g. date, echo, ls, write, etc. Test the script from the shell prompt first, then finally try it from Netscape. )---------- test3-unix.txt -- Tue 2002-Feb-12 22:17:25 ===================================== What to Study for the Third Unix Test ===================================== -IAN! idallen@ncf.ca The Third Unix Test takes place in the last part of your Lab period in Week 8 of the course. (See the Course Outline.) The test is 90 minutes long. It is worth 10% of your term mark. The test covers Chapters 1-5, 7, 8, and 10 in the Linux text, with emphasis on Chapters 7, 8, and 10. (Omit Chapters 6 and 9.) The test also covers web page "Notes" files for Weeks 1-6 and the Chapter notes files and Floppix labs mentioned in those weekly files. You need to know how to write the Script Header given in the Week 7-8 notes. (Don't worry about setting exit codes for this test.) What you must bring to the test: 1. You will need your Floppix diskettes (and your backup copies). You must know how to boot Floppix with DHCP networking enabled. 2. You will need a pencil (only a pencil) to write in answers to multiple-choice questions on a mark-sense sheet. You must use pencil. 3. Your name, EMail, and web password must be registered with me at least 24 hours before the test begins, using the registration form on the course web page. If you are not registered, you won't have a userid on the Linux test machine and won't be able to log in to that machine to answer questions. You will work on Floppix, on ACADAIX, and on a Linux test machine. You must know how to use telnet on Floppix to telnet to ACADAIX and to the Linux test machine. No Windows access is permitted during the test. The address of the Linux test machine has been given out in the announcements news group. Make sure you can login to the machine. Knowing how to switch among the three Floppix consoles will save you time, since you can have one console logged into ACADAIX and another logged in to the Linux test machine at the same time. The test is open-book, open-note, and open-terminal. You may use any online or written resources available to you; but, you may not consult with or communicate with anyone except the instructor in any way during the test period. Multiple-choice answers will be written down on mark sense forms. Some questions will require you to execute commands on any of the three machines and save the results into files. You may be asked to download and upload files using FTP between any of the three machines. You will not have enough time to try every multiple-choice question on the computer or to look up all the terms in the textbook. You must know your material before the test. The test will appear to be "too long" if you try to type in all the questions to see the output. Thinking about the correct answer will be faster. Most of the multiple-choice questions can be answered quickly, without a computer, from the knowledge you have acquired in lectures and by doing the weekly readings and homework material. Your Floppix lab Exercises and Questions & Answers are very important. Strategy: Answer the questions you know, first. Come back later to the questions for which you require a computer. For more strategy on writing Unix tests see: test_preparation.txt )---------- test_preparation.txt -- Tue 2002-Feb-12 22:22:47 ========================================= Advice in Preparing for and Writing Tests ========================================= -IAN! idallen@ncf.ca How to succeed on tests in this course: Do the text and note examples: Type in the examples and do them to develop "muscle memory" about what the commands do. Don't just read the book and notes! If you never type in the "finger" command, you won't realize that you mis-spell it until the test. If you never use the "man" command, you'll get stuck at the "(END)" prompt in the test, and then it's too late. Type in the sample commands given in the book as you review the posted chapter review questions. Make sure your output is what you expect it to be. Do the Labs: Do the Floppix labs, and record your answers. Don't just read the labs. Some questions come from the labs, and you don't have time to do the lab online for the first time during the test. Make sure your output is what you expect it to be. Asking questions about how something works during the test is too late. Watch your prompts: Look at your screen and the prompt to know what program is expecting command input from you. If you haven't learned to type without looking at the keyboard, this advice is critically important. (Many students look at the keyboard instead of the screen when they type - they don't notice that the prompt has changed and that they are no longer typing input into the program they were expecting.) This is not a shell prompt: ftp> date ?Invalid command Invalid command: You can't type Unix commands into the FTP program. This is not an FTP prompt: $ put penguin put: Command not found Command not found: You can't type FTP commands into the Unix shell. This is not a login prompt for a telnet session: $ abcd0000 abcd0000: Command not found Command not found: You can't type a login userid (or your password or PIN number) into the shell. Watch your prompts! Prepare your computer: You will have to write your test while logged in to several computers at once (Floppix, ACADAIX, and the Linux test machine). You will save time if you know how to switch among the several Floppix consoles. Under Linux (including Floppix), you can see previous screens of text that have scrolled up on a console by holding down the SHIFT key and using the PageUp and PageDown keys. Remember - Linux runs the BASH shell (and you can use BASH on ACADAIX too). Don't waste time retyping your commands when you can use the arrow keys to scroll back up and edit them or re-execute them. Use of Windows is not permitted during tests. (If you use Windows telnet at home, you run the risk of deleting your entire file if you use Windows TELNET without reading my "Using Telnet" web page first.) ! What's the intended contrast between "Windows telnet" and "Windows TELNET"? Answer what you know first: You will not have enough time to try every question on the computer or to look up all the terms in the textbook. Answer what you know first. Most of the questions can be answered quickly from the knowledge you have acquired in lectures and by doing the weekly readings and homework material. Give the answers that you know first. Come back later to the questions that require computer work or fishing in documentation. Do not attempt to try every question on the computer! Think first! - Some questions, when typed in, may destroy your files or your shell. This can be a catastrophic loss during a test. Think first! - You will run out of time, especially if you are not yet skilled at typing, if you try to type in all the questions to see what the output might be. Answer the questions based on your understanding of the theory. You can come back later to "try" some questions on the computer, if they seem safe. Think! Think before you type something into the computer - you could wipe out your files or render your session unusable. If you don't know what it means, don't do it! If you do damage your session, you will have to either log out (if that is possible); or, you will have to reboot Floppix and start over. Prepare for problems: If you have ever sent a binary file to your terminal and turned it into graphics mode, you know how important it is to have handy the ways of resetting your terminal given at the end of the Week 1 notes. )---------- practiceCh7-10.txt -- Tue 2002-Feb-12 23:42:00 --------------------------------------------- Practice Unix/Linux Questions - Chapters 7-10 --------------------------------------------- -IAN! idallen@ncf.ca All scripts written must begin with a proper script header, including the correct interpreter, comments, and a proper PATH and umask. All scripts must work on both Linux (Linux Test Machine) and on AIX Unix. (The same script must work in both places without modification.) All prompts for user input must appear on the terminal, even if standard output is redirected. (Do not send prompts for input into the output file!) Test this! Where possible, commands should open their own files. (Do not use "cat" needlessly - read more on this point in the week4.txt notes.) Never let the shell expand glob patterns (wildcards) unexpectedly. Test this! Don't assume you can write temporary files into the current directory. Test this by changing directories to a directory into which you cannot write (or, create such a directory) and running your script from there. Pay close attention to the names and locations of output files. Real scripts must check the error returns on the commands they use. You may disregard this requirement until after we discuss Chapter 11, at which point you must always check for errors. Not all of these scripts are equally difficult to write. Find the easy ones first! TimeSaver: Use filename completion in the shell to complete the names of these scripts when you want to test them or edit them. Don't keep typing the full path names of the scripts! Let your Shell do your typing for you! (0) Type in all the scripts in the parts of Chapter 10 that we study (starting with "whoson"). Add the correct script headers to the scripts. Test them to make sure that they work. (1) Write this executable script named "1_linux_finger.sh": Run "finger" to the Linux Test Machine (use the IP address) to show all logins, and delete from the display the logins that are by idallen. (The output must not contain idallen logins.) (2) Write this executable script named "2_aix_rusers.sh": Run the "rusers" command with a single argument that is the AIX Unix machine name (use the full domain name). Translate blanks to newlines. Delete the first line. Sort the remaining lines. Hints: - use "tail -n +2" (see the man page) - see the example "tr" commands in week4.txt notes The rusers command is probably not available on Floppix. (3) Write this executable script named "3_rhostmaker.sh": Create the file ".rhosts" in the HOME directory of the person running the script. (Use a variable for this directory name.) The file must contain the host name of the machine on which the script is running. Make the permissions on the file exactly this: read for the owner of the file, nothing for anyone else. The script must work correctly no matter in which directory it is executed, no matter on which machine it is run. It must create the .rhosts file in the HOME directory of the person running the script; it must put the current machine name in the file. Hints: - the Unix "hostname" command will output the current host name - command substitution (Ch.10) will let you capture command output - one of the keyword variables (Ch.10) may be useful (4) Write this executable script named "4_pinger.sh": Prompt to read a host name from the script user and then read the host name. Send only 4 ping packets (a count of 4) to the host name supplied by the user. Display the exit status (return code) of the ping command. Have the script exit with that return status. Hints: - "man ping" in the Linux man pages (the AIX man page for ping is displayed far below the man page for "ntx_ping") - Chapter 10 Special Parameters (exit status) - on AIX, ping is not in /bin or /usr/bin; find where it is located and set the script PATH accordingly (5) Write this executable script named "5_nameservers.sh": Extract all lines containing the word "nameserver" from the system file /etc/resolv.conf and display them with line numbers. Hints: - "man cat" on Linux or AIX (6) Write this executable script named "6_secure_directories.sh": Remove all group and other permissions on the current directory and on all subdirectories of the current directory that don't begin with periods (the non-hidden directories). Do not change any files. Hints: - echo */. (in the notes for Chapter 4 and Chapter 5) Note: This is tricky to do correctly if the current directory has *no* non-hidden subdirectories. You need control structures from Chapter 11 to handle that case. To keep this script simple, you may assume that there will always be at least one non-hidden subdirectory. (7) Write this executable script named "7_backgrounder.sh": Prompt to read a command name from the script user, and then read the command name. Run the command that the user enters in the background. Hints: - "the read builtin" (Ch10) REMEMBER: Always test your scripts with input that contain blanks and other special characters. (8) Write this executable script named "8_standard_error.sh": Prompt to read a line from the script user and then read the line. Display one copy of the line on standard output. Display a second copy of the line on standard error with all the characters changed to UPPER CASE. (This last item will require some thought - you can't blindly copy anything you've seen to date.) Test your script as follows: $ ./8_standard_error.sh >out If you don't see a prompt, fix your script so that the prompt does not get redirected into the file "out". The prompt should be visible, even with standard output redirected into a file. (Advice: Prompts for input should always be visible, even if output is redirected. You never want the prompt to be redirected into the output file along with the script output!) Hints: - last page of "Redirecting Standard Error" (Ch10) - in a command pipeline, which command is the one that needs to have its standard output sent to standard error? (9) Write this executable script named "9_show_processes.sh": Prompt to read a text string from the script user and then read the text string. Look for that text string in the output of the "ps" command. Use both the "f" and "e" options to the "ps" command, so that "ps" outputs *all* the processes on the machine in a nice format. Your output will be only the lines from "ps" that contain the text string. Bonus: Fix the script so that it works even if the user enters a text string that starts with a dash. (Hint: the man page.) 10) Try all the script examples that use different command interpreters (e.g. #!/bin/cat) given in the chapter10.txt file. Also try these command interpreter lines and see what they do: #!/bin/wc #!/bin/cat -n #!/bin/sort -r #!/bin/grep # Reminder: These first lines are interpreted by the Unix kernel only if the script is executable and you try to execute it. The lines are comment lines and are ignored by all Unix shells that read the file. 11) Write this executable script named "91_environment_variables.sh": Produce an output line similar to the following: The value of $HOME is: /thome/alleni for each of the shell variables: HOME, MAIL, PATH, and SHELL. The script will generate four lines of output, showing the name and current values of each of the four shell variables. Test the script by setting your MAIL variable to be an asterisk, exporting MAIL, then running your script. Your script must show the correct value - an asterisk. (Never let shell glob patterns expand unintentionally.) 12) Write this executable script named "92_UPPER_translator.sh": Prompt to read a line from the script user and then read the line. The script user should respond by entering a line that contains two file names, separated by blanks. The script will place each file name into its own shell script variable. Translate all the characters from the first file to UPPER-CASE and put the result into the second file. (Translate the file contents, not the file names!) Your script doesn't have to handle the errors that might result if the user doesn't supply two names, or if s/he supplies more than two names, or if the input file doesn't exist, or if the output name is not writable. (You'll deal with all that in Chapter 11.) 13) Write this executable script named "93_host_finder.sh": Look for the name of the current host in the file /etc/hosts and print only the lines containing the current host name. The script must work on any machine and print the lines from /etc/hosts that match the current machine name. Hints: - the Unix "hostname" command will output the current host name - command substitution (Ch.10) will let you capture command output 14) Write this executable script named "94_make_email.sh": Generate and output the full username@domain.name email address of the person who runs this script on this machine. The script must work for any user and must work on any Unix machine. (Test it on Floppix, AIX Unix, and the Linux Test machine.) Hints: - see the examples in the Chapter 10 notes related to capturing command output using Command Substitution - the "whoami" command outputs your user name to stdout - the file /etc/resolv.conf contains a line with the domain name - "awk '{print $2}'" will extract the 2nd field of its input, e.g. $ echo one two three | awk '{print $2}' two (the single quotes are required to protect the $2 field number) - command substitution (Ch.10) will let you capture command output Advanced Bonus: Some versions of /etc/resolv.conf have a "search" line instead of a "domain" line. Modify the script to find *either* line ("search" or "domain") and extract the second field (the domain name). (Hint: Look up the "-e" option. What if you used two "-e" options?) Double Advanced Bonus: Some versions of /etc/resolv.conf contain *both* a "search" *and* a "domain" line. Modify the script to work even if both lines are present. (Hint: You only want the first line you find, not both lines. What command will do that for you?) 15) Write this executable script named "95_PATH_lister.sh": Display all the directories in your PATH variable, one per line, line numbered. Hints: - translate colons into newlines see the example "tr" commands in week4.txt notes - "man cat" on Linux or AIX 16) Write this executable script named "96_script_history.sh": Look for and display lines containing the current name of this shell script that is being run that are contained in the file ".bash_history" in the HOME directory of the user running the script. The script must work no matter what the name of the shell script is. (Do not hard-code the name into the script! Use the correct shell positional parameter variable.) The HOME directory must be the HOME directory of the user running the script. (Use a variable!) If you test this on ACADAIX, the default shell is the Korn shell and it uses a file named ".sh_history" instead. You can link that file name to the name ".bash_history" for testing. (Or, you can simply type "bash" and bash will create its own history file for you to examine.) Hints: - one of the keyword variables (Ch.10) may be useful - one of the positional parameter variables (Ch.10) will be needed 17) Write this executable script named "97_backwards_args.sh": Display the number of arguments found on the command line. Next, display three of the arguments from the command line, one per line, in reverse order (last to first). The script doesn't need to detect or handle more or less than three arguments. Assume that the users of the script will always type exactly three and don't worry about errors that might occur if they do not. Warning: Make sure that shell glob characters inside the arguments do not expand when you display the arguments! 18) Write this executable script named "98_sort_diff.sh": Display on the screen the strings that are the first and second command line arguments to the script. Take the first argument as a file name and sort it into a temporary file. Take the second argument as a file name and sort it into a different temporary file. (Put the temporary files under the /tmp directory.) Use the "diff" command to show the differences (if any) between the two sorted files. Remove the temporary files. The script should exit with the same return code as the "diff" command, whatever that is. The script doesn't need to detect or handle more or less than two command line arguments. Assume that the users of the script will always type exactly two and don't worry about errors that might occur if they do not. Don't worry about errors that might occur because of invalid, unreadable, or missing file names. (You'll learn how to do that in Chapter 11.) Warning: Make sure that shell glob characters inside the arguments do not expand when you process the arguments! 19) Write this executable script named "99_pid_finder.sh": Look for the process ID number of the currently running script in the output of the "ps" command with the "f" option. Display the matching lines along with line numbers. Hints: - "man cat" on Linux or AIX Bonus: Find a way to display the first title/header line of "ps" before you display the matching lines. (Hint: - you might want to run "ps" twice in the same script, using the first run to obtain just the title line.) 20) Write this executable script named "9a_who_from.sh": Extract all the "From:" header lines from your mailbox and display them with line numbers. Bonus: Make sure the word "From:" is at the *start* of the line. Hints: - "man cat" on Linux or AIX - use a shell variable to locate the file that is your mailbox (Ch.10) - Floppix labs dealing with special characters in patterns =================== Command syntax used =================== You probably need to use most of the variables, syntax, and commands listed below to write the above scripts; though, your scripts won't necessarily use them in the way that they are shown in the list below: $input & cat "$HOME/.bash_history" cat "$HOME/.rhosts" cat -b file who | cat -n grep nameserver /etc/resolv.conf | head -1 date >/tmp/temp1$$ ; who >/tmp/temp2$$ ; cat /tmp/temp1$$ /tmp/temp2$$ echo "Some shell variables:" "$#" "$$" "$?" echo "$(echo one two three | tr ' ' '\n' | sort | head -1)" echo "$(grep 'pattern' file)" echo "$(hostname)" echo "$(whoami)" echo "$0" "$1" "$2" "$3" echo "HOME $HOME" "MAIL $MAIL" "PATH $PATH" "SHELL $SHELL" echo */. echo 1>&2 "some error message or prompt" month=$(date | awk '{print $2}') ; echo "Month is $month" diff one two ; status=$? ; exit "$status" finger @205.211.32.96 grep -e 'pattern1' -e 'pattern2' grep -e "$input" grep -v grep '^pattern' head -1 ls /usr/sbin ping -c ps -ef read var1 var2 ; echo "var1 is $var1 and var2 is $var2" tr ' ' '\n' tr ':' '\n' tr 'a-z' 'A-Z' whereis ping )---------- test2-answers-A.txt -- Wed 2002-Feb-13 22:43:36 =================================== DAT2330 Test Two A - Unix - Answers =================================== -IAN! idallen@ncf.ca Problem A - Marks: 3 On the Test Machine: Put the five-word sentence $1 isn't "easy money". into a new output file named easy_approx.txt in your HOME directory. Copy the sentence and all the punctuation exactly! Also append to this same output file a copy of the last line of the file /home/ian/dat2330/boat. The output file will contain exactly 212 characters. A - Chapter 2, Chapter 5, quotes.txt echo '$1 isn'"'"'t "easy money".' >easy_approx.txt -OR- echo \$1 \ isn\'t \"easy money\". >easy_approx.txt -OR- cat >easy_approx.txt $1 isn't "easy money". ^D -OR- ...use the vi text editor... tail -1 /home/ian/dat2330/boat >>easy_approx.txt Problem B - Marks: 3 On the Test Machine: Create a sub-directory named .copyme in your HOME directory that has exactly the same permissions as the existing directory /home/ian/dat2330/.copyme. B - Chapter 4 ls -ld /home/ian/dat2330/.copyme -OR- ls -la /home/ian/dat2330 mkdir .copyme chmod u=rx,g=x,o=r .copyme -OR- chmod 514 .copyme Problem C - Marks: 3 On the Test Machine: Put a translated copy of the file /home/ian/dat2330/untranslated.txt into the file named linux_translated.txt in your HOME directory. The translation should change the all the letters except z to their upper-case equivalents. C - Chapter 5 tr a-y A-Y linux_translated.txt Problem D - Marks: 3 On the Test Machine: From the file /home/ian/dat2330/boat extract the first 100 lines of the file into a file named top and the last 90 lines of the file into a file named bottom. Count the number of lines that contain the digit 1 (one) in the two new files. Which file contains the most matches? Place only those (most) matching lines into the file named linux_boat_most.txt in your HOME directory. D - Chapter 3, Floppix Labs head -100 /home/ian/dat2330/boat >top tail -90 /home/ian/dat2330/boat >bottom grep 1 top | wc -OR- grep -c 1 top grep 1 bottom | wc -OR- grep -c 1 bottom grep 1 bottom >linux_boat_most.txt Problem E - Marks: 3 On the Test Machine: Create a reverse-sorted list of all the names (just the names, also including any hidden names) contained in the directory /home/ian/dat2330/bigdir. Put the reverse-sorted output into the file named linux_bigdir_reverse_sorted.txt in your HOME directory. E - Chapter 3, Chapter 4 ls -a /home/ian/dat2330/bigdir | sort -r >linux_bigdir_reverse_sorted.txt -OR- ls -ar /home/ian/dat2330/bigdir >linux_bigdir_reverse_sorted.txt Problem F - Marks: 3 On the Test Machine: Show the full listing (permissions, owner, etc.) for all the non-hidden names (including files and directories) in the directory /home/ian/dat2330/bigdir that end in the letters n through z (inclusive). Put the output into the file named linux_bigdir_nz_list.txt in your HOME directory. F - Chapter 4, Chapter 5 ls -ld /home/ian/dat2330/bigdir/*[n-z] >linux_bigdir_nz_list.txt Problem G - Marks: 3 On the Test Machine: Create a directory named io under your HOME directory. Under the io directory, create two directories named phobos and mars. Under the phobos directory, create two files (any size) named jupiter and venus. Under the mars directory, create two files (any size) named metis and thebe; but, the name thebe should be a second name for the file named jupiter. G - Chapter 4 mkdir io io/phobos io/mars touch io/phobos/jupiter io/phobos/venus touch io/mars/metis ln io/phobos/jupiter io/mars/thebe Problem H - Marks: 3 On the Test Machine: there is a partially protected directory named /home/ian/dat2330/unix_test_two_DAT2330 containing five files with one-letter file names: a, b, c, d, and e. If any of the five file names are names for the same data, put only the full absolute pathnames that point to the same data into output file same_data.txt in your HOME directory. If all five names are names of different file data, put the word none into the output file. H - Chapter 4, Web page "File Nodes" ls -i /home/ian/dat2330/unix_test_two_DAT2330/a ls -i /home/ian/dat2330/unix_test_two_DAT2330/b ls -i /home/ian/dat2330/unix_test_two_DAT2330/c ls -i /home/ian/dat2330/unix_test_two_DAT2330/d ls -i /home/ian/dat2330/unix_test_two_DAT2330/e echo /home/ian/dat2330/unix_test_two_DAT2330/b \ /home/ian/dat2330/unix_test_two_DAT2330/c >same_data.txt Problem I - Marks: 3 From the Floppix file /etc/services select all lines that do not contain the character string udp and put those lines into the file floppix_service_no_udp.txt in your HOME directory on the Test Machine. I - Chapter 3, Floppix Labs grep -v udp /etc/services >floppix_service_no_udp.txt ftp 205.211.32.96 ftp> put floppix_service_no_udp.txt ftp> quit Problem J - Marks: 3 On the machine acadaix there is a readable compressed text file named /thome/alleni/dat2330/two/secone.gz containing a compressed text message. Extract a copy of the text from the file, read it, and do what the message says. J - Chapter 3 gzip -d tmp -OR- cp /thome/alleni/dat2330/two/secone.gz tmp.gz ; gzip -d tmp.gz more tmp mv tmp acadaix_onehide.txt ftp 205.211.32.96 ftp> put acadaix_onehide.txt ftp> quit )---------- test2-answers-B.txt -- Wed 2002-Feb-13 22:43:36 =================================== DAT2330 Test Two B - Unix - Answers =================================== -IAN! idallen@ncf.ca Problem A - Marks: 3 On the Test Machine: Put the five-word sentence $0 isn't "any money". into a new output file named zero_money.txt in your HOME directory. Copy the sentence and all the punctuation exactly! Also append to this same output file a copy of the last line of the file /home/ian/dat2330/train. The output file will contain exactly 209 characters. A - Chapter 2, Chapter 5, quotes.txt echo '$0 isn'"'"'t "any money".' >zero_money.txt -OR- echo \$0 \ isn\'t \"any money\".' >zero_money.txt -OR- cat >zero_money.txt $0 isn't "any money". ^D -OR- ...use the vi text editor... tail -1 /home/ian/dat2330/train >>zero_money.txt Problem B - Marks: 3 On the Test Machine: Create a sub-direc tory named .mycopy in your HOME directory that has exactly the same permissions as the existing directory /home/ian/dat2330/.mycopy. B - Chapter 4 ls -ld /home/ian/dat2330/.mycopy -OR- ls -la /home/ian/dat2330 mkdir .mycopy chmod u=r,g=rx,o=x .mycopy -OR- chmod 451 .mycopy Problem C - Marks: 3 On the Test Machine: Put a translated copy of the file /home/ian/dat2330/nottranslated.txt into the file named linux_translatedone.txt in your HOME directory. The translation should change the all the letters except a to their upper-case equivalents. C - Chapter 5 tr b-z B-Z linux_translatedone.txt Problem D - Marks: 3 On the Test Machine: From the file /home/ian/dat2330/train extract the first 80 lines of the file into a file named first and the last 110 lines of the file into a file named last. Count the number of lines that contain the digit 0 (zero) in the two new files. Which file contains the most matches? Place only those (most) matching lines into the file named linux_train_most.txt in your HOME directory. D - Chapter 3, Floppix Labs head -80 /home/ian/dat2330/train >first tail -110 /home/ian/dat2330/train >last grep 0 first | wc -OR- grep -c 0 first grep 0 last | wc -OR- grep -c 0 last grep 0 first >linux_train_most.txt Problem E - Marks: 3 On the Test Machine: Create a reverse-sorted list of all the names (just the names, also including any hidden names) contained in the directory /home/ian/dat2330/largedir. Put the reverse-sorted output into the file named linux_largedir_reverse_sorted.txt in your HOME directory. E - Chapter 3, Chapter 4 ls -a /home/ian/dat2330/largedir | sort -r >linux_largedir_reverse_sorted.txt -OR- ls -ar /home/ian/dat2330/largedir >linux_largedir_reverse_sorted.txt Problem F - Marks: 3 On the Test Machine: Show the full listing (permissions, owner, etc.) for all the non-hidden names (including files and directories) in the directory /home/ian/dat2330/largedir that end in the letters a through m (inclusive). Put the output into the file named linux_largedir_am_list.txt in your HOME directory. F - Chapter 4, Chapter 5 ls -l /home/ian/dat2330/largedir/*[a-m] >linux_largedir_am_list.txt Problem G - Marks: 3 On the Test Machine: Create a directory named thebe under your HOME directory. Under the thebe directory, create two directories named jupiter and venus. Under the jupiter directory, create two files (any size) named phobos and mars. Under the venus directory, create two files (any size) named metis and io; but, the name io should be a second name for the file named phobos. G - Chapter 4 mkdir thebe thebe/jupiter thebe/venus touch thebe/jupiter/phobos thebe/jupiter/mars touch thebe/venus/metis ln thebe/jupiter/phobos thebe/venus/io Problem H - Marks: 3 On the Test Machine: there is a partially protected directory named /home/ian/dat2330/unix_test_two_DAT2330 containing five files with one-digit file names: 1, 2, 3, 4, and 5. If any of the five file names are names for the same data, put only the full absolute pathnames that point to the same data into output file same_digit.txt in your HOME directory. If all five names are names of different file data, put the word none into the output file. H - Chapter 4, Web page "File Nodes" ls -i /home/ian/dat2330/unix_test_two_DAT2330/1 ls -i /home/ian/dat2330/unix_test_two_DAT2330/2 ls -i /home/ian/dat2330/unix_test_two_DAT2330/3 ls -i /home/ian/dat2330/unix_test_two_DAT2330/4 ls -i /home/ian/dat2330/unix_test_two_DAT2330/5 echo /home/ian/dat2330/unix_test_two_DAT2330/4 \ /home/ian/dat2330/unix_test_two_DAT2330/5 >same_digit.txt Problem I - Marks: 3 From the Floppix file /etc/services select all lines that do not contain the character string tcp and put those lines into the file floppix_service_no_tcp.txt in your HOME directory on the Test Machine. I - Chapter 3, Floppix Labs grep -v tcp /etc/services >floppix_service_no_tcp.txt ftp 205.211.32.96 ftp> put floppix_service_no_tcp.txt ftp> quit Problem J - Marks: 3 On the machine acadaix there is a readable compressed text file named /thome/alleni/dat2330/two/twosec.gz containing a compressed text message. Extract a copy of the text from the file, read it, and do what the message says. J - Chapter 3 gzip -d tmp -OR- cp /thome/alleni/dat2330/two/twosec.gz tmp.gz ; gzip -d tmp.gz more tmp mv tmp acadaix_twohide.txt ftp 205.211.32.96 ftp> put acadaix_twohide.txt ftp> quit )---------- badSpelling.txt -- Sun 2002-Feb-17 19:39:35 ====================================== Correcting Spelling Errors after Tests ====================================== -IAN! idallen@ncf.ca If you have made a spelling error naming some script or output file in the test, and you want partial credit, read this. For those of you who got one or more of the answer file names wrong (or who put the answer files in the wrong subdirectories), you can redeem yourselves partially by sending me by email a properly constructed shell script that makes links to the correct names in the correct directory. I will run the script you send me, in my copy of your HOME directory and then I will re-run the marking program. For part marks, send me a properly constructed repair shell script that will make a link from the incorrect name to the correct name in the correct directory: 1. I will run your repair script in my copy of your HOME directory. The script must be designed to run in your HOME directory. 2. The body of the repair script must contain relative pathnames only. No absolute pathnames are allowed. 3. It must not remove or copy the existing mis-named file. Only links are allowed. 4. The repair script must meet course standards for comments and header. See the week7.txt notes. This is your second and last chance to get the name correct, for part marks. If the repair script does not meet all of the above four criteria, the existing mark stands. For example, the body of your script (after a properly constructed template Shell Script Header as given in the week7.txt notes) might read like this: ln Linux_three_zeros.txt linux_three_zeroes.txt ln jupiter/io/easy.txt easy.txt ln -s copy .copy Use hard links for files with incorrect names. Use symbolic links ("-s") for an io/ or thebe/ directory with an incorrect name. Your script is only allowed to make additional links in your HOME directory; it is not allowed to move, copy, or alter file content. You also can't use the script to repair anything under your io/ or thebe/ subdirectory. Your emailed scripts must have a complete Shell Script Header. Use paths relative to your HOME directory only, since my saved location for your HOME directory isn't even on the same machine as the test machine. I will run the script in my copy of your HOME directory. Do not use any "cd" commands inside your script - only "ln" commands with relative pathnames (relative to your HOME directory) are allowed. To send the completed shell script, you may use the Unix "mail" command. The Unix email program "mail" takes an email address as its command line argument and (like most Unix programs) reads from standard input. You only get one chance to get the script correct. )---------- scriptChecklist10.txt -- Mon 2002-Feb-18 09:38:37 ================ Script Checklist (to Chapter 10) -IAN! idallen@ncf.ca ================ This is a list of things to verify in your shell scripts: - script must start with a complete script header (interpreter, syntax, purpose, author, date, PATH, umask) - issue prompts before reading input - prompt messages must appear on stderr so they don't get redirected - note: some menus might also be considered as prompts - quote every use of a shell variable, and don't miss any - quote every use, every time, to prevent wildcard expansion - I mean every use, every time - I mean it - you might get away with not quoting $$, $#, and $?, but that's all - I *really* do mean it - use good programming style - avoid lines longer than 80 characters (break and continue long lines using a backslash to escape the newline) - add useful, helpful comments (not useless ones) in front of code blocks (don't comment in front of every single line) - use good variable names, related to your algorithm (don't copy the poor names used by your instructor in examples) - don't repeat code (parametrize your algorithm) - test your scripts with problematic input - try wildcards, blanks, dashes, punctuation, empty strings, EOF, etc. - try no command line arguments and then hundreds of arguments - try short inputs, medium inputs, and huge long inputs - look for unquoted variables and put wildcards in them and watch your script blow up, then fix it by quoting all your variables - no input should cause a shell script to generate an internal error )---------- homeHOME.txt -- Fri 2002-Feb-22 13:45:03 ===================================== Directories: current, HOME, and /home ===================================== -IAN! idallen@ncf.ca Some people are still confused about: - "your HOME directory" - the directory given to you by the system administrator and into which you are placed when you first log in - this is the directory you go to when you type "cd" with no arguments - the name of your home directory is available in the shell keyword variable named "$HOME" (HOME is UPPER CASE) - on many Unix systems, your home directory is a directory under /home e.g. /home/abcd0001, /home/zyxw0002, etc.; but, this is not always true - /home - a pathname (probably a directory) under the root directory, named "home" - this is NOT your HOME directory; it is (usually) a directory named /home - (/home might be a file name, not a directory; but, this is rare) - some other names (directories) under root are /etc, /bin, and /usr - "current directory" - the directory that your shell (or any Unix process) is currently in - relative pathnames are relative to this current directory - the name of this directory is always "." (dot) - the current directory of a shell can be changed by using the "cd" command - every process has its own current directory; changing current directory in one process does not change it for other processes - the current directory is passed on to child processes when they are first started (e.g. to shell scripts that you invoke) - changing directories inside a shell script will not affect the parent process that called the shell script (different processes) Problem: "Put the date into a file out.txt in the current directory." Solution: date >out.txt Problem: "Put the date into a file out.txt in your HOME directory." Solution: date >"$HOME"/out.txt Problem: "List the names (including hidden names) in the /home directory." Solution: ls -a /home )---------- openSource.txt -- Wed 2002-Feb-27 13:22:47 ---------------------------------------- E-Week Editorial - Software Independence ---------------------------------------- July 2, 2001 12:00 AM ET Revolutions happen when people make a big deal of little things. When they do, they are willing to risk much and endure much. It has happened many times in human history, and it's commemorated in the holiday, July 4, we [Americans] celebrate the week this issue of eWeek is published. It has been well-documented that Great Britain's tea and stamp taxes that were so galling to the American colonists represented a tiny portion of their per-capita income-far smaller than today's tax burden. No, the colonists weren't overtaxed; it was their having no say in the taxes that they found unacceptable. It was a detail, really, but one ultimately worth fighting and possibly dying for to correct. We don't want to trivialize our nation's history; however, we do see some parallels in today's open-source software movement that harken back to revolutionary events. Open-source software, although viewed by some as risky and uncertain, does empower users in ways that proprietary software does not. Sure, many needs can be satisfied by good-quality, reasonably priced proprietary software. But the absence of the right to access and change source code and then redistribute those changes is one of those things that some people are willing to fight and make sacrifices for. The ability to fix bugs or customize source code is tremendously valuable for some businesses. In the case of Google, the vaunted search engine would scarcely be able to exist as a business if it were burdened by conventional software licensing costs and its engineers were unable to take apart and optimize Linux operating system code. Although Bill Gates and Microsoft are now arrayed against the open-source movement, some recent spin control notwithstanding, it was only a little over a decade ago that a bespectacled nerd from the Northwest promised a very low-cost platform as a de facto standard, welcoming any and all developers to take advantage of the widely published APIs. IBM offered a better-engineered product in its OS/2 operating system, but Microsoft's Windows gave users more flexibility in choosing their own timing for moving forward from DOS to a next-generation platform. Users chose Windows because it gave them more freedom. Now, it's open source that offers the dual promise of lower cost and greater empowerment. Sure, there are many flavors of open-source licenses, and they must be read carefully-but so must conventional licenses. The open-source process may seem chaotic to those who aren't used to it, but so did democracy to those used to the rule of monarchs. http://www.zdnet.com/eweek/stories/general/0,11011,2780122,00.html )---------- echoCommands.txt -- Wed 2002-Feb-27 13:24:10 ================================== No standard for the "echo" command ================================== -IAN! idallen@ncf.ca In the beginning, the Unix "echo" command had no way of suppressing the newline that it always added after outputting its arguments. The Berkeley people added a "-n" option to suppress the newline; the AT&T people chose to use an embedded "\c" to do the same thing. (To make money, you must distinguish your product from your competitors.) Most shells contain a built-in "echo" command, and depending on the origin of the shell, it either honours the "-n" or the "\c". The C Shell (including tcsh) always uses "-n". The Bourne shell derivatives usually use "\c", except for "bash", which uses "-n" by default; but, can be told to use "\c" using the non-standard "-e" option to "echo". Depending on from where you get your version of Unix, and which shell you use, you will have to use either "-n" or "\c" to suppress a newline. Unfortunately, if you want to write a portable shell script that will work on any machine, you can't use either of these. One way around the problem is to use Unix commands that always work on all versions of Unix. Here's one such work-around to suppress the newline: echo "This has no newline:" | tr -d '\n' I believe this works in all popular shells and on all versions of Unix you're likely to encounter, at least until Microsoft adopts their own version of Unix and "embraces and extends" the interface to include a third incompatible way of doing things. If the text message is intended to appear on standard error (not standard output), such as you would intend for a prompt, you must redirect the output of the translate, not the output of the echo: Correct redirection (message appears on standard error): echo "This has no newline:" | tr 1>&2 -d '\n' Incorrect (message appears on standard error but is NOT translated): echo 1>&2 "This is wrong; it still has the newline:" | tr -d '\n' )---------- chapter11-revised.txt -- Wed 2002-Feb-27 14:03:14 -------------------------------- DAT2330 - Unix - Prof. Ian Allen -------------------------------- REVISED (February 2002) notes on part of Chapter 11. Here are questions you should be able to answer. You will likely find questions similar to these on tests and exams. Chapter 11 - Shell Programming (not all - excerpts) General Notes: For much of the material, the Korn Shell, Bourne Shell, and Bash Shell behave identically. (The C Shell and TCSH are different.) You should fix all the Linux text example shell scripts to conform to the rules for shell scripts. In particular, you should add the missing interpreter line ("#!/bin/sh -u") and set PATH and umask. Error messages should appear on Standard Error (1>&2), and should include the name of the script ($0). Scripts should be run using "./scriptname"; because, you should never rely on "." being in your search PATH. Also, a script's name might match an existing Unix command name, and the Unix command might execute instead of your script. Skip over most of the "Optional" sections in this chapter. Concentrate on the text sections related to material covered in lectures, weekly notes, and in these questions. We are concentrating on basic IF/ELSE and WHILE/UNTIL this term. That means you can skip over these sections of Chapter 11: - "if...then...elif" "for..in" "for" p.374-381 - from "break and continue" to end of chapter p.386-405 Skip over the above sections. - The text puts the "then" keyword of an "if" statement on a separate line. Many people prefer to put it on the same line as the "if": if test "$word1" = "$word2" ; then echo "$0: '$word1' matches '$word2'" else echo "$0: '$word1' does not match '$word2'" fi The semicolon is necessary to end the "test-command". (See the Linux text, bottom of p.373.) - True or False: The "test" command that often immediately follows the word "if" in an "if" statement is a shell built-in command in bash. - What is the usual output (on STDOUT or your screen) of the "test" command? (Try typing some test commands and see what you get.) - True or False: The "test-command" that immediately followes the word "if" in an "if" statement must *always* be a shell built-in command. - True or False: The "test-command" in an "if" statement cannot be a pipe line or contain any redirection. - Look under "Shell variables" in the index for the explanation of the "$#" and "$?" variables. What do these variables contain? - True or False: The "test" command prints "1" or "0" on STDOUT depending on whether the test condition is FALSE or TRUE. - You should double-quote all shell variables to prevent shell glob characters from expanding. The $$, $#, and $? variables are exceptions - they don't really ever need to be quoted. Why? - Why should a script first check to see if a user has supplied all the required arguments (positional parameters) to the script? - What is the difference in return status (exit status) between the commands: $ test 0 = 00 $ test 0 -eq 00 $ test 1 = 01 $ test 1 -eq 01 What shell variable contains the exit status of the previous command? How can you see the contents of this (or any) shell variable? - What happens if you compare strings using -eq in the "test" command? $ test abc -eq def - What is wrong with the following IF statement when used with bash V2? bash# echo "$BASH_VERSION" 2.04.12(1)-release bash$ x=" 123 " bash$ y=" 456 " bash$ if [ "$x" -lt "$y" ]; then echo TRUE ; fi [: 123 : integer expression expected Why is the test command complaining about the value in "$x"? (Not all versions of the test command are this fussy about blanks.) - What happens if you compare numbers using '=' in the "test" command? $ if test 100 = 0100 ; then echo TRUE ; fi - What happens if you give too many arguments to the "test" command? $ test abc def = ghi $ x=* $ test "$x" = "*" $ test $x = "*" This is why you must remember to double-quote your variables! - Which of these commands tests to see if name $x is a directory? $ test -directory "$x" $ test "$x" -d $ test "$x" = "-d" $ test -d "$x" $ test "$x" = "d" $ test "d" -eq "$x" $ test -dir = "$x" How do you test to see if name $x is a file? How do you test to see if name $x is not empty? Are the quotes around $x necessary? - What is wrong with this statement? (This is a frequent student error.) if -w "$x" ; then echo "$0: '$x' is writable" fi Hint: The IF must always be followed by Unix command line to execute. What command name is being executed in the "if" statement? - True or False: The command "test" and the command "[" have identical functionality. They are different names for the same thing. - Note the syntax difference between these two identical-in-function versions of the "test" command: if test A = B ; then ... fi if [ A = B ]; then ... fi The behaviour of the above commands is identical; the second one just looks a bit neater because of the brackets. (If you use the bracket form, you must have the closing bracket. If you use the command-name "test" form, you must NOT have any brackets.) REMEMBER: The brackets need to be seen as individual, separate arguments. Use blanks around them to make them into separate tokens. "if testA = B" is as wrong as "if [A = B]". <== WRONG WRONG WRONG Neither one works because of the missing blanks. (There are no Unix commands with names "testA" or "[A". A valid Unix command name must always follow the "if" keyword.) (Why don't you need blanks around semicolons and pipes?) - Which one of these command lines are not valid Unix command lines? $ [ a = b ] || echo $? $ [ 2 -lt 9 ] || echo $? $ [ -d . ] || echo $? $ [ -f /etc/passwd ] || echo $? $ [ -s /etc/motd ] || echo $? (Hint: Convert the commands from "square bracket" to Unix "test" command syntax.) - What is the output of these command lines? Why? $ [ -f /etc/passwd ] || echo hello $ test -f /etc/passwd || echo hello $ [ -d .. ] || echo hello $ test -d .. || echo hello $ [ -d /etc/passwd ] || echo hello $ test -d /etc/passwd || echo hello $ [ -f .. ] || echo hello $ test -f .. || echo hello - In many shells, this command doesn't output "hi": $ test 9999999999999999999 -gt 999999999999999999 && echo hi (The first number contains 19 digits; the second 18.) Why might this command fail, even though the first number is truly greater than the second number? - True or False: To keep the scripts short, example scripts in the text leave out code that verifies arguments. You must not forget this verification code in your own scripts. - True or False: Unix commands on ACADAIX also use the --help convention to get help on a command's usage, e.g. "ls --help". - The "shift" command (p.374) is mentioned in Chapter 10. Scripts in this course won't need it; but, you can use it if you find it useful. - Many people put the "do" keyword on the same line as the "while" keyword: n=0 while [ "$n" -lt 10 ]; do echo "$0: Number variable n is '$n'" let n=n+1 done The semicolon is necessary to end the "test-command". As with the IF statement, this command can be any Unix command. The return status is all that matters. - Think of the "while" statement as an "if" statement that loops. - True or False: The "test-command" that immediately followes the word "while" in a "while" statement must *always* be a shell built-in command. - True or False: The following loops have identical function: while who | grep alleni >/dev/null ; do echo "alleni is still logged in" sleep 60 done until ! who | grep alleni >/dev/null ; do echo "alleni is still logged in" sleep 60 done (See the explanation of the "!" operator in the weekly notes.) ---------------- The FIND command ---------------- The "find" utility is partially explained on p.378. The examples under "find" in the back of the book are also helpful. Note this (from the man page, and from the first sentence under "Critera" in the back of the Linux text): ! Is "Criteria" misspelled in whichever text you refer to? Numeric arguments can be specified as +n for greater than n, -n for less than n, n for exactly n. Thus: "-size +5" means "size greater than 5 blocks". Thus: "-mtime 5" means "last modified time of exactly 5 days ago". Thus: "-group -5" means "group number less than 5". Concentrate on knowing how to build simple "find" commands (no parentheses or "-or" options), e.g. $ find /bin -mtime -300 -print $ find /bin -mtime 300 -print $ find /bin -mtime +300 -print $ find "$HOME" -size -2 -print $ find "$HOME" -size 2 -print $ find "$HOME" -size +2 -print $ find /tmp /usr/tmp "$HOME/tmp" -name '.bash*' -print $ find /tmp /usr/tmp "$HOME/tmp" -name '*dada*' -print $ find /tmp /usr/tmp "$HOME/tmp" -name '*.c' -print Warning: The AIX version of "find" has a more restricted "-size" option syntax and far fewer options than GNU find. The above examples will work on both systems (and on most other Unix systems since about 1980). - Chapter 11 Chapter Review questions: 1,2,6,9a,9b )---------- week7-9.txt -- Wed 2002-Feb-27 14:10:00 Week 7-9 Notes for DAT2330 - Ian Allen Supplement to Text, outlining the material done in class and lab. Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM![*] ([*] Read The Fine Manual) Look under the "Notes" button on the course web page for these study notes for the Linux textbook: chapter10.txt (The Bourne Again Shell [not all]) chapter11-revised.txt (Shell Script Programming [excerpts]) chapter14.txt (Programming Tools [excerpts]) Complete these Floppix Labs on http://floppix.ccai.com/ Floppix Lab 26: BASH Scripts Note: An early version of "bash" is also available on ACADAIX. In this file: * Shell Script Programming * Shell Script Header * Commenting Shell Programs * Code Commenting Style * Testing Return Codes of Commands * Complementing Return Status * Writing and Debugging Shell Scripts * Naming Shell Scripts ======================== Shell Script Programming ======================== You are computer programmers. When writing programs (scripts are programs), you are not simply trying to "get it to work"; but, you are also (and most importantly) practicing and demonstrating good programming techniques. Employers tell us that programming techniques are more important than the question of which languages you know. Good programming techniques include such things as: 1. Correct documentation (using guidelines given here) 2. Writing clean, simple, robust code: "Less code is better code"! 3. Using correct indentation, style, and mnemonic names 4. Using correct modularization and scope (i.e. local vs. global variables) 5. Correct use of arguments and/or prompting for user input 6. Correct use of exit codes on success/failure 7. Choosing the correct control structures 8. Presenting accurate and useful help if user makes an error 9. Checking the return status of important commands used in the script These criteria will be assessed in marking your programs (shell scripts). =================== Shell Script Header =================== All Shell Scripts in this course must contain a set of comment lines similar to the following (my explanatory remarks are on the right): #!/bin/sh -u <== specify interpreter # $0 [ args... ] <== show arguments and syntax # This script echoes its arguments. <== give purpose of script # Ian Allen - idallen@ncf.ca <== author and email # Mon Jun 11 17:06:04 EDT 2001 <== date (optional) export PATH=/bin:/usr/bin <== set and export correct PATH umask 022 <== correct file creation mask echo "$@" <== your script goes here status=$? <== your script goes here exit $status <== set exit status on exit Don't put the name of the shell script inside the shell script, since the name will be wrong if you rename the script. Use $0 instead. You will need to set variable $status to the appropriate exit value for your script. (Zero means "okay", non-zero means "something went wrong".) Choose PATH to include the system directories that contain the commands your script needs. Directories /bin and /usr/bin are almost always necessary. System scripts may need /sbin and /usr/sbin. GUI programs will need the X11 directories. Choose appropriately. Choose the umask to permit or restrict access to files and directories created by the shell script. "022" is a customary value, allowing read and execute by group and others. "077" is used in high-security scripts; since, it blocks all permissions for group and others. You must test the return codes of important commands inside the script. Your script must exit non-zero if an important command inside the script fails. (Read more on how to do this, below.) Prompts and error messages should be sent to Standard Error, not to Standard Output (Linux text p.308). Variables must be quoted correctly to prevent unexpected special character expansion by the shell. If you prepare a template file containing the above script model, you can copy and use it to begin your scripts on tests and save time (and avoid errors and omissions). Do not include my remarks. ========================= Commenting Shell Programs ========================= Comments should add to a programmer's understanding of the code. They don't comment on the syntax or language mechanism used in the code; since, both these things are obvious to programmers who know the language. (Don't comment that which is obvious to anyone who knows the language.) ! Mostly true, but it's useful to comment novel idioms (including use of ! side effects) despite that a reader who knows the language fluently may ! pick up on your subtlety without having his attention drawn to it in a ! comment. A not-strictly-necessary comment will reassure a reader that ! you intend to do what he thinks you intend to do. Programmer comments deal with what the line of code means in the *algorithm* used, not with syntax or how the language *works*. Thus: Do not use comments that state things that relate to the syntax or language mechanism used and are obvious to a programmer, e.g. # THESE ARE OBVIOUS AND NOT HELPFUL COMMENTS: x=$# # set x to $# <== OBVIOUS; NOT HELPFUL date >x # put date in x <== OBVIOUS; NOT HELPFUL test "$a" = "$b" # see if $a equals $b <== OBVIOUS; NOT HELPFUL cp /dev/null x # copy /dev/null to x <== OBVIOUS; NOT HELPFUL Better, programmer-style comments: loop=$# # initialize loop index to max num arguments date >tmpdate # put account starting date in temp file test "$itm" = "$arg" # see if search list item matches command arg cp /dev/null tmpdate # reset account starting date to empty Do not copy "instructor-style" comments into your code. Instructor-style comments are put on lines of code by teachers to explain the language and syntax functions to people unfamiliar with programming (e.g. to students of the language). Instructor-style comments are "obvious" comments to anyone who knows how to program; they should never appear in your own programs (unless you become an instructor!). ===================== Code Commenting Style ===================== Comments should be grouped in blocks, ahead of blocks of related code to which the comments apply, e.g. # Set standard PATH and secure umask for accounting file output. # PATH=/bin:/usr/bin ; export PATH umask 077 # Verify that arguments exist and are non-empty. # NARGS=3 if test "$#" -ne $NARGS ; then echo 1>&2 "$0: Expecting $NARGS arguments; you gave: $#" exit 1 fi for arg do if ! test -s "$arg" -a -f "$arg" ; then echo 1>&2 "$0: arg '$arg' is missing, empty, or a directory" exit 1 fi done Do not alternate comments and single lines of code! This makes the code hard to read: # THIS IS A BAD EXAMPLE OF COMMENTS MIXED WITH CODE !! # THIS IS A BAD EXAMPLE OF COMMENTS MIXED WITH CODE !! # Set a standard PATH plus system admin directories PATH=/bin:/usr/bin:/sbin:/usr/sbin # export the PATH for other programs export PATH # Set secure umask for accounting file output. umask 077 # create empty lock file >lockfile || exit $? # attempt to create link to lock file ln lockfile lockfile.tmp || exit $? # copy password file in case of error cp -p /etc/passwd /tmp/savepasswd$$ || exit $? # remove guest account (can't quick-check return code on grep) grep -v '^guest:' /etc/passwd >lockfile.tmp # copy new file back to password file file system cp lockfile.tmp /etc/passwd.tmp || exit $? # fix the mode to be readable chmod 444 /etc/passwd.tmp || exit $? # use mv to do atomic update of passwd file mv /etc/passwd.tmp /etc/passwd || exit $? # remove lock file rm lockfile.tmp lockfile # THIS IS A BAD EXAMPLE OF COMMENTS MIXED WITH CODE !! # THIS IS A BAD EXAMPLE OF COMMENTS MIXED WITH CODE !! ! Heh, this is a point of taste. I (usually, I think) want comments as close ! as possible to the code. I have a habit of writing things as editor scripts ! and I detest being forced off my preferred scripting editor onto an editor ! that does not provide for in-line comments so I have to block-comment the ! whole script as a single chunk. The validity of your argument depends ! somewhat on the programming language and what the author is doing with it-- ! if several lines of code are inextricably needed to implement one concept, ! then I buy that a single block comment may be optimal. Where one line of ! code implements one concept (such as, set PATH; set umask) then I dispute ! your argument--the unrelated concepts and code lines should be commented ! (if at all) separately. It's misleading to imply by paragraphing and ! comment positioning that they're somehow related. Block comments and code are easier to read. Here is a block-comment version of the above code: # Set and export standard PATH plus system admin directories. # Secure umask protects files. # export PATH=/bin:/usr/bin:/sbin:/usr/sbin umask 077 # Create empty lock file and attempt to create link to lock file. # >lockfile || exit $? ln lockfile lockfile.tmp || exit $? # Copy password file in case of error. # cp -p /etc/passwd /tmp/savepasswd$$ || exit $? # Remove guest account. (Can't quick-check return code on grep.) # Copy new file back to password file file system. # Fix the mode to be readable. # Use mv to do atomic update of passwd file. # Remove the lock file. # grep -v '^guest:' /etc/passwd >lockfile.tmp cp lockfile.tmp /etc/passwd.tmp || exit $? chmod 444 /etc/passwd.tmp || exit $? mv /etc/passwd.tmp /etc/passwd || exit $? rm lockfile.tmp lockfile Why can't you exit the script if grep returns a non-zero status code? ================================ Testing Return Codes of Commands ================================ Just as you would never use a C Library function without checking its return code, you must never use commands in important shell scripts without at least a minimal checking of their return codes. At minimum, the shell script should exit non-zero if a command fails unexpectedly: grep -v '^guest:' /etc/passwd >lockfile.tmp cp lockfile.tmp /etc/passwd.tmp || exit $? chmod 444 /etc/passwd.tmp || exit $? mv /etc/passwd.tmp /etc/passwd || exit $? rm lockfile.tmp lockfile || exit $? The shell conditional execution syntax "||" is used here to test the return codes of the commands on the left and execute the command on the right if the command on the left returns a bad status (non-zero). Some commands naturally return a non-zero exit status even when they are doing what you expect (e.g. grep might not find what you were looking for - this might be okay), and cannot be tested using this simple method. Do not exit the script after a "grep", "diff", or "cmp" command retuns non-zero! More complex testing -------------------- Unfortunately, simply exiting non-zero doesn't tell the user of the script which script contained the command that failed: $ ./myscript1 & <== start in background $ ./myscript2 & <== start in background $ ./myscript3 & <== start in background [...time passes...] cp: /etc/passwd.tmp: No space left on device From which script did the error message come? We don't know! If a script is being run in the background along with several other scripts also containing similar commands, or if a script is being run by a system daemon or delayed execution scheduler (atd or crond), we won't know from which script the actual "cp" error message came. More work is needed to produce a truly useful error message when a command inside a script fails. The full and proper way to handle non-zero return codes in scripts is by using error messages that contain the script name. This means you need "if" statements around *every* command that might fail! This is probably overkill for most hobby scripts; but, it is necessary for systems programming: grep -v '^guest:' /etc/passwd >lockfile.tmp status="$?" if [ "$status" -ne 1 -a "$status" -ne 0 ] ; then # grep returns 2 on serious error echo 1>&2 "$0: grep guest /etc/passwd failed; status $status" exit 1 fi if ! cp lockfile.tmp /etc/passwd.tmp ; then echo 1>&2 "$0: cp lockfile.tmp /etc/passwd.tmp failed; status $?" exit 1 fi if ! chmod 444 /etc/passwd.tmp ; then echo 1>&2 "$0: chmod /etc/passwd.tmp failed; status $?" exit 1 fi if ! mv /etc/passwd.tmp /etc/passwd ; then echo 1>&2 "$0: mv /etc/passwd.tmp /etc/passwd failed; status $?" exit 1 fi rm lockfile.tmp lockfile Now, the script tells you its name in the error message: $ ./myscript cp: /etc/passwd.tmp: No space left on device ./myscript: cp lockfile.tmp /etc/passwd.tmp failed; status 2 Now it's easy to tell from which script the above "cp" error came. Making your scripts detect errors and issue clear error messages is tedious but not difficult. Adding all the error checking makes the code much longer and harder to read and modify. If the script isn't doing anything important, simply exiting after a failed command may be sufficient; it only adds a few words to the end of each line of the script. For a system script that must detect errors under all conditions (including "too many processes", "file system full", etc.), you must have all the additional error checking. The reward is a script that won't let you down when things go wrong, and that will tell you exactly what the problem is when one develops. =========================== Complementing Return Status =========================== Any command or command pipeline's return status can be complemented (reversed from good to bad or bad to good) using a leading "!" before the command, e.g. $ false $ echo $? 1 $ ! false $ echo $? 0 $ grep nosuchxxx /etc/passwd $ echo $? 1 $ ! grep nosuchxxx /etc/passwd $ echo $? 0 This is useful in shell scripts to simplify this: if grep "$var" /etc/passwd ; then : do nothing else echo 1>&2 "$0: Cannot find '$var' in /etc/passwd" fi to this: if ! grep "$var" /etc/passwd ; then echo 1>&2 "$0: Cannot find '$var' in /etc/passwd" fi The "!" prefix is also useful in turning "while" loops into "until" loops or vice-versa. =================================== Writing and Debugging Shell Scripts =================================== The Number One rule of writing shell scripts is: Start Small and Add One Line at a Time! Students who write a 10- or 100-line script and then try to test it all at once usually run out of time. An unmatched quote at the start of a script can eat the entire script until the next matching quote! Start your script with the Script Header (name of interpreter, PATH, umask, comments) and the single command "date". If that doesn't work, you know something fundamental is wrong, and you only have a few lines of code that you need to debug. (Is your interpreter correct? your PATH?) Add to this simple script one or two lines at a time, so that when an error occurs you know it must be in the last line or two that you added. Do not add 10 lines to a script! You won't know what you did wrong! You can ask the a shell to show you the lines of the script it is reading and executing by using the "-v" or "-x" (or both) option to the shell: $ sh -v -u ./myscript arg1 arg2 ... $ sh -x -u ./myscript arg1 arg2 ... The "-v" options displays the command lines as they are read by the shell (without any shell expansion). The "-x" option displays the command lines as they are passed to the commands being executed, after the shell has done all the command line expansion and processing. These options will allow you to see the commands as they execute, and may help you locate errors in your script. (Double-quote your variables!) Of course you can use -v and -x with an interactive shell too: $ sh -v $ echo $SHELL echo $SHELL /usr/bin/ksh $ echo * echo * a b c d $ sh -x $ echo $SHELL + echo /usr/bin/ksh /usr/bin/ksh $ echo * + echo a b c d a b c d $ sh -v -x $ echo $SHELL echo $SHELL + echo /usr/bin/ksh /usr/bin/ksh $ echo * echo * + echo a b c d a b c d Remember that if you use a shell to read a shell script ("sh scriptname"), instead of executing it directly ("./scriptname"), the shell will treat all the comments at the start of the shell script as comments. In particular, the comment that specifies the interpreter to use when executing the script ("#!/bin/sh -u") will be ignored, as will all of the options listed beside that interpreter. Only by actually *executing* the script will you cause the Unix kernel to use the interpreter and options given on the first line of the script. For example: $ cat test #!/bin/sh -u echo 1>&2 "$0: This is '$undefined'" $ ./test ./test: undefined: unbound variable $ sh test test: This is '' $ sh -u test test: undefined: unbound variable $ csh test Bad : modifier in $ ( ). All shells treat #-lines as comments and ignore them. Only the Unix kernel treats #! specially, and only for executable scripts. ==================== Naming Shell Scripts ==================== Often, you will want to put an example of how to run a shell script inside the shell script as a comment. You might create a script called "doexec" and write it as follows: #!/bin/sh -u # doexec [ files... ] # This script sets execute permissions on all its arguments. # -IAN! idallen@ncf.ca Mon Jun 11 23:02:38 EDT 2001 export PATH=/bin:/usr/bin umask 022 if [ "$#" -eq 0 ]; then echo 1>&2 "$0: No arguments; nothing done" status=0 else if chmod +x "$@" ; then status=0 # it worked else status="$?" echo 1>&2 "$0: chmod exit status: $status" echo 1>&2 "$0: Could not change mode of some argument: $*" fi fi exit "$status" You would execute this script by typing: $ ./doexec filename1 filename2 filename3... This comment line in the doexec script: # doexec [ files... ] tells the reader that the script name is "doexec" and the files are the (optional) arguments to the script. This is the "syntax" of the command. But what if you rename the script to be something other than "doexec"? $ mv doexec fixperm $ ./fixperm foo bar The use of "$0" in the echo line for the error message ensures that the shell will print the actual script name in the error message, but the comment in the script is now wrong, since the program name is no longer "doexec". I don't want to have to edit the script and make a change such as "doexec" to "fixperm" every time I change the name of the script. The solution is never to put the actual name of a script inside the script, even as a comment. Wherever you refer to the name of the script, even in a comment, use the "$0" convention instead. So, the comment changes from: # doexec [ files... ] to be: # $0 [ files... ] In the comment, "$0" just means "whatever the name of this script is", without my having to actually write the script name. I don't want to use the actual script name, because I might change it. Since the line is a comment, ignored by the shell, the shell will never actually expand that "$0" to be the real name of the shell; it's just a convenient way of ! script specifying the program name without actually naming it inside the script. Never put the name of a program inside the program; it might change! )---------- chapter14.txt -- Wed 2002-Feb-27 16:13:46 -------------------------------- DAT2330 - Unix - Prof. Ian Allen -------------------------------- Chapter 14 - Programming Tools (excerpts) We will concentrate on the basic Compile, Link, and Go sequence. Sections covered: Intro p.537-541 Compiling and Linking p.541-543 Compiler Warnings p.551 Notes: - Usually when you are asked to "compile" a program, the intent is to compile it and link it into a runnable module. - The indentation used in the tabs.c program on p.539 is not standard. (The author did not indent inside functions.) Since GNU/Linux is an open-source operating system, you can see "standard" C indentation style by reading the source code available. You can find the Linux kernel source code under /usr/src/linux/ on the Linux Test Machine. A sample file would be: /usr/src/linux/kernel/time.c NOTE: The IBM ACADAIX native compiler is named "cc", not "gcc", and it has a very different set of options. The basic use of "-c", "-l", "-o" and a few others are the same as Linux (as they have been in use since 1975); but, the other options may be different. The ACADAIX "cc" compiler will accept both C language (*.c) and C++ lanugage (*.C) source files. Note the upper-case suffix! Chapter summary: - Use the "-Wall" option to gcc (or to "g++") to turn on warnings about possible coding errors. Use the "-g" option to generate symbol tables for symbolic debuggers such as "gdb". - If you don't use the "-o" option, the output load module, linked and ready to run, is left in the file "a.out" in the current directory. Compile and link a C program from several C source files: gcc -Wall -g one.c two.c three.c Compile several C source files into object code without linking: gcc -c -Wall -g one.c two.c three.c Link several object files (*.o) into one program without compiling: gcc -Wall -g one.o two.o three.o Compile and link a C++ program from several C++ source files: g++ -Wall -g -o myprog a.C b.C c.C (note the upper-case suffix!) Here is some source code for some test programs: This version is C language: /******************************************************************/ /* A program to print command line arguments -IAN! idallen@ncf.ca */ /******************************************************************/ #include int main(int argc, char **argv){ int i; for( i=0; i int main(int argc, char **argv){ int i; for( i=0; iWhy do we use " || ", for examples: > test -f /etc/passwd || echo hello >I tried using " | ", only this one works fine. The token "||" is not at all related to the token "|", just as in C language "&&" is not related to "&", "||" is not related to "|", and "==" is not related to "=". They mean different things. In the week7-9.txt notes you'll find a section titled "Testing Return Codes of Commands" that explains briefly how "||" works to test the return codes of commands in shell scripts. Here are some more examples. Here is how the return code (exit status) is set by the Unix "test" command, for a file that exists and one that does not exist: $ test -f /etc/passwd <== set the return code $ echo $? <== display it 0 $ test -f /nosuchfile <== set the return code $ echo $? <== display it 1 Here's how to use the "||" token to have the shell conditionally execute the command to the right if the command on the left returns a bad (non-zero) exit status: $ test -f /etc/passwd || echo "return code $? - no such file" $ (Above: good return status, do *NOT* execute command after "||") $ test -f /nosuchfile || echo "return code $? - no such file" return code 1 - no such file $ (Above: bad return status, do execute command after "||") $ test -f /bin || echo "return code $? - no such file" return code 1 - no such file $ (Above: bad return status [not a file], do execute command after "||") The "||" operator provides a quick way to test the return code of the command on the left, without using a full shell "if" statement. To see the difference, compare the following two identical tests: test -f /nosuchfile || echo "return code $? - no such file" if ! test -f /nosuchfile ; then echo "return code $? - no such file" fi One test takes one line to program; the other takes three. Other than that, the output is identical. We use "||" because it's easy and short. The "&&" token works the opposite way to "||". The command on the right is executed only if the command on the left returns a good (zero) exit status: $ test -f /etc/passwd && echo "good status" good status $ (Above: good return status, do execute command after "&&") $ test -f /nosuchfile && echo "good status" $ (Above: bad return status, do *NOT* execute command after "&&") $ test -f /bin && echo "good status" $ (Above: bad return status, do *NOT* execute command after "&&") The "&&" operator also provides a quick way to test the return code of the command on the left, without using a full shell "if" statement. Here are two identical tests, comparing "if" and "&&": test -f /etc/passwd && echo "file exists" if test -f /etc/passwd ; then echo "file exists" fi One test takes one line to program; the other takes three. Other than that, the output is identical. We use "&&" because it's easy and short. The most common use of "||" is as a "quick exit" if an important command in a shell script fails. In the script fragment below, if the copy doesn't work, there is no point in continuing with the rest of the script: cp "$1" foo || exit $? <== exit immediately if cp fails chmod 700 foo wc foo >foo.count || exit $? <== exit immediately if wc fails chmod 755 foo.count We add "|| exit $?" to the ends of critical lines in the script. If the command on the left fails, there is no point in continuing with the rest of the script. We could write the same fragment using shell "if" statements; but, the code is three times longer for each test: if ! cp "$1" foo ; then exit $? fi chmod 700 foo if ! wc foo >foo.count ; then exit $? fi chmod 755 foo.count The above code is identical to the code that uses "||"; but, it's longer. In the week7-9.txt notes you'll find a section titled "Testing Return Codes of Commands" that has examples of using "||" to test the return codes of important commands in shell scripts. ! (I'm reading this document standalone, without access to the textbook it ! enhances, so this comment may be irrelevant if the book covers it.) ! This (and the previous week7-9) would be a lot more helpful if it included ! explanation of the concept that the shell's interest is in evaluating the ! truth value of the "if" expression, and the shell stops evaluating as soon ! as it determines the truth value of the "if" expression. That's preferable ! to the presentation here of "||" and "&&" as arbitrary symbols that must ! be brute-forced memorized. Who knows the "stop as soon as" concept can ! derive on the fly which of "||" and "&&" to use. ! (Speaking of textbooks, I try to always use a somewhat disambiguating ! reference to a book--"Sobell p 234" rather than "the textbook p 234".) )---------- test3-questions-B.txt -- Fri 2002-Mar- 1 23:51:01 ======================================= DAT2330 Test Three B - Unix - Questions ======================================= -IAN! idallen@ncf.ca You will need to be running Floppix with networking to do this test. You must have network access to use telnet and ftp to reach the machines mentioned in this test. Your directory test_three is where you must put your scripts and your script output if you want the instructor to mark them. This directory will be created by your instructor before you start your test. Your scripts do not have to create this directory; you may assume it exists any time a script specification requires you to use it. Only scripts and output found under directory test_three on the Test Machine will be marked. Only correctly-spelled script and file names will be marked. Your scripts must start with a valid Script Header, as given in class and in the Week 7 notes. You do not have to detect or handle errors in the scripts you write on this test. Error handling will be covered in Chapter 11. For example, your script does not need to check to see if a file or directory already exists before it tries to make one. Many of the scripts you write during this test will not run with out errors if you run them twice; because, the scripts will have created files and/or directories and a second attempt to re-cre ate the same files and/or directories may generate error mes sages. Make sure you clean out the results of one script run before you test your script a second time. Problem A - Marks: 14 On the Test Machine: Write an executable shell script named inputecho.sh that will do the following actions (in the order given below): 1. [Marks: 1] Display this exact sentence: My shell script is starting now. 2. [Marks: 2] Display this exact prompt for user input: A 2 word input: 3. [Marks: 2] Input one line from the user into two new shell variables. 4. [Marks: 3] Display the contents of the two shell variables in this exact format: Number one is: OOO Number two is: TTT where OOO is the contents of the first shell variable and TTT is the contents of the second shell variable. 5. [Marks: 2] Display this exact sentence: Your shell's using: KKK where KKK is the contents of the predefined shell keyword variable that contains your shell's search path. (Use the exact punctuation.) 6. [Marks: 2] Display this exact sentence: The name is SSS when you are here. where SSS is the contents of the predefined shell keyword variable that contains the name of your login shell. 7. [Marks: 1] Display the current date. 8. [Marks: 1] Display this exact sentence: My script is ending at the end. Problem B - Marks: 26 On the Test Machine: Write an executable shell script named commander.sh that will do the following actions (in the order given below): 1. [Marks: 2] Extract the first 50 lines of the file /etc/pjunk2 into a file named fiftyout.txt in the current directory. 2. [Marks: 2] Generate a long listing (showing permissions, owner, etc.) of all names in the /bin directory that end in the two characters sh into the file shells.txt in the current directory. 3. [Marks: 4] Display these two exact sentences: First is: 'FFF' Second is: 'SSS' where FFF is the text of the first positional command-line parameter to this script and SSS is the text of the second positional command-line parameter to this script. Use the exact output punctuation shown. 4. [Marks: 2] Make a sub-directory in the current directory, named using the text of the first command-line argument. (The first positional parameter contains the name of the new sub-directory that you must create.) 5. [Marks: 2] Move the existing file fiftyout.txt that you created earlier, into the sub-directory you just created in the previous step. 6. [Marks: 5] In the existing file fiftyout.txt that you already created, count the number of lines (only the lines) that contain the 4-character string (DAT and put the count into a new file named fcount.txt in the current directory. 7. [Marks: 2] Create a sub-sub-directory under the sub- directory you just created, named using the text of the second argument. (The second positional parameter contains the name of the new sub-sub-directory.) 8. [Marks: 2] Move the existing file fcount.txt that you created earlier, into the sub-sub-directory that you just created in the previous step. 9. [Marks: 5] Add group and other read permissions to the three files you have created in this script. (Each file is now in a different directory.) Problem C - Marks: 18 On the Test Machine: Write an executable shell script named namer.sh that will do the following actions (in the order given below): 1. [Marks: 2] Use a command to generate the host name of the current computer. Put the command output (the host name) into a shell variable. 2. [Marks: 2] Display this exact sentence: The computer is: XXX where XXX is the contents of the variable (the host name) from the previous step. 3. [Marks: 5] Select from the file /etc/hosts the one line that contains the host name of the current computer. Use the variable (the host name) from the previous step as your search pattern. Extract just the IP address (the first tab- delimited field) from that line. Put the output of this step (just the IP address) into a shell variable. 4. [Marks: 3] Use the two shell variables you just created and display this exact output: Address PPP belongs to machine NNN where PPP is the IP address and NNN is the host name of this computer. 5. [Marks: 2] Use a command to get the userid of the person running this script. Put the command output (the userid) into a shell variable. 6. [Marks: 2] Display this exact sentence: The person is: YYY where YYY is the contents of the shell variable (the userid) from the previous step. 7. [Marks: 2] Display the current date in ALL UPPER CASE LETTERS. ! Heh, I suspect you may have been fishing for: date | tr a-z A-Z. ! I habitually display the date as: date '+%Y-%m-%d'; that led to the ! interpretation "all displayable characters in the result must be upper-case ! letters", which is probably too messy to have been what you had in mind. )---------- test3-questions-A.txt -- Fri 2002-Mar- 1 23:51:25 ======================================= DAT2330 Test Three A - Unix - Questions ======================================= -IAN! idallen@ncf.ca You will need to be running Floppix with networking to do this test. You must have network access to use telnet and ftp to reach the machines mentioned in this test. Your directory test_three is where you must put your scripts and your script output if you want the instructor to mark them. This directory will be created by your instructor before you start your test. Your scripts do not have to create this directory; you may assume it exists any time a script specification requires you to use it. Only scripts and output found under directory test_three on the Test Machine will be marked. Only correctly-spelled script and file names will be marked. Your scripts must start with a valid Script Header, as given in class and in the Week 7 notes. You do not have to detect or handle errors in the scripts you write on this test. Error handling will be covered in Chapter 11. For example, your script does not need to check to see if a file or directory already exists before it tries to make one. Many of the scripts you write during this test will not run with out errors if you run them twice; because, the scripts will have created files and/or directories and a second attempt to re-cre ate the same files and/or directories may generate error mes sages. Make sure you clean out the results of one script run before you test your script a second time. Problem A - Marks: 14 On the Test Machine: Write an executable shell script named echoinput.sh that will do the following actions (in the order given below): 1. [Marks: 1] Display this exact sentence: This is the start of my script. 2. [Marks: 1] Display the current date. 3. [Marks: 2] Display this exact sentence: It has the value XXX at login. where XXX is the contents of the predefined shell keyword variable that contains the name of your login shell. 4. [Marks: 2] Display this exact sentence: The search's here: YYY where YYY is the contents of the predefined shell keyword variable that contains your shell's search path. (Use the exact punctuation.) 5. [Marks: 2] Display this exact prompt for user input: Enter two words: 6. [Marks: 2] Input one line from the user into two new shell variables. 7. [Marks: 3] Display the contents of the two shell variables in this exact format: First is: FFF Second is: SSS where FFF is the contents of the first shell variable and SSS is the contents of the second shell variable. 8. [Marks: 1] Display this exact sentence: This is the end of my script. Problem B - Marks: 18 On the Test Machine: Write an executable shell script named hoster.sh that will do the following actions (in the order given below): 1. [Marks: 2] Use a command to get the userid of the person running this script. Put the command output (the userid) into a shell variable. 2. [Marks: 2] Display this exact sentence: The userid is: XXX where XXX is the contents of the shell variable (the userid) from the previous step. 3. [Marks: 2] Display the current date in ALL UPPER CASE LETTERS. 4. [Marks: 2] Use a command to generate the host name of the current computer. Put the command output (the host name) into a shell variable. 5. [Marks: 2] Display this exact sentence: The name is: YYY where YYY is the contents of the variable (the host name) from the previous step. 6. [Marks: 5] Select from the file /etc/hosts the one line that contains the host name of the current computer. Use the variable (the host name) from the previous step as your search pattern. Extract just the IP address (the first tab- delimited field) from that line. Put the output of this step (just the IP address) into a shell variable. 7. [Marks: 3] Use the two shell variables you just created and display this exact output: The address of HHH is III where HHH is the host name of this computer and III is the IP address. Problem C - Marks: 26 On the Test Machine: Write an executable shell script named argumenter.sh that will do the following actions (in the order given below): 1. [Marks: 4] Display these two exact sentences: Number one is: 'OOO' Number two is: 'TTT' where OOO is the text of the first positional command-line parameter to this script and TTT is the text of the second positional command-line parameter to this script. Use the exact output punctuation shown. 2. [Marks: 2] Make a sub-directory in the current directory, named using the text of the first command-line argument. (The first positional parameter contains the name of the new sub-directory that you must create.) 3. [Marks: 2] Generate a long listing (showing permissions, owner, etc.) of all names in the /usr/bin directory that end in the three characters csh into the file cshells.txt in the current directory. 4. [Marks: 2] Extract the last 20 lines of the file /etc/pjunk1 into a file named twentyout.txt in the current directory. 5. [Marks: 2] Move the existing file twentyout.txt that you just created in the previous step, into the sub-directory you created in an earlier step. 6. [Marks: 5] In the existing file twentyout.txt that you already created, count the number of lines (only the lines) that contain the 4-character string (DAT and put the count into a new file named tcount.txt in the current directory. 7. [Marks: 2] Create a sub-sub-directory under the sub- directory you created earlier, named using the text of the second argument. (The second positional parameter contains the name of the new sub-sub-directory.) 8. [Marks: 2] Move the existing file tcount.txt that you created earlier, into the sub-sub-directory that you just created in the previous step. 9. [Marks: 5] Add user and group write permissions to the three files you have created in this script. (Each file is now in a different directory.) )---------- practiceCh11-14.txt -- Mon 2002-Mar- 4 00:50:24 ------------------------------------------------------ Practice Unix/Linux Questions - Chapters 5, 10, 11, 14 ------------------------------------------------------ -IAN! idallen@ncf.ca Not all of these scripts are equally difficult to write. Find the easy ones first! Problems in this file have been given a complexity rating from 1-5, with 1 being "really simple" and 5 being "complex". You can expect complexity 1, 2, and 3 on your Unix tests. All scripts written must conform to the scriptChecklist11.txt checklist. All user input (arguments or via "read") must be validated before being used. Echo user input (including arguments given) back to the user. This is usually a good idea both for debugging your script and giving the user feedback on what the script is processing. Avoid Linux-only commands and command options. The same script should work without modification on both Linux and ACADAIX, if possible. Don't assume you can write temporary files into the current directory, unless the script directions say it is safe to do so. Test this by changing directories to a directory into which you cannot write (e.g. /etc or /home) and running your script from there. Real-world scripts must check the error returns on the commands they use. (This is illustrated in the week7-9.txt notes.) At minimum, exit the script if an important command fails. Don't check the exit status of "unimportant" commands, such as most "echo" commands that print on the screen. TimeSaver: Use filename completion in the shell to complete the names of these scripts when you want to test them or edit them. Don't keep typing the full path names of the scripts! Let your Shell do your typing for you! --) Type in all the scripts in the parts of Chapter 11 that we study. Complexity: 1 Start with "if1" and "chkargs". Add the correct script headers to the scripts. Fix the prompts. Test them to make sure that they work. --) Write this executable script named "10_top_five.sh" Complexity: 1 Verify that the script has exactly one command line argument that is a readable file. Sort the file and display the first five lines. --) Write this executable script named "11_string_compare.sh" Complexity: 1 Prompt for and read a string. Echo the string back to the user. Compare this string against the first command line argument (test to make sure it exists!) and print whether or not the string is an exact match: $ ./foo 'this is four' Enter your one line of input: this is three You entered: this is three String 'this is three' does not match argument 'this is four'. $ ./foo "happy coding" Enter your one line of input: happy coding You entered: happy coding String 'happy coding' is a match. $ ./foo ./foo: Expecting one argument, found 0: ()" --) Write this executable script named "12_file_mailer.sh" Complexity: 1 Verify that the single command line argument exists and is a non-empty, readable file. Prompt for and read an email userid. Echo the userid back to the user. Email the content of the file named on the command line to the given userid. The script must check to see that the mail command works (has a good return status). Print an error message if the mail command has an error. --) Write this executable script named "13_comment_extractor.sh" Complexity: 1 Verify that the script has exactly one command line argument, that it is a file, and that it is readable. Extract and display on standard output all the comment lines (lines starting with "#") from the argument file. Bonus: Don't display lines that begin "#!". --) Write this executable script named "14_lazy_compiler.sh" Complexity: 2 (based on Chapter 14) Expect a single name as a command line argument, e.g. "dodo". Make sure that the name with the suffix ".c" appended, exists as a readable file, and is not empty. For example, if the command argument name is "dodo", test for the file named "dodo.c". Compile the C program (e.g. "dodo.c") with full warnings and debugging symbols enabled, and leave the output in the file named by the command ! "output" is ambiguous, especially since you've just emphasized "warnings". line argument (e.g. "dodo"). Exit the script if the compile fails. Set the executable output file (e.g. "dodo") to have read/write/execute permissions for you and group, and execute only permissions for others. Examples: $ ls dodo.c foo $ ./foo dodo $ ls dodo dodo.c foo $ ./foo nosuchfile ./foo: error: 'nosuchfile.c' is not a readable file. $ chmod 000 dodo.c $ ./foo dodo ./foo: error: 'dodo.c' is not a readable file. Bonus: If the output file already exists (e.g. "dodo"), prompt and ask the user if it should be overwritten before continuing. --) Write this executable script named "15_optional_arguments_demo.sh" Complexity: 2 This script takes zero to three arguments. Without arguments, the contents of three predefined shell keyword variables are displayed. Each command line argument replaces one of the variables in the output: With no arguments, display: PWD MAIL HOME With 1 argument, display: arg1 MAIL HOME With 2 arguments, display: arg1 arg2 HOME With 3 arguments, display: arg1 arg2 arg3 Examples: $ ./foo Output is: /home/idallen/tmp /var/spool/mail/idallen /home/idallen $ ./foo blue Output is: blue /var/spool/mail/idallen /home/idallen $ ./foo blue green Output is: blue green /home/idallen $ ./foo blue green red Output is: blue green red $ ./foo blue green red yellow ./foo: Expecting less than 4 arguments; found 4 (blue green red yellow) Restrictions: You may only expand $PWD, $MAIL, and $HOME *once* each in the script. You may only use *one* echo command to produce the output and *one* echo command to produce the error message. Here are some hints, encrypted using rot13 on all the letters. Try to do the script without looking at the hints. If you must look at the hints, you will need to cut out this rot13 paragraph and translate it back to readable English (see the file rotatePuzzle.txt for help on rot13): Hfr guerr inevnoyrf gb ubyq gur guerr bhgchg inyhrf. Vavgvnyvmr gur inevnoyrf gb gur inyhrf bs CJQ, ZNVY, naq UBZR ng gur fgneg bs gur fpevcg. Qrcraqvat ba gur ahzore bs nethzragf, ercynpr bar be zber bs gur inevnoyrf jvgu gur inyhrf sebz gur pbzznaq yvar. Ng gur raq bs gur fpevcg, hfr gur guerr inevnoyrf va lbhe "rpub" bhgchg yvar. --) Write this executable script named "16_file_sort_self_compare_v1.sh" Complexity: 2 Verify that the script has exactly one argument. Echo the argument that will be processed back to the user. Verify that the argument is a file, and that it is readable. Sort the file into the /tmp directory under a unique temporary name. (Use the pid $$ somewhere in the name to help ensure uniqueness.) The script must check exit status to make sure that the sort worked. Run a "diff" on the original file and the temporary file and count the number of lines of output. The output should look like this: $ ./foo /etc/resolv.conf Processing: /etc/resolv.conf The file differs from its sorted version by 4 lines. $ ./foo /etc/passwd /etc/group ./foo: Expecting 1 argument; found 2 (/etc/passwd /etc/group) Remove the temporary file when you are done with it. --) Write this executable script named "17_path_validator.sh" Complexity: 2 The purpose of this script is to validate a single command line argument. If the pathname is a file, make sure it is readable, writable, and not zero size. If the pathname is a directory, make sure it is readable and searchable. Print appropriate messages if any of these validations fail. --) Write this executable script named "18_file_size_classer.sh" Complexity: 2 Process one command line argument. Make sure it is readable and that it is a file (not a directory). Classify the file according to the count of the number of lines it contains: 0-99: small 100-499: medium over 499: large Sample output: $ ./foo a File 'a' contains 4 lines and is: small $ ./foo b File 'b' contains 500 lines and is: large $ ./foo c ./foo: Argument 'c' is not a readable file. $ ./foo /bin ./foo: Argument '/bin' is is a directory; nothing done. $ ./foo a b c ./foo: Expecting 1 argument; found 3 (a b c) --) Write this executable script named "19_two_number_sort.sh" Complexity: 2 Get two inputs. You may use two command line arguments or you may prompt for two inputs from standard input (your choice). The two inputs will be treated as two unsigned integer numbers. Your script does not have to check that the inputs are really numbers; but, you do have to make sure they both exist and are not empty strings (or missing arguments, if you use arguments). Output the smaller number followed by the larger number, in this form: You entered 100 and 12. In order, they are: 12 100 You may use only one "echo" statement to produce the above message. The "echo" statement that produces the standard output of the script must appear in only one place in the script. Hint: Use variables to hold the smaller and larger numbers. Use the two variables in the "echo" statment at the end of the script. --) Write this executable script named "20_range_tester_v1.sh" Complexity: 2 Prompt for and read two "range" integers. In the prompts, tell the user that the second range integer must be bigger than the first range integer. Your script does not have to make sure that the inputs are numbers; assume the user will always enter numbers. (However - You cannot assume that the user will enter both inputs - make sure that neither of the inputs is empty.) Validate to make sure the second integer is bigger than the first one. Compare a single command-line argument (an integer) against the range and print where it lies in the range. The output must look like this (except that your prompts should be much more informative): $ ./foo 12 Input: 20 40 Argument 12 lies below low range 20. $ ./foo 300 Input: 99 1000 Argument 300 lies between 99 and 1000. $ ./foo 2000 Input: 500 600 Argument 2000 lies above high range 600. $ ./foo a b c ./foo: Expecting 1 argument; found 3 (a b c) $ ./foo ./foo: Expecting 1 argument; found 0 () --) Write this executable script named "21_range_tester_v2.sh" Complexity: 2 As above; but, use two command line arguments as the two range integers and read the integer to be tested from standard input. $ ./foo 20 40 Input: 12 Input 12 lies below low range 20. $ ./foo 99 1000 Input: 300 Input 300 lies between 99 and 1000. $ ./foo 500 600 Input: 2000 Input 2000 lies above high range 600. $ ./foo 500 600 Input: ./foo: Expecting 1 input number - nothing found. $ ./foo 123 ./foo: Expecting 2 arguments; found 1 (123) Remember: Validate your inputs. Do not use any input that is missing or empty. Do not ignore "extra" arguments - produce an error message. --) Write this executable script named "22_range_tester_v3.sh" Complexity: 2 As above, but allow the user to enter the two range integers in any order. (Smallest could be first *or* second.) Your program will put the two integers in order before using them as the range. Just to be sure, echo back the low range integer and the high range integer after your program has put them in order. Generate the same output as the previous problem. --) Write this executable script named "23_ascending_series.sh" Complexity: 2 Expect three non-negative integers on the command line. (You don't have to validate that the arguments are integers; but, you do have to make sure that there are exactly three of them.) Make sure the third integer is not zero. Make sure the second integer is not less than the first integer (equal is okay). Output all numbers from the first number to the second (inclusive), using the third number as the increment. $ ./foo 1 3 1 1 2 3 $ ./foo 1000 1007 2 1000 1002 1004 1006 Note: To add 2 to a shell variable named x, use the syntax: let x=x+2 (for details, see your Linux text, bottom of p.355, top of p.356) --) Write this executable script named "24_both_series.sh" Complexity: 2 As above; but, if the second number is less than the first number, count *down* instead of up. $ ./foo 10 24 5 10 15 20 $ ./foo 24 10 5 24 19 14 Bonus: Do not duplicate code! Set shell variables for your loop start value, loop end value, test condition ("-le" or "-ge"), and increment value and write the loop code only *once*. See the example while scripts on the course web site. --) Write this executable script named "25_file_sort_self_compare_v2.sh" Complexity: 3 Produce the same output as the file_sort_self_compare_v1 script; but, accept either zero or one command line argument. (The argument is now optional.) If there is no command line argument, prompt for and read a filename. If there is a command line argument, use that for the filename. $ ./foo /etc/resolv.conf Processing: /etc/resolv.conf The file differs from its sorted version by 4 lines. $ ./foo Input file name: /etc/resolv.conf Processing: /etc/resolv.conf The file differs from its sorted version by 4 lines. $ ./foo a b ./foo: Expecting 0 or 1 argument; found 2 (a b) Notes: Do not duplicate code! The processing part of the script should *not* be written twice, once for command line arguments and a second time for standard input. Do not duplicate the code. Write it only once. Hint: If using a command line argument, put the command line argument into a shell variable. If there is no command line argument, read standard input into the same variable. Use the shell variable in the rest of the script. Do not duplicate the code. --) Complexity: +1 (add one) for any script with optional arguments Following the model of the previous problem that used optional command line arguments (file_sort_self_compare_v2), go back and modify all the scripts you have written so far and make their command line arguments optional. Read the missing arguments from standard input if there are arguments missing on the command line. Do not duplicate the code; modify the code to use shell variables and put the command line arguments (if any) into those variables. --) Write this executable script named "26_script_permission_validator.sh" Complexity: 3 Make sure that the given single pathname command line argument is a readable file and that the first line of the file starts with "#!". (Hint: The grep command [like most Unix commands] can read standard input. If the grep pattern is found in standard input, grep returns zero, otherwise it returns non-zero.) You may echo the line found for debugging purposes. Next, make sure that this pathname is executable. Print a warning message and a line showing the permissions of the file if it is not. $ ./foo goodscript.sh Found this first line: #!/bin/sh -u $ ./foo wrongexec.sh Found this first line: #!/bin/sh -u Script 'wrongexec.sh' is not executable by you. -rw-r--r-- 1 alleni alleni 123 Jul 31 15:13 wrongexec.sh $ whoami alleni $ ./foo bad.sh Found this first line: !#/bin/sh -u Script 'bad.sh' does not start with '#!'. Script 'bad.sh' is not executable by you. -rw-r-xr-x 1 alleni alleni 123 Jul 31 13:15 bad.sh $ ./foo /bin ./foo: Argument '/bin' is is a directory; nothing done. Bonus: Prompt for a file name if none is given on the command line. --) Write this executable script named "27_script_interpreter_validator.sh" Complexity: 3 Make sure that the script has a single command line argument and that it is a readable file - don't try to process directories or unreadable files given as arguments. Extract the first line of the file. Validate this first line as a valid first line for a shell script. (Validate the entire line, not just the beginning of the line.) To be valid, the line must be one of the lines you use in your scripts to date. Print a special message for invalid first lines that are blank. Sample usage (showing the first three lines of each of my test files): bash$ head -3 script1.sh <== show my test file #!/bin/sh -u # $0 (no arguments) # This script does the following: bash$ ./foo script1.sh Found this first line: #!/bin/sh -u This is a valid first line of a script. bash$ head -3 script2.sh <== show my test file #/bin/sh -u # $0 pathname # This script does the following: bash$ ./foo script2.sh Found this first line: #/bin/sh -u *** '#/bin/sh -u' is not a valid first line of a shell script. *** bash$ touch empty.sh bash$ ./foo empty.sh Found this first line: *** The first line of this script is missing or empty. *** Suggestion: Use a set of simple IF statements to compare the line you found with each possible valid first line, one after the other. If the line matches in any of the IF statements, exit the script immediately with a good return code. If the first line does not match any of the valid lines (if it fails all the IF tests), then after all the IF statements, print an error message at the end of the script (include the text of the line that doesn't match) and exit non-zero. Bonus: Prompt for a file name if none is given on the command line. --) Write this executable script named "28_script_path_validator.sh" Complexity: 3 As above, except the script must pick out the "PATH=" line from the script file that is the first command line argument and validate it as being a correct PATH line. --) Write this executable script named "29_script_umask_validator.sh" Complexity: 3 As above, except the script must pick out the "umask" line from the script file that is the first command line argument and validate it as being a correct umask line. --) Write this executable script named "30_script_validator.sh" Complexity: 2 Write a master script that validates its first argument (a filename) as being a valid script by passing the filename to each of the four permission, interpreter, path, and umask validator scripts that you wrote above. This is a very handy script to have, especially for tests. --) Write this executable script named "31_empty_directory_tester_v1.sh" Complexity: 3 This script validates its only command-line argument as a directory, tells whether or not it is searchable (has execute permissions), and counts the number of non-hidden names in it. The output should look like this: $ ./foo /bin Directory '/bin' contains 90 non-hidden names. $ ./foo /nosuchfile ./foo: error: argument '/nosuchfile' is not a directory. $ mkdir empty $ touch empty/.hidden empty/.morehidden $ ./foo empty Directory 'empty' contains 0 non-hidden names. $ chmod u-x empty $ ./foo empty ./foo: warning: Directory 'empty' is not searchable. Directory 'empty' contains 0 non-hidden names. $ chmod u-rx empty $ ./foo empty ./foo: error: directory 'empty' is not readable; nothing done. Notes: - warnings and errors print on standard error, not standard output Bonus: In your output, say that the directory contains no non-hidden names instead of displaying the number 0. (Hint: Put the count of names into a variable. Change the variable contents to "no" if it has the value zero.) Bonus: Prompt for a directory name if none is given on the command line. --) Write this executable script named "32_empty_directory_tester_v2.sh" Complexity: 3 This produces output similar to the previous problem; but, hidden names except "." and ".." are now included in the count. Include hidden names in the name count; but, don't include the names "." and "..". If a directory contains only "." and ".." you must still call it "empty" in the output: $ mkdir empty $ ./foo empty Directory 'empty' is empty. $ touch empty/.hidden empty/.another $ ./foo empty Directory 'empty' contains 2 names. Hints: Read about the "-A" option to the "ls" command. Bonus: If there is only one hidden name, say "1 name" not "1 names". (Hint: put either "s" or "" [nothing] into a variable, depending on whether the count is 1 or not 1. Use the variable at the end of the word "name" in the output message.) --) Write this executable script named "33_make_backup.sh" Complexity: 3 Verify that the script has exactly one command line argument. Verify that the argument is a file, and that it is readable. See if a "backup" version of the given file exists. The backup version is named with ".bak" appended to the file name. If no backup exists, create a backup version of the file by copying it to the same name with ".bak" appended and then exit the script. Use the "-p" option of the copy command to preserve the modify time on the backup copy of the file. Remember: Test the return code of the copy. If a backup copy already exists, determine if it is identical to the original file. (Hint: use diff, and optionally wc.) If it is identical, print a message saying it is identical and exit the script. If the existing backup copy is different from the original, prompt and ask the user if the existing backup copy should be overwritten. (Bonus: Tell the user how many lines of differences there are between the original file and its backup version.) If the backup copy should be overwritten (if the user responds yes), do so, otherwise exit the script. $ ls foo zork $ ./foo zork Creating new backup: zork.bak $ ls foo zork zork.bak $ ./foo zork Files 'zork' and 'zork.bak' are identical; nothing done. $ date >zork.bak $ ./foo zork File 'zork.bak' already exists; should I overwrite it (yes/no)? yes Creating new backup: zork.bak $ Bonus: Structure the code so that the copy code that creates the backup copy is only present in one place in the script; don't duplicate the code. Bonus: Prompt for a file name if none is given on the command line. --) Write this executable script named "34_integer_sorter_v1.sh" Complexity: 3 This scripting is easy; the sorting algorithm is the item most students find difficult to get right. Prompt the user and read three integers from standard input. Your script does not have to check that the three inputs are really integers; you may assume that they are numbers. (However - You cannot assume that the user will enter all three inputs - make sure none of the three inputs are empty.) Output the three integers in order from smallest to largest, in this form: The numbers are, from smallest to largest: 23 156 9823 You probably need to write a correct pseudocode algorithm to do this first, before you write the script to do it. You might also verify your pseudocode with a small C program, first. Hint: The shortest solution is based on the "bubble sort" algorithm. Bubble the largest integer to the top of the list and then repeat for the remaining two numbers. You don't need a loop or array here, since there are only 3 numbers. Bonus: Check that the user entered only three inputs, not four or more. (Hint: If you use four varibles in the shell "read" statement, the fourth variable must be empty; if not, there are more than three words typed as input and you should issue an error message.) --) Write this executable script named "35_integer_sorter_v2.sh" Complexity: 3 Same as above; but, use exactly three command line arguments instead of reading three inputs from standard input. Your script does not have to check that the three arguments are really numbers; you may assume that they are numbers. (However - You cannot assume that there will be three of them - validate the number of arguments!) Generate the same output as the previous problem. --) Write this executable script named "36_integer_sorter_v3.sh" Complexity: 4 Generate the same output as above; but, allow a mixture of command line arguments and prompts for input. For example: If the user supplies three command line arguments, don't prompt for any input; use the command line arguments. If the user supplies only two command line arguments, prompt for and read the missing third integer. If the user supplies only one command line argument, prompt for and read the missing two ingegers. If the user supplies no command line arguments, prompt for and read all three missing integers. Echo the input numbers back to the user before sorting them. Note: Do not duplicate code! Use shell variables to hold the command line arguments (if any). See the optional_arguments_demo and file_sort_self_compare scripts for examples of how to do this. --) Write this executable script named "37_file_ranger.sh" Complexity: 3 Expect two command arguments to be integers establishing a number range. The two integers may be supplied either smallest and largest or largest and smallest. Output the smaller range number and then the larger range number. (e.g. both "5 9" and "9 5" as command line arguments would generate the ordered output "5 9".) Prompt for and read a file name from standard input. Generate a special error message if the user's supplied input is empty (if the user didn't enter any name). Make sure the file is a readable file (not a directory). Count the number of lines in the file. Output the name of the file, the number of lines in the file, and whether or not the number of lines lies within the range numbers given on the command line. $ ./foo 500 10 The range is from 10 to 500 Input: /etc/passwd Yes '/etc/passwd' has 101 lines and is in range 10 to 500. $ ./foo 99 1000 The range is from 99 to 1000 Input: /etc/passwd Yes '/etc/passwd' has 101 lines and is in range 99 to 1000. $ ./foo 9 3 The range is from 3 to 9 Input: /etc/passwd No '/etc/passwd' has 101 lines, which is greater than 9. $ ./foo 1000 99 The range is from 99 to 1000 Input: /bin ./foo: unknown input: '/bin' is a directory, not a file. $ ./foo 99 1000 The range is from 99 to 1000 Input: ./foo: missing file name; nothing done --) Write this executable script named "38_file_ranger_v2.sh" Complexity: 3 As above; but, reverse where you obtain the integers and the pathname. Prompt for and read the two numbers from standard input. Get the pathname as the only command line argument. Generate the same output as the previous problem. As with all scripts, you must validate all user input, whether coming from the command line arguments or when read from standard input. The script should give good error messages if any input is missing or empty. Bonus: Make sure the user enters exactly two numbers, not more. --) Write this executable script named "39_file_duplicator.sh" Complexity: 3 This script requires one command line argument, the output file. This output file must be a non-existent file name; the pathname must not already exist - the script must verify this. Prompt for and read a second file name, the input file. This input file must be readable and be a non-zero size. (The script must verify this.) Prompt for and read a number. This number will be the desired number of lines in the output file. This user input must not be empty; but, you don't have to validate that it is an integer (which is hard to do in a shell script). ! if test "$1" = "`echo $1 | tr -c -d 0-9`" ! if test -z "`echo -n $1 | tr -d 0-9`" ! if test -z "`echo -n $1 | sed 's/[0-9]*//'`" ! if test `expr length "$1"` -eq `expr match "$1" '[0-9]*'` ! ! I can't explain why the second and third suggestions work (in all of ! test, ash, ksh, bash) without the "-n". Bug in all those versions ! of "test", bug in the documentation, or bug in my understanding. Copy the input file to the output file. (Make sure the copy works.) Loop: While the output file contains less than the desired number of lines, append a copy of the input file to the output file. When the loop finishes, print the number of lines in the output file. $ ./foo /etc/passwd ./foo: error: output file '/etc/passwd' already exists; nothing done. $ ./foo newout.txt Enter input file: /etc/passwd Enter desired number of lines: 300 Output file 'newout.txt' contains 303 lines. Hint: The loop is a WHILE loop. Test the count of the number of lines in the output file against the desired number and exit the loop when the count is greater than or equal to the desired number. )---------- test4-unix.txt -- Thu 2002-Mar- 7 03:56:41 ======================================== What to Study for the Fourth Test (Unix) ======================================== -IAN! idallen@ncf.ca The Fourth Test (Unix) takes place in the last part of your Lab period in Week 10 of the course. (See the Course Outline; but, note that the test takes place in your lab, not in your lecture.) The test is 100 minutes long. It is worth 30% of your term mark. The test contains material from the relevant parts of Chapters 1-5, 10, 11, and 14 in the Linux text, with emphasis on Chapters 10 and 11. (Omit Chapters 6 and 9 and material not covered in the practice questions and weekly lectures.) I am not testing you on your mastery of the VI editor; if you can't use the editor, you won't be able to write and test the scripts during the test. The test also covers web page "Notes" files for Weeks 1-9 and the Chapter notes files and Floppix labs mentioned in those weekly files. You need to know how to write the Script Header given in the Week 7-9 notes. Comments, exit codes, and programming style (as given in the Week 7-9 notes) will be considered when your scripts are marked. What you must bring to the test: 1. You will need your Floppix diskettes (and your backup copies). You must know how to boot Floppix with DHCP networking enabled. 2. You will need a pencil (only a pencil) to write in answers to multiple-choice questions on a mark-sense sheet. 3. Your name, EMail, and web password must be registered with me at least 24 hours before the test begins, using the registration form on the course web page. If you are not registered, you won't have a userid on the Linux test machine and won't be able to log in to that machine to answer questions. You will work on Floppix, on ACADAIX, and on a Linux test machine. You must know how to use telnet on Floppix to telnet to ACADAIX and to the Linux test machine. No Windows access is permitted during the test. The address of the Linux test machine has been given out in the announcements news group. Make sure you can login to the machine. Knowing how to switch among the three Floppix consoles will save you time, since you can have one console logged into ACADAIX and another logged in to the Linux test machine at the same time. The test is open-book, open-note, and open-terminal. You may use any online or written resources available to you; but, you may not consult with or communicate with anyone except the instructor in any way during the test period. Multiple-choice answers will be written down on mark sense forms. Some questions will require you to execute commands on any of the three machines and save the results into files. You may be asked to download and upload files using FTP between any of the three machines. You will not have enough time to try every multiple-choice question on the computer or to look up all the terms in the textbook. You must know your material before the test. The test will appear to be "too long" if you try to type in all the questions to see the output. Thinking about the correct answer will be faster. Most of the multiple-choice questions can be answered quickly, without a computer, from the knowledge you have acquired in lectures and by doing the weekly readings and homework material. Your Floppix lab Exercises and Questions & Answers, and the practice questions for each chapter, are very important. Strategy: Answer the questions you know, first. Come back later to the questions for which you require a computer. For more strategy on writing Unix tests see: test_preparation.txt )---------- lessCodeIsBetter.txt -- Thu 2002-Mar- 7 03:59:29 ======================== Less Code is Better Code ======================== -IAN! idallen@ncf.ca The ability to cut-and-paste code easily has produced a whole generation of programmers who write too much code. You must minimize the amount of code you write, to minimize the effort needed to maintain the code. If you duplicate code, you more than duplicate the amount of effort needed to maintain it. Here are some examples of "too much code". -------------------------------------------------------------------------- You want to use a temporary file name. WRONG WAY: grep foo /etc/passwd >"/tmp/$USER.$$.txt" grep bar /etc/passwd >>"/tmp/$USER.$$.txt" diff /etc/passwd "/tmp/$USER.$$.txt" rm "/tmp/$USER.$$.txt" - changing the file name requires changing four lines of code! - you have three chances to spell the name wrong ! This implies (very) naive use of full-screen "vi". Surely not... RIGHT WAY: tmpfile="/tmp/$USER.$$.txt" grep foo /etc/passwd >"$tmpfile" grep bar /etc/passwd >>"$tmpfile" diff /etc/passwd "$tmpfile" rm "$tmpfile" - the file name is set in one place - changing it is easy - spelling the name wrong in some of the commands is impossible ! I agree with your "set it one place", but not with your arguments. ! If I change the name, it'll be with a single substitute, something like: ! ---,.s/".*/beldar$$.txt/ ! so if all the script's attempts to use the name are identical before my ! change, they'll be identical after. As for "impossible", I can as well write: ! diff /etc/passwd "$tmofile" ! as whatever error you had in mind to prevent. -------------------------------------------------------------------------- You want to process an input file using the above commands. The file might come from a command line argument or by prompting and reading. WRONG WAY: if [ "$#" -ge 1 ]; then tmpfile="/tmp/$USER.$$.txt" grep foo "$1" >"$tmpfile" grep bar "$1" >>"$tmpfile" diff "$1" "$tmpfile" rm "$tmpfile" else echo 1>&2 "Enter a file name:" read filename tmpfile="/tmp/$USER.$$.txt" grep foo "$filename" >"$tmpfile" grep bar "$filename" >>"$tmpfile" diff "$filename" "$tmpfile" rm "$tmpfile" fi - you have written the entire algorithm twice over, once for the command line argument case and once for the prompted input case - changing the algorithm requires changing the code in two places RIGHT WAY: if [ "$#" -ge 1 ]; then filename="$1" else echo 1>&2 "Enter a file name:" read filename fi tmpfile="/tmp/$USER.$$.txt" grep foo "$filename" >"$tmpfile" grep bar "$filename" >>"$tmpfile" diff "$filename" "$tmpfile" rm "$tmpfile" - the duplicated code has been removed - the algorithm is implemented in only one place, not two - the obtaining of the file name is now separate from the algorithm - the algorithm is generalized and parametrized to work no matter how you might obtain a file name Don't duplicate code. Parametrize the code and set the parameters using conditional logic ahead of the code. Use functions, if your programming language allows them. Less code is better code! )---------- abbreviations.txt -- Tue 2002-Mar-19 12:29:58 ================================== Marking Code comment abbreviations ================================== -IAN! idallen@ncf.ca I use these abbreviations when marking your shell scripts: com - missing or incorrect comments mis - missing code required by specification nca - no "cat" needed; superfluous use of "cat" command (not necessary) ncs - no command substitution $(...) needed; use command directly nnc - not necessary (probably correct; but, should be left out) nop - not optional; this argument is not optional; use arg not [arg] obv - obvious comments; comments add nothing to what is already in the code quo - quoting is incorrect (including missing quotes) sim - simplify; code is too complex syn - syntax is incorrect (see the man page) unk - unknown purpose (code, comment, or syntax) If you don't understand why I made the comment, come see me. "Less code is better code" )---------- test4-questions-A.txt -- Tue 2002-Mar-19 13:36:06 ====================================== DAT2330 Test Four A - Unix - Questions ====================================== -IAN! idallen@ncf.ca You will need to be running Floppix with networking to do this test. You must have network access to use telnet and ftp to reach the machines mentioned in this test. Your directory test_four is where you must put your scripts and your script output if you want the instructor to mark them. This directory will be created by your instructor before you start your test. Your scripts do not have to create this directory; you may assume it exists any time a script specification requires you to use it. Only scripts and output found under directory test_four on the Test Machine will be marked. Only correctly-spelled script and file names will be marked. Your scripts must start with a valid Script Header, as given in class and in the Week 7 notes. Scripts without a valid header will be penalized. Scripts containing useful comments will earn bonus marks, to a maximum of five marks per script. Problem A - Marks: 50 On the Test Machine: Write an executable shell script named whodater.sh that will do the following actions (in the order given below): 1. [Marks: 6] Verify that there are exactly two command line arguments; otherwise, exit with an error message. (i.e. Exit if there are not two arguments.) 2. [Marks: 2] Put the first argument into a variable named mydir. 3. [Marks: 2] Put the second argument into a variable named myfile. 4. [Marks: 5] Verify that the path named by the mydir variable is a directory; otherwise, exit with an error message. (i.e. Exit if the path is not a directory.) 5. [Marks: 5] Verify that the path named by the mydir variable is writable; otherwise, exit with an error message. (i.e. Exit if the directory is not writable.) 6. [Marks: 5] Verify that the path named by the mydir variable is searchable; otherwise, exit with an error message. (i.e. Exit if the directory is not searchable.) 7. [Marks: 2] Put the file pathname $mydir/$myfile into a variable named newfile. (Do not create this file yet.) 8. [Marks: 5] Verify that the file named by the newfile variable is zero size or nonexistent; otherwise, exit with an error message. (i.e. Exit if non-empty.) 9. [Marks: 3] Put the current date into the file named by the newfile variable. Exit with a non-zero exit status if the command fails. 10. [Marks: 3] Put the pathname for a temporary file into a variable named tmp. Use the process ID of the current process in the pathname. The pathname of the temporary file should be under the /tmp directory. (Do not create the file yet.) 11. [Marks: 4] Put a sorted list of online users into the temporary file named by the tmp variable. Exit with a non- zero exit status if the command fails. 12. [Marks: 3] Append the contents of the temporary file (the list of online users) to the file that contains the date (created in a previous step). Exit with a non-zero exit status if the command fails. 13. [Marks: 3] Also append this exact sentence to the same file: The number of users is XXX. where XXX is the number of lines in the temporary file. 14. [Marks: 2] Remove the temporary file. Problem B - Marks: 58 On the Test Machine: Write an executable shell script named compiler.sh that will do the following actions (in the order given below): 1. [Marks: 6] Verify that there is zero or one command line argument; otherwise, exit with an error message. (i.e. Exit if there is more than one argument.) 2. [Marks: 7] If there are no command line arguments, prompt the user to enter the missing file name and read the name from standard input. Put the file name (either the command line argument or what the user entered from standard input) into the variable myprog for use in the rest of the script. 3. [Marks: 2] Display this exact sentence: You are processing file 'XXX'. where XXX is the contents of the myprog variable containing the file name that the user specified. Use the exact punctuation. 4. [Marks: 5] Verify that the path named by the myprog variable is a file; otherwise, exit with an error message. (i.e. Exit if the path is not a file.) 5. [Marks: 5] Verify that the file named by the myprog variable is readable; otherwise, exit with an error message. (i.e. Exit if the file is not readable.) 6. [Marks: 5] Verify that the file named by the myprog variable is not empty; otherwise, exit with an error message. (i.e. Exit if the file is empty.) 7. [Marks: 9] If the file named by the myprog variable contains the 7-character string cout << then compile the file using the C++ language compiler. Turn on all warnings during the compile. Name the compiled output program newcpp. Save any error or warning messages that may be generated on standard error by the C++ compiler into a file named warnings.out in the current directory. Save the exit status of the C++ compiler in a variable named cstatus. 8. [Marks: 9] If the file named by the myprog variable does not contain the 9-character string