Week 11 Notes for DAT2330 - Ian Allen Supplement to Text, outlining the material done in class and lab. Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM![*] ([*] Read The Fine Manual) ==================================== MVS - O/S 390 - Job Control Language ==================================== -IAN! idallen@ncf.ca July 2001 These notes are largely based on lectures by Harold Smith: - DAT2330 - Harold Smith Class - Thu Jan 20, 2000 Bring your MVS textbook to all classes and labs - note which things are important, and which are not - how much to study in each the chapter? - enough to do the homework! - see the course outline for the statements and parameters MVS vs OS/390 - MVS - old name dating back to 1970's - OS/390 is new name for MVS (OS/390 merges UNIX with MVS) See the studyGuide.txt file before reading Chapters 1-4 - You must study Chapters 1-4 to learn the precise meanings of the terms as they come up in the JCL - Inroduction (Chapter 1) - MVS Concepts and Vocabulary (Chapter 2) - JCL Within a Job (Chapter 3) - JCL Statement Formats and Rules (Chapter 4) - Don't worry about terms not covered in the lectures, labs, and homework MVS - The SHOCK of JCL (page 1) - book is effective (with help); but, don't read it in detail - JCL dates from an era where machines were expensive and people cheap - MVS is still the O/S of the future for big mainframes Examine the JCL on page 1 - author objects to amount of JCL used to copy a file - not all the complexity is superfluous - MVS has more control - MVS has good reasons for the complexity - you get very fine control - multi-user, need to bill to particular programmers/users - cost recovery - special input/output requirements - detailed (and thus complex) control over resource allocaton - priority relative to other jobs (can't do in windows) - give a good name to the job, e.g. SMITHPAY instead of RT83838 (8 char) - need to name jobs when there are many of them - give a readable job title in quotes - execute a program - might have many parameters on this statement - suppose I submit a program that might not run properly - I'm not watching it (it's a batch job) - it might eat the machine - e.g. suppose it loops for hours and bankrupts my account - Solution: specify a parameter to EXEC to limit cpu time - all these parameters make the JCL look complex - MVS calls files datasets - SYSOUT=A controls the type of printing (spool) queue - windows has very primitive spool control (in system tray) - MVS JCL can specify to decollate and burst carbon-copy fan-fold - DISP=(SHR,PASS) ; can specify whether or not to share the dataset - DISP=(NEW,CATLG) ; you can catalog everything, including offline storage; you can have a pointer to every file on every tape and disk - UNIT=SYSDA, UNIT=SYSSEQ ; what type of output medium (disk or tape) - Blocking (necessary to avoid wasting space on output device): - bit->byte(character)->field->record->dataset(file)->database - Can put multiple logical records into one physical disk block. - Gaps between blocks on disk/tape may be several hundred bytes long - poor usage of space - MVS lets you adjust block size - logical record vs. physical record - blocking factor (number of grouped logical records) - how many logical records per physical block? - if your record is 80 bytes, how many fit in a 4K block? - if your record is 100 bytes, how many fit in a 4K block? - SPACE= ; how to allocate space on output device Supose I know that a file will grow to some size; can't reserve space in windows - files fragment. MVS will let me reserve size; if job overflows, specify how much new space to alocate. We usually care about allocation, since that's why we use MVS. (You really care about allocation when you're allocating tens or hundreds of millions of data items!) - the MVS IDCAMS utility can copy a file (among other things) - IDCAMS has half a dozen operations! - IDCAMS is a "DEBE - Did Everything But Eat" program - Unix would have separate commands for the functions in IDCAMS - need a way to pass program options to programs such as IDCAMS - Windows: can specify moves and copies depending on shift and ctrl keys! - behaviour also depends on destination! - is that intuitive and user-friendly? - Unix: uses command-line flags - DOS: uses comand-line switches - OS/390: IDCAMS reads its options from a file (dataset) - that "file" can be in the JCL stream itself, similar to a Unix shell "Here Document" - the /* indicates the end of the instream data (like EOF in Unix) - the SYSIN DDname specifies the dataset that IDCAMS will read to get its control statements - REPRO - specifies what thing IDCAMS will do (of the many) - must be indented one or more spaces (see text) - uses the JCL DDnames of the input file and the output file - you can pick these DDnames as long as they match your JCL names - commas at end of third field continue lines to next card - Rules for Continuation: see section 4.5 on p.57 - embedded blanks cause JCL errors or cause trailing items to be takes as "comments" and ignored - BE CAREFUL! Text: Virtual Storage (p.17 has an error; fix the error) - program cannot be loaded from disk to disk; must go through memory - part of the program will page to disk if memory fills up - disk is not the same as virtual storage - could run a 5MB program in 2MB, with 3MB on swap - bringing missing pages into memory is a Page Fault - windows loses track of pages - invalid page fault - Windows swap file name: win386.swp - the swap file in MVS is named the: PDS - Page Data Set - PDS is hierarchical: stepped on fast and slow disk IBM Mainframe Jargon - SYSGEN - system generation, installation, and tailoring (naming things) - I/O, CPU, main storage, DASD - direct access storage device, tape - "hard drive" not used because mainframes only had hard drives - 5-platter, 14-inch, one pound arm - joke program to rock the disk - "random access devices" - "random" is a poor word, says IBM - "floppy disk" isn't good, call them "diskettes", says IBM - Application Programmers (you) vs System Programmers vs Operators - may have to ask the System Pgmr who installed MVS about some things - NOTE: in Windows and Unix we teach you some system programming In MVS we do not attempt to teach you any system programming - not installation of MVS, nor tuning, nor debugging - MVS system Operators watch the system constantly, responding to requests from all the JCL - put paper in printer; fetch tape; cancel job; adjust forms, etc. - DSORG - DataSet ORGanization - sequential, direct access, VSAM, partitioned, etc. - not well supported by O/S in Windows or Unix (mostly sequential) - VSAM - virtual storage access method (better than ISAM) - index to file - like a dictionary of where things are on disk - EBCDIC vs ASCII - MVS uses EBCDIC! - ASCII and EBCDIC files have no characters in common - MVS/SP (16MB addr space) vs MVS/XA (2GB addr space) - MVS/ESA (16TB addr space, more than one addr space per pgm, VIO datasets) - SMS (Storage Management Subsystem - extension) - if SMS is installed, it affects how the JCL is coded - may allow you to have more "defaults" and specify less - e.g. with SMS you can say BLKSIZE=0 to get the "best" blocking factor The Compile Link and Go (CLG) process (p.44) - Figure 3.5 - study the C.L.G. diagram: compile, link-edit, execute (GO) - you should be somewhat familiar with this process - how much did your program development environment hide? - MVS "PROCs" hide some of it; but, you have to understand it to code JCL for it - MVS program development cycle: edit, compile, link, and GO; repeat - this task will make up most of your future MVS job - 1) compile source into object code - 2) link object and libraries (subroutine library) into a load module - why have link libraries? - could supply *all* the source and eliminate the linkage editor - don't reinvent the wheel (VSAM access code is a subroutine) - compiling all that extra code is expensive - compiled subroutines are already translated, ready to link - 3) load and run your compiled program - using extensive JCL resource limits and specifications - we will learn to code the CLG process under MVS MVS Job Flow diagram - what is it like to be an MVS programmer? - see MVS Job Flow diagram ("MVS Job Flow" on course home page) - Example: update exiting COBOL programs - sit at a PC (emulating terminal) and connect to MVS via TSO/ISPF (Time-Sharing Option, Interactive Structured Productivity Facility) - edit COBOL program on screen (interactive; but, not graphical!) - prepare JCL and maybe the COBOL source "in stream" - send job JCL from TSO to JES (Job Entry Subsystem) - JES is doorway from interactive to batch world of MVS - JES checks the JCL and queues it based on resources and priority - your job goes in a queue - you wait milliseconds or hours for the resources you requested to become available and allocated to you - your JCL may include a predefined "PROC" (JCL Procedure) - system's stored JCL procedures come from dataset SYS1.PROCLIB - you can "pull a PROC" and merge your JCL with it - you only specify the essential things that differ from the PROC - less code is better code! - JES loads your program into an Application Program Region (APR) - virtual storage space - you can pull in system programs from dataset SYS1.LINKLIB - your program can access tape and disk (datasets) - your output is not sent directly to the printer, it is queued in a "SPOOL" queue (Simultaneous Peripheral Operations On Line) - JES decides when resources are available to print your ouput - your output may be routed back to your TSO screen for viewing - Summary: 0) Edit your source on-screen via TSO 1) pull a JCL "PROC" to do the traditional CLG steps 2) CLG PROC pulls compiler, linkage editor from SYS1.LINKLIB 3) the "GO" step pulls my program off DASD and executes it (need a loader) 4) my output goes to the output spool queues that I have chosen 5) spool queue may lead to printer, or back to the TSO screen Learning JCL - look at job streams and imitate - front of textbook has all the parameters - this index is condensed and very easy to use - JOB statement starts the JOB stream - EXEC indicates a new STEP - a step is an execution of a program - DD statements attach I/O devices to the programs - the name to left of keyword "DD" is the Data Definition name (DDname) - each part of a DDname is limited to 8 characters; but, you may have several parts, e.g. GO.SYSIN, COBOL.SYSPRINT, etc. - DD names are chosen by the programmers who wrote the programs - you will find them mentioned in the program source - e.g. fopen("SYSPRINT","w") Flow Diagrams - you may find it helpful to draw "flow diagrams" before coding the JCL - flow diagrams show each step in the JOB - they show how the steps work together - one step's output may become next step's input - you use arrows to show each input and output stream for each step - show the DDnames of each input and output by putting the DDnames on the arrows between the boxes Example: Flow Diagram of "PAYROLL" JOB (11347),SMITH - (develop flow diagram on blackboard) - the JOB name is PAYROLL - Box: put job name as title of this flow diagram along with programmer name (SMITH) - has three steps (three EXEC statements) that will be three boxes on this page - draw a box for each of the three EXEC steps: - programs (PGM=) get solid line boxes - cataloged procedures (PROC=) get dotted boxes - arrows specify the I/O done by this step - arrows lead to or from datasets (disk, tape) - labels on the arrows correspond to the JCL DDnames of those datasets - the first of the three EXEC steps in this job is named CRETAP - draw a solid line box - put the step name CRETAP in the top left of the box - put the name of the program being executed in this step (IDCAMS) in the middle of the box - Arrows: specify the I/O done by this step - four DD statements mean four arrows - two input arrows (SYSIN, SYSUT1) - two output arrows (SYSPRINT, SYSUT2) - SYSPRINT - very common DDname for printed output - likely for IDCAMS error messages - goes to SYSOUT in the "A" output class - what output class "A" means is defined locally - see your sysadmin for the list of classes - SYSUT2 - has DISP=(NEW,PASS) therefore it is an output DDname - the dataset name is given as DSN=PAYDATA - in the real world, the name will likely be much more complex, requiring your account name and other details - SYSIN - IDCAMS reads control records from SYSIN to know what to do - e.g. to choose REPRO - instructions to IDCAMS are short and usualy embedded directly into the JCL as "instream" data using the "DD *" syntax - Oval: an oval in a Flow Diagram indicates instream data - label this oval as "control statements" - SYSUT1 - more "instream" data - instream data has a restrictive, punch-card-like format - Oval: this data is also represented as an oval - label it as "payroll data" (you have to know what the job step does to label its data) - the second of the three EXEC steps in this job is named PRTCHKS - Box: put the step name in box along with name of what program or procedure is being executed in this step (PAYROLL1, here) - Arrows: specify the I/O done by this step - three DD statements mean three arrows - one input arrow (PAYIN) - two output arrows (SYSPRINT, CHKOUT) - SYSPRINT - goes to SYSOUT - is it the paycheques being printed(?) - no - the next DD statement would seem to be the cheques - SYSPRINT DDname is probably for errors or a log file; not cheques - can re-use same DDname in different steps - CHKOUT - SYSOUT=B - goes to the output spool queue named "B" - this is not the usual "A" queue; it's probably the the cheques being printed on a special printer - you must consult your local print queue definitions to find out how your MVS system is configured - PAYIN - DD statement mentions as input DSN=PAYDATA, the same dataset that was output in the previous (CRETAP) step - each DSN must be unique (so the names are ususally much longer!) - the third of the three EXEC steps in this job is named PRTREG - print the payroll register created in step 1? - Arrows: specify the I/O done by this step - three DD statements mean three arrows - one input arrow (PAYIN) - can re-use same DDname in different steps - two output arrows (SYSPRINT, REGOUT) - SYSPRINT - this would appear to be an error listing? - can re-use same DDname in different steps - REGOUT - probably the payroll registry, sent to yet another spool queue "C" - a limit is placed on the number of print lines - if you generated a million lines of output, you would get it, and you would be billed for it! - PAYIN - this is, again, the same dataset generated by step 1 - we read it a second time - the DISP indicates the dataset exists (OLD) and that we should put it into the system catalog when the step ends - the end of the JCL is indicated by //