Week 11-B Notes for DAT2330 - Ian Allen Supplement to Text, outlining the material done in class and lab. Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM![*] ([*] Read The Fine Manual) ==================================== MVS - O/S 390 - Job Control Language ==================================== -IAN! idallen@ncf.ca Spring 2002 These notes are largely based on lectures by Harold Smith: - DAT2330 - Harold Smith Class - Wed Jan 26, 2000 - DAT2330 - Harold Smith Class - Thu Jan 27, 2000 Do the homework questions; they relate to the tests - don't study the JCL book - too much stuff Components of an O/S used to test a program - critical C.L.G. sequence - see p.44 - memorize diagram on p.44 (edit, compile, link, go) - review these components wrt. testing a job: (boot, shell, task mgmt, memory, i/o, security, net, utilities, help) - need boot to load kernel - don't load all of O/S at boot; load KERNEL or NUCLEUS - might not include all device drivers in kernel (might not have choice!) - need shell to start up the compiler - compiler calls on O/S (data mgmt) to read the data - need memory management (virtual memory - bigger than RAM) - need some CPU cycles to run the job - task manager will schedule me - device drivers let O/S access physical devices - need linkage editor (linker) to link your code with already existing code - after linking, load the program itself Review the O/S terms wrt. testing a program: kernel, batch vs. interactive processing, spooling, batch files (procs) Last class: sample JCL, flow diagrammed and analysed - JCL checklist - Basics (see course outline for details) - see the course outline for the list of things we learn each week - see also the file MVSweekly.txt under the JCL Notes button Using pre-defined JCL in your shop - a procedure (PROC) is used for long and/or complex JCL - the system's COBOL CLG PROC is maybe 50-100 lines of JCL! - use existing PROC - don't re-invent it! - see the end of Chapter 3 for an example of system PROC usage --------------------------------------------------- Review of JCL homework: Find the errors in TESTPROG --------------------------------------------------- - UPPER CASE ONLY for JCL - learn JCL by looking at existing JCL - teach yourself syntax by finding the most errors - style: - put output statements first (output never has instream data) - put input statements last (input DD may have instream data) - put instream data very last in the step, when possible - order of DD doesn't matter to JES (except when over-riding a PROC) JOB statement for TESTPROGRAM: - remove leading blanks - shorten job name to 8 chars or less - remove the blanks after commas - put two positional parameters first - move CLASS=B after the positional parameters - MESGLVL spelled wrong: MSGLEVEL - Flow Diagram: title top of diagram TESTPROG Step ONE - first EXEC statement: PGM=IDCAMS - shorten step name to 8 characters - Style: use meaningful 8-character job and step names, not "STEP1" - Flow Diagram: make this a solid box containing MAKETAPE, IDCAMS - four DD statements (after EXEC, before next EXEC in next step): - SYSPRINT DD - label the flow arrow going out to print spool queue A - some queues may route print to screen, not to printer - label SYSPRINT print sheet "error messages" from the IDCAMS program - "IDC AMS": Access Method Services (IDC - IBM prefix) - IN DD DSN=TESTDATA - error: DDName "IN" is used twice in same job step - this is a "NEW" dataset; therefore it must be output - rename it to OUT (look at the IDCAMS input stream REPRO line) - label arrow OUT going to TAPE - unit name "TAPE" is an MVS sysgen parameter (SYSSQ, CART, etc.) - this unit name is different for each MVS shop; ask your operator - next continuation line must start with slash slash blank - indent continuation over to line up with third column - but stop before column 16! - SYSIN DD * - this is called "in-stream data" - input for this dataset comes from JCL input stream - put instream data last, so that it doesn't space the rest of the JCL down the page - use oval shape on flow diagram, label incoming arrow SYSIN - this SYSIN data is for IDCAMS, telling it what to do - label the oval "IDCAMS control statements" - these are equivalent to DOS flags, switches, UNIX options - having to put all these flags on the EXEC statement would make the statement too long, so we use a separate SYSIN data stream to control IDCAMS - IN DD * - also instream data - draw an oval, label it as "test data" - look down to next step (next EXEC) to see how data will be used - looks like test data for input to our COBOL program - Use "/*" to end instream data; must start in first column - JES *always* splits jobs on //XXX JOB - you cannot ever have //XXX JOB in instream data Step TWO - second EXEC statement: PGM=IDCAMS - change comment to have //* instead of // - remove blanks around "=" - FlowDiagram: use a square, solid box titled PRTTAPE, IDCAMS - four DD statements: - SYSPRINT DD - use this label on flow arrow going out to print spool queue A - remove blanks around "=" - label this print output as "error messages" from the IDCAMS program - WRITE DD - another output to SYSOUT=A - draw a second flow arrow going out to another print spool queue A - remove blanks around "=" - label as "tape records" (look at what IDCAMS is copying) - SYSIN DD * - make sure there is a space before the asterisk - draw an oval for instream data "IDCAMS control statements" - REPRO statemnt (IDCAMS) cannot start in column 1 - missing /* must follow REPRO line - names must match DDnames: change IN to READ and OUT to WRITE (or, change DDname WRITE to OUT and DDname READ to IN) - READ DD - label on arrow coming from passed tape from previous step - DSN=TEST DATA - no blanks, 8 characters for each "qualified" name - qualified names can contain dots to separate components - similar to slashes in a directory hierarchy - maximum length of 5 times 8 chars in qualified names, e.g. DSN=ABCDE.FGH.IJKL.MNOPQ.RSTUVWXY - usually need to have qualified name - ask the sysadmin - simple name not permitted in system catalog - Windows limits full pathnames to 260 bytes from root to end! - Unix limits pathnames to 1024 bytes - DISP=(,PASS) must have leading comma to select positional default for first parameter; but, prefer to use (OLD,PASS) for clarity - see chapter 4 for rules on commas in parameters Step THREE - third EXEC statement: PROC=COBOLCLG - change comment to have //* instead of // - remove blanks around "=" - Flow Diagram: PROC's get a dotted box, not a solid box - Remember: "dotted boxes need dotted DD names" on their arrows - dotted DD names are of form //stepname.DDname DD ... - stepname is the name of the step inside the PROC - DDname is the DDname used in that step (inside the PROC) - label this dotted box: COMLKTST COBOLCLG - four DD statements: - one DD statement is for the COBOL step (the compiler step of C.L.G.) - three DD statements are for the GO step (my running program) - nothing needed for the LINK step (use the defaults in the PROC) - COBOL.SYSIN DD * - COBOL is the stepname of the step in the PROC "COBOLCLG" - this DD statement is followed by instream COBOL program, ending in /* - IMPORTANT: - DD statements for a PROC must appear in same order as the steps in the PROC (find out the order from the manual or a sysadmin) - oval for instream data with arrow into PROC box labelled COBOL.SYSIN - oval itself is labelled "COBOL source program" - GO.PRTOUT DD - GO is the name of the "go" step in the PROC - GO step is where my COBOL program runs and accesses its DDNames - how do you find out the GO step name? - try something and look at JCL errors, or ask someone, or look it up - fix the continuation (can't continue after "="; must split after comma) - put OUTLIM=2000 together on the next line, or move 2000 up - arrow leading to print spool queue B - label the arrow with its DDname: GO.PRTOUT - label print queue "test output" - GO.DSKOUT DD - this must be an output dataset because of DISP=(NEW,....) - arrow leading out from proc labelled GO.DSKOUT - destination is a disk dataset, name the dataset TESTOUT - fix the unbalanced parentheses on SPACE - cannot indent continuation cards past column 16 - must stop before column 72 (71 is last usable column) - GO.TPIN DD - arrow from TESTDATA tape leading into PROC dotted box - label the arrow with the DDname GO.TPIN - this TPIN is the input data for the running program - fix spelling: CATALOG --> CATLG ----------- Study Guide ----------- - I'm not teaching everything you need to know - I concentrate on things that are difficult to understand - you learn the basic things (e.g. continuation rule) Go through each job stream, identifying parameters (look them up) - VOL=SER=127536 - a serial number never changes for a tape - DSN may give it a new name - COBOLCLG is three programs - Compile, Link, Go (see p.44) - why specify OUTLIM=2000 ? - because nobody is watching the spool queue and if your program loops and prints a million lines, MVS will print it and send you the bill! - SPACE= - allocate a bit more than you need - you control growth of the dataset - more details on the SPACE parameter in later examples --------------------------------------- MVS JCL Example 1 and attached homework --------------------------------------- - the written instructions are to test a program (usually COBOL) - how do we actually code the JCL (as on the job)? - we will assume many defaults - few things to learn to start out - this will be a real, working job - look up each of the parameters in the text as you first use it - start with a flow diagram (not formal; only to help visualize) - flow diagrams are never required, never marked Step One - Flow Diagram ----------------------- - EXEC PROC=COBOLCLG step - what are we being asked to do? - develop JCL to compile and run a program - start with title on flow diagram - JOB name is 8 chars - read chapter 4 as you code (I won't teach it) - flow diagram: put the JOB title on top: CBLTSTE1 - Your Name - how many boxes for compile, link, and run COBOL? - C.L.G. requires three programs: 1. compiler, 2. linker, 3. my program - refer to "the CLG process" flow diagram Figure 3.5 on p.44 - three programs means three steps (three EXEC PGM= needed) - but I can do all three steps in one EXEC, using a single system PROC - use a "catalogued procedure" (a "PROC") named: COBOLCLG - system procedures come from SYS1.PROCLIB (remember this name) - wc can call in this already-written procedure (similar to a "macro") - use one box for this CGL step, a dotted box in the flow diagram - EXEC PGM= uses solid boxes in diagram; EXEC PROC= uses dotted boxes - find out the three step names (C.L.G.) used inside the PROC - the three C.L.G. step names are given in the JCL specifications: Compile: COMP; Link: LINK; Go: TEST - these are the names of the three hidden steps inside the PROC - our DDnames must be given *in this step order* in our own JCL - all the COMP DD statements must come before the TEST DD statements when we code the JCL, because step COMP happens before step TEST - different coding for procedure (PROC=) and program (PGM=) steps - a PROC has most of the JCL; but, not all the info about the job - e.g. missing OUTLIM for printing? - e.g. what kind of test data is needed? - e.g. where does output go? - e.g. where does the input come from? - our JCL can supply DD statements for information not given in the PROC - e.g. from where does the COBOL source come? - our JCL can specify information that over-rides default assumptions made by the PROC; we can over-ride DD statements used in the PROC - e.g. we could over-ride the default destination for the compile listing - flow diagram for a PROC: create a dotted box - we get to choose our own step name for the dotted box: CLG - put the name of the PROC in the middle of the box: COBOLCLG - order of DDnames we specify in our JCL must match order of steps in the PROC: - the internal PROC steps are given as: COMP, LINK, TEST - draw instream COBOL source program as an oval labelled "cobol source" - program is on SYSIN DDname, but SYSIN to which step in the PROC? - must qualify DD name; must know names of three steps inside the PROC - we *must* use the step name chosen by PROC writer - how to find out what DDname was used by the PROC writer? - Ask the sysadmin; or, just try something and look at the errors! - JCL error messages will tell us if our guess doesn't match PROC - arrow labelled COMP.SYSIN goes from source oval to main PROC box - COMP is the step name inside the PROC - SYSIN is the DDname used by the compiler in the COMP step - you must supply compiler input using the expected DDname - HINT: cross out each sentence in the specification as you do it on a test - anything left over will need doing - nothing more to do for the COMP step in the PROC - the only item missing from the PROC was the SYSIN DDname used by the compiler to read the source program that needs compiling - the rest of JCL for the COMP step is supplied by the PROC - there is nothing at all to do for the LINK step in the PROC - the default JCL used for linking already in the PROC will work for us - later, we will see how to modify the JCL of the link step - the "Go" step of CLG is named "TEST" according to the specifications - the specifications indicate that it will require four DDnames - the DDnames are: INDATA, UPDTMST, LSTIN, PRINTOP - code this in the JCL as: TEST.INDATA, TEST.UPDTMST, TEST.LSTIN, etc. - draw another oval for instream Test Data, label "test data" - where does test data come from? From DD name INDATA in step TEST - DD name is TEST.INDATA on arrow from oval to dotted box - "dotted boxes require dotted DDnames on their arrows" - why not test programs with live data on datasets? - maybe it might damage the real data (could back it up?) - live data may not test all my conditions - may only test the IF part of IF ELSE - design the test data to exercise all parts of the program - an "update" step is both input and output - dataset must already exist for an update step - actual dataset has a name (DSN=) of TESTMAST - find the DDname needed by examining the actual COBOL source code - find the DDname used at the end of the COBOL "SELECT" statement - SELECT MAST-FILE ASSIGN TO UPDTMST <= UPDTMST is the DDname FD MAST-FILE <== MAST-FILE is the internal, COBOL name RECORD CONTAINS 100 CHARACTERS BLOCK CONTAINS 10 RECORDS - we must specify a dataset for the DDname UPDTMST in step TEST - arrow labelled as TEST.UPDTMST points to TESTMAST disk symbol - why not put DSname (actual name of dataset) right in the source? - would hard-wire the program to that dataset - have to rewrite the program to change datasets - want one level of indirection: DDname to dataset name - another level of redirection gives COBOL a MAST-FILE name - the DSname may have to change if we port to another o/s - insulate COBOL from change in DDname - tape dataset name is PRODLIST for the LSTIN DDname - draw a tape symbol and label it with the name PRODLIST - we must specify the DDname LSTIN, used in step TEST - draw an arrow labelled TEST.LSTIN - print some output - DDname is PRINTOP in step TEST - flow diagram arrow is labelled TEST.PRINTOP - Remember: dotted boxes (PROCs) require dotted DDnames - write the DDnames for PROC's with dots, as . Step Two - Flow Diagram ----------------------- - EXEC PGM=VERDATA step - this is an actual program, not a cataloged procedure (not a PROC) - draw a solid (not dotted) box for a PGM= step - label the step box "VERIFY" and put the program name VERDATA inside - draw an input arrow from disk symbol and the dataset labelled TESTMAST that was created in the previous step (step one) - the arrow needs a DDname (the VERDATA program expects to use it); but, the input DDname is not given in the specifications! - you cannot invent your own DDName - you must use the DDname expected by the given program (VERDATA) - go "fish" for name of DDname for program VERDATA - program probably in SYS1.LINKLIB - find the documentation - old production programs may not have source code any more - you can always: run the job and see what run-time errors you get - make a *guess* at INPUT or SYSIN for the DDname, label the arrow - when VERDATA runs, it will look for its correct DDnames and will issue run-time error messages if it cannot find them - also need the missing the DDname for print output (to be held for TSO) - the output DDname is also not given in the specifications! - guess at OUTPUT or SYSPRINT for the DDname, label the arrow - run-time errors will tell us what DDname the program (VERDATA) expects - VERDATA must actually execute before the error will be found - end of Flow Diagram Coding the JCL for Example 1 ---------------------------- - UPPER CASE ONLY (run your JCL through a Unix tr command!) - use fixed-width (monospace) font for JCL so that blanks are clear - UNIX VI does this automatically (text only) - choose a Courier or Terminal font, if using a word processor - don't go past column 71 - use ":set ruler" in Unix VIM (not available in VI) - no extraneous blanks (blanks start comments!) - continuation cards must start in cols 4-16 - blanks start comments after third column - be careful Start of Step One ----------------- First is the JOB statement. It always has two positional parameters (account code and programmer name) and is followed by optional keyword parameters. We always select and code a CLASS= parameter. //CBLTSTE1 JOB 1000EX,'MY NAME',CLASS=B - you get to choose a meaningful 8-character job name for this job - watch out for positional parameters on JOB statement - keyword parms follow two positional parameters on JOB card - use specified account code - job account billing is often done even if it's your own computer - the MVS shop tracks which team uses more/less CPU Picking a CLASS= for the JOB card - not a random choice - you must calculate this correctly - CLASS: indicates resources required by the *entire* job (all steps) - similar to priority, but technically not a priority - classes are defined by each MVS shop; check with your sysadmin (classes will be different for each homework and on tests) - see the JCL examples for sample job classes - complex jobs may take longer to set up and run - use letter closest to 'A' for best turn around time - check out which class satisfies job, closest to A - operator may kill or extend job when it runs out of resources - automated software may do this, not a human - Depending on the resource limits of your job classes, chosing a job CLASS is determined by ether the *maximum* or the *total* resources used by your job. Job CLASS is determined by: - the sum *total* number of print (SYSOUT) lines, in all steps, no matter whether the lines are sent to a real printer or back to you via to TSO - the sum *total* amount of computer time used, in all steps - the *maximum* number of tape drives being used *simultaneously* in any *one* step - NOTE: *not* the *total* number, only the *maximum* number - NOTE: PROCs containing multiple steps are seen as multiple steps; three tape drives used in a step that calls a PROC may still only count as "one tape drive per step" if the PROC only uses one tape drive at a time in each of the internal steps in the PROC - a job that uses ten tapes may be a "one-tape-drive" job, if the job has ten steps and only uses one tape drive in each step - the *maximum* memory used in any one step NOTE: *not* the sum total of all memory, only the *maximum* memory //CLG EXEC PROC=COBOLCLG - you get to choose a meaningful 8-character step name for this step - don't call it an unhelpful name such as STEP1 or FIRST - a catalogued PROC must be from the system library (use the exact name) - remember the name: SYS1.PROCLIB is where PROC's come from - check your flow diagram to count DD statements (one for each arrow) - we count 5 DD statements (two output, three input) - all three internal proc step names were given in the specification: COBOL, LINK, and TEST (compile step, link step, go step) - if these weren't given, you'd have to fish for them by asking around or by running your JCL and looking at the PROC listing when it aborts - bad guesses for step names generate JCL errors, not run-time errors - what is the output of the COBOL step inside the COBOLCLG proc? - what output does a compiler produce (refer to CLG diagram p.44)? - should I put in my JCL a DD statement to handle this output? - no, that's handled by the PROC (which is why we use a PROC!) - don't specify DD statements for things done correctly by the PROC - what is the ordering of DD statements for MVS? - order of DD statements when using a PROC must be in PROC step order - put COBOL DD statements before LINK statements before TEST statements - the DD order *within* each step doesn't matter to MVS - however, it matters to instructors who want large instream data *last* (Guideline: put output DD before input DD, with instream data very last) //COMP.SYSIN DD * ...your actual COBOL source program goes here (see example p.76)... ...this is fed to the COBOL compiler via the SYSIN DDname... /* - This instream data DD has to come first, because it's part of the COMP step. All the COMP DD statements must precede all the TEST DD statements; because, that's the order of the steps in the COBOLCLG PROC we're using. - you must specify DD statements in the order of the steps in the PROC - this is an MVS rule - remember the /* delimiter to end the COBOL (and all other) instream data - see also the COBOL "COBUCLG" example at the end of Chapter 5 (p.76) The order of the DD statements within the TEST step can follow the guideline of "output first, input last, instream data very last". (MVS doesn't care; it's just good style to put instream data at the bottom where it doesn't break up your JCL as much.) //TEST.PRINTOP DD SYSOUT=A - the SYSOUT parameter always means an output DD (output spool queue) - SYSOUT= classes are different from job CLASS= classes - don't get JOB CLASS= classes confused with SYSOUT= classes - both classes are defined locally for each MVS shop (and each test) - use the classes given and defined for this example - pick the SYSOUT class that matches the output and the requirements - how much output? special forms? hold the output or not? //TEST.UPDTMST DD DSN=TESTMAST,DISP=(OLD,PASS) - output DD: this is an "update" - write or read/write an existing dataset - OLD is used to read or update an existing dataset in place - MOD is only used to "extend the length of a sequential dataset" - MOD is never used in this course - DISP has positional parameters (but it, itself, is not positional) - second parm says what to do with dataset if step succeeds - third says what to do if step fails (default is usually okay; leave it out) - leave it out; but, no trailing commas allowed (see Chapter 4) - DISP=(OLD,KEEP) would have the dataset deallocated - a tape would be dismounted and put back in storage - next step would request the same tape and the operator would have to go back to the store room and get it again (annoying) - don't leave the tape mounted unnecessarily; but, don't dismount it if it will be needed in the next step - this abbreviated JCL only works because dataset is *cataloged* - no need to specify more detail to find the dataset because the catalog has the VOL, SER, UNIT, etc. (see p.121 for list) - no need to type in parameters that are retained in catalog - even offline (dismounted; in storage) datasets are catalogued - existing dataset has DCB information stored in its label - no need to specify output DCB information here either - see p.121 for list of DD parameters stored in dataset label - cataloged datasets are easy to find and use! //TEST.LSTIN DD DSN=PRODLIST,DISP=(OLD,KEEP) - input DD: dataset is cataloged, making it easy to find - never need to specify DCB parameters for an input dataset - the DCB info is in the existing dataset label (see p.121) - what do we do with the dataset after the step ends - question doesn't say what to do - safe thing is to keep it - if you code "PASS" at end of a job it means "KEEP" //TEST.INDATA DD * ...lots of test data appears in the input stream here... ...this data is fed to the test program via DDname INDATA... /* - Instream data goes *last* in the JCL stream, so that it doesn't separate the rest of the JCL. (This is a style guideline; not an MVS JCL rule.) End of Step One - Beginning of Step Two --------------------------------------- //VERIFY EXEC PGM=VERDATA //* Note: I am guessing both of the DDnames; they were not provided. //SYSPRINT DD SYSOUT=D //SYSIN DD DSN=TESTMAST,DISP=(OLD,DELETE) // - you get to choose a meaningful 8-character step name for this step - check your flow diagram to count DD statements (one for each arrow) - we count 2 DD statements (one output, one input) - prefer output statements to go first (style issue; not a JCL rule) - we are guessing at the DDnames used by the VERDATA program - the DDnames weren't given in the specifications - program run-time error messages will tell us the real names - IMPORTANT: JES JCL processor cannot detect incorrect DDnames - only when you *run* the program will it detect the missing DDname - see example specification for classes of SYSOUT spool queues - we are asked to hold the printe output for TSO viewing - choose the sysout hold class: SYSOUT=D - do NOT choose class A, or job will end up on paper! - the specification says to DELETE the dataset at the end of the job - this is the last job step, we code "DELETE" - the null statement "//" marks the end of this JOB To get a listing of just the JCL from this file, use the Unix command: $ grep '^/' thisfile Do the "MVS JCL Example 1" homework question; we will review it next class.