--------------------------- Week 11-A Notes for DAT2330 --------------------------- -Ian! D. Allen - idallen@idallen.ca Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM! (Read The Fine Manual) These notes are largely based on lectures by Harold Smith: - DAT2330 - Harold Smith Class - Wed Jan 26, 2000 ----------------------- Reading the IBM Manuals ----------------------- ----------- Writing JCL ----------- - commas at end of third field continue lines to next card - column 4-16 Rules for Continuation: see IBM Reference Manual - embedded blanks either cause JCL errors or cause trailing items to be takes as "comments" and ignored - BE CAREFUL WITH BLANKS! - un-learn your English habit of putting blanks after commas! -------------------------- Blocking Control: BLKSIZE= -------------------------- Blocking is necessary to avoid wasting space on output device: - bit->byte(character)->field->record->dataset(file)->database - Can put multiple logical records into one physical disk block. - Gaps between blocks on disk/tape may be several hundred bytes long - poor usage of space - MVS lets you adjust block size - logical record vs. physical record - blocking factor (number of grouped logical records) - how many logical records per physical block? - Exercise: if your record is 80 bytes, how many fit in a 4K block? 4096/80 = 51 records per block (integer truncate; do not round!) 51 records use up 4080 bytes of the 4K (16 bytes wasted per block) - if your record is 100 bytes, how many fit in a 4K block? 4096/100 = 40 records 40 records use up 4000 bytes of the 4K (96 bytes wasted per block) - some devices have a block size other than 4K (e.g. ANSI tapes) - adjust your calculations accordingly ----------------------------- Fragmentation Control: SPACE= ----------------------------- Supose I know that a file will grow to some size; can't reserve space in Windows - files fragment all over the disk platters. MVS will let me reserve size; if my job overflows the given space, we can specify how much new space to allocate. We usually care about allocation, since that's why we use MVS. (You really care about allocation when you're allocating tens or hundreds of millions of data items!) SPACE= ; how to allocate space on output device "Extents" - a dataset is allocated a "primary" extent of disk space and some number of "secondary" extents for "overflow" - you specify the size of the primary and secondary extents in the JCL when you first create the dataset; the sizes are remembered from then onward - number of secondary extents is limited to 15 unless SMS in in use ------------------------------------- IDCAMS - how to copy a dataset (file) ------------------------------------- - the MVS IDCAMS utility can copy a file (among other things) - IDCAMS has half a dozen operations! - IDCAMS is a "DEBE - Did Everything But Eat" program - Unix would have separate commands for the functions in IDCAMS - need a way to pass program options to programs such as IDCAMS - Windows: can specify moves and copies depending on shift and ctrl keys! - copy/move/shortcut behaviour also depends on destination! - is that intuitive and user-friendly? - Unix: uses command-line flags to influence program behaviour - DOS: uses comand-line switches to do same thing - OS/390: IDCAMS reads its options from a file (dataset) - that "file" can be read from the JCL input stream itself, similar to a Unix shell "Here Document", hence "instream" data - the /* indicates the end of the instream data (like EOF in Unix) - the SYSIN DDname specifies the dataset that IDCAMS will read to get its control statements that tell it what to do - most IBM-written utilities read from DDname "SYSIN" - REPRO - specifies what thing IDCAMS will do (of the many) - REPRO means "copy" in IBM-speak jargon - the REPRO line must be indented one or more spaces - this REPRO line is *not* a line of JCL; it is *input* to IDCAMS - REPRO specifies the JCL DDnames of the input file and the output file, instead of using the file names themselves - you can choose your own REPRO DDnames as long as they match your JCL DDnames (follow the syntax and length restrictions) - I always use INFILE(IN) and OUTFILE(OUT) because they are simple - IDCAMS produces status messages on DDname SYSPRINT - most IBM-written utilities produce status output on DDname "SYSPRINT" - we always need 4 DDnames for a REPRO operation: SYSIN, SYSPRINT, IN, OUT - every I/O unit must have a DD statement in MVS Flow Diagrams ------------- - you may find it helpful to draw "flow diagrams" before coding the JCL - flow diagrams use boxes to show each step (each EXEC) in the JOB - they show how the steps work together - one step's output may become next step's input - you use arrows to show each input and output stream for each step - show the DDnames of each input and output by putting the DDnames on the arrows connecting the boxes to the storage devices (disk, tape) Flow Diagram for PAYROLL - a Three Step Job (no PROC used) ---------------------------------------------------------- Example: Flow Diagram (payrollflow.jpg) of "PAYROLL" JOB (11347),SMITH - the JOB name is PAYROLL - Box: put job name PAYROLL as the title of this flow diagram - this JOB has three steps (three EXEC statements) that will be represtented by three boxes on this diagram page - draw a solid box for each of the three EXEC steps: - programs (PGM=) get solid line boxes - cataloged procedures (PROC=) get dotted boxes (see PROGTST, below) - compare files payrollflow.jpg and progtstflow.jpg - arrows between items in diagram specify the I/O done by this step - arrows connect boxes (programs) to dataset symbols (disk, tape, output) - direction of the arrow indicates Input (arrow leads toward box) or Output (arrow leads away from box) - labels on the arrows correspond to the JCL DDnames of those datasets (the DDnames must match the names inside the program used in this step) - the first of the three EXEC steps in this job is named CRETAP - draw a solid line box for this job step - put the step name CRETAP in the top left of the box - put the name of the program being executed (PGM=) in this step (IDCAMS) in the middle of the box - Arrows: specify the I/O done by this step - four DD statements mean four arrows, one for each DDname - two input arrows labelled with DDnames SYSIN and IN - two output arrows labelled with DDnames SYSPRINT and OUT - DDname SYSPRINT: - very common DDname for printed output - SYSOUT=A controls the type of printing (spool) queue - windows has very primitive spool control (in system tray) - MVS JCL can specify to decollate and burst carbon-copy fan-fold - for IDCAMS status and error messages - goes to SYSOUT in the "A" output class - what output class "A" means is defined locally - see your sysadmin for the list of classes - DDname OUT: - has DISP=(NEW,PASS) therefore it is an output DDname - the dataset name is given as DSN=PAYDATA - in the real world, the name will likely be much more complex, requiring your account name and other details, e.g. ER93754.GRP8824.PRJ92.PAYDATA - UNIT=SYSDA, UNIT=SYSSEQ, UNIT=TAPE - defined for each shop - what type of output medium (disk or tape) - DDname SYSIN: - IDCAMS reads control records from SYSIN to know what to do - e.g. to choose the REPRO function (a copy operation) - instructions to IDCAMS are short and usualy embedded directly into the JCL as "instream" data using the "DD *" syntax - Oval: an oval in a Flow Diagram indicates instream data - label this oval as "control statements" - DDname IN: - more "instream" data included right in the JCL stream - instream data has a restrictive, punch-card-like format - Oval: this data is also represented as an oval - label it as "payroll data" (you have to know what the job step does to label its data) - the second of the three EXEC steps in this job is named PRTCHKS - Box: put the step name in box along with name of what program or procedure is being executed in this step (PAYROLL1, here) - Arrows: specify the I/O done by this step - three DD statements mean three arrows - one input arrow labeled with DDname PAYIN - two output arrows labeled with DDnames SYSPRINT and CHKOUT - DDname SYSPRINT: - goes to SYSOUT - is it the paycheques being printed(?) - no - the next DD statement would seem to be the cheques - SYSPRINT DDname is probably for errors or a log file; not cheques - can re-use same DDname in different steps (different programs) - DDname CHKOUT: - SYSOUT=B - goes to the output spool queue named "B" - this is not the usual "A" queue with cheap paper; queue "B" is probably a special printer with special paper for the cheques - you must consult your local print queue definitions to find out how your MVS system was configured at SYSGEN time - DDname PAYIN: - DD statement mentions as input DSN=PAYDATA, the same dataset that was output in the previous (CRETAP) step - each DSN must be unique (so the names are ususally much longer!) - the third of the three EXEC steps in this job is named PRTREG - perhaps this is to print the payroll register created in step 1? - Arrows: specify the I/O done by this step - three DD statements mean three arrows - one input arrow labelled with DDname PAYIN - can use the same DDname in different steps; because, different steps run different programs and the programs may or may not use the same DDnames internally - two output arrows labelled with DDnames SYSPRINT and REGOUT - DDname SYSPRINT: - this would appear to be a status or error message listing? - can re-use same DDname in different steps, since DD statements apply only to the immediately preceding EXEC program - DDname REGOUT: - probably the payroll registry, sent to yet another spool queue "C" - a limit is placed on the number of print lines - if you generated a million lines of output, you would get it, and you would be billed for it! - DDname PAYIN: - this is, again, the same dataset generated by step 1 - we read it a second time, using a different program this time - the DISP indicates the dataset exists (OLD) and that we should put it into the system catalog (CATLG) when the step ends - CATLG: you can catalog everything, including offline storage; you can have a pointer to every file on every tape and disk - the end of the JCL is indicated by the "null" statement: // ---------------------------------------------------------- Flow Diagram for "PROGTST" - a "one-step" JOB using a PROC ---------------------------------------------------------- JCL allows "macros" of stored JCL called "Procedures" to be called up from system libraries in EXEC statements using the "PROC=" syntax. The PGM= syntax calls a single executable program; PROC= calls up a canned procedure that may contain many internal program steps. Using pre-defined JCL in your shop: - a procedure (PROC) is used for long and/or complex JCL - the system's COBOL CLG PROC is maybe 50-500 lines of JCL! - use the existing PROC in your shop - don't re-invent it! These procedures ("PROC"s) may contain many internal steps (many internal EXEC statements). You may or may not be told about all the internal steps hidden inside a PROC. You often don't care about many of the internal steps. Example: Flow Diagram (progtstflow.jpg) of "PROGTST" JOB (11348),SMITH In the PROGTST job example, a PROC named "COBCLG" is being called up. This PROC has three internal steps; but, in the particular JCL we are given we can only detect the names of two of the steps: "COBOL" and "GO". Use a dotted box to draw a PROC step. (Use a solid box for a PGM step.) DDnames for PROC steps must also be labelled with the internal step name inside the PROC in which the DDname is used - more on this later. Mnemonic Rule: Dotted PROC boxes need dotted DDnames on the arrows! ------------------- JCL Syntax Practice ------------------- Find the errors in badjcl.txt JOB "TESTPROGRAM" - UPPER CASE ONLY for JCL - learn JCL by looking at existing JCL - teach yourself JCL syntax by finding the errors JOB statement for TESTPROGRAM: - remove leading blanks - shorten job name to 8 chars or less - remove the blanks after commas - put two positional parameters first - move CLASS=B after the positional parameters (positional always first) - MESGLVL spelled wrong: MSGLEVEL - Flow Diagram: title top of diagram TESTPROG Step ONE - first EXEC statement: PGM=IDCAMS - shorten step name to 8 characters - Style: use meaningful 8-character job and step names, not "STEP1" - Flow Diagram: make this a solid box containing MAKETAPE, IDCAMS - four DD statements (after EXEC, before next EXEC in next step): - style: - order of DD in a step doesn't matter (except when over-riding a PROC) - put output statements first (output never has instream data) - put input statements last (input DD may have instream data) - put instream data very last in the step, when possible - SYSPRINT DD - label the flow arrow going out to print spool queue A - some queues may route print to screen, not to printer - label SYSPRINT print sheet "status messages" from the IDCAMS program - "IDC AMS": Access Method Services (IDC - IBM prefix) - IN DD DSN=TESTDATA - error: DDName "IN" is used twice in same job step - this is a "NEW" dataset; therefore it must be output - rename it to OUT (look at the IDCAMS input stream REPRO line) - label arrow OUT going to TAPE - unit name "TAPE" is an MVS sysgen parameter (SYSSQ, CART, etc.) - this unit name is different for each MVS shop; ask your operator - next continuation line must start with slash slash blank - indent continuation over to line up with third column - but stop before column 16! - VOL=SER=127536 is correct syntax - a serial number never changes for a tape - DSN may give it a new name - SYSIN DD * - this is called "in-stream data" - input for this dataset comes from JCL input stream - put instream data last, so that it doesn't space the rest of the JCL down the page - use oval shape on flow diagram, label incoming arrow SYSIN - this SYSIN data is for IDCAMS, telling it what to do - label the oval "IDCAMS control statements" - these are equivalent to DOS flags, switches, UNIX options - having to put all these flags on the EXEC statement would make the statement too long, so we use a separate SYSIN data stream to control IDCAMS - IN DD * - also instream data - draw an oval, label it as "test data" - look down to next step (next EXEC) to see how data will be used - looks like test data for input to our COBOL program - Use "/*" to end instream data; must start in first column - JES *always* splits jobs on //XXX JOB - you cannot ever have //XXX JOB in instream data Step TWO - second EXEC statement: PGM=IDCAMS - change comment to have //* instead of // - remove blanks around "=" - FlowDiagram: use a square, solid box titled PRTTAPE, IDCAMS - four DD statements: - SYSPRINT DD - use this label on flow arrow going out to print spool queue A - remove blanks around "=" - label this print output as "status messages" from the IDCAMS program - WRITE DD - another output to SYSOUT=A - draw a second flow arrow going out to another print spool queue A - remove blanks around "=" - label as "tape records" (look at what IDCAMS is copying) - SYSIN DD * - make sure there is a space before the asterisk - draw an oval for instream data "IDCAMS control statements" - REPRO statemnt (IDCAMS) cannot start in column 1 - missing /* must follow REPRO line - names must match DDnames: change IN to READ and OUT to WRITE (or, change DDname WRITE to OUT and DDname READ to IN) - READ DD - label on arrow coming from passed tape from previous step - DSN=TEST DATA - no blanks, 8 characters for each "qualified" name - qualified names can contain dots to separate components - similar to slashes in a directory hierarchy - maximum length of 5 times 8 chars in qualified names, e.g. DSN=ABCDE.FGH.IJKL.MNOPQ.RSTUVWXY - usually need to have qualified name - ask the sysadmin - simple name not permitted in system catalog - Windows limits full pathnames to 260 bytes from root to end! - Unix limits pathnames to 1024 bytes - DISP=(,PASS) must have leading comma to select positional default for first parameter; but, prefer to use (OLD,PASS) for clarity - see chapter 4 for rules on commas in parameters Step THREE - third EXEC statement: PROC=COBOLCLG - procedure COBOLCLG is three programs - Compile, Link, Go - change comment to have //* instead of // - remove blanks around "=" - Flow Diagram: PROC's get a dotted box, not a solid box - Remember: "dotted boxes need dotted DD names" on their arrows - dotted DD names are of form //stepname.DDname DD ... - stepname is the name of the internal step inside the PROC - DDname is the DDname used in the program executed in that step - label this dotted box: COMLKTST COBOLCLG - four DD statements: - one DD statement is for the COBOL step (the compiler step of C.L.G.) - three DD statements are for the GO step (my running program) - nothing needed for the LINK step (use the defaults in the PROC) - COBOL.SYSIN DD * - COBOL is the stepname of the step in the PROC "COBOLCLG" - this DD statement is followed by instream COBOL program, ending in /* - IMPORTANT: - DD statements for a PROC must appear in same order as the steps in the PROC (find out the order from the manual or a sysadmin) - oval for instream data with arrow into PROC box labelled COBOL.SYSIN - oval itself is labelled "COBOL source program" - GO.PRTOUT DD - GO is the name of the "go" step in the PROC - GO step is where my COBOL program runs and accesses its DDNames - how do you find out the GO step name? - try something and look at JCL errors, or ask someone, or look it up - fix the continuation (can't continue after "="; must split after comma) - put OUTLIM=2000 together on the next line, or move 2000 up - why specify OUTLIM=2000 ? - because nobody is watching the spool queue and if your program loops and prints a million lines, MVS will print it and send your boss the bill! - arrow leading to print spool queue B - label the arrow with its DDname: GO.PRTOUT - label print queue "test output" - GO.DSKOUT DD - this must be an output dataset because of DISP=(NEW,....) - arrow leading out from proc labelled GO.DSKOUT - destination is a disk dataset, name the dataset TESTOUT - fix the unbalanced parentheses on SPACE - you control space growth requirements of the dataset - allocate a bit more space than you need - more details on the SPACE parameter in later examples - cannot indent continuation cards past column 16 - must stop before column 72 (71 is last usable column) - GO.TPIN DD - arrow from TESTDATA tape leading into PROC dotted box - label the arrow with the DDname GO.TPIN - this TPIN is the input data for the running program - fix spelling: CATALOG --> CATLG