------------------------- Week 10 Notes for DAT2330 ------------------------- -Ian! D. Allen - idallen@idallen.ca Remember - knowing how to find out the answer is more important than memorizing the answer. Learn to fish! RTFM! (Read The Fine Manual) ===================================== OS/390 MVS JCL - Job Control Language ===================================== ------------------------------------- Textbook, Manuals, Online Web Manuals ------------------------------------- No purchased textbook is required for the MVS JCL section of this course. (You can buy the $125 JCL textbook if/when you get a job using JCL.) IBM has given us permission to use their online manuals and BookServer: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/library Type in "V2R10.0 MVS JCL" as the search term and you will find two online manuals: OS/390 V2R10.0 MVS JCL User's Guide IEA1B540 OS/390 V2R10.0 MVS JCL Reference IEA1B640 The "book" icon leads to online HTML versions of the manual. The stylized "A" icon leads to Adobe PDF versions (about 2-3MB each). These two PDF manuals are also available in compressed archive format on the course Notes page (IBM_JCL_Manuals_pdf.zip and IBM_JCL_Manuals_pdf.tar.gz). The online HTML manuals are usable and have nice working cross-links; but, they have formatting problems, especially with some of the figures containing captures of output with long lines. The PDF versions of the manual (326 pages and 700 pages!) look much better. ---------------- Learning MVS JCL ---------------- Only read enough from the manual to do the exercises and homework! Don't read the whole manual (and especially don't print it all)! The IBM manuals are reference material; not tutorials. Note which things are important for this course, and which are not. If you must print, print only a few selected pages. See the file weekly.txt (under Notes) for the JCL statements and parameters that we will be learning each week. Look up the given words (e.g. DSORG, LRECL, etc.) in the indexes of the IBM manuals and concentrate on learning the basics of these terms. Read and understand the JCL job streams given in the notes and learn to imitate them. Do the homework questions each week; they relate to the test questions you will be given. Go through each sample job stream, identifying parameters that you don't recognize and look them up in the IBM Reference Manual index. I'm not teaching in class everything you need to know; I concentrate on things that are difficult to understand. Don't worry about terms in the manuals that are not covered in the lectures, labs, and homework; there is lots to learn "on the job". ------------ Introduction ------------ These notes are largely based on lectures by Harold Smith: - DAT2330 - Harold Smith Class - Thu Jan 20, 2000 MVS vs OS/390 - MVS ("Multiple Virtual Storage") is the old name dating back to 1970's - OS/390 is a newer name for MVS (was OS/360 in 1960 then OS/370 in 1970) - but there never was an OS/380 - z/OS is now the new name for OS/390 - JCL dates from an era where machines were expensive and people were cheap - MVS is still the O/S of the future for big mainframes that need to deal with large amounts of input/output (e.g. governments, insurance companies, mutual funds, etc.) - no other O/S will handle printing (and re-printing) millions of lines The complexity of MVS JCL is necessary - you get very fine control over things not possible in other O/S - detailed (and thus complex) control over resource allocaton - multi-user: ability to bill to particular programmers/users - can bill for input/output as well as CPU and memory used - detailed cost recovery (even for companies that own their own systems) - special input/output requirements - you can manage millions of lines of input/output - can set priority relative to other jobs on the system - priority of CPU usage, printer usage, etc. - overall job cost is often relative to priority MVS is a batch system, not interactive - you submit files of JCL to the system and wait - cannot "watch" the JCL execute; you only get to see the results after the whole job is done - a runaway program might consume large amounts of resources - e.g. suppose it loops for hours and bankrupts my boss' account - solution: specify a parameter to EXEC to limit cpu time - all these resource limit parameters make the JCL complex - no concept of "standard input" or "standard output" - programmer must specify *all* I/O units for *all* programs in the JCL - JCL "DD" statements connect the program I/O units to I/O devices - without any "DD" statements, the output from a program goes nowhere! MVS introduced Virtual Storage - memory was very very expensive; even big machines had only a few megabytes - Virtual Storage was a way to run more programs than would fit in available RAM - could run a 5MB program in 2MB, with 3MB stored temporarily on disk - different parts of programs would "swap" places from disk to memory - programs are divided into equal-sized "pages" (4KB each for IBM) - the currently executing "pages" of the program are in memory - the parts of the program that are not in RAM will "page" to disk - bringing missing parts of the program into memory is a "Page Fault" - MS Windows bug: loses track of pages - invalid page fault - Windows swap file name: win386.swp - the swap file in MVS is named the "PDS" - Page Data Set Application Programmers (you) vs System Programmers vs Operators - you (Application Programmers) write applications - you write business programs in C++ or COBOL or Java - the System Programmers SYSGEN'd MVS and maintain it for you - often write in low-level IBM Assembly Language (for speed) - NOTE: in Windows and Unix course we teach you some system programming In MVS we do not attempt to teach you any system programming; not installation of MVS, nor tuning MVS, nor debugging MVS - MVS system Operators watch the system constantly, responding to special requests from all the running programs - put paper in printer; fetch tape; cancel job; adjust special forms; burst pages (split at perforations and stack); decollate (separate multi-part forms into individual forms), etc. ------------------------------------- The Compile Link and Go (CLG) process ------------------------------------- - study well the critical Compile, Link-Edit, GO (C.L.G.) sequence - know the inputs/outputs of each step - we will learn to code the CLG process under MVS - use extensive JCL resource limits and specifications for each step - you should be somewhat familiar with this process - how much did your program development environment hide? - MVS "PROCs" (canned JCL procedures) hide some of it; but, you have to understand the process to code JCL for it - MVS program development cycle: edit, compile, link, and GO; repeat - this task will make up most of your future MVS job - Why do we have libraries of object code that need linking? Could we supply *all* the source and eliminate the linkage editor? - compiling *all* that extra code is expensive (takes too long!) - compiled subroutines are already translated, ready to link - we want to re-use good code written by other people - use existing subroutines - don't rewrite or reinvent the wheel Summary: 1) Compile source code into object code (machine code) - some compilers have libraries of source code (COBOL, PL/1, etc.) - e.g. C compiler uses #include to bring in source code 2) Link-edit the compiled object code with system subroutine libraries (libraries are also object code) into an executable load module 3) GO: load and run the executable load module O/S components used for "Compile, Link, Go": - a "bootstrap" loader to first load the O/S - IBM calls it IPL, not "boot" - mainframes are not rebooted very often! - a job entry subsystem (JES) that finds and runs programs based on the JCL you submit (similar to the role of the Unix shell) - e.g. find the COBOL compiler and attach datasets to it - memory management (virtual memory) to allocate memory for the compiler - O/S 390 can monitor and restrict memory usage - task manager will schedule CPU cycles for your job - O/S 390 can monitor and restrict CPU usage - compiler calls on O/S to do I/O to read the program to compile - device drivers let O/S access physical devices - your JCL calls on the linkage editor (linker) to link your compiled code with already existing code in system libraries - after linking, your JCL asks JES to load the linked program itself and execute it ------------ MVS Job Flow ------------ - what is it like to be an MVS programmer? - see MVS Job Flow diagram ("MVS Job Flow" on course home page) - Example: update an existing COBOL program - sit at a PC (emulating a 3270 terminal) and connect to MVS via TSO/ISPF (Time-Sharing Option, Interactive Structured Productivity Facility) - edit COBOL program on screen (interactive; but, not graphical!) - prepare JCL and maybe the COBOL source "in stream" - send job JCL from TSO to JES (Job Entry Subsystem) - JES is doorway from interactive to batch world of MVS - JES checks the JCL and queues it based on resources and priority - your job goes into an input queue - you wait milliseconds or hours for the resources you requested to become available and allocated to you - your JCL may include a predefined "PROC" (JCL Procedure) - system's stored JCL procedures come from dataset SYS1.PROCLIB - you can "pull a PROC" and merge your JCL with it - you only specify the essential things that differ from the PROC - less code is better code! - see example progtstjcl.txt for a one-step job using a PROC - JES loads your program into an Application Program Region (APR) - virtual storage space - you can pull in system programs from dataset SYS1.LINKLIB - your program can access tape and disk (datasets) - your output is not sent directly to the printer, it is "spooled" (sent) to an output queue to wait for an available output device - "SPOOL" queue (Simultaneous Peripheral Operations On Line) - JES decides when resources are available to print your ouput - your output may be routed back to your TSO screen for viewing MVS Job Flow Summary (see numbers in diagram MVS_Job_Flow.gif): 1) Edit your source and JCL file on-screen via TSO/ISPF 2) Submit the JCL file from TSO to the Job Entry Subsystem (JES) - your JCL will usually "pull" (invoke) a system JCL "PROC" from SYS1.PROCLIB to request the traditional Compile/Link/Go (CLG) - JES puts your job into the appropriate priority input queue 3) When your job is selected for execution by JES, the CLG PROC pulls the compiler, linkage editor from system library SYS1.LINKLIB - the final "GO" step of the PROC executes my compiled program 4) all output goes to the output spool queues; you may direct your job to print the output or send it back up to your TSO screen -------------------- IBM Mainframe Jargon -------------------- - a dataset: a file - SYSGEN: system generation, installation, and tailoring (naming things) - a very complex process done *ONCE* when first installing OS/390 - IPL: Initial Program Load - booting the computer with the O/S - became a verb: "you need to re-IPL MVS to install the changes" - JES: Job Entry Subsystem (JES2 and JES3) - a program that schedules and runs all the other programs - TSO: Time Sharing Option - a MVS batch program to which you can connect interactive terminals - this is how you prepare and submit programs to JES - SPF and ISPF: Integrated Structured Programming Facility - the suite of TSO programs that let you edit and submit jobs to the mainframe downstairs - in core: in main storage (memory) - kernel or nucleus: the part of the O/S that is always in memory - the absolutely necessary part of the O/S - may not include all the code to manage all the devices (device drivers) - DASD: direct access storage device (hard disk) - IBM didn't like the term "random access devices" - "random" is a poor business word, says IBM - call them "direct access" instead - "hard disk" term is not used because mainframes only had hard drives - IBM didn't like the term "floppy disk" - "floppy" isn't a good business word, says IBM - call them "diskettes" instead - DSORG: DataSet ORGanization - how data is stored and accessed on the storage media (disks/tapes) - disks are slow; a poor choice of data layout can make a job run many orders of magnitude slower! - for this course know "PS": physical sequential (IBM has many others, e.g. VSAM, ISAM, partitioned, etc.) - DSORG not well supported by O/S in Windows or Unix (mostly sequential) - EBCDIC vs ASCII: MVS uses the EBCDIC character set! - ASCII and EBCDIC files have no printable characters in common - SMS: Storage Management Subsystem - added (extra cost!) program product extension to manage storage - if SMS is installed, it affects how the JCL is coded - may allow you to have more "defaults" and specify less - e.g. with SMS you can say BLKSIZE=0 to get the "best" blocking factor - we will assume SMS is *NOT* available and we will code everything - SPOOL: "Simultaneous Peripheral Operations On Line" - "spool queues" are the queues holding your job output - became a verb, meaning "to send output", e.g. "spool your print to queue A" - burst: split fan-fold paper at perforations and stack the individual sheets - decollate: separate multi-part fan-fold forms (with carbon paper between!) into individual sheets - PROC: a "procedure" - a set of already-written JCL statements that can be copied ("pulled") into your JCL - similar to C++ "include" files; but, more powerful - "to pull": to access or invoke, e.g. "pull a PROC" or "pull the compiler" -------------------- Basic JCL Statements -------------------- Readings in "IBM MVS JCL Reference": 1.1 JCL Statements: Only these three: JOB, EXEC, DD Three basic JCL statements are all we cover in this course: - JOB statement is always first; it starts the job stream - has the accounting information (who gets billed) - has the overall job resource information (priority, CPU limit, etc.) - usually only one JOB card per file - EXEC statement indicates a new "step" in the job - EXEC (a step) is an execution of a single program - a job may have many EXEC and run many programs (like a shell script) - a CLG job needs at least three steps for the three programs used - DD statements attach I/O devices to the program named by EXEC - the identification name in the first column (to left of keyword "DD") is called the Data Definition name (DDname); e.g. in //SYSPRINT DD SYSOUT=A the DDname is "SYSPRINT". - MVS programs do not open real file names; they open DDnames instead and depend on the JCL to connect the DDname to a real dataset (file) name - DD names are chosen by the programmers who wrote the programs; you will find them mentioned in the program source code: - e.g. fopen("SYSPRINT","w") will match //SYSPRINT DD SYSOUT=A - thus, the DDnames in the JCL *must match what is in the program* - the JCL connects the name used in the program to a *real* dataset e.g. the name SYSPRINT is connected to "SYSOUT=A" (a printer queue) - each part of a DDname is limited to 8 characters; but, you may often have two parts, e.g. GO.SYSIN, COBOL.SYSPRINT, etc. Summary: - one JOB card - one EXEC card per program being executed ("steps") - one or more DD cards to attach I/O units to each program ---------- JCL Syntax ---------- Readings in "IBM MVS JCL Reference": 3.0 Chapter 3. Format of Statements 3.1 JCL Statement Fields: All syntax descriptions. In Figure 3-1, only: Comment, DD, Delimiter, EXEC, JOB, Null 3.4 Continuing Statements 3.4.1 Continuing JCL Statements 3.4.1.1 Continuing the Parameter Field 3.4.2 Examples of Continued Statements - Example 1 and 2 only 4.0 Chapter 4. Syntax of Parameters 4.1 Notation Used to Show Syntax 4.2 Character Sets 4.3 Syntax Notes Summary - maximum 71 characters of UPPER CASE ONLY input per 80-character input line - JCL lines always begin with // (even continuation lines) - three column format: //NAME OPERATION PARAMETER - fourth column and beyond is ignored (is a comment field) - columns are separated by blanks - accidentally putting a blank after a comma may create a fourth column and everything after the blank will be ignored! - when entering a parameter that contains blanks, use single quotes - *only* single quotes allowed - all JCL identifiers and dataset name components are MAX 8 characters each - dataset names can have multiple 8-char components separated by periods, e.g. PROJ234.REGION23.MYDATA.BIN - JCL statements may have both positional and keyword parameters - positional parameters must be coded in order, before all keyword parameters - keyword parameters (KEYWORD=value) may be coded in any order - a parameter may contain sub-parameters in parentheses, e.g. MSGLEVEL=(1,0) DCB=(BLKSIZE=1024,RECFM=FB) - depending on the parameter, the sub-paramters may be positional or keyword - if there is only one parameter in the parentheses, omit the parentheses - you may omit leading or embedded positional parameters and leave the comma, e.g. TIME=(,10) SYSOUT=(A,,GRNL) - if you omit trailing positional pararameters, remove trailing commas, e.g. TIME=(1) or TIME=1 not TIME=(1,) [WRONG!] - to continue a line of JCL, split it after a comma and start the next card in postions 4 through 16 (inclusive), e.g. //MYJOB JOB 123456,'IAN D ALLEN', // CLASS=X,MSGLEVEL=(0,1),PRTY=5 (See section 3.4.1.1 in the IBM Reference Manual for examples.) - in this course we won't be continuing parameters inside quoted strings, or continuing fourth-column comment lines