All topics
16 / 60
JCLintermediate

Generation Data Groups (GDGs)

Understand how GDGs manage a rolling series of related datasets (like daily extracts) by relative generation number instead of hardcoded names.

A GDG is a series of datasets that share a name and are referred to by position rather than by date. Today's extract, yesterday's, the one before.

A GDG is like a stack of dated newspapers on a porch where you never refer to the actual date printed on the page — you just say 'today's paper' (+1) when adding a new one or 'yesterday's paper' (-1 or 0) when picking one up to read, and the oldest paper at the bottom eventually gets recycled once the stack hits its limit.

Key Concepts

1
    PROD.DAILY.EXTRACT.G0041V00     <- the real names
    PROD.DAILY.EXTRACT.G0042V00
    PROD.DAILY.EXTRACT.G0043V00     <- most recent
2
The JCL never uses those. It uses relative numbers:
3
    (0)    the most recent generation
    (-1)   the one before that
    (+1)   a NEW generation, created by this step
4
Which is the whole point. No job hardcodes a date. It says "create the newest" or "read the most recent", and the catalog keeps track.
5
The two directions, and the mistake people make.
    //OUTFILE DD DSN=PROD.DAILY.EXTRACT(+1),
    //           DISP=(NEW,CATLG,DELETE)        writing today's
6
    //INFILE  DD DSN=PROD.DAILY.EXTRACT(0),
    //           DISP=SHR                        reading the latest
7
Using (+1) to read finds nothing, because that generation does not exist yet. Using (0) to write fails, because it already does.
(+1)(0)
8
A subtlety worth knowing. Within a single job, (+1) refers to the same generation throughout — it is resolved once at job start. So a later step can read back what an earlier step wrote by also saying (+1), and it does not become (0) until the job ends.
(+1)(0)
9
Defining the base, with IDCAMS:
10
    DEFINE GDG (NAME(PROD.DAILY.EXTRACT) -
                LIMIT(7) -
                SCRATCH -
                NOEMPTY)
11
    LIMIT      how many generations to keep
    SCRATCH    physically delete the ones that roll off
    NOSCRATCH  just remove them from the index; the data stays
    EMPTY      when the limit is hit, uncatalog ALL generations
    NOEMPTY    uncatalog only the oldest
12
SCRATCH against NOSCRATCH is the recurring confusion. With NOSCRATCH the rolled-off dataset still occupies disk and still exists — it is simply no longer part of the group. Shops that use NOSCRATCH without a cleanup process accumulate orphaned datasets for years.
SCRATCHNOSCRATCH
13
EMPTY is the more dangerous one. It uncatalogs every generation when the limit is reached, not just the oldest, so the whole history disappears from the group at once. It is rarely what anyone wants and it is occasionally what somebody typed.
EMPTY
14
What the interviewer is probing.1. "What problem does a GDG solve?" Probing: the purpose. Stalls: "Naming." Moves up: automatic versioning of a periodically created dataset, so yesterday's file is (-1) without anyone renaming anything.
15
2. "What does (+1) mean inside a job?" Probing: the semantics. Stalls: "The next one." Moves up: the new generation being created by this job — and it stays (+1) for the whole job, only becoming (0) afterwards.
16
3. "What do LIMIT and the SCRATCH option control?" Probing: the operational detail. Stalls: "The size." Moves up: how many generations are kept and whether rolled-off ones are deleted or merely uncataloged — which is how history is retained or lost.
17
4. "What is the classic restart problem with GDGs?" Probing: experience. Stalls: "None." Moves up: a rerun after a partial failure can create another generation, shifting the relative numbers so later steps read the wrong file.
Use cases:gdggeneration-data-groupsdataset-managementbatch-processing