z/OS Mainframe TPT RECFM=FB versus LRECL=VB

Tools & Utilities
Enthusiast

z/OS Mainframe TPT RECFM=FB versus LRECL=VB

Hi,

This is a mainframe z/OS TPT version Version 13.10.00.04 question

Running this JCL and TPT script with INDICATORMODE='N' but I'm getting what looks like the indicator bytes at end of file  

//MYUSERIT JOB (TESTT),'UNLOAD',                                       

//            MSGCLASS=V,

//            NOTIFY=&SYSUID

//*

//  JCLLIB ORDER=(MYUSERI.PROCLIB,PROD.PROCLIB)

//*

//JOBLIB  DD DSN=TEST.LOADLIB

//        DD DSN=PROD.LOADLIB

//*

//ALLOC   EXEC PGM=IEFBR14

//SYSPRINT  DD SYSOUT=*

//SYSOUT    DD SYSOUT=*

//DD5       DD DSNAME=MYUSERI.SORT,

//             DISP=(MOD,DELETE,DELETE),

//             UNIT=SYSDA,SPACE=(CYL,(1,2),RLSE)

//*******************************************************************

//*******************************************************************

//JS030   EXEC DWBTBEXP,HLQ=DWBT,MODULE=DXXXXXXX,PARM='-S U'

//PS0010.ATTRFILE  DD DATA,DLM='@@'

 JOBVAR_TDPID = 'TDXX'

,JOBVAR_USERNAME = 'MYUSERI'

@@

//PS0010.SYSIN     DD DATA,DLM='@@'

/*-----------------------------------------------*/

/* COMMENTS                                      */

/*-----------------------------------------------*/

DEFINE JOB UNLOD_ENV

DESCRIPTION 'UNLOAD FROM APP_WORK.CARD TABLE'

(

  /*  DEFINING SCHEMA OF INPUT TABLE             */

  DEFINE SCHEMA CRD_SCHEMA

  DESCRIPTION 'TABLE SCHEMA'

  (

   DATE1               (CHAR(6)

  ,DATE2               (CHAR(10)

  ,COL1                (DECIMAL(4,0)

  ,COL2                (DECIMAL(9,0)

  ,COL3                (DECIMAL(11,0)

  ,COL4                (CHAR(02)

  ,COL5                (DECIMAL(15,2)

  ,COL6                (DECIMAL(15,2)

  ,COL7                (CHAR(02)

  ,COL8                (CHAR(01)

  ,COL9                (CHAR(04)

  ,COL10               (DECIMAL(15,2)

  ,COL11               (DECIMAL(15,2)

  ,COL12               (DECIMAL(15,2)

  ,COL13               (DECIMAL(15,2)

  ,COL14               (DECIMAL(15,2)

  ,COL15               (CHAR(01)

  ,COL16               (CHAR(03)

  ,COL17               (CHAR(05)

  ,COL18               (CHAR(04)

  ,COL19               (DECIMAL(15,2)

  ,COL20               (CHAR(01)

  ,COL21               (CHAR(01)

  ,COL22               (DECIMAL(06,0)

  ,COL23               (DECIMAL(09,0)

  ,COL24               (CHAR(03)

  ,COL25               (DECIMAL(15,2)

  ,COL26               (CHAR(03)

  ,COL27               (DECIMAL(05,0)

  ,COL28               (CHAR(06)

  ,COL29               (DECIMAL(04,0)

  ,COL30               (CHAR(02)

  ,COL31               (CHAR(01)

  ,COL32               (CHAR(04)

  ,COL33               (DECIMAL(18,6)

  ,COL34               (DECIMAL(18,6)

  ,COL35               (DECIMAL(18,6)

  ,COL36               (DECIMAL(18,6)

  ,COL37               (DECIMAL(18,6)

  ,COL38               (DECIMAL(18,6)

  ,COL39               (DECIMAL(18,6)

  ,COL40               (DECIMAL(18,6)

  ,COL41               (DECIMAL(15,2)

  ,COL42               (CHAR(10)

  ,COL43               (DECIMAL(05,0)

  ,COL44               (CHAR(01)

  ,COL45               (CHAR(02)

  ,COL46               (DECIMAL(15,2)

  ,COL47               (CHAR(02)

  ,COL48               (CHAR(04)

  ,COL49               (DECIMAL(15,2)

  ,COL50               (DECIMAL(15,2)

  ,COL51               (DECIMAL(15,2)

  ,COL52               (DECIMAL(15,2)

  ,COL53               (DECIMAL(05,0)

  ,COL54               (CHAR(04)

  ,COL55               (DECIMAL(10,0)

  ,COL56               (DECIMAL(10,0)

  ,COL57               (DECIMAL(10,0)

  ,COL58               (CHAR(12)

  ,COL59               (DECIMAL(10,0)

  ,COL60               (CHAR(15)

  ,COL61               (CHAR(03)

  ,COL62               (CHAR(04)

  ) ;

/* DEFINE PRODUCER OPERATOR    */

 DEFINE OPERATOR EXPORT_OPERATOR()

  DESCRIPTION 'READ TABLE'

  TYPE EXPORT

  SCHEMA CRD_SCHEMA

  ATTRIBUTES

  (     VARCHAR USERNAME= @JOBVAR_USERNAME,

        INTEGER MAXSESSIONS = 4,

        INTEGER MINSESSIONS = 1,

        VARCHAR PRIVATELOGNAME = 'EXP_LOG',

        VARCHAR TDPID = @JOBVAR_TDPID,

        VARCHAR SELECTSTMT='SELECT

   DATE1               (CHAR(6))

  ,DATE2               (CHAR(10))

  ,COL1                (DECIMAL(4,0))

  ,COL2                (DECIMAL(9,0))

  ,COL3                (DECIMAL(11,0))

  ,COL4                (CHAR(02))

  ,COL5                (DECIMAL(15,2))

  ,COL6                (DECIMAL(15,2))

  ,COL7                (CHAR(02))

  ,COL8                (CHAR(01))

  ,COL9                (CHAR(04))

  ,COL10               (DECIMAL(15,2))

  ,COL11               (DECIMAL(15,2))

  ,COL12               (DECIMAL(15,2))

  ,COL13               (DECIMAL(15,2))

  ,COL14               (DECIMAL(15,2))

  ,COL15               (CHAR(01))

  ,COL16               (CHAR(03))

  ,COL17               (CHAR(05))

  ,COL18               (CHAR(04))

  ,COL19               (DECIMAL(15,2))

  ,COL20               (CHAR(01))

  ,COL21               (CHAR(01))

  ,COL22               (DECIMAL(06,0))

  ,COL23               (DECIMAL(09,0))

  ,COL24               (CHAR(03))

  ,COL25               (DECIMAL(15,2))

  ,COL26               (CHAR(03))

  ,COL27               (DECIMAL(05,0))

  ,COL28               (CHAR(06))

  ,COL29               (DECIMAL(04,0))

  ,COL30               (CHAR(02))

  ,COL31               (CHAR(01))

  ,COL32               (CHAR(04))

  ,COL33               (DECIMAL(18,6))

  ,COL34               (DECIMAL(18,6))

  ,COL35               (DECIMAL(18,6))

  ,COL36               (DECIMAL(18,6))

  ,COL37               (DECIMAL(18,6))

  ,COL38               (DECIMAL(18,6))

  ,COL39               (DECIMAL(18,6))

  ,COL40               (DECIMAL(18,6))

  ,COL41               (DECIMAL(15,2))

  ,COL42               (CHAR(10))

  ,COL43               (DECIMAL(05,0))

  ,COL44               (CHAR(01))

  ,COL45               (CHAR(02))

  ,COL46               (DECIMAL(15,2))

  ,COL47               (CHAR(02))

  ,COL48               (CHAR(04))

  ,COL49               (DECIMAL(15,2))

  ,COL50               (DECIMAL(15,2))

  ,COL51               (DECIMAL(15,2))

  ,COL52               (DECIMAL(15,2))

  ,COL53               (DECIMAL(05,0))

  ,COL54               (CHAR(04))

  ,COL55               (DECIMAL(10,0))

  ,COL56               (DECIMAL(10,0))

  ,COL57               (DECIMAL(10,0))

  ,COL58               (CHAR(12))

  ,COL59               (DECIMAL(10,0))

  ,COL60               (CHAR(15))

  ,COL61               (CHAR(03))

  ,COL62               (CHAR(04))

  FROM DB.TABLE

  ) ;

  DEFINE OPERATOR FILE_WRITER()

  DESCRIPTION 'TERADATA PARALLEL TRANSPORTER DATA CONNECTOR OPERATOR'

  TYPE DATACONNECTOR CONSUMER

  SCHEMA CRD_SCHEMA

  ATTRIBUTES

  (

  VARCHAR FILENAME            = 'DD:FILEOUT',

  VARCHAR INDICATORMODE = 'N',

  VARCHAR FORMAT              = 'UNFORMATTED',

  VARCHAR OPENMODE         = 'WRITE'

  );

  STEP EXPORT_TO_FILE

   (

      APPLY TO OPERATOR (FILE_WRITER() [1] )

      SELECT * FROM OPERATOR (EXPORT_OPERATOR() [1] );

   );

);

@@

//*

//FILEOUT  DD DSN=MYUSERI.SORT,

//       DISP=(NEW,CATLG,DELETE),

//       RECFM=FB,BLKSIZE=0,LRECL=383,BUFNO=160,

//       SPACE=(CYL,(25,25),RLSE)

//*

-7----+----8---

***************

TUU    ........

TUU    ........

TUU    ........

TUU    ........

TUU    ........

If I change the output file (FILELOUT) to RECFM=VB the ........ go away at end of the file

Why are the indicator bytes being written at the end when the file is RECFM=FB and INDICATROMODE='N' ?  

-7----+----

***********

TUU       

TUU       

TUU       

TUU       

TUU       

TUU       

11 REPLIES
Enthusiast

Re: z/OS Mainframe TPT RECFM=FB versus LRECL=VB

Sorry, typo....subject should be z/OS Mainframe TPT RECFM=FB versus RECFM=VB 

Teradata Employee

Re: z/OS Mainframe TPT RECFM=FB versus LRECL=VB

Those are not indicator bytes. They might be garbage characters.

We have made several bug fixes in this area.

Please upgrade to the latest version of TPT 13.10 and re-run the job and see if the problem is fixed.

If not, post here and we will investigate.

-- SteveF
Enthusiast

Re: z/OS Mainframe TPT RECFM=FB versus LRECL=VB

What type of bug fixes have you impemented ? We are currently at TPT Version 13.10.00.04 on a Enterprise system used by hundereds of people ...it is not easy to upgrade to latest versionu unless for a very good reason

We are new to TPT and have just started replacing the old generation load/unload utilities. The way I understand this indicator bit....TPT creates 1 bit for each field that is nullable and then 8 fields/8 bits make a byte which is an indicator byte. This looks to be true because the jobs private log shows             

INDICATOR BYTES NEEDED: 8          

EXPECTED RECORD LENGTH: 383        

My schema has 64 coulmns and all are NULLable but none contain NULL values....

Did some experimenting and turned on INDICATORMODE='Y' with RECFM=FB and I'm assuming these are indicator bytes at front of file, like the manual states 

----+----1-

***********

.......2013

In HEX mode they are x'00' which is LOW-VALUES in mainframe world

.......2013

0000000FFFF

00000002013

The value I'm getting at end of record when INDICATORMODE='N' is same x'00'  

TUU    ........

EEE444400000000

344000000000000

And private log still shows

INDICATOR BYTES NEEDED: 8        

EXPECTED RECORD LENGTH: 383      

If you need the logs or a trace let me know and I can send.....

Teradata Employee

Re: z/OS Mainframe TPT RECFM=FB versus LRECL=VB

I believe the latest patch is #15, #16 or #17.

I cannot list all of the bug fixes (too many).

I understand that you are on an enterprise system, but if you have access to a single test system and can try out the latest patch with this scenario, you can determine if the issue has been fixed.

We cannot assist with a version that old (you are at patch #4).

As for the inidcator byte messages you see, the existence of those messages does not imply that your input data must have them. We transfer rows of data from the producer operator (the file reader) to the consumer operator (the operator that loads data to Teradata) in indicator mode. The consumer operator is just reporting the number of bytes it needs to receive internally.

The file writer will not write out the indicator bytes if INDICATORMODE = 'N'.

-- SteveF
Teradata Employee

Re: z/OS Mainframe TPT RECFM=FB versus LRECL=VB

Just checked. Here are the latest patches:

TPT Infrastructure 13.10.00.19

TPT Export Operator 13.10.00.07

TPT Load Operator 13.10.00.05

TPT Stream Operator 13.10.00.09

TPT Update Operator 13.10.00.07

For sure you need to install the infrastructure package.

You then need to install the operator packages based on the operators you use.

(You cannot install just the infrastructure and then re-run your test.)

-- SteveF
Enthusiast

Re: z/OS Mainframe TPT RECFM=FB versus LRECL=VB

Ok let me do some checking with the teradata guys...we may have a test system that i'm not aware of....just seems strange when trying to do an export it puts the ........ at end of each record when indicatormode=n and if i change to indicatormode=y it puts the ........at beginning of each record with RECFM=FB and if I change to RECFM=VB i dont get the ........

-Rick

Teradata Employee

Re: z/OS Mainframe TPT RECFM=FB versus LRECL=VB

Did you try adding up the column sizes by hand?

The operator is reporting 383 bytes.

That 383 might include the indicator bytes.

Which means the actual record length being written out is 375.

With an LRECL of 383, the file writer might be padding with 0x00.

-- SteveF
Enthusiast

Re: z/OS Mainframe TPT RECFM=FB versus LRECL=VB

Yes the LRECL=375. If I change the LRECL to 375 I get error message

FILE_WRITER: TPT19438 pmOpen failed. LRECL does not match length defined by data schema (38)

 So that tells me TPT needs the extra 8 bytes for the indicatormode and it is padding at end of record when INDICATORMODE='N'

 INDICATOR BYTES NEEDED: 8  

 EXPECTED RECORD LENGTH: 383

 To prove this I took one of the examples from the TERADATA samplib (GT95) and ran some different tests with RECFM=FB and RECFM=VB ....RECFM=VB doesn't pad the indicator byte with INDICATORMODE='N' but you still have to add 1 to record length for the indicator byte and add 4 to record length for RECFM=VB, see results below

/**************************************************************/

 /*                                                            */

 /* Copyright 2007-2009, Teradata Corporation.                 */

 /* All Rights Reserved.                                       */

 /*                                                            */

 /* Script Name: GT95.txt                                      */

 /**************************************************************/

 DEFINE JOB UPSERT_EMPLOYEE_TABLE_FROM_FILE

 DESCRIPTION 'UPSERT SAMPLE EMPLOYEE TABLE FROM A FILE'

 (

    DEFINE SCHEMA EMPLOYEE_SCHEMA

    DESCRIPTION 'SAMPLE EMPLOYEE SCHEMA'

    (

       EMP_ID    CHAR(5),

       EMP_NAME  CHAR(10),

       DEPT_NAME CHAR(10)

    );

    DEFINE OPERATOR DDL_OPERATOR()

    DESCRIPTION 'TERADATA PARALLEL TRANSPORTER DDL OPERATOR'

    TYPE DDL

    ATTRIBUTES

    (

       VARCHAR PrivateLogName = 'GT95_ddloper_log',

       VARCHAR TdpId          = @MyTdpId,

       VARCHAR UserName       = @MyUserName,

       VARCHAR AccountID,

       VARCHAR ErrorList      = '3807'

    );

    DEFINE OPERATOR FILE_WRITER()

    DESCRIPTION 'TERADATA PARALLEL TRANSPORTER DATA CONNECTOR OPERATOR'

    TYPE DATACONNECTOR CONSUMER

    SCHEMA EMPLOYEE_SCHEMA

    ATTRIBUTES

    (

       VARCHAR PrivateLogName    = 'GT95_dataconnoper_writer_privatelog',

       VARCHAR FileName          = @GT95_FileName,

       VARCHAR INDICATORMODE     = 'N',

       VARCHAR OpenMode          = 'Write',

       VARCHAR FORMAT            = 'UNFORMATTED'

    );

    DEFINE OPERATOR EXPORT_OPERATOR()

    DESCRIPTION 'TERADATA PARALLEL TRANSPORTER EXPORT OPERATOR'

    TYPE EXPORT

    SCHEMA EMPLOYEE_SCHEMA

    ATTRIBUTES

    (

       VARCHAR PrivateLogName    = 'GT95_exportoper_privatelog',

       INTEGER MaxSessions       =  32,

       INTEGER MinSessions       =  1,

       VARCHAR TdpId             = @MyTdpId,

       VARCHAR UserName          = @MyUserName,

       VARCHAR AccountId,

       VARCHAR SelectStmt        = 'SELECT * FROM TEMP_EMP_TABLE;'

    );

    STEP setup_tables

    (

       APPLY

       ('DROP TABLE TEMP_EMP_TABLE;'),

       ('DROP TABLE GT95_STREAMOPER_ERRTABLE;'),

       ('DROP TABLE GT95_STREAMOPER_LOGTABLE;'),

       ('CREATE TABLE TEMP_EMP_TABLE(EMP_ID CHAR(5), EMP_NAME CHAR(10),

                                     DEPT_NAME CHAR(10));'),

       ('INSERT INTO TEMP_EMP_TABLE(''119'',''RALPH'',''LAW'');'),

       ('INSERT INTO TEMP_EMP_TABLE(''114'',''ZACH'',''SERVICE'');'),

       ('INSERT INTO TEMP_EMP_TABLE(''105'',''ELLEN'',''PARTS'');'),

       ('INSERT INTO TEMP_EMP_TABLE(''110'',''ALICE'',''LAW'');'),

       ('INSERT INTO TEMP_EMP_TABLE(''115'',''GEORGE'',''SERVICE'');'),

       ('INSERT INTO TEMP_EMP_TABLE(''121'',''HENRY'',''PARTS'');'),

       ('INSERT INTO TEMP_EMP_TABLE(''122'',''RAY'',''FINANCE'');'),

       ('INSERT INTO TEMP_EMP_TABLE(''123'',''LAURA'',''PARTS'');'),

       ('INSERT INTO TEMP_EMP_TABLE(''124'',''SHEILA'',''SERVICE'');'),

       ('INSERT INTO TEMP_EMP_TABLE(''125'',''IVAN'',''PARTS'');')

       TO OPERATOR (DDL_OPERATOR () );

    );

    STEP setup_export_to_file

    (

       APPLY TO OPERATOR (FILE_WRITER() Ý1¨ )

       SELECT * FROM OPERATOR (EXPORT_OPERATOR() Ý1¨ );

    );

 );

With INDICATORMODE='Y' RECFM=FB LRECL=26, puts indicator byte at beginning of record 

===================================================================    

 Column Name                    Offset Length Type                      

 ============================== ====== ====== ========================  

 EMP_ID                              0      5 CHAR                      

 EMP_NAME                            5     10 CHAR                      

 DEPT_NAME                          15     10 CHAR                      

 ============================== ====== ====== ========================  

 INDICATOR BYTES NEEDED: 1                                              

 EXPECTED RECORD LENGTH: 26                                             

 .123  LAURA     PARTS

 .124  SHEILA    SERVICE

 .121  HENRY     PARTS

 .125  IVAN      PARTS

 .110  ALICE     LAW  

 .115  GEORGE    SERVICE

 .114  ZACH      SERVICE

 .119  RALPH     LAW  

 .122  RAY       FINANCE

 .105  ELLEN     PARTS

With INDICATORMODE='N' RECFM=FB LRECL=26, puts indicator byte at end of record

======================================================

 Column Name                    Offset Length Type   

 ============================== ====== ====== =========

 EMP_ID                              0      5 CHAR   

 EMP_NAME                            5     10 CHAR   

 DEPT_NAME                          15     10 CHAR   

 ============================== ====== ====== =========

 INDICATOR BYTES NEEDED: 1                           

 EXPECTED RECORD LENGTH: 26                          

 105  ELLEN     PARTS     .

 114  ZACH      SERVICE   .

 115  GEORGE    SERVICE   .

 124  SHEILA    SERVICE   .

 110  ALICE     LAW       .

 121  HENRY     PARTS     .

 125  IVAN      PARTS     .

 123  LAURA     PARTS     .

 122  RAY       FINANCE   .

 119  RALPH     LAW       .

With INDICATORMODE='N' RECFM=VB LRECL=30 (extra 4 bytes added for RECFM=VB), no indicator bytes are written/padded

 ======================================================

 Column Name Offset Length Type

 ============================== ====== ====== =========

 EMP_ID 0 5 CHAR

 EMP_NAME 5 10 CHAR

 DEPT_NAME 15 10 CHAR

 ============================== ====== ====== =========

 INDICATOR BYTES NEEDED: 1

 EXPECTED RECORD LENGTH: 26

 124  SHEILA    SERVICE

 115  GEORGE    SERVICE

 121  HENRY     PARTS

 114  ZACH      SERVICE

 122  RAY       FINANCE

 110  ALICE     LAW  

 125  IVAN      PARTS

 123  LAURA     PARTS

 105  ELLEN     PARTS

 119  RALPH     LAW  

With INDICATORMODE='Y' RECFM=VB LRECL=30 (extra 4 bytes added for RECFM=VB), indicator bytes are written at beginning of record

 ======================================================

 Column Name Offset Length Type

 ============================== ====== ====== =========

 EMP_ID 0 5 CHAR

 EMP_NAME 5 10 CHAR

 DEPT_NAME 15 10 CHAR

 ============================== ====== ====== =========

 INDICATOR BYTES NEEDED: 1

 EXPECTED RECORD LENGTH: 26

 .119  RALPH     LAW    

 .114  ZACH      SERVICE

 .115  GEORGE    SERVICE

 .122  RAY       FINANCE

 .110  ALICE     LAW    

 .105  ELLEN     PARTS  

 .124  SHEILA    SERVICE

 .123  LAURA     PARTS  

 .125  IVAN      PARTS  

 .121  HENRY     PARTS  

Teradata Employee

Re: z/OS Mainframe TPT RECFM=FB versus LRECL=VB

First let me clear up one thing.

Indicator bytes are never written at the end of a record.

What you are seeing are pad bytes written out.

I have our mainframe developer looking into this.

Clearly, we have an issue.

When INDICATORMODE='N', we should not be accounting for the indicator bytes when determining the proper LRECL.

-- SteveF