TPT Script - Not all records loaded from a file with SkipRowsEveryFile option

Tools
Enthusiast

TPT Script - Not all records loaded from a file with SkipRowsEveryFile option

Hi,

I am trying to load a file which has 2181224338 records to a Teradata table using TPT Script. The Data Connector attributes are as below and I am using LOAD operator. Version of TTU is 14.10.00.11.

DEFINE OPERATOR File_Rd_Table

TYPE DATACONNECTOR PRODUCER

SCHEMA Table_S

ATTRIBUTES

(

VARCHAR FileName            = 'Transactions.txt',

VARCHAR MultipleReaders     = 'N',

VARCHAR Format              = 'Delimited',

VARCHAR OpenMode            = 'Read',

VARCHAR TextDelimiter       = '|',

INTEGER SkipRows            = 1

VARCHAR SkipRowsEveryFile   = 'Y'

);

But once the script is completed, only 2147483646 records are loaded and the remaining are skipped.

Load_X: Total Rows Sent To RDBMS:2147483646

Load_X: Total Rows Applied:            2147483646

Load_X: Total Rows in Error Table 1:   0

Load_X: Total Rows in Error Table 2:   0

Load_X: Total Duplicate Rows:          0

What is the reason for this? If I remove the attribute SkipRowsEveryFile (default is N) then all the records are loaded

Load_X: Total Rows Applied:            2181224338

Load_X: Total Rows in Error Table 1:   0

Load_X: Total Rows in Error Table 2:   0

Load_X: Total Duplicate Rows:          0

I see that 2181224338 is greater than the maximum value a INTEGER data type can hold. INTEGER - Range:  -2,147,483,648 to 2,147,483,647

Can you suggest any solution for this.

Thanks

Manjeeth
Tags (1)
5 REPLIES
Teradata Employee

Re: TPT Script - Not all records loaded from a file with SkipRowsEveryFile option

Did you check the table to see how many rows are actually in the table?

When you do a SELECT COUNT(*), the default result is an integer value.

To get the true value you will have to CAST the result to a BIGINT (if your database version supoprts that data type) or DECIMAL.

-- SteveF
Enthusiast

Re: TPT Script - Not all records loaded from a file with SkipRowsEveryFile option

Yes Steve. I checked the record count as select cast(count(1) as bigint) and the value is still 2147483646

Teradata Employee

Re: TPT Script - Not all records loaded from a file with SkipRowsEveryFile option

We have located the source of the bug and will fix in the near future.

The workaround (as you well know), if you only have a single file, then do not set SkipRowsEveryFile to 'Y'.

-- SteveF
Enthusiast

Re: TPT Script - Not all records loaded from a file with SkipRowsEveryFile option

Thank you Steve. I will work on my scripts

Teradata Employee

Re: TPT Script - Not all records loaded from a file with SkipRowsEveryFile option

We have fixed the bug.

The fix will be available in the next set of 14.10, 15.0 and 15.10 efixes.

Dates for those releases have not yet been announced.

-- SteveF