TPT load failure - Please suggest

Tools
Enthusiast

TPT load failure - Please suggest

Dear Experts,

We are facing a failure during loading a file into a table using TPT utility. We loaded files a lot of times successfully but this time, its a very huge file (>100 GB) probably causing the issue.

Errror Observed :

FILE_READER: TPT19350 I/O error on file '/dev/aps/file123.dat.

FILE_READER: TPT19417 pmRead failed. Buffer overflow (17)

FILE_READER: TPT19305 Fatal error reading data.

I did some research based on the error codes but couldn't get anything solid to narrow down to the root cause.

Could you please throw some light on what the actual issue could be ?

33 REPLIES
Teradata Employee

Re: TPT load failure - Please suggest

What is the version of TPT?

What is the platform?

-- SteveF
Enthusiast

Re: TPT load failure - Please suggest

Hi , Sorry i was out for a couple of weeks and couldn't respond.

The teradata client version is 13.0.

Platform : AIX

We are calling the TPT in a shell script with parameters.

We loaded 100s of files successfully without any issues till date. I think its the colume of the huge file (>100 GB) thats causing the issue.

Teradata Employee

Re: TPT load failure - Please suggest

What exact version of 13.0?

We have released several patches since its GCA version.

And 13.0 is not an officially supported version of TPT, so my first response is for you to update TPT to the latest patch.

Then re-run the job.

If there is still an issue, let me know and we can dig further.

-- SteveF

Re: TPT load failure - Please suggest

We are using the TPT version 13.00.00.02 and facing the same issue while loading 320GB file. Immediate help is required here.

Teradata Employee

Re: TPT load failure - Please suggest

Please upgrade to the latest efix version and try again.

If the problem persists, let me know (and let me know what version you used).

-- SteveF
Teradata Employee

Re: TPT load failure - Please suggest

Please post the script you are using.

Thanks!

-- SteveF
Enthusiast

Re: TPT load failure - Please suggest

Hi Steve, Thanks for your responses.

I have a general question. When loads for smaller files (upto 80-90 GB) are going through successfully using the same scripts and the same version of TPT, it is indicating the the script and version are actually working.  If the framework works for none of the loads we can assume that the version doesn't support the load process.

Could you please share your thoughts on the version/patch impacts only higher volumes?  Also could you please share if addressing bulk volumes was one of the considerations for the efix ?

Thanks much!

Teradata Employee

Re: TPT load failure - Please suggest

We fix issues all of the time.

We are currently in development for the 15.10 release.

This means that bug fixes have occurred for 13.0, 13.10, 14.0, 14.10, and 15.0.

When bugs are found in newer releases, we ofte back-ship the fixes to older releases.

It is nearly impossible for me to determine whether any of the 13.0 efixes will take care of the problem you are experiencing.

However, it does not make sense to have you generate traces and logs for us to look at because the code has most likely changed so much between the version you have and the most recent efix.

Yes, you are correct that if the load process works on an 80GB file, it should work on a 300GB file. But I have seen stranger things happen.

That particular error usually means the row encountered is larger than our block size. On the surface, it does not make much sense, but the file might have a bad row in it and we are having trouble processing it. It is possible that we made changes in one of the efixes to handle it better (not continue, but maybe provide a better error message?

-- SteveF
Teradata Employee

Re: TPT load failure - Please suggest

Hi Experts.

I have a problen which I think is also a version issue.

I detail the problem (I have posted it in other topic too):

TPT 14.10 problems loading data from SQL Server using TPT Wizard

===============================================================================

Recently I made an version upgrade to TTU 14.10 but I'm having some problems that I didn't have in 13.10.

I tried to use the TPT Wizard to create a load Job to retrieve data from "SQL Server", but the Log File says that no rows are exported. That is weird because If I submit the SQL in the source, It returns data.

Do you know if there are some bugs in this version about that?

I copy the TPT JOB and the log, If for some case you need them

Script

======

/* 1 */ 

/* 2 */ 

/* 3 */  USING CHARACTER SET UTF8

/* 4 */  DEFINE JOB "Carga_tablas_Paradata"

/* 5 */  (

/* 6 */   DEFINE OPERATOR "W_1_o_Carga_tablas_Paradata"

/* 7 */   TYPE LOAD

/* 8 */   SCHEMA *

/* 9 */   ATTRIBUTES

/* 10 */   (

/* 11 */    VARCHAR "UserName",

/* 12 */    VARCHAR "UserPassword",

/* 13 */    VARCHAR "LogTable",

/* 14 */    VARCHAR "TargetTable",

/* 15 */    INTEGER "BufferSize",

/* 16 */    INTEGER "ErrorLimit",

/* 17 */    INTEGER "MaxSessions",

/* 18 */    INTEGER "MinSessions",

/* 19 */    INTEGER "TenacityHours",

/* 20 */    INTEGER "TenacitySleep",

/* 21 */    VARCHAR "AccountID",

/* 22 */    VARCHAR "DateForm",

/* 23 */    VARCHAR "ErrorTable1",

/* 24 */    VARCHAR "ErrorTable2",

/* 25 */    VARCHAR "NotifyExit",

/* 26 */    VARCHAR "NotifyExitIsDLL",

/* 27 */    VARCHAR "NotifyLevel",

/* 28 */    VARCHAR "NotifyMethod",

/* 29 */    VARCHAR "NotifyString",

/* 30 */    VARCHAR "PauseAcq",

/* 31 */    VARCHAR "PrivateLogName",

/* 32 */    VARCHAR "TdpId",

/* 33 */    VARCHAR "TraceLevel",

/* 34 */    VARCHAR "WorkingDatabase"

/* 35 */   );

/* 36 */ 

/* 37 */   DEFINE SCHEMA "W_0_s_Carga_tablas_Paradata"

/* 38 */   (

/* 39 */    "Lote_id" INTEGER,

/* 40 */    "Lote_descripcion" VARCHAR(100),

/* 41 */    "Lote_Cliente" VARCHAR(20),

/* 42 */    "dedu_fecha_proceso" CHARACTER(23),

/* 43 */    "dedu_cantidad_registros" BIGINT,

/* 44 */    "dedu_veto_total" BIGINT,

/* 45 */    "dedu_cliente_vigente_tot" BIGINT

/* 46 */   );

/* 47 */ 

/* 48 */   DEFINE OPERATOR "W_0_o_Carga_tablas_Paradata"

/* 49 */   TYPE ODBC

/* 50 */   SCHEMA "W_0_s_Carga_tablas_Paradata"

/* 51 */   ATTRIBUTES

/* 52 */   (

/* 53 */    VARCHAR "UserName",

/* 54 */    VARCHAR "UserPassword",

/* 55 */    VARCHAR "SelectStmt",

/* 56 */    VARCHAR "PrivateLogName",

/* 57 */    VARCHAR "DSNName",

/* 58 */    VARCHAR "ConnectString",

/* 59 */    VARCHAR "TruncateData"

/* 60 */   );

/* 61 */ 

/* 62 */   APPLY

/* 63 */    (

/* 64 */     'INSERT INTO P_DW_PROSPECTOS_TEMP_TABLES.mktd_lotes_v1 (Lote_id,Lote_descripcion,Lote_Cliente,dedu_fecha_proc eso,dedu_cantidad_registros,dedu_veto_total,dedu_clien te_vigente_tot) VALUES (:Lote_id,:Lote_descripcion,:Lote_Cliente,: dedu_fecha_proceso,:dedu_cantidad_registros,: dedu_veto_total,:dedu_cliente_vigente_tot);'

/* 65 */    )

/* 66 */   TO OPERATOR

/* 67 */   (

/* 68 */    "W_1_o_Carga_tablas_Paradata"[1]

/* 69 */ 

/* 70 */    ATTRIBUTES

/* 71 */    (

/* 72 */     "UserName" = 'L0607460',

/* 73 */     "UserPassword" = 'XXXXXXXXXX',

/* 74 */     "LogTable" = 'P_DW_PROSPECTOS_TEMP_TABLES.mktd_lotes_v1_log',

/* 75 */     "TargetTable" = 'P_DW_PROSPECTOS_TEMP_TABLES.mktd_lotes_v1',

/* 76 */     "TdpId" = '10.0.xx.xx'

/* 77 */    )

/* 78 */   )

/* 79 */   SELECT * FROM OPERATOR

/* 80 */   (

/* 81 */    "W_0_o_Carga_tablas_Paradata"[1]

/* 82 */ 

/* 83 */    ATTRIBUTES

/* 84 */    (

/* 85 */     "UserName" = 'XXXXXXXXXXXXXX',

/* 86 */     "UserPassword" = 'XXXXXXXXXXXXXX',

/* 87 */     "SelectStmt" = 'SELECT Lote_id,Lote_descripcion,Lote_Cliente,dedu_fecha_proce so,dedu_cantidad_registros,dedu_veto_total,dedu_client e_vigente_tot FROM MKTD_DEDUPLICACION.dbo.Lotes;',

/* 88 */     "DSNName" = 'paradata'

/* 89 */    )

/* 90 */   );

/* 91 */  ); 

LOG FILE:

===========

Teradata Parallel Transporter Version 14.10.00.04

Job log: C:\Archivos de programa\Teradata\client\14.10\Teradata Parallel Transporter/logs/LOTE_V1_2-31.out

Job id is LOTE_V1_2-31, running on GAL082262

Teradata Parallel Transporter Load Operator Version 14.10.00.04

W_1_o_Carga_tablas_Paradata: private log not specified

Teradata Parallel Transporter ODBC Operator Version 14.10.00.04

W_0_o_Carga_tablas_Paradata: private log not specified

W_0_o_Carga_tablas_Paradata: connecting sessions

W_0_o_Carga_tablas_Paradata: TPT17176: Info: Message received from ODBC driver:

STATE=01000, CODE=5701,

MSG='[Microsoft][ODBC SQL Server Driver][SQL Server]Changed database context to 'MKTD_DEDUPLICACION'.'

W_0_o_Carga_tablas_Paradata: TPT17176: Info: Message received from ODBC driver:

STATE=01000, CODE=5703,

MSG='[Microsoft][ODBC SQL Server Driver][SQL Server]Changed language setting to us_english.'

W_0_o_Carga_tablas_Paradata: TPT17175: Warning: Message received from ODBC driver:

STATE=S1092, CODE=0,

MSG='[Microsoft][ODBC SQL Server Driver]Identificador de opción o atributo no válido'

W_1_o_Carga_tablas_Paradata: connecting sessions

W_1_o_Carga_tablas_Paradata: preparing target table

W_1_o_Carga_tablas_Paradata: entering Acquisition Phase

W_0_o_Carga_tablas_Paradata: sending SELECT request

W_0_o_Carga_tablas_Paradata: data retrieval complete

W_0_o_Carga_tablas_Paradata: Total Rows Exported:  0

W_1_o_Carga_tablas_Paradata: entering Application Phase

W_1_o_Carga_tablas_Paradata: Statistics for Target Table:  'P_DW_PROSPECTOS_TEMP_TABLES.mktd_lotes_v1'

W_1_o_Carga_tablas_Paradata: Total Rows Sent To RDBMS:      0

W_1_o_Carga_tablas_Paradata: Total Rows Applied:            0

W_1_o_Carga_tablas_Paradata: Total Rows in Error Table 1:   0

W_1_o_Carga_tablas_Paradata: Total Rows in Error Table 2:   0

W_1_o_Carga_tablas_Paradata: Total Duplicate Rows:          0

W_0_o_Carga_tablas_Paradata: disconnecting sessions

W_1_o_Carga_tablas_Paradata: disconnecting sessions

W_0_o_Carga_tablas_Paradata: Total processor time used = '0.859375 Second(s)'

W_0_o_Carga_tablas_Paradata: Start : Tue Jul 22 18:28:37 2014

W_0_o_Carga_tablas_Paradata: End   : Tue Jul 22 18:29:00 2014

W_1_o_Carga_tablas_Paradata: Total processor time used = '4.57813 Second(s)'

W_1_o_Carga_tablas_Paradata: Start : Tue Jul 22 18:28:37 2014

W_1_o_Carga_tablas_Paradata: End   : Tue Jul 22 18:29:06 2014

Job step MAIN_STEP terminated (status 4)

Job LOTE_V1_2 completed successfully, but with warning(s).

Job start: Tue Jul 22 18:28:34 2014

Job end:   Tue Jul 22 18:29:06 2014

Thanks