TPT Export issue -- Not able to export large tables

Tools
Enthusiast

TPT Export issue -- Not able to export large tables

Good morning  Gentlemen,

I am facing a weird issue while doing a TPT export on linux . I am facing this issue when I am trying to export more than 4million records(not sure of the row size) .

The same code ran fine for  fewer records (apprx ~ 3 mil) .  

I verified the DBQL logs for this session and found that there are no error codes related to it (this being an export job is there nothing to be err'ed out ?.) 

My tpt file is below :

DEFINE JOB EXPORT_RN_TABLE_TO_FILE

DESCRIPTION 'EXPORT  RN VIEWS TO FILES'

(

DEFINE SCHEMA RN_SCHEMA FROM TABLE @MyTableName;

   DEFINE OPERATOR FILE_WRITER()

   DESCRIPTION 'TERADATA PARALLEL TRANSPORTER DATA CONNECTOR OPERATOR'

   TYPE DATACONNECTOR CONSUMER

   SCHEMA *

   ATTRIBUTES

   (

      VARCHAR PrivateLogName    = 'RN_LODG_DATA',

      VARCHAR FileName          = @OutputFileName,

      VARCHAR IndicatorMode     = 'N',

      VARCHAR OpenMode          = 'Write',

      VARCHAR Format            = 'Delimited',

      VARCHAR TextDelimiter     = '|'

   );

   DEFINE OPERATOR EXPORT_OPERATOR()

   DESCRIPTION 'TERADATA PARALLEL TRANSPORTER EXPORT OPERATOR'

   TYPE EXPORT

   SCHEMA RN_SCHEMA

   ATTRIBUTES

   (

      VARCHAR PrivateLogName    = 'EXPORT_RN_LODG_DATA',

      INTEGER MaxSessions       =  32,

      INTEGER MinSessions       =  1,

      INTEGER MaxDecimalDigits  = 38,

      VARCHAR DateForm       = 'ANSIDATE',

      VARCHAR TdpId          = @MyTdpId,

      VARCHAR UserName       = @MyUserName,

      VARCHAR UserPassword   = @MyPassword,

      VARCHAR AccountId,

      VARCHAR SelectStmt        = 'select * from <table_name>'

   );

   STEP export_to_file

   (

      APPLY TO OPERATOR (FILE_WRITER()[1])

      SELECT * FROM OPERATOR (EXPORT_OPERATOR() [1] );

   );

);

-----

[username@servername ~]$ cat log_RN_tpt_RN_td_test_3_6.log

Teradata Parallel Transporter Version 15.00.00.02

Job log: /home/<username>/tmp/tbuild/logs/TEST_JOB_RN_36-7053.out

Job id is TEST_JOB_RN_36-7053, running on servername.domainname

Teradata Parallel Transporter Export Operator Version 15.00.00.02

EXPORT_OPERATOR: private log specified: EXPORT_RN_LODG_DATA

Teradata Parallel Transporter FILE_WRITER[1]: TPT19006 Version 15.00.00.02

FILE_WRITER[1]: TPT19010 Instance 1 directing private log report to 'RN_LODG_DATA-1'.

FILE_WRITER[1]: TPT19007 DataConnector Consumer operator Instances: 1

FILE_WRITER[1]: TPT19003 ECI operator ID: 'FILE_WRITER-11137'

FILE_WRITER[1]: TPT19222 Operator instance 1 processing file '/tmp/outputfile_RN_3_6.dat'.

EXPORT_OPERATOR: connecting sessions

EXPORT_OPERATOR: sending SELECT request

Unexpected error.

TPT_INFRA: TPT01036: Error: Task (TaskID: 4, Task Name: INSERT_1[0001]) terminated due to the receipt of signal number 6

TPT_INFRA: TPT01037: Error: Task (TaskID: 4, Task Name: INSERT_1[0001]) core dumped

TPT_INFRA: TPT02595: Error: DSAC_DataStreamSingularOutput - send error

TPT_INFRA: TPT02268: Error: Cannot write message to Data Stream, status = DataStream Error

TPT_INFRA: TPT02269: Error: Data Stream status = -3197160

OS_GetLocalShmAddr: Invalid cntl shm address 31303030

TPT_INFRA: TPT01036: Error: Task (TaskID: 5, Task Name: SELECT_2[0001]) terminated due to the receipt of signal number 6

TPT_INFRA: TPT01037: Error: Task (TaskID: 5, Task Name: SELECT_2[0001]) core dumped

Job step export_to_file terminated (status 8)

Job TEST_JOB_RN_36 terminated (status 8)

Job start: Tue Apr 19 16:25:42 2016

Job end:  Tue Apr 19 16:28:26 2016

[username@servername ~]$

--------------

Any ideas or suggestions to resolve this ?.

6 REPLIES
Enthusiast

Re: TPT Export issue -- Not able to export large tables

Just an update to the above issue . I ran it with a different user on the same server and job ran fine . I am thinking this is a unix related issue .

Any Ideas on what I should check for this user ?

Enthusiast

Re: TPT Export issue -- Not able to export large tables

Did you ever find a solution for this error?

Enthusiast

Re: TPT Export issue -- Not able to export large tables

Yes, a dude named Bob from Texas solved the issue for us :) 

Just to update the forum with the solution that worked in our case .

The problem is with the tpt binaries that are running this job . Our current environment is on 32bit libraries which is throwing this core dump issue for this export .

Switching the libraries to 64 bit helped resolve the issue .

Steps to update the path:

PATH=/opt/teradata/client/15.00/tbuild/bin64:$PATH

Enthusiast

Re: TPT Export issue -- Not able to export large tables

Hi All,

We are facing the same issue. Below is the error details:

Teradata Parallel Transporter Version 14.10.00.10
Job log: /opt/teradata/client/14.10/tbuild/logs/EVEREST_LEGACY_CONTRACT_ATTRIBU-213.out
Job id is EVEREST_LEGACY_CONTRACT_ATTRIBU-213, running on ebdp-ch2-e047p.sys.comcast.net
Teradata Parallel Transporter Export Operator Version 14.10.00.10
o_ExportOper: private log specified: Export
Teradata Parallel Transporter DataConnector Operator Version 14.10.00.10
o_FileWritter: Instance 1 directing private log report to 'DataConnector-1'.
o_FileWritter: DataConnector Consumer operator Instances: 1
o_FileWritter: ECI operator ID: 'o_FileWritter-5401'
o_FileWritter: Operator instance 1 processing file '/home/edpintevrp/ExtractFramework/TgtFile/FullExtract/LEGACY_CONTRACT_ATTRIBU.txt'.
o_ExportOper: connecting sessions
o_ExportOper: sending SELECT request
Unexpected error.
TPT_INFRA: TPT01036: Error: Task (TaskID: 4, Task Name: INSERT_1[0001]) terminated due to the receipt of signal number 6

TPT_INFRA: TPT01037: Error: Task (TaskID: 4, Task Name: INSERT_1[0001]) core dumped

OS_GetLocalShmAddr: Invalid cntl shm address ac00001

TPT_INFRA: TPT01036: Error: Task (TaskID: 5, Task Name: SELECT_2[0001]) terminated due to the receipt of signal number 6

TPT_INFRA: TPT01037: Error: Task (TaskID: 5, Task Name: SELECT_2[0001]) core dumped

Job step MAIN_STEP terminated (status 8)
Job EVEREST_LEGACY_CONTRACT_ATTRIBU terminated (status 8)

We are using "TTU 14.10.00.10" which allows only 32 bit TPT. OS version is: "CentOS release 6.6 (Final)"

Can someone please help in this issue.

 

Thanks & Regards,

Arpan.

Re: TPT Export issue -- Not able to export large tables

Hi All,

To overcome this issue for 32 bit environment, limit the output file size to less than 1244 MB.

Not sure why, but its working. 

Ty

Satish

Tags (1)
Teradata Employee

Re: TPT Export issue -- Not able to export large tables

Arpan,

This is a known issue and we have not yet determined the root cause, nor resolved this issue.

Thanks!

 

-- SteveF