TDCH-TPT Interface

Tools
Enthusiast

TDCH-TPT Interface

Hey Guys,

                     I was trying to export a large table from Teradata to HDFS as a textfile using TDCH-TPT Interface i was able to export around 3Mil records with any problems but when i am trying to export entire table i.e 10Mil records its failing saying that "TPT_INFRA: TPT02270: Error: Cannot read from Data Stream, status = DataStream Error" i am using TPT-15.10 and TDCH 1.4.3, i am stuck on this for long time 

i have seen the same question here http://teradataerror.com/TPT2270-Error-Cannot-read-from-Data-Stream-status--s.html but the remedy says contact global support.

Thank you

9 REPLIES
Teradata Employee

Re: TDCH-TPT Interface

If you are writing the data to HDFS, why not just use TPT's HDFS interface?

-- SteveF
Enthusiast

Re: TDCH-TPT Interface

Do you have any sample scripts to invoke TPT's HDFS interface, the reason for using TDCH-TPT interface was because of better parllelism where we have more mappers writing simultaneously to hdfs, is there any issue with scalability of TDCH-TPT interface.?

Teradata Employee

Re: TDCH-TPT Interface

Reading/writing from/to Hadoop files via HDFS are no different than accessing them from the local file system.

You give us the name and the directory and the format.

The only difference that is mandatory is to provide us with the HadoopHost name in the DataConnector operator's HadoopHost attribute.

This is documented in the TPT Reference Manual.

There should also be examples in the TPT User Guide.

-- SteveF
Enthusiast

Re: TDCH-TPT Interface

I tried the example in the TPT User Guide but instead of wrting to hdfs it was writing to local eventhough i have set HadoopHost for the Dataconnector operator may be i have missed some other configuration there, only way i was able to get the file to hdfs via TDC-TPT Interface

Teradata Employee

Re: TDCH-TPT Interface

Please provide your script and version of TPT.

-- SteveF
Enthusiast

Re: TDCH-TPT Interface

Hey I was able to make TPT-HDFS Api work there was some issue with my classpath, but i somehow feel tpt-tdch interface is  much faster when compared with TPT-HDFS becuase parllelism will be better in tpt-tdch since it runs multiple mappers. Our main goal is to reduce load on teradata while transfering data from teradata to hadoop. what would you suggest.?

Teradata Employee

Re: TDCH-TPT Interface

I am glad you were able to get the HDFS method working.

As to which method to use, that would be up to you.

We do not run performance comparison tests.

The method that performs the best for you and is the easiest to use would probably be your best solution.

-- SteveF
Enthusiast

Re: TDCH-TPT Interface

The main goal is to reduce AMP CPU Seconds while exporting the data, if you can point me to some performace tuning parameters of TPT that would be great help for me.

Teradata Employee

Re: TDCH-TPT Interface

TPT is a client application and we have no control over AMP CPU usage.

I would not be able to help there.

-- SteveF