Loading data files from hadoop to teradata using TPT

Tools & Utilities

Loading data files from hadoop to teradata using TPT

I am trying to load data from hadoop to Teradata using tpt.

The delimiter in these files is the default hadoop delimiter (Cntrl-A - hex value 01).

Does tpt support this delimiter and how do I specify it?

Tags (3)
5 REPLIES

Re: Loading data files from hadoop to teradata using TPT

We use Teradata Parallel Transporter Version 13.10.00.12

Teradata Employee

Re: Loading data files from hadoop to teradata using TPT

Hi there,

TPT currently does not support loads / unloads to and from Hadoop. This feature will be available in TPT 15.0.

If you'd like to use TPT 13.10 to read data from Hadoop into Teradata, you'll have to use the command 'hadoop fs -get' to move the file onto your local machine from HDFS. One the file is local, you can use TPT to read the file using the DATACONNECTOR PRODUCER operator. TPT 13.10 does allow you to specify single byte hex values for the TextDelimiter attribute, but you'll have to use a program like hexedit to encode the hex byte '01' into the TPT script. Later versions of TPT allow multibyte hex delimiters to be specified via the TextDelimiterHEX attribute.

If you're open to utilizing a different load / unload product, we've recently released the Teradata Connector for Hadoop. For more information, see http://developer.teradata.com/connectivity/articles/teradata-connector-for-hadoop-now-available

Ryan

Re: Loading data files from hadoop to teradata using TPT

Hi,

Whether the feature 'TPT for Hadoop' available in TD 14.0 ?

I'm able to unload data from Teradata to Hadoop using TDCH. But, Customer prefers to unload huge volume of data using TPT for Hadoop only.

Can someone provide me TD TPT for Hadoop - sample script or syntax to load/unload to and from Hadoop. Am not able to find one. It's quite urgent.

Thanks !

Teradata Employee

Re: Loading data files from hadoop to teradata using TPT

TPT - Hadoop integration is supported in TPT 15.0.

The TPT documentation User Guide has sample scripts.

-- SteveF
Enthusiast

Re: Loading data files from hadoop to teradata using TPT

This link maybe of help : http://developer.teradata.com/connectivity/articles/teradata-connector-for-hadoop-now-available

I am thinking people are using hadapt, because it can bifurcate works into MR and SQL operators. Hadapt is already Teradata's.