TDCH-TPT Interface--For loading into Hadoop

Tools
Enthusiast

TDCH-TPT Interface--For loading into Hadoop

I need to ingest volume of data from Teradata to Hadoop using TPT.I saw in the TPT documentation that we can achieve this using TDCH-TPT interface.I would like to know the following about the process:

  1. Whether it follows the same process and extracts data block by block.
  2. Whether it utilizes all the nodes in the cluster while loading into Hadoop.
  3. In this case whether TPT needs to be installed in all the nodes in the hadoop cluster?
  4. For 1 single table ingestion and export to hadoop whether both the read(Teradata) and write(Hadoop) whether both the process are multithreaded while using TDCH-TPT interface.
5 REPLIES
Teradata Employee

Re: TDCH-TPT Interface--For loading into Hadoop

The use of TDCH by TPT is basically performed by TPT sending a command to the name node of the Hadoop cluster to execute the TDCH command, passing the needed information as command line options to TDCH.

If you are using the Export operator to extract from Teradata, then the data is processed block by block.

TDCH will use as many mappers as needed (or as indicated by the user).

TPT does not need to be installed on any of the Hadoop nodes.

It is generally installed (with the rest of TTU) on the client server.

The Export operator will run as a multi-process operator if you tell it to use more than 1 instance. TPT is not a multi-threaded application.

For information about TDCH and whether it is multi-threaded, you would have to refer to the TDCH documentation.

-- SteveF
Enthusiast

Re: TDCH-TPT Interface--For loading into Hadoop

Hi Steve,

To use TDCH-TPT API, whether Teradata Connector for Hadoop needs to be installed in the node where TPT is installed?

Does it needs to be configured as well after installation?

Is there any documentation for installation and configuration?

 

Thanks & Regards,

Arpan. 

Teradata Employee

Re: TDCH-TPT Interface--For loading into Hadoop

Hi

 

I'm trying to export a table from Teradata to Hive using TDCH-TPT, but I get the following error:

 

$DATACONNECTOR_CONSUMER[1]: TPT19608 The TDCH-TPT interface is only available on LINUX platforms

 

Is because of my TPT version or actually TDCH-TPT can be used only on Linux?

 

Here the whole log:

 

Teradata Parallel Transporter Version 15.10.01.02 64-Bit
Job log: C:\Program Files\Teradata\client\15.10\Teradata Parallel Transporter/lo
gs/L0607460-3.out
WARN:Failed to lookup account administrators
WARN:Failed to lookup account administrators
Job id is L0607460-3, running on gal116261
Found CheckPoint file: C:\Program Files\Teradata\client\15.10\Teradata Parallel
Transporter/checkpoint\L0607460LVCP
This is a restart job; it restarts at step MAIN_STEP.
Teradata Parallel Transporter Export Operator Version 15.10.01.02
$EXPORT: private log not specified
Teradata Parallel Transporter DataConnector Operator Version 15.10.01.02
$DATACONNECTOR_CONSUMER[1]: Instance 1 directing private log report to 'dtacop-L
0607460-7328-1'.
$DATACONNECTOR_CONSUMER[1]: DataConnector Consumer operator Instances: 1
$DATACONNECTOR_CONSUMER[1]: ECI operator ID: '$DATACONNECTOR_CONSUMER-7328'
$DATACONNECTOR_CONSUMER[1]: TPT19608 The TDCH-TPT interface is only available on
LINUX platforms
$DATACONNECTOR_CONSUMER[1]: TPT19424 pmOpen failed. Request unsupported by Acces
s Module (24)
$DATACONNECTOR_CONSUMER[1]: TPT19304 Fatal error opening file.
$DATACONNECTOR_CONSUMER[1]: TPT19015 TPT Exit code set to 12.
$EXPORT: connecting sessions
$EXPORT: disconnecting sessions
$DATACONNECTOR_CONSUMER[1]: Total files processed: 0.
$EXPORT: Total processor time used = '2.875 Second(s)'
$EXPORT: Start : Thu Feb 02 11:57:12 2017
$EXPORT: End : Thu Feb 02 11:58:15 2017
Job step MAIN_STEP terminated (status 12)
Job L0607460 terminated (status 12)
Job start: Thu Feb 02 11:57:09 2017
Job end: Thu Feb 02 11:58:15 2017

 

Thanks

Tags (3)
Teradata Employee

Re: TDCH-TPT Interface--For loading into Hadoop

Right now, the TPT-TDCH integration is only certified and available on Linux.

 

-- SteveF
Teradata Employee

Re: TDCH-TPT Interface--For loading into Hadoop

"Que macana". It's a shame. I can't install Linux on my customer's workstation.

 

Do you know If exists other option for TPT on Windows?