TDCH won't load ORC DATE columns using fastload

Database
The Teradata Database channel includes discussions around advanced Teradata features such as high-performance parallel database technology, the optimizer, mixed workload management solutions, and other related technologies.

TDCH won't load ORC DATE columns using fastload

Hi All,

I'm trying to load a single column table from an ORC format HIVE table using TDCH with the internal.fastload method.  All rows end up in the ERR1 table.  TDCH version is 1.5.4, TD release is 15.10.07.15, version is 15.10.07.31.  Here's a specific example.
In HIVE:

create table orctest (DateCol date) stored as orc;

insert into orctest values ('2019-01-01');

 

Loaded using TDCH method internal.fastload.  Exit code 0, ERR_1 table is not empty.

In Teradata:

sel * from orctest_err_1;

2,665 DateCol 00-00-0A-32-30-31-39-2D-30-31-2D-30-31

The error number is usually "2673 : The source parcel length does not match data that was defined" not sure why this test was different.

 

If the load method is changed to batch.insert, or the HIVE table is RCFILE or just not specified then it works.  Seems to be specifically dates in ORC tables using fastload.

 

Any help very gratefully recieved as I'm running out of hair to tear out!

 

Regards, Jim Hudd.

2 REPLIES 2
Teradata Employee

Re: TDCH won't load ORC DATE columns using fastload

Hi Jim,

 

Try with :

-fileformat orcfile
-jobtype hive

Re: TDCH won't load ORC DATE columns using fastload

Hi, the jobtype is set to HIVE for all my tests and the fileformat is set to either textfile, rcfile or orcfile depending whether my HIVE table is text, rc or orc.

 

Here's the command line. i've masked a few bits but the important stuff is unchanged.

 

hadoop jar /usr/lib/tdch/1.5/lib/teradata-connector-1.5.4.jar com.teradata.connector.common.tool.ConnectorExportTool -libjars "$HIVE_LIB_JARS" -classname com.teradata.jdbc.TeraDriver -url jdbc:teradata://tdhostname/DATABASE=TDDBNAME -username myuser -password "$TD_PWD" -jobtype hive -fileformat orcfile -method internal.fastload -sourcedatabase hivesourcedb -sourcetable JimOrcTest -sourcefieldnames "DateCol " -targettable JimOrcTest -targetfieldnames "DateCol "

 

 

 

Jim.