Sqoop export fails with untranslatable character error

Hadoop

Sqoop export fails with untranslatable character error

HI, we have a sqoop job that exports data to Teradata hourly. Sometimes job fails with untranslatable character error. We would like to know, if there is any Java Function to check if the string is translatable by teradata. This is of higher priority. Please let us know, if there are any questions.

2015-04-19 22:59:56,734 WARN org.apache.hadoop.mapred.Child: Error running child

com.teradata.hadoop.exception.TeradataHadoopSQLException: java.sql.BatchUpdateException: [Teradata JDBC Driver] [TeraJDBC 14.00.00.01] [Error 1338] [SQLState HY000] A failure occurred while executing a PreparedStatement batch request. Details of the failure can be found in the exception chain that is accessible with getNextException.

   at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeBatchUpdateException(ErrorFactory.java:147)

 at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeBatchUpdateException(ErrorFactory.java:136)

       at com.teradata.jdbc.jdbc_4.TDPreparedStatement.executeBatchDMLArray(TDPreparedStatement.java:239)

        at com.teradata.jdbc.jdbc_4.TDPreparedStatement.executeBatch(TDPreparedStatement.java:1951)

        at com.teradata.hadoop.mapreduce.TeradataRecordWriter.write(TeradataRecordWriter.java:60)

  at com.teradata.hadoop.mapreduce.TeradataRecordWriter.write(TeradataRecordWriter.java:23)

    at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:558)

   at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:85)

      at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:106)

       at com.teradata.hadoop.mapreduce.TeradataTextFileExportMapper.map(TeradataTextFileExportMapper.java:32)

   at com.teradata.hadoop.mapreduce.TeradataTextFileExportMapper.map(TeradataTextFileExportMapper.java:12)

       at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)

 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)

   at org.apache.hadoop.mapred.Child$4.run(Child.java:268)

       at java.security.AccessController.doPrivileged(Native Method)

       at javax.security.auth.Subject.doAs(Subject.java:415)

     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)

 at org.apache.hadoop.mapred.Child.main(Child.java:262)

Caused by: com.teradata.jdbc.jdbc_4.util.JDBCException: [Teradata Database] [TeraJDBC 14.00.00.01] [Error 6706] [SQLState HY000] The string contains an untranslatable character.

     at com.teradata.jdbc.jdbc_4.util.ErrorFactory.cloneJDBCException(ErrorFactory.java:169)

 at com.teradata.jdbc.jdbc_4.statemachine.PreparedBatchStatementController.handleRunException(PreparedBatchStatementController.java:93)

      at com.teradata.jdbc.jdbc_4.statemachine.StatementController.runBody(StatementController.java:126)

       at com.teradata.jdbc.jdbc_4.statemachine.PreparedBatchStatementController.run(PreparedBatchStatementController.java:56)

   at com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:372)

        at com.teradata.jdbc.jdbc_4.TDPreparedStatement.executeBatchDMLArray(TDPreparedStatement.java:219)

 ... 16 more

   at com.teradata.hadoop.mapreduce.TeradataRecordWriter.write(TeradataRecordWriter.java:64)

     at com.teradata.hadoop.mapreduce.TeradataRecordWriter.write(TeradataRecordWriter.java:23)

       at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:558)

      at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:85)

 at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:106)

  at com.teradata.hadoop.mapreduce.TeradataTextFileExportMapper.map(TeradataTextFileExportMapper.java:32)

      at com.teradata.hadoop.mapreduce.TeradataTextFileExportMapper.map(TeradataTextFileExportMapper.java:12)

  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)

   at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)

    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)

      at org.apache.hadoop.mapred.Child$4.run(Child.java:268)

  at java.security.AccessController.doPrivileged(Native Method)

  at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)

    at org.apache.hadoop.mapred.Child.main(Child.java:262)

2015-04-19 22:59:56,737 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task

6 REPLIES

Re: Sqoop export fails with untranslatable character error

Karthik,

Assume here you Export data from Teradata using Sqoop utility.

Check if any UNICODE column is part of export column list in Teradata. Use TRANSLATE_CHK on unicode data to verify translate conversion can be performed without production errors.

Cheers!!

Re: Sqoop export fails with untranslatable character error

Hi nandkarthik,

I am also facing same issue while exporting data from Hdfs to Teradata using sqoop export (internally uses teradata connector for hadoop).

are you able to find any solution for the above issue?

Re: Sqoop export fails with untranslatable character error

Hi,

I am also facing similar issue. I am able to load data with 1K bytes but data is more than 1MB it throws following error 

Error: com.teradata.hadoop.exception.TeradataHadoopSQLException: java.sql.BatchUpdateException: [Teradata JDBC Driver] [TeraJDBC 14.10.00.26] [Error 1338] [SQLState HY000] A failure occurred while executing a PreparedStatement batch request. Details of the failure can be found in the exception chain that is accessible with getNextException.

Anybody found solution for this issue.

Thanks

Re: Sqoop export fails with untranslatable character error

Hi,

We haven't found the solution. We restricted characters only to ASCII solved this issue on our side.

-Karthik

Teradata Employee

Re: Sqoop export fails with untranslatable character error

I have a similar issue.  The data is already ASCII and per the customers request we are using ASCII \037 as a seperator in our tdch cli scripts.  Upon execution we error out because the data is being stuffed into the  first column and is not understanding the ASCII seperator.  It does work when we translate the seperator to a \t.  Any suggestions if we can support ASCII seperators in TDCH because it works in pure, non-TD sqoop.

--Josh

Teradata Employee

Re: Sqoop export fails with untranslatable character error

We found a solution to this problem.  TDCH will take an ASCII seperator in the form of unicode.  '\037' = '\u001F'

--Josh