Thanks in advance for reading this.
I am trying to pull data from Teradata and load it into MySQL without hitting the disk.
I am currently doing it with a perl script and connecting BTEQ to LOAD DATA INFILE with a named pipe in linux.
Now I am trying to convert this perl script into java so that I can integrate it with our other ETL processes and would also like to get away from manipulating command line executables. Now I know I can do LOAD DATA INFILE through the MySQL JDBC driver and read from a pipe. What I have yet to figure out is how to read the data, using a pipe (or PipedOutputStream), from Teradata.
Can anyone point me in the right direction? What tool or library should I be using here?
Once I figure this out, I will be sure to come back and post my results.
You would need to write a Java program that uses the Teradata JDBC Driver to execute a SQL query and read the rows from the result set. As the Java program reads each row from the Teradata result set, it would write the data to the named pipe.
Teradata JDBC Driver sample programs are available here:
In particular, sample program T20300JD shows how to execute a SQL query and read rows from the result set: