How long it will take to insert 20 million rows to teradata table from java using jdbc drivers TYPE=FASTLOADCSV

Database

How long it will take to insert 20 million rows to teradata table from java using jdbc drivers TYPE=FASTLOADCSV

Hi All,

We want to  insert the bulk amount of data to Teradata tables from java application for Cost analytics project (it is a new project)  . As per my analysis & understanding , JDBC FastLoad CSV provides a method for quickly loading large amounts of data into an empty destination table in a Teradata Database.   JDBC FastLoad CSV is enabled with TYPE=FASTLOADCSV in the URL connection string. When enabled, the FastLoad protocol is used with the Teradata Database for FastLoadCSV-capable SQL INSERT statements

My questions is . How long it will take to insert the 20M rows to Teradata database from java class( using jdbc driver : ---TYPE=FASTLOADCSV)? Please advice

3 REPLIES
Enthusiast

Re: How long it will take to insert 20 million rows to teradata table from java using jdbc drivers TYPE=FASTLOADCSV

I don't think there is an absolute answer of this, you might need to look at some bench marking results for this but still it will be highly dependent upon I/O and network/communication capacity. 

http://developer.teradata.com/connectivity/articles/speed-up-your-jdbcodbc-applications might help in this.

Enthusiast

Re: How long it will take to insert 20 million rows to teradata table from java using jdbc drivers TYPE=FASTLOADCSV

Teradata Employee

Re: How long it will take to insert 20 million rows to teradata table from java using jdbc drivers TYPE=FASTLOADCSV

As Kawish said, the throughput that you obtain will be dependent on many factors, such as the number of columns, the average amount of data per row, the available network bandwidth, the CPU and I/O speed of the client system, and the performance (TPERF) of the Teradata Database system.

For example, if you are loading records with ten columns or fewer, where the average row size is 1KB or less, from a fast powerful client machine, over a quiet 1GB Ethernet network, then loading one million rows with JDBC FastLoad CSV would probably take somewhere between one and two minutes.

Extrapolating from that, loading 20 million rows might take somewhere between 20 and 40 minutes. Your mileage may vary.