FastLoad's useful -r option when loading a very large number of rows

Tools
Tools covers the tools and utilities you use to work with Teradata and its supporting ecosystem. You'll find information on everything from the Teradata Eclipse plug-in to load/extract tools.
Teradata Employee

FastLoad's useful -r option when loading a very large number of rows

While loading rows into a table, FastLoad displays, by default, an output message for every 100,000 rows loaded,  for example:

**** 14:47:20 Starting row 100000
**** 14:47:21 Starting row 200000
**** 14:47:22 Starting row 300000
...

If the number of rows being loaded is less than 100,000, FastLoad will not output any of these messages.

These messages are used as heartbeats that indicate that FastLoad is working for a long running job.

That was fine for jobs decades ago, however, for today's jobs where millions (and even billions) of rows are being loaded this much output may not be sensible. Can we imagine what the console output would look like with the default output message rate of every 100,000 rows?

Therefore, FastLoad has provided an option to control this output message rate. This option is -r specified on the command line when FastLoad is invoked.

The syntax is:

fastload -r outputRate

Where outputRate is an integer within the range of [1, 4294967295] inclusive.

If outputRate is specified outside this valid range or is not specified as an integer then a message indicating that an invalid rate has been entered will be displayed and the output rate is set back to the default, which is 100,000.

For example: if outputRate is specified as 20000000, and fl.txt is a script that loads 100 millions rows, the job is run as follows:

fastload -r 2000000 < fl.txt

The output rate messages will look like:

**** 19:49:07 Starting row 20000000
**** 19:49:12 Starting row 40000000
**** 19:49:17 Starting row 60000000
....

Five messages are displayed instead of about 1000 messages (which is what would occur if the default rate of 100000 was used).

Clearly the "-r" option is useful when loading a very large number of rows.

7 REPLIES
Enthusiast

Re: FastLoad's useful -r option when loading a very large number of rows

Dear Thomas,

Can you tell me the disadvantages of this option when loading small files?

Thanks a lot

BR

Roland
Teradata Employee

Re: FastLoad's useful -r option when loading a very large number of rows

Hi Roland,

This option is useful when loading a very large number of rows. I don't think there are any disadvantages of using it when loading small number of rows. In fact, we don't need this option for loading small number of rows.

Thanks,
Thomas
Enthusiast

Re: FastLoad's useful -r option when loading a very large number of rows

Hi Thomas, Thanks for the very good article. Is this option introduced with 13.10? My HP-UX fastload version 13.0.0.5 don't know this option. Thanks, Roland
Teradata Employee

Re: FastLoad's useful -r option when loading a very large number of rows

Hi Roland,
This option has been introduced since 13.01 on all platforms.
Thanks,
Thomas

Re: FastLoad's useful -r option when loading a very large number of rows

HI,

Thanks for the above inputs. could you please help me understand my fastload loader log is writing as
**** 06:47:20 Starting row 100000

like the above mentined way it is writing for every one lakh records.. does this mean teradata is committing for every 1L rows or as said above its just to say that session is still alive and showing the progress of longrun??
Teradata Employee

Re: FastLoad's useful -r option when loading a very large number of rows

Raja888,

Yes, it is just to say that everthing is working and showing the loading is in progess.

Thomas

Re: FastLoad's useful -r option when loading a very large number of rows

Many Thanks Thomas for clarifying..,

Satish.