I need to load numerous flat files using FastLoad. Is it possible to loop through the file names in FastLoad so I don’t have to change the file name for each data file that needs to be loaded in the FastLoad script?
Let’s say I have flat files FileName1, FileName2, FileName3, etc. They all need to be loaded to the same table TableName1.
, Field2 ...
BEGIN LOADING TableName1 ERRORFILES TableName1_E1, TableName_E2;
INSERT INTO TableName1 VALUES ( . . . . );
Do I have to change the "FILE=FileName1" for each file and run the script? Or, is there a way to provide a list of file names for the FastLoad to loop through?
This is a sitting duck for TPT LOAD operation!!! You can setup a dataconnector (it reads flat files) with a wildcard on the filename.
TPT will suck up the files in parallel vs. one at time if you looped fastload. If the files are large, take the time to do TPT.
I think you can go one step further.
If you use the Easy Loader feature of TPT, you can do this from a command line without a script, using "tdload".
The syntax would be something like this:
$ tdload -f "*.dat" -t <tablename> -h <tdpid> -u <username> -p <password>
There is a DevX article on how to use "tdload". We do not document that you can use the wildcard syntax but I believe it works.
"tdload" will generate the TPT script for you (and then delete it when it is done).
Thanks Q, mnylin, and feinholz for your responses! TPT seems to be the answer. I am very new to Teradata. So I need to do some digging and learn more about TPT.
Also, I found the article on Easy Loader for TPT (http://developer.teradata.com/tools/articles/easy-loader-for-teradata-parallel-transporter) mentioned by feinholz.