Processing files from S3 using Teradata TPT

Tools & Utilities

Processing files from S3 using Teradata TPT

We have a scenario to process a large set of files from S3. we are using Teradata Access Module for Amazon S3. Looks like this utility has some limitation on the file size as when we are trying to process large set of files. After processing 20 files each of size 160 MB it is showing below messages and the connection is getting closed.

S3AXM [25510]: AWS S3 ACCESS Module Shutdown Completed is not an error , its closing access module connection.

 

We are using below option in our script 

S3ConnectionCount=50

S3BufferSize=128M 

still its not working.

 

I would like to know the usage of S3SinglePartFile = false. In documentation I didn't find any example. It was mentioned we can use this option in case of processing large set of files which matches my requirment.

If we mention S3SinglePartFile = false what we need to mention in below parameters. 

 

S3Object=

S3Prefix=

Any example please. 

 

Thanks & Regards,

Shayan