Exporting to Multiple files in Fast Export

Tools
N/A

Exporting to Multiple files in Fast Export

Hi  All,



Can we export the output of multiple select statements specified with in .begin export and .end export to multiple outfiles ?

If we create multiple .begin export and .end export blocks with in a single fast export job, are they executed in parallel or sequencially ?

Thanks

ambuj

10 REPLIES
Teradata Employee

Re: Exporting to Multiple files in Fast Export

Not with FastExport.

But you can do this with TPT.

Multiple .begin export/.end export are executed sequentially.

-- SteveF
N/A

Re: Exporting to Multiple files in Fast Export

thanks Steve

Re: Exporting to Multiple files in Fast Export

Steve,

I tried it using tpt script with Max sessions as 5. It created 5 files but only one file had all the data and all other remaining files were empty.

Can you please let me know if I need to take care of anything else in the script to make it extract parallely?

USING CHARACTER SET ASCII

DEFINE JOB ec_rgtry_rlatnp_dat_extract

DESCRIPTION 'export ec_rgtry_rlatnp_dat_Job'

(

  DEFINE SCHEMA SCHEMA_ec_rgtry_rlatnp_dat_extract (

RGTRY_RLATNP_ID VARCHAR(40),

RGTRY_RLATNP_ASOCN_TYPE_CD VARCHAR(4),

SOR_ID VARCHAR(10),

ENT_CUST_ID VARCHAR(38)

);

DEFINE OPERATOR EXPORT_OPERATOR

TYPE EXPORT

SCHEMA SCHEMA_ec_rgtry_rlatnp_dat_extract

ATTRIBUTES (

UserName='AAAA',

UserPassword='BBBB',

TdpId='DB',

MaxSessions=5,

MinSessions=1,

SpoolMode='NoSpool',

VARCHAR DateForm = 'ANSIDATE',

SelectStmt = 'SELECT

 trim((OREPLACE(RGTRY_RLATNP_ID,x''0A'','''')) (VARCHAR(40))),

trim((OREPLACE(RGTRY_RLATNP_ASOCN_TYPE_CD,x''0A'','''')) (VARCHAR(4))),

TRIM(SOR_ID),

trim(((ENT_CUST_ID) (BIGINT) (VARCHAR(20))))

FROM DB.EC_RGTRY_RLATNP_PT

;',

VARCHAR ReportModeOn

);

DEFINE OPERATOR FILE_WRITER_ec_rgtry_rlatnp_dat_extract

TYPE DATACONNECTOR CONSUMER

SCHEMA SCHEMA_ec_rgtry_rlatnp_dat_extract

ATTRIBUTES (

FileName='20140505104506_abc.dat',

Format='DELIMITED',

TextDelimiter='|',

IndicatorMode='N',

OpenMode='Write'

);

APPLY TO OPERATOR (FILE_WRITER_ec_rgtry_rlatnp_dat_extract[5])

SELECT

RGTRY_RLATNP_ID ,

RGTRY_RLATNP_ASOCN_TYPE_CD ,

SOR_ID ,

ENT_CUST_ID

FROM OPERATOR (EXPORT_OPERATOR[5]);

);

Teradata Employee

Re: Exporting to Multiple files in Fast Export

Use the -C (uppercase) on the command line to round-robin the records to the files.

-- SteveF

Re: Exporting to Multiple files in Fast Export

I do not see -C at all as an option. Is it because of the version?

Usage: tbuild -f ScriptFile [-d | -t] [-e UTF16] [-m StatisticsInterval]

             [-n] [-r CheckpointDirectory] [-R RetryLimit] [-l secs] [-s JobStepID]

             [-h memorysize | -h memorysizeK | -h memorysizeM]

             [-S DD:ddname | -S Dataset name | -S SYSOUT class] [-u JobVariables]

             [-v JobVariablesFile] [-V] [-z CheckPointInterval] [JobName]

             [-c] [-L LogFilePath]

 Options description:

     -f ScriptFile            - Read Teradata Parallel Transporter script file

   [ -d ]                     - Debug trace from all tasks

   [ -t ]                     - Trace all tasks

   [ -e UTF16]                - Specify UTF16 encoding of script file

   [ -m StatisticsInterval ]  - Seconds between statistics collections

   [ -n ]                     - Continue job even if a step returns an error

   [ -c ]                     - Cyclic (round robin) data delivery

   [ -L LogFilePath ]         - Log file directory path (non-MVS only)

   [ -r CheckpointDirectory ] - Create checkpoint/restart files in this directory

   [ -l LatencyInterval ]     - Seconds between latency processing

   [ -R RetryLimit ]          - Number of automatic retries after restart

   [ -s JobStepID ]           - Start job execution at this job step

   [ -h memorysize ]          - Shared memory size specified in bytes

   [ -h memorysizeK ]         - Shared memory size specified in kilobytes

   [ -h memorysizeM ]         - Shared memory size specified in megabytes

   [ -S DD:ddname ]           - Log report ddname (MVS only)

   [ -S Dataset name ]        - Log report Dataset name (MVS only)

   [ -S SYSOUT class ]        - Log report SYSOUT class (MVS only)

   [ -u JobAttributes ]       - String of job variables

   [ -v JobAttributesFile ]   - Name of a file containing job variables

   [ -V ]                     - Display Teradata Parallel Transporter version

   [ -z CheckpointInterval ]  - Seconds between checkpoints

   [ JobName ]                - The name to be assigned to this job

Teradata Parallel Transporter Version 13.00.00.12

Re: Exporting to Multiple files in Fast Export

With lower case it did not export data into multiple files. Is it because of I had less data. I had only 25,000 rows.

Re: Exporting to Multiple files in Fast Export

it worked when I exported 13.3 million rows but with -c (lower case). Thanks Steve !!!

Teradata Employee

Re: Exporting to Multiple files in Fast Export

The lowercase -c is not what you think it is.  :)

What version of TPT are you using?

The only reason why rows were distributed when exporting 13.3 million is because there was enough work to spread to other file writer instances (but it was not round-robin).

The -C (uppercase) will round-robin the data.

(It might be missing from the usage notes, and I will have to look into that.)

I do not remember when we put -C into the code, but I think it was at least 14.0.

-- SteveF

Re: Exporting to Multiple files in Fast Export

Teradata Parallel Transporter Version 13.00.00.12