TPT help - Too few columns

Tools
Visitor

TPT help - Too few columns

I am working on a TPT script to load some large files (10GB) to tables. The files are '|' delimited.

 

For the table I am trying to load, the last 200 records are 'optional' for the line of business to send us. So we have a file that has the first 62 fields or so, and then maybe a few of the optional fields. So every record in the file will vary in how many fields are sent for that particular record.

 

The problem I am having is that the majority of the records do NOT have all 262 fields, but also do not send delimiters for the unused fields at the end of each record. So what we see is this (example)

A|B|C|||||||||D||||1|0||||||||||||||||||||||||||E|999|F|G|||H|1||||||2017-04-18|2017-08-13|I|||49|J|I|U|
A|B|C|||||||||D||||1|0||||||||||||||||||||||||||E|999|F|G|||H|1||||||2017-04-18|2017-08-13|I|||49|J|I|U|I|L|I|L|
A|B|C|||||||||D||||1|0||||||||||||||||||||||||||E|999|F|G|||H|1||||||2017-04-18|2017-08-13|I|||49|J|I|U|I|L|I|L||I|L|I|L|I|L|I|L|I|L|I|L|J|I|U|I|L|I|L||I|L|I|L|I|L|I|L|I|L|I|L|J|I|U|I|L|I|L||I|L|I|L|I|L|I|L|I|L|I|L|J|I|U|I|L|I|L||I|L|I|L|I|L|I|L|I|L|I|L|
A|B|C|||||||||D||||1|0||||||||||||||||||||||||||E|999|F|G|||H|1||||||2017-04-18|2017-08-13|I|||49|J|I|U|I|L|I|
A|B|C|||||||||D||||1|0||||||||||||||||||||||||||E|999|F|G|||H|1||||||2017-04-18|2017-08-13|I|||49|J|I|U|I|L|I|L|
A|B|C|||||||||D||||1|0||||||||||||||||||||||||||E|999|F|G|||H|1||||||2017-04-18|2017-08-13|I|||49|J|I|U|I|L|I|L||I|L|I|L|I|L|I|L|I|L|I|L|

 

Is there a way around that? Does TPT and/or FASTLOAD require that there be a delimiter/value for EVERY field in the define statement?

1 REPLY
Junior Contributor

Re: TPT help - Too few columns

Have a look at the AcceptMissingColumns option of the DataConnector.

 

Optional attribute that determines how rows in which the column count is less than defined in the schema are treated.
Valid values are:
'Y[es]' = the row is to be extended to the correct number of columns. Each appended column will be a zero length column and be processed according to the value of the NullColumns attribute. The edited record is sent to the Teradata Database and the original record is saved in the record error file.
'N[o]' = AcceptMissingColumns is not invoked (default).
‘YesWithoutLog’ = the edited row is sent to the Teradata Database, but the original record is not saved in the record error file.