Loading carriage returns with TPT Load Operator

Tools & Utilities

Loading carriage returns with TPT Load Operator

Hey Guys,

I am loading a delimited data file (using '|') with newline characters embedded within the fields, which the customer would like to preserve. How do I do that?

There doesn't appear to be any option in TPT Producer Operator for changing the record delimiter. I've tried escaping the newline characters but that just inserts the literal two characters "\n" in place of each newline character... Any ideas would be appreciated.
10 REPLIES
Teradata Employee

Re: Loading carriage returns with TPT Load Operator

We are working on this in the 14.0 release. Once that is complete, we will hopefully be able to backport that feature to earlier versions of TPT but that effort has not been committed yet.
-- SteveF

Re: Loading carriage returns with TPT Load Operator

Is this feature implemented in TPT 14.

Teradata Employee

Re: Loading carriage returns with TPT Load Operator

No. It did not make it in to 14.0 or 14.10.

We will look into this as a post-14.10 activity.

-- SteveF
Senior Supporter

Re: Loading carriage returns with TPT Load Operator

Will it make in 15.0?

It creates so much overhead if you have to deal with it in preprosessing. These social network sources are full of this as it all about free text fields...

Teradata Employee

Re: Loading carriage returns with TPT Load Operator

No, it did not make it into 15.0, and may not even make it into 15.10.

-- SteveF
Senior Supporter

Re: Loading carriage returns with TPT Load Operator

sad, sad...

Enthusiast

Re: Loading carriage returns with TPT Load Operator

For this scenario of loading \n embedded in data fields, is that possible with any other operators in TPT at all ? 

Teradata Employee

Re: Loading carriage returns with TPT Load Operator

The DataConnector is the only operator that processes data from flat files and it does not currently support newlines embedded in data.

(We are actually working on that feature now as part of our 16.0 release, and will eventually roll that feature back to prior releases, but no timetable yet.)

-- SteveF
Enthusiast

Re: Loading carriage returns with TPT Load Operator

Is this feature any closer to being implemented yet?

We are looking at loading some files which contain email text as a field.  We could quote the field using some obscure unicode character, but if tpt still thinks the end of line is an end of record then it isn't going to work.  Alternatively if we had AppendDelimiter = 'Y' set then TPT could only end the record if it had the delimiter character immediately before the end of line.

 

I think we may be able to load the data if we change it so that the records have a windows format CR-LF end of line marker, and the email fields have just unix format LF for the new line.  This will take some preprocessing of the file in unix, which is not ideal but should work.  I think we would need to make sure that the first row is a header record which didn't have the unix LF end of line marker in it.

Tags (1)