XML data format support by the TPT DataConnector Operator

Tools & Utilities
Enthusiast

XML data format support by the TPT DataConnector Operator

Hi,

I need to write the TPT script, which will read from JMS queue and load the data to Teradata table. I am using Teradata databse v12 and JMS access module for teradata was installed. As far as I know (in most of the cases) , body of the JMS queue message will be in XML format. However looks like TPT DataConnector Operator will not support the XML operator. (See here http://developer.teradata.com/tools/articles/external-data-formats-supported-by-the-tpt-dataconnector-operator). Please advise me on how to handle this situation. Thanks for all your help.

-Reddy
8 REPLIES
Enthusiast

Re: XML data format support by the TPT DataConnector Operator

Small correction..looks like TPT DataConnector Operator will not support the XML *format*.My question is "Can we read the JMS queue data (which is having XML format) , using the TPT scipt, which uses the JMS access module to read the data from JMS queue
Teradata Employee

Re: XML data format support by the TPT DataConnector Operator

The TPT DataConnector operator does not support the processing of data that exceeds 64000 bytes per row, and currently does not support any type of XML "data type".

I will have to investigate the JMS access module, but all of our access modules have to provide the data in a format that is understood by our load/unload utilities. So, if the JMS access module is converting the XML formatted data into CHAR/VARCHAR, then our load utilities would be able to load the data.

Afterall, we developed the JMS access module ourselves, and it is developed to work with our load tools.

However, I am not familiar with how it does any type of conversions and would have to look into that.

-- SteveF
Teradata Employee

Re: XML data format support by the TPT DataConnector Operator

Addition:

The rule using the axsmod is ‘the message has to be in a format compatible with our utility layout’.

The original problem statement seems to infer that data is somehow (by convention) in a certain format when it is placed on the queue. I believe the user is responsible for providing the data on the queue in the proper format, a format understood by our load tools.

Since our load tools do not support XML formats, or LOBs, you either need to provide the data in VARCHAR format, or as has been suggested to me by others, provide the data in an XML or LOB format and use TD XML services to shred.

By definition, though, the JMS access module (as with all of our access modules) expects the data on the queue to be ina format compatible with our utility layout.

-- SteveF
Enthusiast

Re: XML data format support by the TPT DataConnector Operator

Thanks feinholz! Informatica PowerCenter ( realtime with JMS license) was used to load the data in the queue. I heard that , with default settings , informatica will load as XML format to JMS message queue. I need to ask the informatica team to load the data in the queue, in some other format (not in XML) , to work with my TPT script. One more observation: Looks like XML data format support was available in Teradata Universal data connector 13, as mentioned in this article
http://developer.teradata.com/connectivity/reference/jms-install-guide/13-00-03/payload-examples-for-loader
Enthusiast

Re: XML data format support by the TPT DataConnector Operator

And even Teradata JMS Universal connector will also load the data into the message queue in XML format (by default). http://developer.teradata.com/connectivity/reference/jms-install-guide/13-00-03/overview
So this behavior is same as Informatica PowerCenter. So looks like everyone prefers the XML as the default format for the JMS message. Not sure how we missed the XML format read functionality in JMS access module +TPT :(
Teradata Employee

Re: XML data format support by the TPT DataConnector Operator

Our access modules are currently constructed to support the existing record formats by our utilities. XML format is not one of their formats, thus this is not something that was "missed". The user has always been responsible for supplying the data in one of our supported record formats, whether the data is placed in a flat file or on a queue.
-- SteveF
Enthusiast

Re: XML data format support by the TPT DataConnector Operator

Hi,

Is there any volume limit on the messages in the queue? I have a scenario to extract millions of rows from Teradata table and put in JMS queue. I am not sure if I can put everything at once?

Thanks

Sundar

Teradata Employee

Re: XML data format support by the TPT DataConnector Operator

If by volume you mean system capacity, you would need to check with the administrator or perhaps the documentation for whatever the provider is.  

-- SteveF