Load Kafka Stream to Teradata via Kafka Access Module

General
Senior Supporter

Load Kafka Stream to Teradata via Kafka Access Module

Hi all,

I got the Kafka Access Module working for TPT.

I am interested in loading an continous stream of events via TPT Stream from an Kafka cluster.

Currently I have a problem to set this up.

Main issue is that the apply phase is only starting once TPT recieves and end of file...

If the stream is not continous the RWAIT parameter can be used to send this signal. But this is not what I need...

If the stream is always putting data into the Kafka topic the signal will never send...

I would like to stream the data as it arrives and don't want to run small batches....

 

Any ideas? Or won't this work with TPT and the Kafka Access Module?

Br Ulrich

1 REPLY
Senior Supporter

Re: Load Kafka Stream to Teradata via Kafka Access Module

Ok, some more clarity!

The examples had been to small of volume.
I wrote a data producer which is now generating any number of rows and now I see the loading also starting....

Where number of rows processed is very slow, but this might have other issues... Will check further.