TDLoad Hangs when invoked from Python

Tools & Utilities
Enthusiast

TDLoad Hangs when invoked from Python

Hi,

I'm wondering if you are aware of some weird bug in TDLOAD that makes it hangs in certain condition.

I have very strange situation:

1) tdload loads data and exits properly when invoked from command line

2) tdload loads data and exits properly when invoked from shell file

3) tdload loads data and exits properly when invoked from a basic python program

4) tdload loads data but hangs when invoked from a more elaborate python program where couple of modules call each other and then one inner one calls tdload

Note:

1) Command line is exactly the same in all 4 scenarios

/opt/teradata/client/13.10/tbuild/bin/tdload  -u freidere -p change_me -h phx2-dw-td-dev-db.cnet.com -t pt_fragment -d '|' -f /var/opt/insight/dev_freidere/data/fragment.load  load_pt_fragment_20111212

2) I printed 'env' and variables are exactly the same in all 4 scenarios

3) When tdload hangs I see a bunch of teradata processes in ps including:

tdload, tbuild, ptmexec, JobLogger, ptmgts, ptmtims, twbmdlgr, ptmcmd

I tried connecting to most of them via gdb and as I can see most of them are just waiting for children to finish but I have no idea why a final child doesn't exit.

4) tdload definitely has rights to write to the current dir as otherwise it just fails to run.

5) I tried this with tdload 13.10.00.04 and latest 13.10.00.07. Both behave the same :-(

At this point I'm quite desperate as I tried multiple ways of a script invocation from Python and all of them work from a simple python program but hang when invoked from a more complicated system :-(

Any ideas what could make tdload hang?

Thanks in advance

Tags (1)
3 REPLIES
Teradata Employee

Re: TDLoad Hangs when invoked from Python

What is different between the "simple" python program and the "complicated" one?

If you can get TPT to run from within a "simple" python program, I fail to see how you would call this failure a "tdload bug".

The hang could be due to several factors I suppose, mostly around signals or semaphores.

You do not indicate which version of TPT you are using, the platform, any output you see from the job, or anything that would give any type of clue to engineers to help diagnose the problem.

-- SteveF
Enthusiast

Re: TDLoad Hangs when invoked from Python

I have no idea what is the difference between a simple python program vs more complicated one. The only difference is that a simple program is just 1 script file while a more complicated one is an object oriented system which defines Python classes/etc however tdload invocation is the same in both.

I got it to work on 32bit CentOs EL5 however it still hangs on 64bit Cent OS El5.

The job output is all successful but it hangs at the end, see the output below:

=================

Teradata Parallel Transporter Version 13.10.00.07

Job log: /var/opt/teradata/tbuild/logs/load_pt_fragment_20111214075854-102.out

Job id is load_pt_fragment_20111214075854-102, running on c13-ad-integ1.cnet.com

Teradata Parallel Transporter DataConnector Version 13.10.00.07

$FILE_READER Instance 1 directing private log report to 'FileReaderLog-1'.

$FILE_READER: TPT19008 DataConnector Producer operator Instances: 1

Teradata Parallel Transporter Load Operator Version 13.10.00.04

$LOAD: private log specified: LoadLog

$FILE_READER: TPT19003 ECI operator ID: $FILE_READER-15666

$FILE_READER: TPT19222 Operator instance 1 processing file '/var/opt/insight/dev_freidere/data/fragment.load'.

$LOAD: connecting sessions

$LOAD: preparing target table

$LOAD: entering Acquisition Phase

$LOAD: entering Application Phase

$LOAD: Statistics for Target Table:  'pt_fragment'

$LOAD: Total Rows Sent To RDBMS:      3

$LOAD: Total Rows Applied:            3

$LOAD: disconnecting sessions

$FILE_READER: TPT19221 Total files processed: 1.

$LOAD: Total processor time used = '1.35 Second(s)'

$LOAD: Start : Wed Dec 14 07:59:00 2011

$LOAD: End   : Wed Dec 14 07:59:23 2011

Job step MAIN_STEP completed successfully

Job load_pt_fragment_20111214075854 completed successfully

Teradata Load Utility Version 13.10.00.07

=================

As noted originally both tdload 13.10.00.04 and latest 13.10.00.07 don't work.

Teradata Employee

Re: TDLoad Hangs when invoked from Python

Sorry. I do not have an answer for you.

I also do not know who is putting out the very last message, "Teradata Load Utility Version 13.10.00.07".

It is not a normal TPT message.

-- SteveF