The NPAM Fallback file

Tools
Tools covers the tools and utilities you use to work with Teradata and its supporting ecosystem. You'll find information on everything from the Teradata Eclipse plug-in to load/extract tools.
Teradata Employee

The NPAM Fallback file

The Teradata Named Pipe Access Module (NPAM) provides an inter-process communication link between a writer process (such as FastExport) and a reader process (such as FastLoad).  The NPAM can also be used by Teradata Parallel Transporter (TPT) for data transfer activity.

This article discusses the Fallback file that is used by the NPAM.

Background of the Fallback file requirement

Checkpoint and restart are not possible with Named pipes because the Named pipes do not allow the user to reposition to an earlier position in the pipe. The Named Pipes Access Modules supplements the speed and convenience of the Named pipes with checkpoint and restart capability by caching data in a fallback data file.  The NPAM module creates a fallback data file that grows large enough to hold a copy of all data passing through the module between two successive checkpoints. The below picture depicts the relationship between the NPAM and the fallback file. The fallback file is the basis for recovering the data during the data transfer errors.  The NPAM access the fallback file to recover the data during the data transfer errors.

The Fallback file

The user can provide the name of the fallback file and its location via the fallback_file  and the fallback_directory parameters respectively in the initialization string.  For example, the following statement can be given in the TPUMP job script to create the Fallback file with the name test.fbf in the current directory.

.IMPORT INFILE np1 /* np1 is the name of the Named pipe used */
LAYOUT LAY1A       /* LAY1A is the name of the data LAYOUT used */
APPLY LABELA       /* LABELA is the name of the LABEL used */
FORMAT text        /* fastload is the type of data FORMAT used */
axsmod np_axsmod.so 'log_level=5 log_directory=. buffer_size=32000 fallback_file=test.fbf fallback_directory=. ;
                         /* “np_axsmod.so” is the NPAM library name */
                        /* ‘Quoted string’ given adjacent to the NPAM library
                            is the Initialization string which is used to
                            Initialize the NPAM module */

The NPAM creates the Named pipe with the name provided in the fallback_file parameter and at the location provided in the fallback_directory parameter.  If the user do not provide the name of the fallback file in the initialization string then the filename is constructed by the NPAM with the combination of the pipe name, the hash of the directory path of the Named pipe and a unique number. The name of the fallback file created by the NPAM is in the ppppppphhhhnnnn.fbf format, where ppppppp is the name of the Named pipe, hhhh is the hash of the Named pipe’s directory path, nnnn is a unique hex value from 0000 to ffff and .fbf is the file extension (which stands for fallback file).

4 REPLIES
Enthusiast

Re: The NPAM Fallback file

Teradata TPT OUTLIMIT value is ignored. BTEQ RETLIMIT and FEXP OUTLIMIT work fine. Anyone knows anything to do with TPT OUTLIMIT?
I defined INTEGER OutLimit = 1,
in TPT but it retrieved all rows anyway.
Enthusiast

Re: The NPAM Fallback file

Hi,

We're currently having an issue with large sizes of the fallback files. I'm wondering if there is any way:

1. To increase frequency of checkpoints so that smaller size fallback files are created. But increasing frequency of checkpoints increases the overall job duration. We don't see the need to do frequent checkpoints for jobs that are non critical

2. To disable this fallback file feature altogether. We have some non-critical jobs that do not require checkpointing. If they fail midway, we can restart them from scratch. Currently we're facing issues with such jobs creating big fallback files and causing space alerts on unix servers.

I need to know how can be disable this fallback file feature. Thanks.

-Suhail

Teradata Employee

Re: The NPAM Fallback file

Hi Suhail,

Please visit my article: https://developer.teradata.com/tools/articles/the-initialization-process-of-the-npam.  This article teaches you about the various parameters that you can initialize with NPAM.  Don't set "fallback_file" parameter.

Thanks,

Sastry

Enthusiast

Re: The NPAM Fallback file

Hi Sastry,

Thank you for the quick response. So essentially you're saying if we remove the paramters:

confirm_fallback_deletion, fallback_directory and fallback_file from the initializing string

the fallback files will not get created, correct?

Also, all these jobs run in ODI (Oracle Data Integrator).  

Would you happen to know how to disable these parameters in ODI?

We will also continue researching this ourselves and check with ODI support on this.

Thank you once again for your responses.

-Suhail