The Teradata Named Pipe Access Module (NPAM) provides an inter-process communication link between a writer process (such as FastExport) and a reader process (such as FastLoad). The NPAM can also be used by Teradata Parallel Transporter (TPT) for data transfer activity.
This article discusses the Fallback file that is used by the NPAM.
Checkpoint and restart are not possible with Named pipes because the Named pipes do not allow the user to reposition to an earlier position in the pipe. The Named Pipes Access Modules supplements the speed and convenience of the Named pipes with checkpoint and restart capability by caching data in a fallback data file. The NPAM module creates a fallback data file that grows large enough to hold a copy of all data passing through the module between two successive checkpoints. The below picture depicts the relationship between the NPAM and the fallback file. The fallback file is the basis for recovering the data during the data transfer errors. The NPAM access the fallback file to recover the data during the data transfer errors.
The Fallback file
The user can provide the name of the fallback file and its location via the
fallback_file and the
fallback_directory parameters respectively in the initialization string. For example, the following statement can be given in the TPUMP job script to create the Fallback file with the name test.fbf in the current directory.
.IMPORT INFILE np1 /* np1 is the name of the Named pipe used */
LAYOUT LAY1A /* LAY1A is the name of the data LAYOUT used */
APPLY LABELA /* LABELA is the name of the LABEL used */
FORMAT text /* fastload is the type of data FORMAT used */
axsmod np_axsmod.so 'log_level=5 log_directory=. buffer_size=32000 fallback_file=test.fbf fallback_directory=. ;
/* “np_axsmod.so” is the NPAM library name */
/* ‘Quoted string’ given adjacent to the NPAM library
is the Initialization string which is used to
Initialize the NPAM module */
The NPAM creates the Named pipe with the name provided in the
fallback_file parameter and at the location provided in the
fallback_directory parameter. If the user do not provide the name of the fallback file in the initialization string then the filename is constructed by the NPAM with the combination of the pipe name, the hash of the directory path of the Named pipe and a unique number. The name of the fallback file created by the NPAM is in the
ppppppphhhhnnnn.fbf format, where ppppppp is the name of the Named pipe, hhhh is the hash of the Named pipe’s directory path, nnnn is a unique hex value from 0000 to ffff and
.fbf is the file extension (which stands for fallback file).
We're currently having an issue with large sizes of the fallback files. I'm wondering if there is any way:
1. To increase frequency of checkpoints so that smaller size fallback files are created. But increasing frequency of checkpoints increases the overall job duration. We don't see the need to do frequent checkpoints for jobs that are non critical
2. To disable this fallback file feature altogether. We have some non-critical jobs that do not require checkpointing. If they fail midway, we can restart them from scratch. Currently we're facing issues with such jobs creating big fallback files and causing space alerts on unix servers.
I need to know how can be disable this fallback file feature. Thanks.
Please visit my article: https://developer.teradata.com/tools/articles/the-initialization-process-of-the-npam. This article teaches you about the various parameters that you can initialize with NPAM. Don't set "fallback_file" parameter.