Understanding Named Pipe Access Module data transfer

Tools
Tools covers the tools and utilities you use to work with Teradata and its supporting ecosystem. You'll find information on everything from the Teradata Eclipse plug-in to load/extract tools.
Teradata Employee

Understanding Named Pipe Access Module data transfer

The Teradata Named Pipe Access Module (NPAM) provides an inter-process communication link between a writer process (such as FastExport) and a reader process (such as FastLoad).

NPAM – Data transfer

The below block diagram (which is self-explanatory) depicts the transfer of data between a writer process (such as the Teradata FastExport utility) and a reader process (such as the Teradata FastLoad utility).  The reader process interacts with Named Pipes Access Module via the Teradata Data Connector Module. It is not necessary for the writer process to use the Teradata Data Connector to interact with the NPAM.

The data transfer between writer process and reader process is performed as follows:

  1. The reader process loads and initializes the NPAM library via the Teradata Data Connector. This step includes the creation of the named pipe.
  2. The writer process writes the data (in the same format that the reader process expects) on to the named pipe. Note that the writer process does not require an instance of the Data Connector Module. The writer process can be any third-party application that supplies data through the named pipe in the format the reader process expects.
  3. The NPAM tracks the data flow, copies the inbound data into a fallback data file. The data is read by the reader process. Note that the NPAM creates and manage the fallback data file throughout the data transfer activity. Note that the fallback data file is deleted by the NPAM after the completion of the job.
  4. The data transfer is performed by the writer and reader processes until the end-of-file (EOF) is reached.
  5. If the reader process determines that it must fall back to an earlier point in the data stream, it issues the standard File Set Position access command to NPAM (via the Teradata Data Connector), which then supplies subsequent reads from the data it saved in the fallback data file.

Please note,

  • The reader and the writer processes should use the same name of the Named pipe for the read operation and write operation respectively.
  • The reader and the writer processes should use the same format of the data being transferred via the Named pipe.  That is, the format of the data written by the writer process should be understandable by the reader process.

Example to illustrate the usage of the NPAM

The reader process (such as the Teradata TPUMP) can use the following statement in the job script to use the NPAM module for the data transfer.

.IMPORT INFILE np1  /* where “np1” is the name of the Named pipe used */
LAYOUT LAY1A /* where “LAY1A” is the name of the data LAYOUT used */
APPLY LABELA /* where “LABELA” is the name of the LABEL used */
FORMAT fastload /* where “fastload” is the type of data FORMAT used */
axsmod np_axsmod.so 'll=5 ld=. b=32000 fd=. cfd=n sc=2' ;
/* “np_axsmod.so” is the NPAM library name */
/* ‘Quoted string’ given adjacent to the NPAM library
is the Initialization string which is used to
Initialize the NPAM module */

Writing data via the Data Connector

The writer process (such as Teradata FastExport) can use the following statement in the job script to write the data (to be transferred) on to the Named pipe.  The Teradata FastExport is dependent on the Data Connector to perform the I/O operations. Hence the data to be transferred is written by the Teradata FastExport via the Data Connector module on to the Named pipe.

.EXPORT OUTFILE np1; /* where “np1” is the name of the Named pipe. */ 

Writing data without using the Data Connector

The data can be written on to the Named pipe by using simple “cat” command as mentioned below.

$ cat data_file > np1  

where,

  • “data_file” contains the data to be transferred to the reader process.
  • “np1” is the name of the Named pipe being used for the data transfer.
9 REPLIES
Enthusiast

Re: Understanding Named Pipe Access Module data transfer

Very nice article. Since new to the NPAM, I would like to understand the usage of it in a script from end to end. what I meant by that is an illustration of the script for the Reader process and the writer process into and from a table.
FastExport-->NPAM world -->FastLoad.
Not sure if I'm asking correctly but I want to understand end to end. Look forward to your response.
Teradata Employee

Re: Understanding Named Pipe Access Module data transfer

Hey Vasu...thanks for your comments.
I am providing you the Fastexport and TPUMP scripts which transfers the data via NPAM. You can follow the similar syntax (to invoke NPAM) provided in TPUMP script into your Fastload script. Please refer the product manual for more details. Hope this helps !!

TPUMP Script
===========
.logtable TPUMP_NPAMLOG;
.LOGON /, ;

CREATE TABLE table1 (c1 integer,c2 char(10))
unique primary index(c1);

.BEGIN LOAD SESSIONS 4;
.LAYOUT LAY1A ;
.FIELD C1 * integer;
.FIELD C2 * char(10);
.DML LABEL LABELA;
INSERT INTO table1 VALUES (:C1,:C2);

/* npp1 is the name of the pipe being used for
the data transfer */

.IMPORT
INFILE \.\pipe\npp1
LAYOUT LAY1A
APPLY LABELA
format fastload
axsmod np_AXSMOD.dll 'll=2 ld=. b=32000 ff=test.fbf fd=. cfd=n sc=2 ' ;
.END LOAD;
.LOGOFF;

FASTEXPORT Script
================

.logtable FEXP_NPAMLOG;
.logon /, ;

ct temp(f1 int, f2 char(10));
ins into temp(1,'hello1');
ins into temp(2,'hello2');

.begin export sessions 4;
.export outfile \.\pipe\npp1 mode record;

/* npp1 is the name of the pipe being used
for data transfer */

sel * from temp;
.end export;
.logoff;
Teradata Employee

Re: Understanding Named Pipe Access Module data transfer

The sample scripts provided by me in the above comment can be executed on WINDOWS platform only (not on UNIX). Different syntax is used for invoking NPAM on UNIX platforms. You can get more related information to this context in the NPAM manual.

Re: Understanding Named Pipe Access Module data transfer

I am trying to load oracle data into teradata using NP do you have any sample script for that? thanks for your help.-Selva
Teradata Employee

Re: Understanding Named Pipe Access Module data transfer

Selva,
I am sorry my friend; Please kindly refer the manuals of NPAM, Fastexport, Fastload and TPT for the sample scripts. Please note, NPAM is also used by TPT for data transfer.
Enthusiast

Re: Understanding Named Pipe Access Module data transfer

Hello Sastry.. Is there a NPAM for HP IA64 to use with TPT ?
Teradata Employee

Re: Understanding Named Pipe Access Module data transfer

Yes. NPAM for HP IA64 is available.
Enthusiast

Re: Understanding Named Pipe Access Module data transfer

how do i get the Version of NP access module installed

Teradata Employee

Re: Understanding Named Pipe Access Module data transfer

Hi Vijay,

You can get the NP access module version information from the log file generated by Data connector module.  In order to make Data connector generate log, you need to add ".debug trace io high" statement in your job script(like multiload/fastexport/fastload/tpump scripts).  After execution of the script, you can find that Data connector generated log (in the current directory with the file name format 'pm................'), which contains version of Data connector and NPAM modules.

Thanks,

Sastry