The Named Pipe Access Module introduces the “pipe_wait” parameter
About the NPAM
The Teradata Named Pipe Access Module provides an inter-process communication link between a writer process (such as FastExport) and a reader process (such as FastLoad).
Where it started:
One of our customers reported a performance problem when data is transferred via Named Pipes access module (NPAM) using non-blocking Named Pipe. In non-blocking mode, the data is read from the pipe until end-of-file (EOF) is received. In order to fix the performance problem, a new feature was implemented in the NPAM where “pipe_wait” parameter was introduced.
The following picture depicts how the NPAM polls the pipe when the data is transferred using Named Pipe in non-blocking mode.
How it happens:
The NPAM functionality before introducing this feature
The sleep interval for polling the pipe when Named Pipe is operated in non-blocking mode was 1 second or 1000 milliseconds (not user configurable). The disadvantage of this behavior is that it increases the waiting time and thereby hinders the performance of the data transfer activity.
The NPAM functionality after introducing this feature
The sleep interval for polling the pipe when Named Pipe is operated in non-blocking mode is made user configurable by introducing a new parameter “pipe_wait”. This parameter takes the time interval value (in milliseconds) which is used for sleep operation while polling the pipe for the availability of the data to be transferred in non-blocking mode. The default value of this parameter is 10 milliseconds. Please note that the value to this parameter is passed by the user in the initialization string while invoking the NPAM library.
The dynamic incremental sleep interval has been implemented (whose logic is depicted in below diagram) to reduce the wait time during the pipe polling activity.
Examples for dynamic incremental sleep interval (based on the logic implemented):
If the “pipe_wait” time given by user is 50 milliseconds, then the sleep interval values will be 1, 2, 4, 8, 16, 32, 50, 50, 50 …
If the “pipe_wait” time is not given by the user then the “pipe_wait” parameter is set to 10 milliseconds (its default value). In this case, the sleep interval values will be 1, 2, 4, 8, 10, 10, …
Benefits of this feature:
With the introduction of this feature, the user can configure the value of the time interval used for sleep operation while polling the pipe.
Before introducing this feature, the NPAM used a hardcoded value of 1 second or 1000 milliseconds which caused too many lengthy delays especially in the cases where short wait duration is sufficient. This feature will overcome this disadvantage.
This feature implemented the logic to dynamically increment the sleep interval starting from 1 millisecond and doubling it steps until it reaches the user configured time interval value (via “pipe_wait” parameter). The shorter sleep interval (used in the logic) further reduces the wait time during polling pipe activity and helps in improving the performance.