The Teradata Parallel Transporter (Teradata PT) operators play a vital role in high-speed data extraction and loading geared towards the Teradata Database. Besides interfacing with the Teradata Database, some of the Teradata PT operators provide access to external sources such as files, ODBC-compliant DBMS, and message-oriented middleware.
There are four different types of operators: 1) producer operators, 2) consumer operators, 3) standalone operators, and 4) user-written custom operators.
Producer operators read data from a source and send the data to the Teradata PT data streams. Data streams are in-memory buffers that allow the data to be passed from a producer operator to a consumer operator without landing the data to disk. Teradata PT provides the following producer operators:
• Export operator: Utilizes multiple sessions of the Teradata FastExport protocol to extract large volumes of data at fast speeds from one or more tables on the Teradata Database.
• SQL Selector operator: Uses a single Teradata SQL session to extract low volumes of data from a table on the Teradata Database. The SQL Selector operator has two features not available in the Export operator. The SQL Selector operator can extract data in “field mode” (i.e. columns in VARCHAR format) and pass the data to the Data Connector operator in order to have the data written in delimited format to an external file. The SQL Selector operator can extract LOB data.
• Data Connector operator: As a producer operator, the Data Connector operator reads data in parallel from multiple sources such as files, named pipes, access modules, and message queue systems.
• FastLoad INMOD Adapter operator: Reads data from any Teradata FastLoad INMOD routines. The INMOD routines are user-written. The INMOD routines preprocess the data before the data are passed to the FastLoad INMOD Adapter operator.
• MultiLoad INMOD Adapter operator: Reads data from a Teradata MultiLoad or TPump INMOD routine. The INMOD routine is written by the user. The INMOD routines preprocess the data before the data are passed to the MultiLoad INMOD Adapter operator.
• ODBC operator: Reads data from many ODBC-compliant DBMS, such as Oracle, SQL Server, and DB2.
Consumer operators “consume” data from the Teradata PT data streams and send data to the Teradata Database or files. Teradata PT provides the following consumer operators:
• Load operator: Utilizes multiple sessions of the Teradata FastLoad protocol to insert large volumes of data at fast speeds into an empty table on the Teradata Database.
• Update operator: Uses multiple sessions of the Teradata MultiLoad protocol to perform Inserts, Updates, Deletes, and Upserts into empty or populated tables (up to 5 tables concurrently) on the Teradata Database.
• Stream operator: Uses multiple Teradata SQL sessions to perform Inserts, Updates, Deletes, and Upserts into empty or populated tables (up to 127 tables concurrently) on the Teradata Database without locking the tables. Locking occurs at the row level.
• SQL Inserter operator: Uses a single Teradata SQL session to insert low volumes of data into an empty or populated table on the Teradata Database. The SQL Inserter operator has a feature not available in the Load, Update, or Stream operator. The SQL Inserter operator can load LOB data.
• Data Connector operator: As a consumer operator, the Data Connector operator writes data to files or access modules.
• FastExport OUTMOD Adapter operator: Writes data to a Teradata FastExport OUTMOD routine. The OUTMOD routine is written by the user. The OUTMOD routines post process the data.
Standalone operators perform unique functions that do not involve sending data to, or receiving data from, the Teradata PT data streams. Teradata PT provides the following standalone operators:
• DDL operator: Sends SQL requests to the Teradata Database to perform setup operations prior to running load or export tasks. For example, before loading the data using the Load operator use the DDL operator to send the DROP TABLE and CREATE TABLE requests to ensure the target table is empty.
• OS Command operator: Sends commands to the client OS. Commands can be operating system commands, executables, or shell scripts.
• Load operator: As a standalone operator, the Load operator sends a request to the Teradata Database to apply the data on the target table. For example, when there is no more data in a staged loading, use the Load operator as a standalone operator to apply the data already in the target table.
• Update operator: As a standalone operator, the Update operator performs the Delete task on a single table on the Teradata Database. The Delete task performs faster than using a single DELETE SQL statement when a large percentage of rows in a table need to be deleted, such as deleting all rows with a transaction date prior to a certain date.
Custom operators are user-developed operators that must follow the requirements of the Teradata PT operator interface. Custom operators must be written in C or C++ programming language. Teradata PT provides a sample custom operator when you install Teradata PT.
I have a big file(50GB) sitting in UNIX which is generated from TPT fast export. The file contains sales details of the individuals. From this file we are generating separate files for each distinct individual using UNIX commands. For Example
Individual ID SalesAmt
We are generating 3 files here for 3 distinct individuals.
can we do this while we generate the file itself using TPT fast export. I heard about OUTMOD routine option but don't know how to use this.
Thanks in advance.
You can use the TPT FastExport OUTMOD Adapter operator with your user-written OUTMOD library.
The TPT FastExport OUTMOD Adapter operator can pass the data to your user-written OUTMOD library.
Your user-written OUTMOD library can generate separate files for each distinct individual.
See the TPT Reference manual for more information about the TPT FastExport OUTMOD Adapter operator.
See the TPT sample user guide #12 for a sample TPT script that uses the TPT FastExport OUTMOD
Adapter operator. The TPT sample user guide is available after you have installed the TPT
See the Teradata FastExport manual for information on how to create and compile your
user-written OUTMOD library.
Your user-written OUTMOD library should be compatible with TPT.
Thanks a lot Tony. Could you have all these documents available online, that would be a great help.
The documents are available online at this URL:
I am trying to develop an outmod c script, and I am getting below error.
The OutmodName attribute refers to the name of your FastExport OUTMOD library name, not the OUTMOD source file.