Teradata Parallel Transporter External Command Interface

Tools
Tools covers the tools and utilities you use to work with Teradata and its supporting ecosystem. You'll find information on everything from the Teradata Eclipse plug-in to load/extract tools.
Teradata Employee

Teradata Parallel Transporter External Command Interface

The Teradata Parallel Transporter (TPT) External Command Interface is a command based interface which allows users to issue commands to TPT jobs. The term “external commands” implies two important implementations. First, it implies that users can issue commands to TPT jobs from outside the TPT address space. Secondly, it implies that commands are processed by TPT while it is in the middle of performing ETL operations. In addition, TPT internal components such as operators (which run under different processes) can also communicate with each other within a job through the same interface by using commands. As a result, ETL and system “events” are not only shared between the TPT and users, but also shared amongst TPT components within a job at runtime.

Using the command interface, you can interrupt a TPT job to perform job monitoring or controlling functions. You can define external procedures or “rules” for issuing these commands. This rule-enabling capability can help improve operational efficiencies, streamline processing of your ETL jobs, and enable the enterprise to react to events in a more timely fashion. All these commands work in an integrated manner with the existing features in TPT. For example, the “STATUS” command can collect the most current status information about a job, which by default, also collects status information at important milestones of the job. The “CHECKPOINT” command would take an immediate checkpoint, in addition to the originally scheduled checkpoints, such that queries right after the “CHECKPOINT” command would pick up the latest information from the database that is in the middle of being loaded. More importantly, you can also issue commands directly to operators for changing some of the execution parameters to improve performance or balance system resource usage. Therefore the use of these commands not only provides more monitoring capability and controllability to the active data warehouse, but also extends the existing TPT features to cope with real-time requests for providing you with the most accurate, consistent, and close-to-real-time information for tactical decision support.

TPT External Job Commands

On z/OS, external commands are sent to the target TPT job using the MODIFY system console command. On all other platforms, commands are sent using the TPT twbcmd command line utility:

For z/OS:

F <jobname>,APPL=<command_identifier> <command_text>

For all other platforms:

twbcmd <jobname> <command_identifier> <command_text>

Where <jobname> is the name of the target TPT job, <command_identifier> is a token identifying the type of command and the task in the target job that will receive and execute the command and <command_text> is the command sent to the processing task. The TPT command processor is case insensitive.

The syntax of the command interface requires the name of the target TPT job. On IBM’s z/OS operating system, the job name can be retrieved using standard system, JES (Job Entry Subsystem), or TSO/E (Time Sharing Option/Extensions) commands. On all other platforms, a list of the names of currently running TPT jobs can be obtained via the TPT twbstat command line utility:

#twbstat
Using job directory /home/cl151001/jobs
Jobs running: 3
cl151001-112
lol42000-133
dcc13370-147

As mentioned above, the TPT command interface is used by some operators to allow the dynamic modification of certain operator attribute values. The operators’ use of the command interface will not be discussed in this article. The command interface’s other use is the implementation of TPT external job commands by the TPT Global Coordinator to allow some management of active TPT jobs. The command_identifier for all job commands is, appropriately, job.

TPT external job commands allow TPT users to manage the execution of TPT jobs. The currently provided functions will:

  • Take a checkpoint and then suspend further processing.
  • Take a checkpoint and then continue the job.
  • Resume a previously suspended job.
  • Take a checkpoint and then terminate a job, retaining the checkpoint files for later restart.
  • View and log the status of a job.

Examples of the TPT external job commands follow.

Take a Checkpoint

An active job can be directed to take a checkpoint as follows, where <jobname> is the name of the target TPT job:

For z/OS:

F <jobname>,APPL=job checkpoint

For all other platforms:

twbcmd <jobname> job checkpoint

Upon receiving the checkpoint request, each operator instance immediately takes a checkpoint rather than waiting for the checkpoint interval to expire. After the checkpoint completes, the job continues processing.

Pause and Resume a Job

To suspend a job, issue the following command, where jobname is the name of the target TPT job:

For z/OS:

F <jobname>,APPL=job pause

For all other platforms:

twbcmd <jobname> job pause

Upon receiving the pause request, each operator instance immediately takes a checkpoint rather than waiting for the checkpoint interval to expire. After the checkpoint completes, the job suspends further processing until a resume command is received:

For z/OS:

F <jobname>,APPL=job resume

For all other platforms:

twbcmd <jobname> job resume

Terminate a Job

When a TPT job is terminated using a system command, such as kill, or the TPT twbkill command line utility, a checkpoint is not taken prior to termination. Restarting such a job may reprocess all data rows produced since the completion of the last checkpoint. As a consequence, the restarted job may experience errors such as the reinsertion of database table rows or the duplicate update of table rows. The termination of a TPT job via the external command interface avoids such problems by instructing each operator instance to take a checkpoint prior to beginning termination. A TPT job can be directed to terminate as follows, where jobname is the name of the target TPT job:

For z/OS:

F <jobname>,APPL=job terminate

For all other platforms:

twbcmd <jobname> job terminate

Upon receiving the terminate request, each operator instance immediately takes a checkpoint. After the checkpoint completes, the job terminates.

View and Log Job Status

A TPT job can be directed to display and log its status as follows, where jobname is the name of the target TPT job:

For z/OS:

F <jobname>,APPL=job status

For all other platforms:

twbcmd <jobname> job status

Upon receiving the status request, each operator instance displays the current count of rows sent and received and then writes a status record to the TWB_STATUS log.

Tags (2)