TRM Server Logging Basics

Applications
Applications covers the world of Teradata apps, including apps offered by Teradata (such as TRM or DCM), as well as best practices and methodologies for building your own Teradata-connected apps.
Teradata Employee

TRM Server Logging Basics

This article describes the basics of TRM's V6 logging in the application server. This includes log file configuration, versioning, location and content.

  • TRM logging is currently configured to generate several logging outputs:

    • The "console" display is the log that is displayed in the application server's “sysout” device:
      • Running Tomcat, the command window used to start Tomcat serves as sysout.
      • Running Websphere Application Server (WAS), the sysout stream is actually written to a file, <WAS root>/profiles/<AppSrv>/dist/logs/system.log. You can use an editor or "tail" to view this, or see Log browser to set up a display con-sole.
  • The file trm-system.log is the main application log file. For development, it is usually configured to provide more detail than a normal production setting. The "root" location of trm-system.log depends on the web server:
    • For Tomcat, launched from command-line, the default root is <tomcat root>\bin\dist, where on-line file logs are written under the directory "logs".
    • For Websphere, the default logging root is
      • <WAS install- root>\AppServer\profiles\AppSrv01\dist,
      • On-line file logs are written under "..dist\logs". Websphere writes several ad-ditional log files that may be useful for diagnosing general Websphere server issues. To locate these additional log files, use the WAS Admin Console to navigate to "Environment > WebSphere variables > LOG_ROOT.
  • TRM's Processing Engines (PE) are defined to handle background or off-line job exe-cution. The PE's write 2 log files for each job which share the same location root as the trm-system.log file, adding additional directories "..dist\trm\processing-logs\<date>" where "date" is in the form "yyyy-mm-dd".
  • TRM also defines a separate log file to protocol any events that the application re-ports at ERROR level. This file, trm-error.log, is currently written to the same di-rectory as the trm-system.log.
  • You can override this root log file location in the configuration file, log4j.xml (see Config file root).

General Logging

General logging (non-PE) is handled by an open-source component called Log4J, and is configured by the file log4j.xml. Logging Processing Engine (PE) activity also uses Log4j internally, though is not user-configurable and does not use the log4j.xml configuration file.

This log4j.xml file declares basically 3 types of objects:

  • appender (specifies the output target)
  • layout (specifies the content of logged line entries)
  • loggers (specifies the source, designation and logging level)

     

Appenders

The appenders specify essentially the "where" of log4j logging. The appender names each specific logging destination; its type of output (file, display, etc.) and the log event content (see layout, below). Common appender types include the file appender and console appender. The file appender specifies its file path (where it logs) and possibly some other attributes that control rolling (versioning and purging) files. The console output writes to the display.

The current log4j configuration specifies three appenders:

Appender Name Output
CONSOLE display (sysout)
SYSLOG ./dist/logs/trm-system.log
ERRLOG ./dist/logs/trm-error.log

The RollingFileAppender, shown below, is a file appender that allows the admin to specify the number of versions (MaxBackupIndex) that Log4j maintains, and the size of each (MaxFile-Size). The most recent data is appended to the log file without version number (trm-system.log). Log4j renames each file with the subsequent version sequence number (e.g. trm-system.log.1, trm-system.log.2, etc) when the recent data file reaches “max size”. Once the number of versions indicated in MaxBackupIndex is reached, the oldest file is simply de-leted.

<appender name="SYSLOG" class="org.apache.log4j.RollingFileAppender">  
<param name="File" value="dist/logs/trm-system.log" />
<param name="Append" value="true" />
<param name="Threshold" value="DEBUG" />
<param name="MaxFileSize" value="10MB" />
<param name="MaxBackupIndex" value="5" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %-5p %X{username}:%t [%c] %m %n" />
</layout>
</appender> 

Layout

The layout specifies what each logged event contains, similar to a format string, and is an attribute of the appender. The appender example shown specifies its format in the "layout" element.

The layout format specified in the existing appenders include all the essential information available at the point of logging. You can, however, tweak the format on appenders as you see fit. Be forewarned, though, that some format items are performance drags (as noted in the following table) and that the Chainsaw log browser (described in an upcoming article) uses a configuration that closely maps to the format specified in the layout. Changes made to the layout will likely cause the log browser parser to fail.

Format Element Format Char Example Notes
Thread "name" %t "Http-Processor25"  
Dates %d{ISO8601} "2007-04-25 16:05:40,517”  
Priority (level) padded to 5 spaces %-5p "INFO "  
Message content %m "your message here..."  
Source code info %l (equivalent to %C.%M(%F:%L)) filename; method, line Warning: potential performance drag
FQ class/path %C "com.teradata.trm.common.db.Query" Warning: potential performance drag
Class/package name; where n is an integer representing number of elements of class/pkg to display %C{n} %C{2}: "db.Query" Warning: potential performance drag
Method name %M Blank Warning: potential performance drag
Logger %c com.mypkg.MyClass  
File name %F Blank Warning: potential performance drag
Source line %L   Warning: potential performance drag

MDC (Mapped Diagnostic Context) allows additional values to be included in logging output. The "username" item is included in the pattern example above. Items are added to the MDC at code level (see TRMContextFilter). The current items and their format string include:

Item Format String
Username %X{username}
Session ID %X{sessionid}
URL %X{url}

Loggers

Loggers specify 3 things:

  • what the source of the logging events (the classes or paths that get logged)
  • their logging level
  • the destination: to which appender(s) the events are to be sent

They have an inheritance-like relationship, where logger named "com.x.y" inherits the ap-penders of the logger "com". This will result in double output where both loggers explicitly de-clare the same "appender-refs", unless the additivity is set to "false". The "root" is like a base logger, inherited by all loggers. If the appender is not declared explicitly in a logger, you don't need to worry about additivity.

In this example, the logger "com.x.y" logs at DEBUG level and declares additivity=false to prevent double logging to CONSOLE. The root specifies default level and appender(s) used for everything that isn't otherwise covered with a specific logger (set to additivity="false").

        <logger name="com" additivity="false">   
<level value="WARN" />
<appender-ref ref="DEFAULT"/>
<appender-ref ref="CONSOLE"/>
</logger>
<logger name="com.x.y" additivity="false">
<level value="DEBUG" />
<appender-ref ref="My-XY-Appender"/>
<appender-ref ref="CONSOLE"/>
</logger>
<root>
<level value="ERROR" />
<appender-ref ref="CONSOLE" />
<appender-ref ref="DEFAULT" />
</root>

Logging levels (priorities) are symbols with numeric relationship, such that TRACE < DEBUG < INFO < WARN < ERROR < FAIL.  Enabling a certain level implicitly enables all levels above it.

The logging levels in the log4j configuration are specified in two places.  The appender states a "Threshold" logging level value that specifies the maximum de-tail level. For example:  <param name="Threshold" value="INFO" /> would limit this appender to log messages of INFO, WARN, ERROR and FATAL levels (i.e. no TRACE or DEBUG).

The logger associates a package path or class with a level, which has a similar affect on logging that particular item. A logger with

<level value="ERROR" /> will only send messages with ERROR or FATAL levels to the appender. Note that the appender has "final say" in this: even though the logger may specify a lower level, the appender will only write what its Threshold specifies and higher.

You should modify and add loggers to fine-tune what you want to see (set level to DEBUG) and what you don't want to see (set level to INFO, WARN, ERROR, etc).

In our next article, Advanced Logging, we will cover SQL logging, logging of user clicks, logging of authentication attempts and a few other more advanced topics.

Tags (3)