Integrating Teradata ADS Generator with Fuzzy Logix DB Lytix

Applications
Applications covers the world of Teradata apps, including apps offered by Teradata (such as TRM or DCM), as well as best practices and methodologies for building your own Teradata-connected apps.
Teradata Employee

Integrating Teradata ADS Generator with Fuzzy Logix DB Lytix

Teradata ADS Generator, part of the Teradata Warehouse Miner family of products, was built to support both comprehensive data profiling as well as analytic data generation for Teradata customers.  Neither the data profiling nor the analytic data set generation capabilities of the product require any movement of data outside of the warehouse and utilize as much of the data as the analyst or data scientist requires.  Results and metadata are stored directly in the database while utilizing the parallel, scalable processing power of the Teradata platforms to perform data intensive operations.

DB Lytix, a product developed by Fuzzy Logix, offers scalable and robust high performance analytical methods that are embedded seamlessly into database systems.  The DB Lytix library of statistical, machine learning, and quantitative methods provide Teradata customers a rich set of in-database components – over 600 in all – which fall into one of the following categories:

    • Cumulative Distribution Functions
    • Data Mining Functions
    • Date Functions
    • Fit Distributions
    • Hypothesis Testing Functions
    • Inverse Cumulative Distribution Functions
    • Mathematical Functions
    • Probability Density Functions
    • Sampling Techniques Functions
    • Simulate Copula Functions
    • Simulate Univariate Functions
    • Sparse Statistics Functions
    • Statistical Functions
    • String Functions
    • Time Series Functions

 

The Fuzzy Logix DB Lytix library of advanced analytic functions can be deployed on Teradata 13.10 through Teradata 15.10 as embedded Teradata database objects in the form of User Defined Functions (UDF’s) and External Stored Procedures (XSP’s) as follows

    1. Scalar UDF’s - for those analytics that require a single result / single row as an answer set.
    1. Aggregate UDF’s - for those analytics that require single result of multiple aggregated rows as an answer set.
    1. Table UDF’s - for those analytics that require a table of results as an answer set.
    1. XSP’s - for those analytics that create multiple results and/or iterate over the input data multiple times.  The XSP controls the iteration of the SQL executed, often with DB Lytix embedded UDF’s in it.

Teradata’s Partner Integration Lab (PIL) has worked closely with Fuzzy Logix to port their DB Lytix advanced analytics library to the Teradata platform.  At the same time PIL collaborated with the Teradata Warehouse Miner Engineering team to integrate the Teradata ADS Generator product with DB Lytix.  The integration of these two products provide an extremely powerful in-database predictive analytics platform providing data profiling, data preparation, machine learning, multivariate statistics and predictive model deployment capabilities for Teradata customers.

As a result of the collaboration, recent changes were made to the Teradata Warehouse Miner family of products, allowing the Teradata ADS Generator front-end to call the DB Lytix database objects, and manage them for production environments.  Teradata Warehouse Miner or ADS Generator Version 05.04 is required for this integration.  The combined ADS Generator/DB Lytix integrated architecture is depicted below:

In this depiction, XSP’s run at the Parsing Engine level, while UDF’s run at the AMP level.  Since there are over 600 (and counting!) DB Lytix functions within their library, it would be too cumbersome to illustrate how each would be called and managed from Teradata ADS Generator.  What we have done though, is put together a tutorial on how to call each type of DB Lytix function, and how to use the Teradata ADS Generator’s Variable Creation Analysis to manage their execution and interrogate the results of their in-database analysis.  This tutorial is available for download here:

This guide describes the integration points between the two products and does not attempt to replicate the information in either the Teradata ADS Generator or Fuzzy Logix DB Lytix user documentation.  For more information on the capabilities of Teradata ADS Generator, please refer to the following:

For simplicity, the Teradata ADS Generator & Fuzzy Logix DB Lytix Integration Guide utilizes the well-known TWM Demonstration Data that is released with each version of Teradata Warehouse Miner or Teradata ADS Generator.  Refer to Volume 1 of the TWM User Guide listed above for instructions on how to install this data.  Attached to this reference is a Zip file containing the supporting ADS Project “.bin” file that can be imported into your Teradata ADS Generator environment:

Explicit instructions for importing the “Teradata ADS Demo Powered by Fuzzy Logix.bin” file are included in the Teradata ADS Generator & Fuzzy Logix DB Lytix Integration Guide.

The Teradata ADS Generator & Fuzzy Logix DB Lytix Integration Guide is meant to provide a functional description on how each type of database object available within the Fuzzy Logix DB Lytixlibrary is called from the Teradata ADS Generator User Interface. It also includes an example of one of the Fuzzy Logix Excel Templates that is available for visualizing the results generated by their database objects.  A thorough description of all the Excel Templates in a use-case oriented format is available in the following companion document which is attached to this Reference Article.  Also attached to this Reference Article is a Zip file containing four supporting ADS Project “.bin” files that can be imported into your Teradata ADS Generator environment.  Both are available for download here:

The Teradata ADS Generator & Fuzzy Logix DB Lytix Integration Guide describes in step-by-step fashion, how to build out the analyses within the “Teradata ADS Demo Powered by Fuzzy Logix” ADS Project.  There are analyses which support the detailed descriptions in the Integration Guide for each of the following:

    • Creating Variables by Executing DB Lytix Scalar and Aggregate UDF’s
    • Creating Variables From DB Lytix Table UDF’s
    • Calling Fuzzy Logix XSP’s from Variable Creation

The Integration Guide also introduces the enabling technology within Teradata ADS Generator required for the DB Lytix integration, namely the Variable Creation analysis.   In specific, the unique features of Variable Creation that enable the integration are Run Units, Literals and Literal Parameters along with Variables for Scalar and Aggregate UDF calls, and Function Tables for Table UDF calls. Also included is an example of using Run Units to launch Microsoft Excel to visualize Fuzzy Logix results.  A sample screen capture illustrating the integration of Fuzzy Logix, Teradata ADS Generator as well as the Fuzzy Logix Excel templates is shown below:

For a complete description of all the functions available in the Fuzzy Logix DB Lytix library, please refer to the following document, available with the Fuzzy Logix DB Lytix software:

    • User Manual for DB Lytixon Teradata Advanced Package v1.3

In order to obtain TWM / ADS or Fuzzy Logix DB LytixSoftware, the following links are provided:

For Teradata Associates

    • TWM or ADS software for version 5.4 can be downloaded from the Teradata Warehouse Miner Sharepoint site located at:

http://sharepoint.teradata.com/eng/twm/default.aspx

    • An evaluation version of Fuzzy Logix software and the documentation listed aboce can be obtained sending an E-Mail to downloads@fuzzyl.com with a with a cc: Klepacz, Christopher with the following information:
    1. Teradata employee name
    1. Title
    1. Department
    1. Teradata platform on which the package is being requested
    1. Teradata Database Version

For Teradata Warehouse Miner / Teradata ADS Generator Customers

    • The TWM 5.4 software can be downloaded from the patch server on T@YS located here:

https://tays.teradata.com/

    • An evaluation version of the DB Lytix software can be obtained by contacting Chris Klepacz, Teradata Global Alliances, at:

christopher.klepacz@teradata.com

For Other Interested Parties

    • An express version of Teradata Warehouse Miner version 5.4 is available on the Teradata Developer’s Exchange located at:

http://downloads.teradata.com/download/applications/twm-express-edition     

    • An evaluation version of the DB Lytix software can be obtained by contacting Chris Klepacz, Teradata Global Alliances, at:

christopher.klepacz@teradata.com

 

2 REPLIES
Teradata Employee

Re: Integrating Teradata ADS Generator with Fuzzy Logix DB Lytix

Currently, we have a slight glitch with the Fuzzy Logix Excel templates.  The ODBC DSN can not have any "special" characters in it (i.e. spaces, dashes, etc.)  Pleasre create an ODBC DSN with only alphanumeric characters, and everything will work fine!

Teradata Employee

Re: Integrating Teradata ADS Generator with Fuzzy Logix DB Lytix

Documentation and project files have been updated for TWM 5.4 and DBLytix 1.3.  If you need a demo for previous versions of TWM (05.03.05 only) and DBLytix (v1.1 and v1.2 only), please contact me.