What's New in 15.10 for Unity Ecosystem Manager

UDA
The UDA channel is for Teradata’s Unified Data Architecture including the Analytical Ecosystem and other UDA influences. This channel provides information specific to the integration and co-existence of multiple systems, in particular when a mix of Aster, Teradata, and Hadoop are present. It is also meant to support information around the UDA enabling technologies so products like Viewpoint, Data Mover, Connectors, QueryGrid, etc.
Teradata Employee

What's New in 15.10 for Unity Ecosystem Manager

Unity Ecosystem Manager 15.10 release is now GCA.  Ecosystem Manager is tightly integrated with Unity Director/Loader, Data Mover, Load utilities and BAR to achieve high availability and disaster prevention/recovery.  It is an integral part of the Unity Portfolio (Unity Director/Loader and Unity Data Mover) providing a holistic view of the entire Unified Data Architecture (UDA).  Ecosystem Manager can be deployed in single and multi-system UDA environment.

  • Ecosystem Manager monitors and controls the health of the Teradata systems (Teradata, Aster and Hadoop), Servers (Unity, TMS and third party) and their supporting software (Unity, ETL, BI) and manages the interdependencies between all applications, jobs and tables.
  • Ecosystem Manager provides a powerful event driven system that monitors operations, generates alerts when thresholds are exceeded and can take actions to push the mission-critical application alerts to operation staff, DBAs and other subscribed EDW staff. 
  • The Ecosystem Manager Explorer Portlet lets you monitor workflows and applications, view component and daemon health status, display job status, track resource dependencies and provide rapid troubleshooting capabilities when issues occur.

The Unity Ecosystem Manager 15.10 release primary feature is the new Workflow Engine which is composed of the Workflow Designer and Monitoring Portlets.  This feature extends the operational control aspects of Ecosystem Manager by providing a visual front end for building and deploying workflows.  The Monitoring portlets have been extended to support the monitoring and administration of deployed workflows.  The complete set of Ecosystem Manager 15.10 features are as follows:

Table Validation Enhancements

The 15.10 release of Unity Ecosystem Manager includes several new customer driven enhancements in the Table Validation Feature

  • First, the user interface has been simplified for Table and Database Entries.  The Table and Database menus are merged into a single functional view eliminating some of the redundant or cumbersome interface selection and entry.  You can Manage existing tables of add new one.  When adding a new table it can either be from a new database when Server and password information are required or added from an existing database where you simply select the appropriate database and systems.
  • Second, Multiple Table validation rules can now be established for a table (using Standalone and Workflow validation rules).  While only one standalone rule can be defined, multiple workflow rules can be establish for a table allow unique comparisons based on the elements of the workflow.  When defining a Workflow validation it must include a workflow name and description along with the additional table validation entry
  • Third, the Table validation threshold sync rule can be either a fixed value or a percent operator.  Leveraging the custom rule a percent threshold value can now be applied.  This means threshold alerts can be triggered base on a percent regardless of table size.
  • Fourth the table validation frequency option is now calendar entry versus crontab entry.  This will make the operation of entering the schedule that much easier.
  • Finally, the Explorer view has been enhance to show a table validation “pending” status.

 

Workflow Engine

Since the introduction of Unity Ecosystem Manager (UEM), then called TMSM 13.00,

  • EM has had the ability to respond to events and run external processes such as ETL jobs, monitor their progress, evaluate their results (through return values, table validation, and timeouts) and execute new jobs, notify users, and send alerts based on these results.   
  • This ability to string jobs (tasks) together into ever more complex event driven workflows has been put to significant use by customers.

Unity Ecosystem Manager 15.10 eliminates some of the control complexity to the existing event driven workflow by adding two portals:

  • Workflow Designer allows the user to define object interaction through a simple GUI interface.  The workflow can be as simple as executing a Table Validation at the end of a Data Mover Job and providing the DBA with an email notification upon failure to something as complex as Monitoring the Load Server and upon receiving the End Event to the load job, triggering the transformation script on Hadoop, upon completion triggering a Data Mover job to copy the table to the production Teradata System, when complete notifying the BI tool users that the data is now available for their application, then triggering a second Data Mover Job to copy the table to DR, upon completion running a Table validation and notification upon error.
  • Workflow Monitor to monitor and manage workflow execution.  Workflows progress can be monitored and managed down the individual task associated with the workflow.  Workflows can be paused at the completion of the current workflow task.  Log files are easily accessed for each task defined in a workflow to ease troubleshooting when problems occur.  A workflow can be restarted or resumed after corrective actions have taken place.  And a complete workflow history is maintained so workflow history can be examined and compared.
  • And all metadata associated with a Workflow is stored in the Unity Ecosystem Repository.

Workflow Designer

  • Ability to design and configure workflows within the analytical ecosystem

  • Leverage existing event subsystems ability to run external processes (ETL Jobs, Table Validations, execute new job, execute a Data Mover job, notify user and send alters)

  • Ability to promote the workflow across Environments (Development or Production)

Workflow Monitor

  • Ability to monitor workflow progress and status

  • Manage alerts that occur within a workflow by pausing, aborting, correcting and resuming or restarting a workflow

  • View the detailed workflow view down to the individually defined workflow tasks. 

Unity Ecosystem Manager 15.10 release and the new Workflow Engine can be leveraged in single and multisystem environments.  The tool can be used to orchestrate dual loads, synchronize data movement between two systems or to administer and monitor all aspects of Unified Data Architecture.