Aster Environments: Part 2: Development - Test - Production The Technology Side of the Story

Learn Aster
Teradata Employee

In part 1 of this blog series we talked about the process of determining when I might need to have a development/test/or production environment.  That blog post is found here:  Aster Environments: Part 1.  In this section we will cover three distinct environment scenarios and how they might look and the pros and cons.

If you remember from the previous blog post we have basically two distinct workload types:

  • Exploratory Discovery: a one time project that may or may not be repeated or produced in the future.
  • Perpetual Discovery: a repeatable analytic project with service level agreements.  This means that analytic models are constantly tweaked, improved, or redesigned based on model decay.


How are these projects different:Exploratory:

  • Project Governance:  Rapid Analytic Development project that may not evolve into a repeatable production project.
  • Service Level Agreements:  Service level agreements are relaxed or non existent.
  • Infrastructure and IT Ops Process:  Discovery platform with relaxed standards and processes.


Perpetual:

  • Project Governance:  PMO driven project with tight standards and practices that enterprise relies on for a business purpose.
  • Service Level Agreements:  There are defined service level agreements for inbound and outbound data/analytic products.
  • Infrastructure and IT Ops Process:  Infrastructure and process are robust to provide adherence to service level agreements.

Now we will go over three scenarios:  Single Environment, Dual Environments, and Triple Environment.

Single Environment:

One single Aster environment supports both Exploratory and Perpetual Discovery.  This means a single Aster environment that supports discovery, test, development and production.

Environment Support:

-Exploratory Analytic projects

-Production/Development/Test Perpetual Discovery projects (4 types of workloads)

Project Governance:  N/A

Service Level Agreements:  Difficult to ensure compliance if they exist due to mixed workloads

Infrastructure and IT Ops:  Having one environment to support production and discovery means that there is a single point of failure.

Advantages:

-Initial Reduction in Rack Space/Investment

-Fewer Queens

Disadvantages:

-Difficult to protect SLA’s

-Production activities compete with exploratory activities (vice-versa)

-Eventually will require more Workers to compensate for both types of workloads

-Exploratory workloads are analytic and data random

-Loss of both workloads if DOS occurs

-Limits on Exploratory analytic projects required to support Perpetual Analytic projects.

-Logical schema objects required/Tight security required

Dual Environments:

One Aster environment dedicated to support Exploratory Discovery and another to support Perpetual Discovery.

Separate Environment Support:

-Stack 1: Exploratory Analytic projects (Development and Test Perpetual Discovery development)

-Stack 2: Production Perpetual Discovery projects

Project Governance:  N/A

Service Level Agreements:  Production is segregated into its own environment so SLA's can be managed.

Infrastructure and IT Ops:  Having one environment to support discovery, dev and test means that there is a single point of failure for those workloads.

Advantages:

-Production only segregation of environments

-Better Production physical/logical security

-Perpetual Discovery growth is controlled

-Better/predictable production SLA adherence

-Perpetual Discovery environments will have tighter Network/Operations controls

Disadvantages:

-Exploratory Discovery environment burdened by Perpetual Analytic development/test efforts

-Bigger Initial Infrastructure Requirements

Triple Environments:

Three separate Aster Environments to support complete segregation of workloads:  discovery, dev/test,and production.

Separate Environment Support:

-Stack 1: Exploratory Analytic projects

-Stack 2: 2 Aster Environments to Production Perpetual Discovery projects (supports Development/Test/Production)

Project Governance:  N/A

Service Level Agreements:  Production is segregated into its own environment so SLA's can be managed.

Infrastructure and IT Ops:  Having one environment to support discovery on its own.  Another Aster environment to support dev and test.  Another to support production environments is more of a traditional model and eliminates the single point of failure.

Advantages:

-Complete segregation of environments

-Better physical/logical security

-Perpetual Discovery growth is controlled

-Better/predictable SLA adherence

-Perpetual Discovery environments will have tighter Network/Operations controls

-Complete Dev/Test/Production software release management controls.

-Test environments accurate simulation of production Perpetual Discovery projects.

-Zero burden to any type of Discovery project

Disadvantages:

-Bigger Initial Infrastructure Requirements/costs

What this might look like as far as infrastructure:

triple.JPG

Conclusion:  There are a variety of ways in which these environments could be configured to support various workloads.  There is no one correct or best approach.  Discovery has a very random and unplanned type of workload where production is very scheduled.  Typically discovery environments are bigger as a result and production starts off very small.  Production environments will grow as more repeatable projects come online.  It is very important to understand the non-functional requirements of a production system and the schedules in which SLA's must adhere too.  You must also understand the data size and computational impact that production systems require.  For now most Aster customers that are discovery only probably only require one environment.  As your needs change and projects migrate into a production mode your needs may change.  This blog post was written to give you something to consider.