Aster Introduction: How Aster is Different (Issue: 1)

Learn Data Science
Teradata Employee

Aster was borne out of a need to do more with large datasets and to go beyond the power ANSI SQL.  ANSI SQL is a paradigm built to perform high velocity transaction processing or online transaction processing (OLTP).  SQL was never intended to work well against large data sets let alone the complex demands of multi-genre analytics.  ANSI SQL does not lend itself well to work against partitions of data sets at scale, developers attempted to solve this problem with stored procedures or user defined functions.  These often worked but not at scale and could be very difficult to build more complex functionality.

Business Intelligence tools enabled some capabilities to offset some of the limitations of ANSI SQL but fell back on being cumbersome multi-pass SQL code generators.  Even with Business Intelligence, UDF’s and other  tools, developers and power users found it very complex to do advanced analytics such as predictive modeling, path and pattern matching, text analytics, or clustering.  Even with the invention of Advanced Analytical tools there were still network challenges and data volume limitations.  This often led to data reduction (aggregation) and sampling thus limiting model accuracy and completeness.

Build it yourself is also a very popular option with languages such as C++, Java, .net, and R.  These languages are extremely flexible and powerful.  These low level languages enable looping, case logic, object orientation, reusability, and other factors that can solve many of the problems faced by traditional analytic data platforms let alone ANSI SQL.   These low level languages do have their limitations.  For instance very few people can develop a solution with these languages thus making them very expensive to build, support, and modify.   With the rate of change in today’s data climate we need tools that enable fail fast as well as change fast.

What is needed is a paradigm that allows people all the flexibility of a low level language with the ease and simplicity of ANSI SQL.  Another requirement is that this platform must scale to petabytes and accept a variety of data shapes and sizes.   The data must be together with the analytics and be architected to minimize the I/O limitations of a network.  Teradata Aster was developed to solve these limitations of traditional ANSI SQL while providing the flexibility of low level languages.