The Evolving Machine Learning/AI mindset

Learn Data Science
Teradata Employee

Machine Learning (as limited AI) is here and now.

Some useful things Machine Learning offers today:

  • Predictive Analytics (on both events and numbers)

- Churn, Adoption, Fraud detection, Propensity to buy, Survival Analysis, Next best action, Estimation of margins, sales forecast, customer spend etc., Product recommendations.

  • Identifying objects and people in videos, pictures, and speech

- Scene detection, Find associated relationships with actors of interest

  • Analyze text and find nuggets/sentiments, topics, associations, translate etc.,
  • Large-scale clustering or grouping of customers using behaviors

All of the above on a boatload of data. As you might've heard before, some of the above require the machine learning models to be trained ahead of time.

ML Models can be trained on:

  • Historical data from transactional systems (holds indicators to past results)
  • Borrowed from the ML community/providers and then improvised/tweaked in your environment (transfer learning).

Popular applications in Machine Learning

As you can imagine, it's a green field on what you can do with above new capabilities. For a lot of businesses, generating top N lists of curated customers, prospects, recommendations is priceless and is an immediate application.

Customer Satisfaction and Retention is another area of interest. If businesses can follow the journey of a customer and offer products and services at the right time, can lead to stickiness.

                               
© Can Stock Photo / aquir

Customer Churn - if only businesses knew the Top N list of customers who are in the last mile of switching out ... Predictions derived from a myriad # of variables, contexts.

Product recommendations - which products customers will most likely to click and how to position it in product search pages for maximum margin.

How not to make good the enemy of perfect

Most machine learning models use probabilities to calculate potential outcomes and thus quantifies uncertainty automatically. Obviously, we want the highest accuracy as possible and it comes with a price. As of today's state of affairs (which is changing as I type), there exists a tradeoff between accuracy and explainability. The more accurate the models are in prediction, the harder it is to explain the ML's decision - why is prospect A ahead of prospect B on the list or why product X is being recommended instead of product Y. The good part is you can have both high accuracy and low accuracy models side by side to get decent insights.

Other tradeoffs beyond accuracy and explainability: performance, the cost of deployment. See example chart for algorithm tradeoffs on a sequence prediction algorithm use case:

Traditional businesses are used to deterministic decisions and it's sometimes hard to communicate the algorithmic decisions which have high perplexity :). This is one of the reasons for slow adoption of ML and uncertainty that surrounds it - pun :)

For new emerging businesses in the last 5 years, you can see that letting ML make decisions is the norm and it's managed by a lot of A/B testing results to justify ML approach. It's often called the Champion->Challenger model approach, where better models have selected automatically in the ML setup depending on the deployment results of an A/B testing cycle. This works really well in the absence of historical data (also known as the cold start problem).

Moving to machine learning world is almost the same as us counting on the plane autopilot (which the pilots rely on) and more so with self-driving cars.

Technologies that can help businesses get there

Both proprietary and open source solutions provide ML options that businesses can leverage. Proprietary solutions such as Teradata's Aster or Open Source solutions around Apache Spark (including Deep Learning)/Python/R are available to quickly bring the businesses to the new age of machine learning/AI.