Recommendation engines and how to get your engine started

Learn Data Science
Teradata Employee

Recommendation engines have been around for few years, but recent advances in multi-genre analytics and AI (Artificial Intelligence) have opened new possibilities to make very powerful recommendation engines

In this blog, I explain the importance of a business approach and functional frameworks required for developing a recommendation engine.

A brief summary of Aster algorithms which are usually used in developing recommendation engine is also given

I also touch upon components of overall recommendation architecture and how Aster fits into this architecture

Approach before Algorithms

As a good recommendation engine can improve customer engagement, a bad recommendation engine can impact very adversely. If clients are recommended products or services which are not what they are interested in, you soon start losing the customers

 

So it pays to have a business oriented approach and recommendation engines need to be very business result oriented. Whatever algorithm you will implement eventually, we should not directly start with algorithms in mind.

 

Business involvement should be sought very early-on and business objectives of recommendation engine should be clarified, as an example:

  • What do you want to recommend: Is it product, service, content or people? 
  • Will the recommendation engine be used to drive traffic or also to increase sales revenue? 
  • What is the objective and how it will be measured: increase in traffic, increase in average order value, increase in number of items per order etc..
  • Do not keep objectives which are elusive. As an example, I recently came across a dating site company which had an objective that profiles which they recommend should result into strong relationship in future. It’s a good vision to have, but there are many non-measurable factors which come into play in people relationship and goes beyond a recommendation engine scope

Once the business has decided on objectives of having a recommendation engine, the next step would be define a functional framework

Functional Framework for recommendation

Recommendations can be done in various ways. I call them functional framework, as they show how functionally a recommendation engine can be implemented. Listing some of the most common functional frameworks:

History-based: Here you recommend based on historical customer transactions. A typical use would be next product/service to buy in a banking industry based on customer’s life-events.

Content-based: Here you recommend a product or service based on its content. Example would be news sites and apps, which recommends news similar to news you have already read.  

Collaborative Filtering: Recommendations are generated by ratings given by user. If explicit ratings are not available, implicit ratings can be created based on user interactions (example time spent on product page , or zoom-in of photo can be used to create implicit rating)

Affinity-based: Generally people tend to buy similar items. This framework is based on affinity seen in customer interactions and transactions. The recommendations are done based on affinity

Context-based: A typical example of this framework would be recommendations based on you location. You location is considered as your context in which you are, and recommendations of products, services, people around are made.

Temporal: In some business, the product can be available only for a short period of time. Examples are a market-place business or news apps. You cannot recommend a product which is already sold or a news items which is few months old. In such a situation, we use temporal recommendation engine framework which takes into account temporary availability of products or services

Profile-based: Here you recommend based on user profile. Most of the dating apps or social networking apps use user profile to recommend other users

Hybrid: In many situations, you may need to combine some of the above mentioned frameworks into a hybrid framework. An example would be movie recommendations, where you might want affinity-based or collaborative filtering for not-so-recent movies, but use a content-based framework for recent movies, as recent movies do not have sufficient data to use affinity or collaborative filtering framework

 

Each of the frameworks has its advantage and disadvantages. So the knowledge of business objectives and context of recommendation would help in deciding which framework to implement

Aster Algorithms to implement the functional framework

All the above frameworks require multi-genre analytic capability and Aster provides all algorithms to implement the recommendation functional framework. Just to state a few:

  • nPath and other predictive algorithms can be used to implement history-based framework
  • Text Analytics algorithms or clustering algorithms for a key part of  content based framework
  • CFilter and WSRecommender, KNNRecommender, pSALSAalgorithms can be used for affinity based or collaborative filtering frameworks
  • Location analytics algorithms can be useful for context based frameworks, if context is location
  • Graph functions and predictive algorithms play a major role in temporal framework
  • Profile-based framework can be implemented using clustering, KNNRecommender algorithms
  • Aster also now provides Neural Networks algorithms which can be useful to build advanced predictive models or AI (Artificial Intelligence) model

Aster within the overall Recommendation Engine Architecture

Recommendation Engine Architecture usually requires multiple components, as listed below:

 Data Capture & integration component: This component captures all digital data, parses it if required and send it for storage or analysis

Advanced Analytics component: This is the Aster advanced analytics where recommendation engine model is developed. All Aster analytic recommendation models can be deployed to external systems using Aster Scoring SDK

Real-time recommendation engine deployment component: This component deploys the recommendation model developed by Aster (using Aster Scoring SDK) and makes the real-time recommendation to customer. Just as a note, recommendation can also be made offline (by mail for example), in which case this system can be used for offline recommendation generation

 

 

So hopefully this blog gives you an idea on recommendation engine approach, as well as how Aster multi-genre analytic capabilities could be used within overall recommendation architecture. Please feel free to comment

 

For more details, I invite you to you my upcoming session at Partners 2016 : https://event.crowdcompass.com/16partsandbox/person/M5OeT8exD2

1 Comment
Teradata Employee

Good information.  Look forward to hearing your session at Teradata PARTNERS 2016.