Introduction To Recommender Systems

In this post, will discuss the recommender systems and their uses.

The explosive growth in the amount of available digital information and the number of visitors to the Internet have created a potential challenge of information overload which hinders timely access to items of interest on the Internet. Information retrieval systems, such as Google, DevilFinder and Altavista have partially solved this problem but prioritization and personalization(where a system maps available content to user’s interests and preferences) of information were absent. This has increased the demand for recommender systems more than ever before.

A recommender Systems is an Information Retrieval technology that improves access and proactively recommends relevant items to users by considering the users’ explicitly mentioned preferences and objective behaviors. The recommendations relate to various decision-making processes, such as what items to buy, what music to listen to, or what online news to read.

Every minute of the Day

Over the past two decades, the Internet has emerged as the mainstream medium for online shopping, social networking, e-mail, and more. It has been observed that customers spend significant amounts of time researching products they seek before purchasing. In an effort to save time and efforts the organization puts recommender systems in place which learn the customer behavior and recommends the products to them based on this past purchase history.

According to Domo, every single minute of the day, YouTube users upload 300 hours of new video. Meanwhile, Netflix subscribers stream more than 77,000 hours of video in one minute.

Apple users download 51,000 apps; Amazon gets 4,310 unique visitors every minute, and Uber passengers take 694 rides.

In one Internet minute, 4,166,667 Facebook likes, 347,222 tweets, 590,279 Tinder swipes, and 284,722 Snapchat snaps are recorded.

Every minute of the Day

Motivation: Why should be care about recommender System?

The key reason why many people seem to care about recommender systems is money. For companies such as Amazon, Netflix, and Spotify, recommender systems drive significant engagement and revenue. But this is the more cynical view of things. The reason these companies (and others) see increased revenue is that they deliver actual value to their customers – recommender systems provide a scalable way of personalizing content for users in scenarios with many items.

E-commerce, such as Amazon or eBay in the US and MercadoLibre in Latin America, are putting a lot of money into it. They are building great teams just to focus on improving the accuracy of their recommenders because by doing so, users are much more tempted to buy more things. That’s why when you buy something like a bicycle you get a warm message like this one: “you might also be interested in buying this wonderful helmet”. I bet you that somehow it has happened to you and, what’s more, you were caught by this powerful trap and ended up buying such a suggested product. In the end, it’s just a matter of happiness. Yes, people like spending money, so the recommendations are just trying to stimulate that part of the brain that makes you feel happier when buying some stuff.

Few Examples

  • Two-thirds of the movies watched by Netflix customers are recommended movies.
  • 38% of click-through rates on Google News are recommended links.
  • 35% of sales at Amazon arise from recommended products.

Impact of Recommendation Systems

Recommender systems can help you retain customers by providing them with tailored suggestions specific to their needs. They can help you increase sales and can also help you create brand loyalty through relevant personalization.

  • Netflix – Recent study by McKinsey suggests that up to 75% of what consumers watch on Netflix comes from the company’s recommender system. According to Netflix executives, their recommender system saves them about $1 billion each year.
  • Amazon – When a retail giant like Amazon credits recommender systems with 35% of their revenue, it’s time to take note of this highly adaptable AI solution. They credit recommender systems with a 29% increase in total sales, bringing their yearly sales volume in 2016 up to (wait for it) 135.99 Billion.
  • Best Buy – Best Buy is another retailer that has boasted big returns from their recommender systems. The company began using recommender systems in 2015. In 2016’s second quarter they reported a 23.7% increase, thanks in part to their recommender system.
  • Spotify – One of Spotify’s most innovative uses of AI and recommendation systems is their popular Discover Weekly playlist. This algorithmically powered tool updates personal playlists on a weekly basis so that users won’t miss newly released music by artists they like. This recommendation system has helped Spotify increase its number of monthly users from 75 million to 100 million at a time, in spite of competition from rival streaming service Apple Music.

Application Domain

There are various filtering and recommender systems has been deployed and used by many companies across the globe. These systems are falling under different application domains. This analysis revealed that some recommender systems are purely research prototypes, some are working on Internet and some systems provide their services for fee. All of the recommender/filtering systems were studied and analyzed based on published technical literature in the form of technical reports, conference papers, and journal articles.

  • Web Recommendation
  • Movie/TV Recommendation
  • Information/Document recommendation
  • Usenet News/Personalized Newspaper Recommendation
  • Music Recommendation
  • Restaurant Recommendation
  • Organizational Expertise Recommendation
  • E-Commerce Product Recommendation
  • Travel Recommendation
  • Learning Resources/Research Grants Recommendation
  • Video Recommendation

Issues & Challenges

Today, several recommender systems have been developed for different domains however, these are not precise enough to fulfill the information needs of users. Therefore, it is necessary to build high-quality recommender systems. In designing such recommenders, designers face several issues and challenges that need proper attention.

  • Cold Start Problem

When new users enter the system or new items are added to the catalog. In such cases, neither the taste of the new users can be predicted nor can the new items be rated or purchased by the users leading to less accurate recommendations.

  • Synonymy

Synonymy arises when an item is represented with two or more different names or entries having similar meanings. In such cases, the recommender cannot identify whether the terms represent different items or the same item.

  • Shilling Attacks

What happens if a malicious user or competitor enters into a system and starts giving false ratings on some items either to increase the item popularity or to decrease its popularity. Such attacks can break the trust in the recommender system as well as decrease the performance and quality of recommenders.

  • Limited Content Analysis and Overspecialization

Content-based recommenders rely on content about items and users to be processed by information retrieval techniques. The limited availability of content leads to problems including overspecialization.

  • Grey sheep

Grey sheep occur in pure CF systems where opinions of a user do not match with any group and therefore, are unable to get the benefit of recommendations.

  • Sparsity

The availability of huge size of data about items the catalog and the disinclination of users to rate items make a dispersed profile matrix leading to less accurate recommendations.

  • Scalability

The rate of growth of nearest-neighbor algorithms shows a linear relation with the number of items and number of users. It becomes difficult for a typical recommender to process such large-scale data. For example, recommends more than 18 million items to more than 20 million customers.

  • Latency Problem

Collaborative filtering recommenders face latency problems when new items are added more frequently to the database, where the recommender suggests only the already rated items as the newly added items are not yet rated.

Research Opportunities

 There are design guidelines that can help design a fine-tuned recommenders that will perform better in mitigating issues like latency, cold-start, scalability, context-awareness, grey-sheep, and sparsity. These guidelines include:

  • Using demographic filtering and clustering, a recommender system may cluster users having similar preferences and demographic features that the system can only look into the appropriate user group rather than the entire dataset. This will minimize latency, increase performance, handle sparsity, and grey sheep problem.
  • The personal information of newly registered users can be obtained through registration. Contextual information such as location, time, etc. can be obtained through their IP address and those items are recommended that have been mostly viewed, downloaded, and purchased by other users having similar contextual information. This can easily avoid cold-start problems.
  • For a user with frequently changing preferences, two recommendations list should be maintained. The first one should be maintained according to the current preferences of the user, while the second one should keep track of the user’s long-term preferences so that the system should recommend items that match the user’s previous transaction history.
  • The recommender system should filter out obsolete and older items. A time threshold should be used to find out such items, and consequently newer items should be presented to users along with accurate suggestions.

Recommender Systems Algorithms

Recommender algorithms are typically implemented in the recommender model, which is responsible
for taking data, such as user preferences and descriptions of the items that can be recommended, and predicting which items will be of interest to a given set of users. Classification of recommender system algorithm as given below:

Collaborative Recommender System

Collaborative recommender systems aggregate ratings or recommendations of objects, recognize commonalities between the users on the basis of their ratings and generate new recommendations based on inter-user comparisons. The greatest strength of collaborative techniques is that they are completely independent of any machine-readable representation of the objects being recommended and work well for complex objects where variations in taste are responsible for much of the variation in preferences.

Collaborative filtering

Content-based Recommender System

In this system, the objects are mainly defined by their associated features. A content-based recommender learns a profile of the new user’s interests based on the features present, in objects the user has rated. It’s basically a keyword specific recommender system here keywords are used to describe the items. Thus, in a content-based recommender system, the algorithms used are such that it recommends users similar items that the user has liked in the past or is examining currently.

Content-based Recommender System

Knowledge-based Recommender System

This type of recommender system attempts to suggest objects based on inferences about a user’s needs and preferences. Knowledge-based recommendation works on functional knowledge: they have knowledge about how a particular item meets a particular user need, and can, therefore, reason about the relationship between a need and a possible recommendation.

Knowledge-based Recommender System​

Hybrid Recommender System

Combining any of the two systems in a manner that suits a particular industry is known as the Hybrid Recommender system. It combines the strengths of more than two Recommender system and also eliminates any weakness which exists when only one recommender system is used.

Hybrid Recommender System​

Demographic-Based Recommender System

This system aims to categorize users based on attributes and make recommendations based on demographic classes. The benefit of a demographic approach is that it does not require a history of user ratings like that in collaborative and content-based recommender systems.

Utility-Based Recommender System

The utility-based recommender system makes suggestions based on the computation of the utility of each object for the user. The main advantage of using a utility-based recommender system is that it can factor non-product attributes, such as vendor reliability and product availability, into the utility computation. This makes it possible to check the real-time inventory of the object and display it to the user.


Recommender systems are powerful for extracting additional value for business from its user databases. These systems help users find items they want to buy from a business. Recommender systems benefit users by enabling them to find items they like. They help the business by generating more sales.


Leave a Comment

Your email address will not be published. Required fields are marked *