An Adaptive Boosting Technique to Mitigate Popularity Bias in Recommender System

Recommendation systems have become an essential part of our lives, from movies we watch on OTT platforms to shopping on e-commerce websites to the news we read on the internet. In this information age, where the information is available in abundance or huge volumes, recommendation systems make our lives simpler by filtering information for us according to our taste.

Recommender systems recommend relevant items to the users based on the user’s previous data. The primary goal of recommendation engines is to identify the items that a particular customer might buy or be interested in based on the previous ratings. They do so by anticipating the ratings that users would give to the items, and the more data we have for a user, the more precisely we can predict their ratings. Netflix (movie recommendation system), Spotify (music recommender system), and Amazon (product recommendation system) are some examples of recommendation systems that we see in our daily lives.

Although recommender systems are quite popular, there are concerns regarding the fairness of these systems amongst the research community. Recommender systems primarily face fairness issues at two levels, user-level (Ekstrand and Kluver, 2021; Mansoury et al., 2020) and item level (Saito, 2020; Ovaisi et al., 2020). This paper considers the most important fairness issue, namely the popularity bias at the item level. The items rated by most of the users or have received high ratings are known as popular items, and the items rated by very few users or not rated at all are known as non-popular items. The underlying notion behind popularity bias is that people are more inclined to offer comments on mainstream or popular products than on non-popular items. As a result, reported user response is skewed towards popular products rather than genuine user interest. Thus, frequently rated items or popular items receive a lot of exposure, but less popular or niche items are underrepresented in a recommendation. Recommending only popular items is problematic for two reasons: first, not everyone wants to follow the taste of the mainstream crowd, and second, it makes it harder for new goods to gain user attention.

One example of how just suggesting popular things affects companies is that a lawsuit was recently filed in the United States against Google for only showing advertising for popular items and not giving opportunities to less popular and newer items on the market (San Ramon, 2020). Because just a few items are displayed most of the time, this sort of activity generates a monopoly in the market, which is unsuitable for any firm. It will hamper the possibilities of creativity and innovation in the products or items.

Exploring and mitigating popularity bias in recommender systems is not a new problem and have been previously explored in many works (Abdollahpouri et al., 2019; Huang et al., 2020; Schnabel et al., 2016; Steck, 2011; Abdollahpouri et al., 2017) . However, the existing works focus either on improving the accuracy of the overall recommendation system or exploring the non-popular items through diversification. Since more ratings are available for the popular items, the accuracy is inherently biased towards the popular items. A highly accurate recommender system may have very high accuracy on popular items but a very low accuracy on non-popular items. On the other hand, naive diversification may lead to poor accuracy of the overall recommender system. In this work, we define popularity bias as the difference between errors on non-popular items and popular items. A fair recommender system should perform equally well on both popular and non-popular items thus ensuring a balanced accuracy of the recommender system. The main aim is to reduce popularity bias from the data and recommend relevant items to the user based on prior data while keeping the error as low as possible.

Fairness in recommender systems has been the center of discussion for a long time since these systems suffer from different types of biases present in the data. Due to these biases, it can not represent the entire population, and due to this, the results produced by recommender systems are biased, so there is an utmost need to solve this problem. The most common types of biases are gender bias (Ekstrand and Kluver, 2021; Mansoury et al., 2020), racial bias(Obermeyer et al., 2019), selection bias (Saito, 2020; Ovaisi et al., 2020), exposure bias (Yadav et al., 2019; Singh and Joachims, 2018), position bias (Collins et al., 2018; Guo et al., 2019), and popularity bias (Abdollahpouri et al., 2019; Huang et al., 2020; Schnabel et al., 2016; Steck, 2011; Abdollahpouri et al., 2017). The focus of this paper is on popularity bias.

It is crucial to recommend non-popular items as they are the ones that are less likely to be discovered. Abdollahpouri et al. (2019) have proposed a customized re-ranking diversification strategy that aims to boost non-popular items representation while retaining acceptable recommendation accuracy. The focus in (Abdollahpouri et al., 2019) was to achieve a trade-off between coverage of popular and non-popular items. To achieve this coverage, non-popular items that consumers could appreciate were randomly selected and presented to the user without being displayed in some order of preference. Thus the proposed method could result in poor accuracy for non-popular items.

To address the popularity bias, Huang et al. (2020) and Schnabel et al. (2016) employed the Inverse Propensity Score (IPS) method. Propensity-based techniques have previously been used in causal inference and observational studies but are used for the first time in the recommender system by Schnabel et al. (2016). To apply the IPS approach, we need propensities, and the authors in (Schnabel et al., 2016) have discussed primarily two propensity estimation models, via Naive Bayes, and the other is via logistic regression. Saito (2020) has also discussed a straightforward approach to measure the propensities. However, the performance of these propensity-based methods is heavily influenced by the model used to estimate propensity. The algorithms require actual probabilities to function correctly which are difficult to estimate (Saito, 2020). As a result, a more effective strategy is required.

Furthermore, all of the above works focus on reducing recommender system errors without explicitly being fair to non-popular items.Two exceptions to this are (Boratto et al., 2021; Zhu et al., 2020). (Boratto et al., 2021) mitigate the issue of popularity bias by providing a metric similar to the individual fairness metric used in machine learning. The idea is to equalise the outcomes across all individual items. On the other hand, (Zhu et al., 2020) focuses on group fairness by grouping the items together and equalizing the true positives across all groups. Our work focuses on group fairness, and instead of just equalizing true positives, we sought to equalize the total error across the groups.

Machine learning algorithms typically discriminate based on biased historical data. Data manipulation is used in certain preprocessing techniques to address this discrimination problem. Massaging (Kamiran and Calders, 2009) , re-weighting (Calders et al., 2009) , uniform or preferential sampling (Kamiran and Calders, 2012) , and data augmentation (Iosifidis and Ntoutsi, 2018) are examples of preprocessing methods that might have an impact on data distribution. By altering instance labels, giving different weights, under or oversampling instances, and producing pseudo instances, they attempt to correct imbalances between protected and non-protected groups within the data. Iosifidis and Ntoutsi (2019) demonstrated Adafair, an Adaboost-based fairness-aware classifier that adjusts the weights of the instances in each subsequent round while explicitly addressing class imbalance by optimizing the number of ensemble models for balanced classification error. Adapting this idea, we examine the use of boosting algorithm in the recommender system. The idea is to increase the weights of non-popular items to maintain the balance between popular and non-popular items. We show that this technique can significantly reduce the popularity bias present in the system.

Let us consider the set of users ${U}$ and the set of items ${M}$, with the number of users $|U|=k$ and the number of items $|M|=l$. $A=U\times{M}$ represents the collection of (user, item) pairs with each entry $A_{u,m}$ denoting the true rating provided by user $u$ to item $m$. Integer values ranging from $1$ to $5$ can be used for representing each entry, with $1$ representing low interest and $5$ indicating strong interest. Because consumers only rate a small portion of the items, many ratings are typically unknown. The main aim is to develop an algorithm that generates the best possible predicted rating matrix $\hat{A}$. In the predicted rating matrix $\hat{A}$, each entry $\hat{A}_{um}$ denotes the probable rating given by user $u$ to item $m$. To achieve the goal, ideal loss function to find predicted rating matrix is defined as:

where $\delta_{um}$ can be taken as mean squared error (MSE) i.e.

or minimum absolute error (MAE) or i.e.

.

The ideal loss function cannot be calculated since the majority of the elements in the actual rating matrix are missing. Therefore, to build an effective recommender system, we need to estimate the ideal loss as accurately as possible using only the ratings present in the true rating matrix. The simplest straightforward estimator for estimating ideal loss, also known as a naïve estimator is used which is defined as follows:

where $B$ is an observed rating matrix and each entry $B_{um}=1$ if user $u$ has given rating to item $m$ otherwise $B_{um}=0$.

The loss over observed ratings is calculated using naive loss, which is defined as the sum of loss overall user-item pairs in the data divided by the total number of user-item pairs in the true rating matrix. It is easy to see that when the data are missing fully at random, this naïve estimator is unbiased i.e.

However, when the data are not missing completely at random which happens in the presence of popularity bias where there is more rating available for popular items as opposed to that of non popular items, it is shown that the naive loss may not exactly correspond to the ideal loss function (Schnabel et al., 2016; Steck, 2010) i.e.

One popular approach to mitigate the popularity bias is by using Inverse Propensity Score (IPS) approach (Huang et al., 2020; Schnabel et al., 2016). The main idea behind this approach is to build a pseudo missing completely at random dataset by weighting all the observed ratings by the inverse of their propensity score. It is easy to theoretically show that IPS loss is an unbiased estimator and hence can be used to remove popularity bias (Saito, 2020).

The IPS estimator’s unbiasedness is desired; nevertheless, this feature is dependent on the true propensity scores which need to be approximated using various approaches. These methods majorly suffers from two problems (Saito et al., 2017). One, the IPS estimator is no longer an unbiased estimator if a propensity estimation model is not stated appropriately. Another, the inverse of the propensities may be substantial, so the IPS estimator has a high variance problem. As a result, building learning methods that are resilient to misspecification of propensity and estimator variation is crucial for applying the approaches to real-world MNAR situations.

Saito (2020) devised an asymmetric tri-training technique based on the asymmetric tri-training approach used in unsupervised domain adaptation to tackle the challenges that an IPS approach faced. It employs three rating predictors, two of which were used to create a pseudo rating dataset and the third of which was used to train the model on these pseudo ratings. The problem with this method is that the size of the dataset reduces after the algorithm is applied, making it impossible to accurately estimate the ratings of all the items. Because data in MNAR datasets are typically sparse, we must make appropriate use of it in order to predict ratings. Boosting technique on the other hand would be ideal because they utilise the majority of their data in different iteration stages, and one can also use upweighting to boost non-popular items in the same way that incorrectly categorised points are boosted in subsequent iterations.

Adequate coverage of non-popular or long-tail items is critical to any business’s success. Because almost all users are familiar with popular items, a recommender system’s ability to recommend non-popular items will determine how well it introduces users to new experiences and products; however, it is well known that recommender systems are biased towards popular items.
This paper proposed and compared FairBoost, an adaptive boosting algorithm, to previously implemented approaches for mitigating popularity bias in recommender systems. We also proposed a new metric to quantify popularity bias, because the error metric was insufficient for this purpose. On the four datasets, we were able to demonstrate that the FairBoost algorithm significantly reduces the popularity bias compared to other algorithms while keeping the error as low as possible. One exciting area for future study would be to test this algorithm on other sorts of biases that might exist in the system such as selection bias, gender bias, and so on. Another thing we can try is to test the algorithm with different base learners.