Density Weighting for Multi-Interest Personalized Recommendation

Module for Multi-Interest Extraction. We extend the standard user tower described in Figure 2 to model MUR using a multi-interest extraction layer. The goal of such a layer is to output multiple user representations, $i.e.$ $f_{\theta}(x_{u})=\left\{z_{u}^{(1)},\dots,z_{u}^{(M)}\right\}$, where $M$ denotes the number of user representations extracted. While $\{z_{u}^{(m)}\}_{m=1}^{M}$ could be learned separately using $M$ separate user towers, such an approach will be inefficient in both computation and the number of parameters. Humeau et al. [20] recently proposed an efficient and scalable approach that uses global attention to extract multiple context embeddings for the natural language sentence matching tasks. We adapt this idea of global attention for a recommender system to extract multiple representations for a given user history. Let the output of the sequence encoder be $\{h^{(1)}_{u},\dots,h^{(n)}_{u}\}$, then to obtain multi-interest user representations, we learn $m$ global query vectors $\{q_{1},\dots,q_{M}\}$, where each $q_{m}$ extracts the user representation $z_{m}$ using attention:

Given multiple user representations, we compute the affinity score for each candidate $y$ as follows:

As the above multi-interest extraction approach uses global query parameters in attention, we refer this representation extraction layer as parametric attention. Note that there are alternate methods for multi-interest extraction layers [10, 27, 31], but in this work we only consider parametric attention to extract MUR due to its simplicity and scalability. More importantly, the focus of this work is on studying the effect of long-tail distribution on MUR in the two-tower DNN architecture [11], which has become a popular framework for recommendations in the recent years.

In this section, we discuss our analysis on the synthetic data, which forms the basis of our density weighting scheme. The synthetic data for sequential recommendation is generated using a finite state hidden Markov model [3, 22], where we consider the hidden state clusters as the so-called user interests. To define the Markovian process for the sequence, we assume a user-specific interest transition matrix which is a function of the user interests, distribution skewness, and user interest volatility i.e., probability of switching between different user interests. A detailed description of the data generating process is provided in Appendix A. Figure  3(a)-3(c) show examples of synthetically generated data distributions. The synthetic data models long-tailed distributions for both interest and item space to capture the power-law behavior of items encountered in real recommendation systems.

While the Markovian assumption may not reflect complex real-world scenarios, we find that our observations are consistent in real-world datasets as described later in Section 4. Our key motivation to use synthetic data is to study the effect of distribution skewness on modeling capabilities of MUR. Moreover, item clusters in the synthetic data can be treated as ground-truth interests for the users, allowing us to quantitatively (using   mean-silhouette (MS) score [41]) and qualitatively (See Figure 1) evaluate the structure of the learned item representations for different models.

Retrieval performance. To study the sensitivity with respect to an imbalanced distribution, we evaluate the performance of the models trained with varying degree of data skewness. Specifically, we vary the imbalance in the user interest distribution and plot the recommendation performance (Hit Ratio, expressed as %) across different splits: tail/infrequent clusters (Figure 3(d)), head/frequent clusters (Figure 3(e)), and all clusters (Figure 3(f)). As seen in Figure 3(d), increasing the skewness causes the performance of MUR-based model to decrease for tail interests, which also results in decreasing average performance across all the interest clusters (Figure 3(f)). In fact, when the user interests are highly skewed, the performance of MUR-based model becomes comparable to the performance obtained using the SUR-based model. This is due to the under-representation of items in the tail interest clusters, causing the model to dominate on the head interest items. On the other hand, using our proposed density weighting with MUR, referred to as MUR-IDW (described in Section 3.3), makes the model robust to imbalance as shown in Figure 3(d) and Figure 3(f).

Clustering quality. We inspect the structure of the representation space learned using different models. A good representation space would have localized clusters representing user interests, such that the items belonging to a similar interest have item representations close to each other. First, we do a qualitative analysis of the clustering structure learned using MUR and SUR. Figure 3(h)- 3(i) shows that MUR leads to a better clustering structure of the representation space compared to the representations learned with SUR on the synthetic data.

Next, we measure the clustering quality of the item representations using the mean-silhouette (MS) [41] score. The MS score measures the average tightness and separation of the clusters. For a point $p$ belonging to the ground truth cluster $c$, we define the silhouette as $s_{p}=\frac{b_{p}-a_{p}}{max(a_{p},b_{p})}$, where $a_{p}$ is the mean distance between $p$ and other points in the cluster $c$, and $b_{p}$ is the mean distance between $p$ and the points in the closest cluster. Specifically, we can calculate $a_{p}$ and $b_{p}$ as:

The MS score is computed by taking the average over all points, $i.e$ $MS:=\mathbb{E}_{p}\left[s_{p}\right]$. The MS score takes the value $\in[-1,1]$, where a higher MS depicts distinct and separated clusters, and a lower MS depicts overlapping clusters. Figure 3(g) shows the MS score for varying degrees of skewness in the synthetic dataset. While the clustering structure obtained with MUR is consistently better than with SUR , the clustering quality in the representation space deteriorates with increasing degree of imbalance.

In this section, we describe our novel density-weighted loss for the two-tower recommendation system, which advocates leveraging the structure of the item representation space to learn and mitigate the effective imbalance of the items. This proposal aims to leverage the item representation space enabled by MUR to enhance the overall recommendation performance.

To determine the effective item imbalance, we use item distribution smoothing that convolves a symmetric kernel with the empirical distribution of the items. This results in a kernel-smoothed version of the item distribution that accounts for the overlap in information of nearby items in the representation space. We then use this kernel-smoothed item distribution to learn a balanced loss that is uniformly weighted in the item representation space.

We first consider the simple frequency-balanced [18] version of the loss function described in (3):

$p(y):=n_{y}/|{\mathcal{U}}|$ corresponds to the empirical distribution of the items and $n_{y}$ denotes the item frequency. Minimizing the balanced loss in  (5) aims to learn a uniform loss function across all the items in the training dataset. However, the imbalance in $p(y)$ may not reflect the effective imbalance of the items in the representation space, and naively optimizing  (5) can lead to overfitting of the model to tail items that have low $p(y)$. As we show in Section 4, such a frequency-based balanced loss can lead to poor overall recommendation performance.

In contrast, our density-weighted loss uses the kernel-smoothed version of the empirical item distribution:

$k(y,y^{\prime})$ is the symmetric Gaussian kernel that characterizes the distance of items in the representation space. Therefore, $\tilde{p}(y)$ leverages the structure of the representation space in determining the effective imbalance of items. In particular, $\tilde{p}(y)$ captures the effective weight of the region defined by the neighborhood of item $y$ in the representation space, thereby reducing the model’s sensitivity to $p(y)$ through kernel smoothing. Note that since $\tilde{p}(y)$ is determined using the item representations learned in the item tower, in practice we use $\tilde{p}(y|\phi)$. As both the kernel-smoothed density $\tilde{p}(y|\phi)$ and the loss used to train the parameters of the two-tower recommender model $\{\theta,\phi$} are inter-dependent, we propose an iterative two-phase procedure (Algorithm 1) to train the two-tower architecture. Given the model parameters $\{\theta^{(t)},\phi^{(t)}\}$ and the density $\tilde{p}(y|\phi^{(t)})$ at iteration $t$,

• •

  In the Sleep Phase, the item representations from the item tower are used to update the density through kernel-smoothing.

• •

  In the Wake Phase, the two-tower model is trained with the balanced loss given by the item density from the previous iteration.

Sleep Phase: Update Item Weights. To learn a balanced loss function with respect to $\tilde{p}(y|\phi^{(t)})$, we introduce item-specific weights $w_{y}^{(t)}$, such that the initial $w_{y}^{(0)}=n_{y}/|{\mathcal{U}}|$. The sleep phase updates the item weights based on the item representations defined by the current state of the item tower. In particular, the density update is defined as:

$v_{k}=g_{\phi^{(t)}}(i_{k})$ refers to the representation of the $k^{th}$ item, ${\mathcal{K}}$ is the Gaussian kernel function with the bandwidth parameter $\ell$ (the bandwidth $\ell$ is selected based on Scott’s Rule [43], which is a standard bandwidth selection method). In practice, we found using relative normalized density [44] as a better estimate:

The relative item density of each item is $\in$ $[0,1]$ and the item weights are updated with $h^{(t)}_{k}:=1-p^{\prime}(y=k|\phi^{(t)})$ resulting in a negative correlation with the relative item density. To avoid a drastic change in the loss function, the weights are updated gradually using a momentum parameter $m$:

where we use the subscript $k$ to denote the $k^{th}$ item. Since the updates are negatively correlated with the item density, they allow the model to progressively upweight/downweight the regions with low/high item density respectively. The complete procedure followed in the sleep phase is summarized in Algorithm 2.

Wake Phase: Train Two-Tower Model. Given $w^{(t+1)}_{y}$, the two-tower model is trained by minimizing the following weighted loss:

Stopping Criterion. To detect convergence in IDW, we track $||w^{(t+1)}-w^{(t)}||$. As the model is trained, the loss contributions becomes uniform in the item representation space. We visualize the evolution of the item representation space for the synthetic data in Figure 4. We observed that making the loss uniform in the representation space, progressively improves the clustering structure of the item representations. The improvement in the clustering structure is also reflected in the mean-silhouette (MS) score in Figure 4 (Top).

User Tower Calibration. While the uniform loss structure improves the clustering structure of the items, the retrieval performance over the head clusters can drop compared to when the model is trained using the unweighted likelihood. Therefore, to mitigate the performance drop for head clusters, once we improve the item clustering structure using IDW, we freeze the item tower and train only the user tower with the unweighted loss. We study the importance of user tower calibration in section 4.6.3.

We employ top-k Hit Ratio (HR@k) and NDCG@K with $k=20$ to evaluate the recommendation performance of all the models. Additional results with $k=5$ are included in the Appendix B. As the goal of IDW is to improve the performance of the tail items without hurting the overall performance across all the items, we further compare the metrics on the tail and head item sets separately. Assuming the Pareto Principle [7], the 20% most frequent items are considered head items, and the remaining items as the tail items. Following the standard evaluation protocol [24, 42], we use the leave-one-out strategy for evaluation. For each item sequence, the last item is used for testing, the item before the last is used for validation, and the rest is used for training. As the item set is typically large, we use randomly sampled negative items [24, 19, 26, 42] at evaluation. Specifically, we pair the ground-truth item with 99 randomly sampled negative items that the user has not previous interacted with to compute the evaluation metrics.

To demonstrate the effectiveness of the proposed IDW on long-tail item distribution, we compare IDW with other commonly used weighting methods that deal with imbalance in recommender models. All the baselines use the two-tower model as the backbone, which has proven to be scalable because of efficient inference through inner products. For extracting multiple user representations to capture diverse user interests, we use parametric attention as described in Section 3.1. We list all the baselines below. We note that methods that rely on knowledge transfer of user/item features [6, 51] or augmentation methods (over/under sampling [8]) to mitigate imbalance can be easily combined with all the following baselines. Our proposed density-weighting and the baselines considered do not depend on user/item features.

• •

  SUR: We use a single user representation as the output of the user tower by using $M=1$ in the parametric attention [20] layer.

• •

  MUR: The multiple user representations are extracted using the parametric attention [20] layer with $M=5$ output representations. We do an ablation to study the effect of $M$ in Section 4.6.1.

• •

  Frequency Balance: We use the frequency-based balancing [18] as described in Equation 5. Note that this is a static weighting scheme, where the item weights do not change during training.

• •

  Focal Balance: A recent work by Lin et al. [33] modifies the loss function to train the model on an imbalanced dataset by down-weighting the loss of well-classified examples. This is a dynamic weighting scheme, where the item weights depend on the loss of each input during training.

• •

  Qmargin: Cao et al. [9] proposed to add a per-class (per-item) margin $\delta_{y}$ into the cross-entropy loss, where $\delta_{y}\propto p(y)^{-1/4}$.

• •

  Item Norm: [25] upweights the score of rare items by normalizing the item representations. We also experiment with a post-hoc version of normalized item, where a standard two-tower model is trained and the item representation normalization is only carried out during the inference. [23].

We set the embedding dimension as 16 across all the models for the three benchmarks. We use the two-tower architecture as the backbone for all the models, where the user tower is a self-attention encoder [45] with two multi-head attention layers and four attention heads per layer. The item tower is an MLP with two fully connected layers each with 16 output nodes. For IDW, the momentum update of item weights (Equation 9) and $\eta$ are determined using grid search in the range of $\{0.7,0.8,0.9,0.99\}$ and $\{1.0,2.0,3.0,4.0\}*10^{-4}$. We use Adam to optimize all the models with a learning rate of 0.01. For all models, we used early stopping based on the validation dataset. The models are implemented with TensorFlow Recommenders (TFRS) [28].