Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation

We represent each item with an elliptical Gaussian distribution governed by a mean vector and one diagonal covariance matrix. To be specific, each item has two embedding representations, which are for mean and covariance. We denote the mean embedding table as $\mathbf{E}^{\mu}\in\mathbb{R}^{|\mathcal{V}|\times d}$ and covariance embedding table as $\mathbf{E}^{\Sigma}\in\mathbb{R}^{|\mathcal{V}|\times d}$, where $\mathcal{V}$ denotes the item set and $d$ is the embedding dimensionality. We also define separate learnable positional embeddings for both Transformers, $\mathbf{P}^{\mu}\in\mathbb{R}^{n\times d}$ for the mean Transformer and $\mathbf{P}^{\Sigma}\in\mathbb{R}^{n\times d}$ for the covariance Transformer respectively. The $n$ denotes the maximum length of the sequence. The elements $p_{i}$ signifies the positional information at position $i$ in the sequence. As the sequence lengths of users vary a lot, we keep the most recent $n$ interactions to format the sequence if a user has more than $n$ interactions. Otherwise, we keep all interactions and apply zero paddings until the sequence has a length of $n$. With all embeddings, given a sequence $\mathcal{S}={[v_{1},v_{2},\dots,v_{n}]}$, we have its mean and covariance sequence representations as:

We introduce a novel distribution-based self-attention mechanism considering numerical stability and its applicability to distribution embeddings. A typical self-attention block introduces query $\mathbf{Q}$, key $\mathbf{K}$, and value $\mathbf{V}$. In sequential recommendation, the query $\mathbf{Q}$, key $\mathbf{K}$, and value $\mathbf{V}$ are linear transformations of the same sequence embedding $\mathbf{E}_{\mathcal{S}}^{*}\mathbf{W}^{Q}$, $\mathbf{E}_{\mathcal{S}}^{*}\mathbf{W}^{K}$, and $\mathbf{E}_{\mathcal{S}}^{*}\mathbf{W}^{V}$, where $\mathbf{E}_{\mathcal{S}}^{*}$ is any sequence embedding defined in Eq. (2). The self-attention adopts the scaled dot-product attention (Vaswani et al., 2017) on the query and key to measure weights from previous steps on the current step. To maintain numerical stability of query, key, and value, we apply “exponential linear unit” (elu) activation (Clevert et al., 2015) after the linear transformations, which presents the distribution-based self-attention as:

where $\sigma(\cdot)$ denotes the softmax function and ‘DSA’ is short for Distribution Self-Attention. We apply the DSA module in the mean Transformer as $\textbf{DSA}^{\mu}$ and the covariance Transformer as $\textbf{DSA}^{\Sigma}$, either taking $\mathbf{E}_{\mathcal{S}}^{*}=\mathbf{E}_{\mathcal{S}}^{\mu}$ or $\mathbf{E}_{\mathcal{S}}^{*}=\mathbf{E}_{\mathcal{S}}^{\Sigma}$ in Eq. (2) as inputs, respectively.

In additional to uncovering sequential patterns, we introduce two novel and adaptive versions of feed-forward layers to endow extra non-linearity. We adapt the FFN layer to distribution representations by replacing the ReLU activation as ELU. The FFN with respect to both $\textbf{DSA}^{\mu}_{i}$ and $\textbf{DSA}^{\Sigma}_{i}$ at position $i$ are defined as:

respectively, where all $W$ are in $\mathbb{R}^{d\times d}$ and all $b$ are in $\mathbb{R}^{d}$.

We retain those techniques used in existing Transformer structures, , including residual connection, dropout operation and layer normalization. Note that we can stack multiple DSA and FFN layers to learn more complex item relationships. In the meantime, the multi-heads mechanism is also applicable in DT4SR framework.

However, a valid distribution requires covariance matrix to be positive definite while the outputs from FFN layers do not guarantee this property. Thus, we add an all ones vector to the output covariance embedding vector after the ELU activation, which is defined as follows:

where $\mathbf{O}^{(L)}_{i}$ denotes the output covariance embedding of item $i$ after $L$ layers of $\textbf{DSA}^{\Sigma}$ and $\mathbf{FFN}^{\Sigma}$, and $\mathbf{1}\in\mathbb{R}^{d}$ is an all ones vector.

Recall that the mean and covariance Transformers’ outputs infer the mean and covariance embeddings of the next-step item at each position in the sequence, respectively. Following the same setting in (Kang and McAuley, 2018), we use the next-step item as ground truth prediction for each position in the sequence. For example, given a sequence $\mathcal{S}^{u}=[v^{u}_{1},v^{u}_{2},v^{u}_{3},\dots,v^{u}_{n}]$, the ground truth item for $v^{u}_{1}$ is $v^{u}_{2}$. Note that if $v^{u}_{i}$ is a padding item, its ground truth item is also a padding item.

We evaluate the proposed DT4SR on three public benchmark datasets from Amazon review datasets (McAuley et al., 2015). Amazon datasets are known for high sparsity and having several categories of rating reviews. Details of datasets statistics are presented in Table 1. Following (Kang and McAuley, 2018; Sun et al., 2019), we treat the presence of ratings as positive implicit feedbacks. We use the timestamps of each rating to sort the interactions of each user to generate the sequence. The most recent interaction is used for test and the last second one is used for validation.

Evaluation Protocol. We evaluate all models with three standard metrics, Recall@N, NDCG@N, and Mean Reciprocal Rank (MRR). Recall@N measures the accuracy of retrieving relevant items in the top-N recommendation. NDCG@N also considers the ranked positions of retrieved relevant items in the top-N list. MRR measures the ranking performance of the entire recommendation list. We set N to be 1 and 5 for evaluation. For each user, we randomly sample 1,000 items without interaction with the user as negative items considering ranking efficiency.
Baselines. We compare DT4SR with several state-of-the-art recommendation methods in three relevant categories: static methods with point embedding vectors, static metric learning methods, and sequential methods. We use LightGCN (He et al., 2020) as the strong baseline with point embedding vectors. We also compare with BPRMF (Rendle et al., 2012), but we omit it in the performance table because of its low values. For metric learning methods, we compare the most recent work DDN (Zheng et al., 2019b), and SML (Li et al., 2020b). For sequential recommendation methods, we adopt the metric learning-based TransRec (He et al., 2017) and Transformer-based SASRec (Kang and McAuley, 2018) as baselines. Note that TransRec belongs to both the metric learning framework and sequential recommendation.
Hyper-parameter Settings. For all baselines, we search the embedding dimension in $\{32,64,128\}$. As the proposed model has both mean and covariance embeddings, we only search for $\{16,32,64\}$ for DT4SR for a fair comparison. We search the $L$-2 regularization weight from $\{0.0,0.001,0.005,0.01,0.05,0.1\}$. The number of layers $L$ is selected from $\{1,2\}$. For all baselines specific hyper-parameters, we do not list all of them because of the space limitations. We grid search all possible combinations for all models and report the test set performance based on the best validation MRR result.

We report the overall performance comparison between the proposed DT4SR and baselines in Table 2. The followings are our observations:

• •

  RQ1: DT4SR significantly outperforms all baselines in all three datasets across all evaluation metrics. The average relative improvements over the second-best baseline is 19.9% on Recall@1, 6.7% in Recall@5, 10.8% in NDCG@5, and 10.4% in MRR. These improvements demonstrate the effectiveness of distribution-based representations with uncertainty in sequential recommendation.

• •

  RQ2: Compared with the point embedding representation sequential model SASRec, the proposed DT4SR still achieves a great improvement margin. The reasons for this improvement are twofold. First, the distribution-based representations provide uncertainty information when we model users with various interests. Moreover, the sequential modeling for mean and covariance can dynamically capture the evolving patterns of uncertainty within the sequence.

• •

  Among all baselines, we can observe that sequential methods achieve outstanding performance, demonstrating the necessity of utilizing sequential information. Moreover, the state-of-the-art CF method LightGCN outperforms other static baselines, owing to its capability of using graph information.

We present the performances of LightGCN, SASRec, and proposed DT4SR with respect to different sequences lengths (i.e., number of interactions of users) in Figure 3. We can observe that DT4SR performs the best on all sequence length intervals of users. Note that for users with only one interaction, the relative performance gain of DT4SR against SASRec is 9% on Recall@5. It demonstrates the effectiveness of DT4SR in cold-start users. We can also observe that the long sequences (i.e., users with more than 20 interactions) achieve considerable improvements, especially on the Toys dataset. Long sequences typically cover diverse interests, and the observation proves the necessity of uncertainty in representing sequences.

We plot the performances of LightGCN, SASRec, and proposed DT4SR on cold start items, which only have limited interactions in training data. For Toys dataset, DT4SR achieves 100% relative improvements on extremely cold start items (i.e., items with only one interaction). For the Games dataset, the performance gain is also more than 50% for extremely cold start items. This experiment shows that distribution representations for items and sequences are more expressive and generalized for cold start items. Note that the DT4SR still achieves comparative performance in frequent items.