DualVAE: Dual Disentangled Variational AutoEncoder for Recommendation

To fit dyadic interaction data, much of the literature on collaborative filtering [7] focuses on latent representation learning. Matrix factorization (MF)[14] is a typical CF framework that directly embeds the IDs of users and items into latent feature space. Later, some studies [21, 26, 10, 30] model both users and items by introducing deep neural networks, which can be viewed as a nonlinear extension of MF. For instance, NeuMF [10] treats the recommendation problem with implicit feedback as a binary classification problem and fuses Generalized Matrix Factorization (GMF) with MLP. JCA [30] employs two separate autoencoders to simultaneously learn both user-user and item-item correlations. To further account for uncertainty in latent space, some researchers introduce VAEs [13] into collaborative filtering to improve model robustness [11, 22, 19, 6, 17, 4]. For instance, MultiVAE [15] extends VAEs to capture the latent variables of users with implicit feedback and estimate parameters via Bayesian inference. BiVAE [23] infers the latent representations for users and items through bilateral VAEs. Despite their achievements, the entanglement of latent factors behind user behaviors, is mostly neglected by these methods, leading to weak interpretable results.

Due to its robust performance and interpretability, disentangled representation learning [2] is gradually being introduced into VAE-based recommender systems. Most of these methods [16, 29, 24] usually decouple users’ diverse preferences based on their collaborative interactions. For example, MacridVAE [16] introduces categorical distributions to disentangle the macro-level and micro-level factors on different items via a variational autoencoder. Zheng et al. [29] propose to structurally disentangle user interest and conformity by training with cause-specific data. Wang et al. [24] utilize structural causal models to generate causal representations that describe the causal relationship between latent factors. Nevertheless, these methods only unilaterally disentangle coarse-grained user preferences, while ignoring the relationship modeling between user preferences and item features in multiple aspects, and hence are not suitable for fitting binary interaction data.

We consider an implicit collaborative recommendation that consists of a user set $\mathcal{U}$ with $m$ users and an item set $\mathcal{I}$ with $n$ items. The interaction data is denoted as $\mathbf{R}\in\mathbb{R}^{m\times n}$, where $r_{u,i}=1$ is the observed interaction of user $u$ on item $i$. We use $\mathbf{r}_{u*}\in\mathbb{R}^{1\times n}$ to denote $u$’s interaction vector corresponding to the $u$-th row in $\mathbf{R}$, and $\mathbf{r}_{*i}\in\mathbb{R}^{m\times 1}$ to denote the $i$-th column in $\mathbf{R}$ towards item $i$. For traditional VAE-based CF models, the latent variables of per user and item are generally denoted by entangled $\mathbf{z}_{u}$ and $\mathbf{z}_{i}$, respectively. Considering that an item in general has multi-aspect features, we denote the item latent representation as $\mathbf{z}_{i}=[\mathbf{z}_{i}^{1};\mathbf{z}_{i}^{2};\dots;\mathbf{z}_{i}^{A}]\in\mathbb{R}^{1\times Ad}$, where $\mathbf{z}_{i}^{a}\in\mathbb{R}^{d}$ corresponds to $i$’s representation towards the $a$-th aspect, $A$ is the number of aspects, and $d$ is the embedding dimension. Similarly, user $u$’s latent representation is denoted as $\mathbf{z}_{u}=[\mathbf{z}_{u}^{1};\mathbf{z}_{u}^{2};\dots;\mathbf{z}_{u}^{A}]$, where $\mathbf{z}_{u}^{a}\in\mathbb{R}^{d}$ reflects $u$’s preference under the $a$-th aspect. Our objective in this work is to learn a robust VAE-based CF model to disentangle and infer multi-aspect latent representations of users and items from user-item interactions. After training, we can exploit the learned multi-aspect latent representations to perform top-$N$ recommendation.

Traditional VAE-based CF models generally define a user-side generative model that generates the observed data from the following distribution,

(3.1)

$$p_{\theta}(\mathbf{r}_{u*})=\int p_{\theta}(\mathbf{r}_{u*}|\mathbf{z}_{u})p(\mathbf{z}_{u})~{}d\mathbf{z}_{u},$$

where $p_{\theta}(\mathbf{r}_{u*}|\mathbf{z}_{u})=\prod_{r_{u,i}\in\mathbf{r}_{u*}}p_{\theta}(r_{u,i}|\mathbf{z}_{u})$ is a probability distribution over the $n$ items learned by a generative model with parameters $\theta$. Each observed interaction $r_{u,i}\in\mathbf{r}_{u*}$ is independently generated from the inferred user latent variable $\mathbf{z}_{u}$. $p(\mathbf{z}_{u})$ is the prior of user variable $\mathbf{z}_{u}$. Different from the one-way generative paradigm, we propose a dual disentangled generative model to generate the observed interaction that encourages multi-aspect disentanglement of both users and items,

(3.2)

$$p_{\theta}(r_{u,i})=\int p_{\theta}(r_{u,i}|\mathbf{z}_{u},\mathbf{z}_{i},\mathbf{C},\mathbf{P})p(\mathbf{z}_{u})p(\mathbf{z}_{i})~{}d\mathbf{z}_{u}\mathbf{z}_{i},$$

where the generation of $r_{u,i}$ is related to two latent variables $\mathbf{z}_{u},\mathbf{z}_{i}$. $\mathbf{C}=\{\mathbf{c}_{i}\}^{n}_{i=1}$ and $\mathbf{P}=\{\mathbf{p}_{m}\}^{m}_{u=1}$ are the item-aspect and user-aspect probability matrices, in which $\mathbf{c}_{i}\in\mathbb{R}^{1\times A}$ and $\mathbf{p}_{u}\in\mathbb{R}^{1\times A}$ are two probability vectors drawn from two aspect distributions $p(\mathbf{c}_{i})$ and $p(\mathbf{p}_{u})$. $A$ is the number of aspects. We assume that $p(\mathbf{z}_{u})=p(\mathbf{z}_{u}|\mathbf{C})$ and $p(\mathbf{z}_{i})=p(\mathbf{z}_{i}|\mathbf{P})$, i.e., $\mathbf{z}_{u}$, $\mathbf{z}_{i}$, $\mathbf{C}$ and $\mathbf{P}$ can be independently generated.

Our proposed DualVAE implements above distributions ($p(\mathbf{c}_{i})$, $p(\mathbf{p}_{u})$, $p(\mathbf{z}_{u})$, $p(\mathbf{z}_{i})$, and $p_{\theta}(r_{u,i}|\mathbf{z}_{u},\mathbf{z}_{i},\mathbf{C},\mathbf{P})$) by four modules: 1) Attention-aware dual disentanglement (ADD) module that generates the aspect-aware probability distributions $p(\mathbf{p}_{u})$, $p(\mathbf{c}_{i})$ of users and items. 2) Disentangled variational inference (DVI) module that infers the posterior $p(\mathbf{z}_{1:|\mathcal{U}|},\mathbf{z}_{1:|\mathcal{I}|}|\mathbf{R})$ over disentangled latent variables $\mathbf{z}_{u}$, $\mathbf{z}_{i}$ of users and items. 3) Joint generation (JG) module that utilizes the inferred user and item latent representations and aspect-aware probability matrices to reconstruct the observed interactions, thereby achieving $p_{\theta}(r_{u,i}|\mathbf{z}_{u},\mathbf{z}_{i},\mathbf{C},\mathbf{P})$. 4) Neighborhood-enhanced representation constraint (NRC) module that maintains the correspondence and independence between latent variables by introducing contrastive learning. Next, we will detail the implementation of each module.

The ADD module is designed to generate the item-aspect and user-aspect probability vectors, which achieves the aspect assignment of users and items.

The DVI module is designed to infer the multi-aspect latent variables of users and items. In general, latent variables are drawn from prior distributions. We follow the convention to represent the prior over both user and item latent variables via the standard multivariate isotropic Gaussians, i.e., $p(\mathbf{z}_{u})=\mathcal{N}(\mathbf{0},\mathbf{I}),p(\mathbf{z}_{i})=\mathcal{N}(\mathbf{0},\mathbf{I})$, where $\mathbf{I}$ is a diagonal matrix. Given the user-item interaction matrix $\mathbf{R}$, the goal of variational inference model is to infer the posterior over the latent variables $p(\mathbf{z}_{1:|\mathcal{U}|},\mathbf{z}_{1:|\mathcal{I}|}|\mathbf{R})$. Proverbially, the true posterior is intractable, and thereby precise inference is infeasible. We therefore exploit Variational Bayes [3] to approximate this posterior distribution by a parameterized function $q_{\phi}(\mathbf{z}_{1:|\mathcal{U}|},\mathbf{z}_{1:|\mathcal{I}|}|\mathbf{R})=q_{\phi_{u}}(\mathbf{z}_{1:|\mathcal{U}|}|\mathbf{R})q_{\phi_{i}}(\mathbf{z}_{1:|\mathcal{I}|}|\mathbf{R})$. Obviously, the variational distribution firstly breaks the coupling between $\mathbf{z}_{u}$ and $\mathbf{z}_{i}$. Considering the statistical independence among users and items, we can set $q_{\phi_{u}}(\mathbf{z}_{1:|\mathcal{U}|}|\mathbf{R})=\prod_{u}q_{\phi_{u}}(\mathbf{z}_{u}|\mathbf{r}_{u*})$ and $q_{\phi_{i}}(\mathbf{z}_{1:|\mathcal{I}|}|\mathbf{R})=\prod_{i}q_{\phi_{i}}(\mathbf{z}_{i}|\mathbf{r}_{*i})$ to achieve the variational inference.

To further obtain disentangled multi-aspect latent representations of users and items, we assume that $q_{\phi_{u}}(\mathbf{z}_{u}|\mathbf{r}_{u*})=q_{\phi_{u}}(\mathbf{z}_{u}|\mathbf{r}_{u*},\mathbf{C})=\prod^{A}_{a=1}q_{\phi_{u}}(\mathbf{z}^{a}_{u}|\mathbf{r}_{u*},\mathbf{C})$ and $q_{\phi_{i}}(\mathbf{z}_{i}|\mathbf{r}_{*i})=q_{\phi_{i}}(\mathbf{z}_{i}|\mathbf{r}_{*i},\mathbf{P})=\prod^{A}_{a=1}q_{\phi_{i}}(\mathbf{z}^{a}_{i}|\mathbf{r}_{*i},\mathbf{P})$. Without loss of generality, we express the variational $q_{\phi_{u}}(\mathbf{z}^{a}_{u}|\mathbf{r}_{u*},\mathbf{C})$ and $q_{\phi_{i}}(\mathbf{z}^{a}_{i}|\mathbf{r}_{*i},\mathbf{P})$ as a multivariate normal distribution with a diagonal covariance matrix,

(3.5)		$\displaystyle q_{\phi_{u}}(\mathbf{z}^{a}_{u}|\mathbf{r}_{u*},\mathbf{C})$	$\displaystyle\sim\mathcal{N}(\boldsymbol{\mu}^{a}_{u},\boldsymbol{\sigma}^{a}_{u}),$
		$\displaystyle q_{\phi_{i}}(\mathbf{z}^{a}_{i}|\mathbf{r}_{*i},\mathbf{P})$	$\displaystyle\sim\mathcal{N}(\boldsymbol{\mu}^{a}_{i},\boldsymbol{\sigma}^{a}_{i}),$

where $\boldsymbol{\mu}^{a}\in\mathbb{R}^{d}$ and $\boldsymbol{\sigma}^{a}\in\mathbb{R}^{d}$ are the mean and covariance of the disentangled variational distributions, parameterized by two shallow networks $f_{\phi_{u}}:\mathbb{R}^{n}\rightarrow\mathbb{R}^{2d}$ and $f_{\phi_{i}}:\mathbb{R}^{m}\rightarrow\mathbb{R}^{2d}$ with parameters $\phi_{u}$ and $\phi_{i}$,

(3.6)		$\displaystyle\left[\boldsymbol{\mu}^{a}_{u};\boldsymbol{\sigma}^{a}_{u}\right]$	$\displaystyle=f_{\phi_{u}}(\mathbf{r}^{a}_{u*}),~{}\mathbf{r}^{a}_{u*}=\mathbf{r}_{u*}^{\top}\otimes{\mathbf{c}^{a}},$
		$\displaystyle\left[\boldsymbol{\mu}^{a}_{i};\boldsymbol{\sigma}^{a}_{i}\right]$	$\displaystyle=f_{\phi_{i}}(\mathbf{r}^{a}_{*i}),~{}\mathbf{r}^{a}_{*i}=\mathbf{r}_{*i}\otimes\mathbf{p}^{a},$

where $\mathbf{c}^{a}\in\mathbb{R}^{n\times 1}$ and $\mathbf{p}^{a}\in\mathbb{R}^{m\times 1}$ are the $a$-th column of $\mathbf{C}$ and $\mathbf{P}$, respectively, representing the $a$-th aspect probability vectors over all items and users. $\otimes$ denotes element-wise product. $\mathbf{r}^{a}_{u*}$ and $\mathbf{r}^{a}_{*i}$ are calculated as the aspect-level interaction vectors of user $u$ and item $i$, respectively. It is clear to see that $\mathbf{r}_{u*}=\sum_{a}\mathbf{r}^{a}_{u*}$, and $\mathbf{r}_{*i}=\sum_{a}\mathbf{r}^{a}_{*i}$. Notably, under aspect $a$, the aspect-level probability vectors of items are employed to learn aspect-level latent representations of users, and vice versa. Such operations ensure that the disentangled representations of users and items are in a one-to-one correspondence at the aspect-level. Further, $\mathbf{z}^{a}_{u}$ and $\mathbf{z}^{a}_{i}$ are sampled by a reparameterization trick [13, 20],

(3.7)

$$\mathbf{z}^{a}_{u}=\boldsymbol{\mu}^{a}_{u}+\boldsymbol{\sigma}^{a}_{u}\otimes\boldsymbol{\epsilon},~{}~{}~{}\mathbf{z}^{a}_{i}=\boldsymbol{\mu}^{a}_{i}+\boldsymbol{\sigma}^{a}_{i}\otimes\boldsymbol{\epsilon},$$

where $\boldsymbol{\epsilon}$ is a noise vector with $\boldsymbol{\epsilon}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$.

Given the user and item latent representations $\mathbf{z}_{u}$ and $\mathbf{z}_{i}$ and the aspect probability matrices $\mathbf{C}$ and $\mathbf{P}$, the JG module is designed to predict the preference of a user towards an item by reconstructing their observed interactions. Specifically, we generate the distribution $p_{\theta}(r_{u,i}|\mathbf{z}_{u},\mathbf{z}_{i})$ of user $u$’s preference towards item $i$ by a Poisson distribution [23],

(3.8)

$$p_{\theta}(r_{u,i}|\mathbf{z}_{u},\mathbf{z}_{i},\mathbf{C},\mathbf{P})=exp(r_{u,i}\rm log\it g_{\theta}(\mathbf{z}_{u},\mathbf{z}_{i})-g_{\theta}(\mathbf{z}_{u},\mathbf{z}_{i})),$$

where $g_{\theta}(\mathbf{z}_{u},\mathbf{z}_{i})$ is a differentiable function that utilizes disentangled latent representations of user $u$ and item $i$ to generate the joint observation $r_{u,i}$,

(3.9)			$\displaystyle g_{\theta}(\mathbf{z}_{u},\mathbf{z}_{i})=\sum\nolimits_{a=1}^{A}p^{a}_{u}\cdot c^{a}_{i}\cdot\sigma\left(\textsc{Skip}_{\theta}(\mathbf{z}^{a}_{u},\mathbf{z}^{a}_{i})\right)$
			$\displaystyle\textsc{Skip}_{\theta}(\mathbf{z}^{a}_{u},\mathbf{z}^{a}_{i})=\mathbf{z}^{a}_{u}\odot\mathbf{z}^{a}_{i}+f_{\theta}(\mathbf{z}^{a}_{u})\odot f_{\theta}(\mathbf{z}^{a}_{i})$

where $\textsc{Skip}_{\theta}(\cdot)$ is a skip-connection operation to avoid the issue of latent variable collapse [5], $f_{\theta}(\cdot)$ is a non-linear function parameterized by $\theta$, $\sigma$ is sigmoid function, and $\odot$ denotes inner product. Notice $p_{\theta}(r_{u,i}|\mathbf{z}_{u},\mathbf{z}_{i},\mathbf{C},\mathbf{P})\propto g_{\theta}(\mathbf{z}_{u},\mathbf{z}_{i})$ and $c^{a}_{i}$ and $p^{a}_{u}$ are regarded as aspect-level weights for item $i$ and user $u$.

To optimize our model parameters $\theta$, $\phi_{u}$ and $\phi_{i}$, we proceed with approximate inference and learning by maximizing the evidence lower bound (ELBO) $\sum_{u,i}\rm log\it p_{\theta}(r_{u,i})$. However, due to the sparsity of observed interactions, directly performing unbiased stochastic optimization on the above objective is inconvenient. We hence exploit the two-way nature of our model and perform alternate optimization in a Gauss-Seidel fashion [23]. Specifically, we split the model parameters into user-related and item-related parts, and then alternately optimize them. Firstly, the user-side objective is updated with fixed item-related parameters,

(3.10)		$\displaystyle\mathcal{L}^{u}_{vae}$	$\displaystyle=\mathbb{E}_{p(\mathbf{C})}[\mathbb{E}_{q_{\phi_{u}}(\mathbf{z}_{u}|\mathbf{r}_{u*},\mathbf{C})}[\rm log\it p_{\theta_{u}}(\mathbf{r}_{u*}|\mathbf{z}_{u},\mathbf{z}_{1:|\mathcal{I}|},\mathbf{C})]$
			$\displaystyle-KL(q_{\phi_{u}}(\mathbf{z}_{u}|\mathbf{r}_{u*},\mathbf{C})\|p(\mathbf{z}_{u}))]$

where $p_{\theta_{u}}(\mathbf{r}_{u*}|\mathbf{z}_{u},\mathbf{z}_{1:|\mathcal{I}|},\mathbf{C})=\prod_{i}p_{\theta_{u}}(r_{u,i}|\mathbf{z}_{u},\mathbf{z}_{i},\mathbf{c}_{i})$. The first term is interpreted as reconstruction error, while the second term defines the Kulback-Leibler divergence. Analogously, when keeping user-related parameters fixed, the item-side objective needs to be optimized,

(3.11)		$\displaystyle\mathcal{L}^{i}_{vae}$	$\displaystyle=\mathbb{E}_{p(\mathbf{P})}[\mathbb{E}_{q_{\phi_{i}}(\mathbf{z}_{i}|\mathbf{r}_{*i},\mathbf{P})}[\rm log\it p_{\theta_{i}}(\mathbf{r}_{*i}|\mathbf{z}_{1:|\mathcal{U}|},\mathbf{z}_{i},\mathbf{P})]$
			$\displaystyle-KL(q_{\phi_{i}}(\mathbf{z}_{i}|\mathbf{r}_{*i},\mathbf{P})\|p(\mathbf{z}_{i}))]$

where $p_{\theta_{i}}(\mathbf{r}_{*i}|\mathbf{z}_{1:|\mathcal{U}|},\mathbf{z}_{i},\mathbf{P})=\prod_{u}p_{\theta_{i}}(r_{u,i}|\mathbf{z}_{u},\mathbf{z}_{i},\mathbf{p}_{i})$. Note the above objectives $\mathcal{L}^{u}_{vae}$ and $\mathcal{L}^{i}_{vae}$ are not conflicting with each other, since maximizing either of them corresponds to the maximization of the overall ELBO.

The NRC module is designed to prevent the confusion between latent representations inferred from different aspects and maintain correspondence between aspect-level representations of users and items. Specifically, we first aggregate the interacted neighbors to calculate the aspect-level neighborhood-based representations $\mathbf{o}^{a}_{u}$ and $\mathbf{o}^{a}_{i}$ of user $u$ and item $i$,

(3.12)

$$\mathbf{o}^{a}_{u}=\sum\nolimits_{i\in\mathcal{N}_{u}}c^{a}_{i}\cdot\mathbf{z}^{a}_{i},~{}~{}~{}\mathbf{o}^{a}_{i}=\sum\nolimits_{u\in\mathcal{N}_{i}}p^{a}_{u}\cdot\mathbf{z}^{a}_{u},$$

where $\mathcal{N}_{u}$ and $\mathcal{N}_{i}$ are the neighbor sets of user $u$ and item $i$, respectively. Afterwards, we treat the inferred aspect-level latent representations $\mathbf{z}^{a}_{u}$ and neighborhood-based representation $\mathbf{o}^{a}_{u}$ of user $u$ as positive sample pair. Moreover, we set two kinds of negative samples. Firstly, to constrain the independence among different aspect representations, we set aspect-level negative samples by taking representations of different aspects for the same user as negative samples. Secondly, we use representations from different users under the same aspect as user-level negative samples to guarantee the discrepancy in aspect-level representations among different users. The InfoNCE [8] loss is employed to achieve the contrastive constraint,

(3.13)

$$\mathcal{L}^{a}_{u}=-\log\frac{e^{\frac{s(\mathbf{z}^{a}_{u},\mathbf{o}^{a}_{u})}{\tau}}}{\underbrace{e^{\frac{s(\mathbf{z}^{a}_{u},\mathbf{o}^{a}_{u})}{\tau}}}_{\text{pos-pair}}+\underbrace{\sum\limits_{b\neq a}^{A}e^{\frac{s(\mathbf{z}^{a}_{u},\mathbf{o}^{b}_{u})}{\tau}}}_{\text{aspect-level neg-pairs}}+\underbrace{\sum\limits_{v\neq u}^{B_{u}}e^{\frac{s(\mathbf{z}^{a}_{u},\mathbf{o}^{a}_{v})}{\tau}}}_{\text{user-level neg-pairs}}}$$

where $\mathcal{L}^{a}_{u}$ is the contrastive loss of each aspect-level representation of user $u$, $s(\cdot)$ denotes the cosine similarity measure, $\tau$ is a temperature parameter (generally set to $0.2$), and $B_{u}$ denotes a set of users within a batch. Similarly, the item disentangled representation can be constrained by,

(3.14)

$$\mathcal{L}^{a}_{i}=-\log\frac{e^{\frac{s(\mathbf{z}^{a}_{i},\mathbf{o}^{a}_{i})}{\tau}}}{\underbrace{e^{\frac{s(\mathbf{z}^{a}_{i},\mathbf{o}^{a}_{i})}{\tau}}}_{\text{pos-pair}}+\underbrace{\sum\limits_{b\neq a}^{A}e^{\frac{s(\mathbf{z}^{a}_{i},\mathbf{o}^{b}_{i})}{\tau}}}_{\text{aspect-level neg-pairs}}+\underbrace{\sum\limits_{j\neq i}^{B_{i}}e^{\frac{s(\mathbf{z}^{a}_{i},\mathbf{o}^{a}_{j})}{\tau}}}_{\text{item-level neg-pairs}}}$$

where $B_{i}$ denotes a set of items within a batch. Intuitively, the auxiliary supervision of positive pairs encourages the direction consistency between latent representations of users and items on the same aspect, while the supervision of negative pairs enforces the divergence among different aspects. Note there are also some methods [25] that use distance correlation as a regularizer to achieve representation independence modeling. We do not use such distance correlation, because it can only ensure the independence of the aspects, but cannot guarantee the correspondence between the aspect-level representations of users and items.

Finally, the overall objects for alternately optimization can be set as,

(3.15)

$$\mathcal{L}_{u}=\mathcal{L}^{u}_{vae}+\gamma\cdot\sum\nolimits_{a=1}^{A}\mathcal{L}^{a}_{u},~{}~{}~{}\mathcal{L}_{i}=\mathcal{L}^{i}_{vae}+\gamma\cdot\sum\nolimits_{a=1}^{A}\mathcal{L}^{a}_{i},$$

where $\gamma$ is an adjusted factor to balance the VAE loss and the contrastive loss.

Table 2 presents the overall performance comparison, from which we have the following key observations:

• •

  DualVAE consistently outperforms all the other baselines on all datasets and metrics. In particular, DualVAE’s improvement over the best baseline is statistically significant in all cases, demonstrating that disentangling multi-aspect features of users and items indeed can help improve performance. Moreover, as the sparsity of datasets increases, the improvement of DualVAE becomes more significant, indicating that capturing multi-aspect uncertainty of user and item feature spaces can further alleviate the data sparsity problem and improve model robustness.

• •

  Dual VAE-based methods outperform user-specific VAE-based methods on all datasets. Specifically, the performance of BiVAE is higher than that of PoissVAE and MultiVAE, while DualVAE achieves better performance than MacridVAE. The results indicate that inferring latent representations of both users and items can better adapt to the two-way nature of interaction data, thereby promoting accuracy.

• •

  Disentanglement-based methods generally achieve better performance than their non-disentanglement counterparts. For instance, MacridVAE outperforms MultiVAE by an average of 3.97% in terms of N@$20$ across all datasets, indicating that disentangling multi-aspect user preferences is beneficial for improving representation quality and performance.

Table 3 reports the results of ablation studies for various variants of DualVAE. Specifically, w/o ID (w/o UD) denotes that our model only retains user(item)-side aspect disentanglement; w/o UNS and w/o ANS are two variants that do not use user-level and aspect-level negative samples in NRC module, respectively. w/o NPS is a variant that removes neighborhood-based representations and uses inferred aspect-level representations and themselves to construct positive pairs. From Table 3, we can find:

• •

  After removing the ADD module, the performance of DualVAE drops (vs. w/o ADD) by an average of 6.65% and 8.46% respectively on R@$20$ and N@$20$, indicating the effectiveness of attention-aware disentanglement. The performance gap between DualVAE and w/o UD (w/o ID) demonstrates the benefit of shaping the aspect distributions of both users and items in improving performance. Moreover, w/o ID performs better than w/o UD on sparser datasets, which suggests that the user-side disentanglement may bring more performance gains than the item-side to alleviate interaction sparsity.

• •

  The variant w/o NRC removes the NRC module and causes an average performance drop of 2.21% and 2.47% in terms of R@$20$ and N@$20$ respectively, which indicates the necessity of ensuring the correspondence and independence of disentangled representations in promoting performance. The comparison between DualVAE and w/o ANS (w/o UNS) reveals that it is beneficial to maintain the independence of disentangled representations by exploiting two-level negative samples. Moreover, w/o ANS performs slightly better than w/o UNS on AKindle and Yelp, showing that aspect-level differences may be more important than user-level differences in keeping the representation independence. w/o NPS performs worse than DualVAE, which demonstrates the significance of maintaining the correspondence between multi-aspect representations of users and items.