Self-supervised Graph Learning for Occasional Group Recommendation

There are three sets of entities in the group recommendation scenario: $U=\ \{u_{1},\cdots,u_{|U|}\}$ denotes a user set, $I=\ \{i_{1},\cdots,i_{|I|}\}$ denotes an item set, and $G=\ \{g_{1},\cdots,g_{|G|}\}$ denotes a group set. There are three kinds of observed interaction graphs, i.e., group-item subgraph $\mathcal{G}_{GI}$, user-item subgraph $\mathcal{G}_{UI}$ and group-user subgraph $\mathcal{G}_{GU}$. Since the social connections of user-user and group-group are also important to depict the user and group profiles, we build two kinds of implicit interaction graphs based on $\mathcal{G}_{GI}$ and $\mathcal{G}_{UI}$, namely user-user subgraph $\mathcal{G}_{UU}$ and group-group subgraph $\mathcal{G}_{GG}$. In $\mathcal{G}_{UU}$, the two users are connected if they share with more than $c_{u}$ items. Similarly, in $\mathcal{G}_{GG}$, the two groups are connected if they share with more than $c_{g}$ items. Formally, notation $\mathcal{G}$ = $\{\mathcal{V},\mathcal{E}\}$ denotes the set of observed and implicit interaction graphs, i.e., $\mathcal{G}$ = $\mathcal{G}_{GI}\cup\mathcal{G}_{UI}\cup\mathcal{G}_{GU}\cup\mathcal{G}_{GG}\cup\mathcal{G}_{UU}$, where $\mathcal{E}$ is the edge set and $\mathcal{V}$ is the nodes set which contains $\{U,I,G\}$.

Although existing GNN-based group recommendation methods show their diversity in modelling group interactions with users and items [13, 12, 14, 16], we notice that they essentially share a general model structure. Based on this finding, we present a base GNN model, which consists of a representation learning module and a jointly training module. The representation learning module learns the representations of groups and users upon their counterpart interaction graphs, while the jointly training module optimizes the user/group preferences over items to compare the likelihood and the true user-item/group-item observations.

We propose embedding reconstruction with GNN, which jointly reconstructs the groups/users/items embeddings from multiple subgraphs under the meta-learning setting. Here we take the group embedding reconstruction as an example. Notably, the user/item embedding reconstruction process is similar to the group embedding reconstruction process. Specifically, first, we need the groups with abundant interactions as the target groups, and use any recommendation model such as AGREE [10] or LightGCN [28] to learn the embeddings of the target groups as the ground-truth embeddings³³3Previous paper [17] has demonstrated that such recommendation models can obtain high-quality embeddings of nodes with enough interactions.. Then we mask a large proportion neighbors of the target group to simulate the occasional group. Based on the remained neighbors, we repeat the graph convolution operation multiple times to learn the ground-truth embeddings. Formally, for each target group $g$, we use notation $\textbf{h}_{g}$ to represent its ground-truth embedding. To mimic the occasional group, in each training episode, for each target group, we randomly sample $K$ items, $K$ users and $K$ groups from the corresponding group-item subgraph $\mathcal{G}_{GI}$, group-user subgraph $\mathcal{G}_{GU}$ and group-group subgraph $\mathcal{G}_{GG}$. We sample neighbors $L$ steps for each subgraph, i.e., for each target group, in each subgraph, we sample from its first-order neighbors to its $L$-order neighbors. After the sampling process is finished, for each target group, and for each subgraph, we can obtain at most $K^{l}(1\leq l\leq L)$ $l$-order neighbors. Next we use Eq. (1) to conduct the graph convolution operation $L$ steps from scratch to obtain the refined group embeddings $\textbf{h}_{g_{GI}}^{L}$, $\textbf{h}_{g_{GU}}^{L}$ and $\textbf{h}_{g_{GG}}^{L}$, use Eq. (2) to obtain the aggregated group embedding $\textbf{h}_{g_{GU^{\prime}}}^{L}$, and use Eq. (3) to obtain the fused group embedding $\textbf{h}_{g}^{L}$. Finally, following [35, 17], we measure the cosine similarity between $\textbf{h}_{g}$ and $\textbf{h}_{g}^{L}$, since the cosine similarity is a popularity indicator for the semantic similarity between embeddings:

Notably, the above proposed embedding reconstruction task is trained under the meta-learning setting, which enables the GNNs rapidly being adapted to new occasional groups. After we trained the model, when it comes to a new occasional group, given its first- and high-order neighbors, the pre-trained GNNs can generate a more accurate embedding for it. However, the above proposed embedding reconstruction task does not explicitly strengthen the high-order cold-start neighbors’ embedding quality. For example, if the high-order cold-start neighbors’ embeddings are biased, they will affect the embeddings of the target group when performing graph convolution. As shown in Fig. 1, for the target group $g_{2}$, its group member $u_{1}$ and high-order neighbor $i_{1}$ only have few interactions. The embeddings of $u_{1}$ and $i_{1}$ are inaccurate, which will hurt the embedding quality of $g_{2}$ after performing graph convolution operation. To solve this problem, we further incorporate an embedding enhancer into the above embedding reconstruction GNN model.

To explicitly strengthen the embedding quality of the high-order cold-start neighbors, we propose the embedding enhancer, which also learns the ground-truth group/user/item embedding, but only based on the first-order neighbors of the target group/user/item sampled from the counterpart graphs. Specifically, before we train the above embedding reconstruction task with GNNs, we train the embedding enhancer $f_{meta}$ under the same meta-learning setting as proposed in Section 3.1. Once the embedding enhancer $f_{meta}$ is trained, we combine the enhanced group/user/item embedding⁴⁴4The enhanced group/user/item embedding is an additional embedding produced by $f_{meta}$. produced by $f_{mata}$ with the original group/user/item embedding at each graph convolution step to improve the cold-start neighbors’ embeddings. Notably, the GNN model is to strengthen the cold-start groups’/users’/items’ embedding quality, while the embedding enhancer is to improve the high-order cold-start neighbors’ embedding quality.

Here we take group embedding as an example. The embedding enhancer tackles the user/item embedding is similar to the group embedding. Specifically, the embedding enhancer $f_{meta}$ is instantiated as a self-attention learner [9]. For each group $g$, the embedding enhancer accepts the randomly initialized first-order embeddings $\{\textbf{h}_{i_{GI_{1}}}^{0},\cdots,\textbf{h}_{i_{GI_{K}}}^{0}\}$, $\{\textbf{h}_{u_{GU_{1}}}^{0},\cdots,\textbf{h}_{u_{GU_{K}}}^{0}\}$ and $\{\textbf{h}_{g_{GG_{1}}}^{0},\cdots,\textbf{h}_{g_{GG_{K}}}^{0}\}$ from the corresponding subgraphs $\mathcal{G}_{GI}$, $\mathcal{G}_{GU}$ and $\mathcal{G}_{GG}$ as input, outputs the smoothed embeddings $\{\textbf{h}_{i_{GI_{1}}},\cdots,\textbf{h}_{i_{GI_{K}}}\}$, $\{\textbf{h}_{u_{GU_{1}}},\cdots,\textbf{h}_{u_{GU_{K}}}\}$ and $\{\textbf{h}_{g_{GG_{1}}},\cdots,\textbf{h}_{g_{GG_{K}}}\}$, and use the average function to obtain the meta embedding $\hat{\textbf{h}}_{g_{GI}}$, $\hat{\textbf{h}}_{g_{GU}}$, $\hat{\textbf{h}}_{g_{GG}}$. The process is:

The advantage of the self-attention learner is to pull the similar nodes more closer, while pushing away dissimilar nodes. Thus, the self-attention technique can capture the major group/user/item preference from its neighbors. Same with Section 3.1, the same cosine similarity (i.e., Eq.(5)) is used between $\hat{\textbf{h}}_{g}$ and $\textbf{h}_{g}$ to measure their similarity. Once the embedding enhancer $f_{meta}$ is trained, we incorporate the meta embedding $\hat{\textbf{h}}_{g_{GI}}$, $\hat{\textbf{h}}_{g_{GU}}$ and $\hat{\textbf{h}}_{g_{GG}}$ produced by the embedding enhancer into the GNN model at each graph convolution step (i.e., we add the meta embedding into Eq. (1)):

For a target group $g$, we repeat Eq. (8) $L$ steps to obtain the embeddings $\textbf{h}_{g_{GI}}^{L}$, $\textbf{h}_{g_{GU}}^{L}$, $\textbf{h}_{g_{GG}}^{L}$. Then, we use Eq. (2) to obtain the aggregated group embedding $\textbf{h}_{g_{GU^{\prime}}}^{L}$, and use Eq. (3) to obtain the final group embedding $\textbf{h}_{g}^{L}$. Finally, we also use cosine similarity (i.e., Eq. (5)) to optimize the model parameters including the GNNs’ parameters $\Theta_{f}$ and the embedding enhancer’s parameters $\Theta_{f_{meta}}$. Similarly, the embedding enhancer $f_{meta}$ can obtain the enhanced user embedding on $\mathcal{G}_{UI}$ and $\mathcal{G}_{UU}$, and the enhanced item embedding on $\mathcal{G}_{UI}$.

We conduct multi-task learning paradigm [45] to optimize the model parameters, i.e., we jointly train the recommendation objective function (cf. Eq. (4)) and the designed SSL objective function (cf. Eq. (6)):

We present the time and space complexity of ${\rm SGG}$, and compare its time and space complexity with the backbone GNN model. Same as LightGCN [28], we implement ${\rm SGG}$ as the matrix form. Suppose the number of edges in the interaction graph $\mathcal{G}$ is $|\mathcal{E}|$. The number of edges in the masked interaction graph $\hat{\mathcal{G}}$ is $|\hat{\mathcal{E}}|$. Since we mask a large proportion neighbors for each node in $\mathcal{G}$ to simulate the cold-start scenario, the size of masked edge set is far less than the original edge set, i.e., $|\hat{\mathcal{E}}|\ll|\mathcal{E}|$. Let $s$ denote the number of epochs, $d$ denote the embedding size and $L$ denote the GCN convolution layers. Since ${\rm SGG}$ introduces meta embedding to enhance the aggregation ability, the space complexity of ${\rm SGG}$ is twice than that of the vanilla GNN model. The time complexity comes from four parts, namely adjacency matrix normalization, graph convolution operation, recommendation objective function and self-supervised objective function. Since we do not change the GNN’s model structure and GNN’s inference process, the time complexity of ${\rm SGG}$ in the graph convolution operation and recommendation objective function is the same as the vanilla GNN model. We present the main differences between the vanilla GNN and ${\rm SGG}$ models as follows:

• •

  Adjacency matrix normalization. In each training epoch, generating the target group embedding needs five corresponding subgraphs. Suppose the non-zero elements in the adjacency matrices of the full training graph and the five subgraphs are $2|\mathcal{E}|$, $2|\hat{\mathcal{E}}_{UU}|$, $2|\hat{\mathcal{E}}_{UI}|$, $2|\hat{\mathcal{E}}_{GG}|$, $2|\hat{\mathcal{E}}_{GU}|$ and $2|\hat{\mathcal{E}}_{GI}|$, respectively. Thus, the total complexity of adjacency matrix normalization is $\mathcal{O}((2|\hat{\mathcal{E}}_{UU}|+2|\hat{\mathcal{E}}_{UI}|+2|\hat{\mathcal{E}}_{GG}|+2|\hat{\mathcal{E}}_{GU}|+2|\hat{\mathcal{E}}_{GI}|)s+2|\mathcal{E}|)\approx 10|\hat{\mathcal{E}}|s+2|\mathcal{E}|$. ⁵⁵5Notation $|\hat{\mathcal{E}}|$ denotes the masked edge length $|\hat{\mathcal{E}}_{UU}|$, $|\hat{\mathcal{E}}_{UI}|$, $|\hat{\mathcal{E}}_{GG}|$, $|\hat{\mathcal{E}}_{GU}|$ and $|\hat{\mathcal{E}}_{GI}|$.

• •

  Self-supervised objective function. We evaluate the self-supervised tasks upon the masked subgraphs. For the user or item embedding reconstruction task, the time complexity is $\mathcal{O}(2d*(2|\hat{\mathcal{E}}_{UU}|+2|\hat{\mathcal{E}}_{UI}|)*s*L)\approx 8|\hat{\mathcal{E}}|Lds$. For the group embedding reconstruction task, the time complexity is $\mathcal{O}(2d*(2|\hat{\mathcal{E}}_{GG}|+2|\hat{\mathcal{E}}_{GU}|+2|\hat{\mathcal{E}}_{GI}|)*s*L)\approx 12|\hat{\mathcal{E}}|Lds$, where $2d$ represents the concatenated embedding size, as we incorporate the meta embedding to the graph convolution process. Thus the total time complexity of self-supervised loss is $8|\hat{\mathcal{E}}|Lds+12|\hat{\mathcal{E}}|Lds=20|\hat{\mathcal{E}}|Lds$.

We summarize the time complexity between the vanilla GNNs and ${\rm SGG}$ in Table 1, from which we observe that the time complexity of ${\rm SGG}$ is in the same magnitude with the vanilla GNNs, which is totally acceptable, since the increased time complexity of ${\rm SGG}$ is only from the self-supervised loss. The details are shown in Section 4.4.1.

We perform the ablation study to explore whether each component in ${\rm SGG}$ is reasonable for the good recommendation performance. To this end, we report the recommendation performance for ${\rm SGG}$ and its variant model in Table 4. We find that: (1) Basic-GNN and Meta-GNN is consistently superior than the vanilla GNNs, which indicates the effectiveness of the proposed SSL tasks. (2) Among all the variant models, Meta-GNN performs the best, which indicates enhancing the cold-start neighbors’ embedding quality is much more important. (3) GNN* performs the best, which verifies the superiority of combining these SSL tasks.

The goal of group recommendation is to recommend proper items to a group. Different from Shared-account recommendation [58, 59], where the members in the shared-account is closely related, the members in the group may formed at hoc. Existing methods for group recommendation can be classified into the following two categories: