FedCL: Federated Contrastive Learning for Privacy-Preserving Recommendation

Contrastive learning is a core technique in many information retrieval (IR) tasks such as web search Xiong et al. (2020) and personalized recommendation Wei et al. (2021). A core problem of contrastive IR model training is selecting proper negative samples to contrast with the positive ones Robinson et al. (2020). One popular way is to randomly sample negatives from the candidate set. For example, the Bayesian Personalized Ranking (BPR) Rendle et al. (2009) method uses a randomly selected negative sample to compare with a positive sample. Many other works explore learning IR models by optimizing the gaps between the positive sample and a certain number of random negative samples Shen et al. (2014); Wu et al. (2019). Another widely used way is in-batch negative sampling. For example, Yi et al. Yi et al. (2019) proposed to use the positive items within the same batch as shared negatives for all queries in this batch. However, random or in-batch negative samples can be somewhat too easy for model training Robinson et al. (2020). To exploit more representative and informative negative samples for discriminative model training, some methods explore to incorporate hard negative samples into model learning Kalantidis et al. (2020). For example, Ding et al. Ding et al. (2019) proposed to use reinforcement learning to train a sampler model to select negative samples for contrastive recommendation model training. Xiong et al. Xiong et al. (2020) proposed to use an asynchronously updated ANN index to select globally hardest negative samples from the candidate document pool for learning the matching model. These methods usually require centralized data storage for negative sampling. However, under privacy-preserving settings there is no centralized data and the local data on user devices can be too sparse to obtain effective negative samples, making it difficult to apply existing contrastive IR model learning methods. Thus, in our work, we propose a federated contrastive learning method for recommendation, which can incorporate various kinds of negative samples into contrastive model training in a privacy-preserving way.

In recent years, the application of federated learning in privacy-preserving recommendation has been extensively studied Chai et al. (2020); Qi et al. (2020); Liang et al. (2021). For example, Ammad et al. Ammad-Ud-Din et al. (2019) proposed to locally learn user and item embeddings on local user data, and upload item embedding gradients to a server for global model updating and delivery. Lin et al. Lin et al. (2020) proposed to randomly sample some unrated items and assign them virtual ratings to avoid communicating the full item embedding table. Qi et al. Qi et al. (2021) proposed a unified method for privacy-preserving news recall and ranking by modeling user interests with linear combinations of basis interest vectors. However, these methods usually assume that there are sufficient negative samples available on local user devices. Unfortunately, in many scenarios such as product recommendation and web search, non-interacted items are typically not locally recorded due to efficiency concerns. It may also be suboptimal to simply use random negative items provided by the server without considering clients’ characteristics. One recent work Ning et al. (2021) studies learning recommendation models under limited locally available negative samples. They proposed to use a batch-insensitive loss that enables using local positive items only to construct training samples. However, useful negative signals cannot be considered by this method. By contrast, our proposed method can effectively exploit various kinds of negative samples under privacy protection to improve contrastive recommendation model learning.

In federated learning, there is no global dataset for hard negative sampling. In addition, since local user data is highly sparse and biased, it is also impractical to conduct hard negative sampling locally. To solve this problem, we propose a federated negative sampling method to choose hard negatives in a privacy-preserving manner. Assume that there are $N$ clients in total. Each client first uses its maintained local user model to infer a user embedding from the local user profile. The user model can be a simple user ID embedding, or a more complicated one that learns user embeddings from the sequence of historical interacted items. We denote the embedding of user $u_{i}$ as $\mathbf{u}_{i}$. Since this user embedding still contains much private user information, we use local differential privacy (LDP) to perturb this embedding. Motivated by Qi et al. (2020), we first clip each user embedding with a threshold $\delta$ according to its $L_{1}$ norm, and add Laplace noise $\mathbf{n}\sim Laplace(0,\frac{2\delta}{\epsilon})$ to protect this embedding ($\epsilon$ is the privacy budget, and a lower $\epsilon$ means better privacy protection). Then the protected user embeddings satisfy $\epsilon$-DP. We refer to the protected user embedding of $u_{i}$ as $\mathbf{\hat{u}}_{i}$.

The protected user embeddings on user clients are uploaded to the server for negative sampling. A simple way is directly measuring the similarity between candidate items and these embeddings to find negative samples. However, these embeddings contain heavy noise if the privacy budget is relatively small, and the selected negative samples are usually inaccurate, as shown in the middle plot in Fig. 2. To mitigate the influence of noise on negative sampling, we propose to cluster the user embeddings and use their cluster centroids to represent users with different types of interests. More specifically, we use the Ward clustering algorithm to aggregate the $N$ user embeddings into $C$ clusters. The centroid of each cluster is the element-wise average of user embeddings in this cluster. We use the cluster centroid to replace the user embeddings in this cluster to conduct hard negative sampling (right plot in Fig. 2). Since the noise added to the user embeddings is zero-mean, the errors brought by the noise can be mitigated to some extent by using cluster centroids. We denote the cluster centroid corresponding to the user embedding $\mathbf{u}_{i}$ as $\mathbf{c}_{i}$.

We evaluate the relevance between $\mathbf{c}_{i}$ and each item embedding in the candidate pool on the server to choose negative samples. A simple way is to pick the globally hardest negative samples Xiong et al. (2020). However, there may be many false negatives because the candidate items in the entire pool have not necessarily been displayed to the users. Thus, in our approach we propose to choose semi-hard negative samples to balance their difficulty and accuracy. For each centroid vector $\mathbf{c}_{i}$, we first retrieve $R$% of candidate samples with the highest difficulties to form a hard candidate subset (a smaller $R$ means harder negatives). We then randomly select $T$ negative samples from this subset as semi-hard negative samples. Note that for clients that are associated with the same cluster centroid, the sampling of negative items from the hard candidate subset is independent, which aims to ensure the comprehensiveness of negative samples. The semi-hard negative samples drawn from $\mathbf{c}_{i}$ are denoted as $[h_{1}^{i},h_{2}^{i},...,h_{T}^{i}]$, which are further distributed to the $i$-th client for local model training. In this way, the user clients can exploit the hard negative samples obtained from the global candidate pool in a privacy-preserving way.

Next, we introduce the proposed negative mix-up method for local contrastive model learning. Motivated by the in-batch negative sampling in prior work Yi et al. (2019), we propose to incorporate in-batch negatives by regarding the local positive items as a mini-batch, where the samples within this batch can be used as negatives for each other. We denote the local positive samples on the client $i$ as $[p_{1}^{i},p_{2}^{i},...,p_{P}^{i}]$. For the positive item $p_{j}^{i}$, its corresponding in-batch negatives are $[p_{1}^{i},...,p_{j-1}^{i},p_{j+1}...,p_{P}^{i}]$. Note that this kind of in-batch negative samples cannot be used for training ID-based user modeling methods because they cannot learn different user embeddings based on the item interaction histories. In addition, in real-world scenarios there can be a small number of negative items stored on user devices or randomly distributed by the server. To help our approach be compatible with this setting, we also incorporate the local negative items into model training, which are denoted as $[n_{1}^{i},n_{2}^{i},...,n_{K}^{i}]$. Given each positive item $p_{j}^{i}$, we mix up the in-batch negatives, local negatives, and semi-hard negatives into a unified list of negative samples, and we denote the list of their embeddings as $\mathcal{N}_{j}^{i}$. The embedding of the positive item $p_{j}^{i}$ is denoted as $\mathbf{p}_{j}^{i}$ The local contrastive training loss function $\mathcal{L}_{i}$ of the $i$-th client is formulated as follows:

where $\mathbf{u}^{i}_{j}$ represents the user embedding for matching with the positive item $p_{j}^{i}$ (inferred by the user model on client $i$).

Finally, we briefly introduce the federated model update process in FedCL. After computing the loss function $\mathcal{L}_{i}$, each client locally optimizes this loss to update the maintained local recommendation model. The local model updates from a certain number of clients are securely uploaded Chai et al. (2020) to the server for aggregation, and the aggregated model updates are used to update the global model maintained by the server. The server further distributes the global model to each client to conduct local update. The cascaded process of federated negative sampling, local negative mix-up for contrastive training, and federated model update is repeated for multiple rounds until the model converges.

We conduct experiments on four widely used benchmark datasets for recommendation. Two of them are taken from the Amazon dataset dump McAuley et al. (2015). We use the “Beauty” and “Sports and Outdoors” (shorten as “Sports”) domains for experiments. The third dataset is collected on Yelp. Following the settings in Zhou et al. (2020), we use the transaction records after January 1st, 2019. The last dataset is MovieLens 1M (denoted as ML-1M) Harper and Konstan (2015), which is also a canonical recommendation dataset. The statistics of the four datasets are listed in Table 1. For each user, the last interacted item is used for test, and the item before the last one is used for validation.

We use the Adam Kingma and Ba (2015) optimizer for model training. The learning rate is 1e-3, and the number of selected users per update is 16. The hidden dimension is 64. The user embedding norm clip threshold is 1. The privacy budget $\epsilon$ is 4. The number of clusters is 25. To simulate local negative items, we randomly sample 100 items on each client, and for each positive we randomly choose 10 local negatives to participate in contrastive learning. The difficulty $R$% of negative samples is set to 25%. The number of semi-hard negatives is 20. These hyperparameters are tuned on the validation sets. Motivated by Krichene and Rendle (2020), we evaluate the recommendation performance on the whole item set rather than the sampled subsets used in Sun et al. (2019). We use the Hit Ratio (HR) and normalized Discounted Cumulative Gain (nDCG) metrics over the top 5 or 10 ranked items. We repeat each experiment 5 times and report the average scores. We will soon release our code for reproducibility.

To verify the effectiveness and generality of FedCL, we compare the results of several widely used methods, including NCF He et al. (2017), GRU4Rec Hidasi et al. (2016) and BERT4Rec Sun et al. (2019), under four different settings as follows: (a) centralized learning, which uses the original data sampling and model learning methods based on centralized data; (b) federated learning (Fed), which directly applies the federated learning framework to the basic model (local negatives are used); (c) federated learning with batch-insensitive loss (BIL) Ning et al. (2021), which uses batch-insensitive loss to learn models on local positive items only; (d) federated contrastive learning (FedCL), our proposed method. The results on the four datasets are shown in Tables 2 and 3. We find that compared with centralized learning, the performance of models learned in the standard federated framework usually decreases. This is mainly due to the non-IID property of user data decentralized on different clients and the scarcity of informative negative samples. In addition, the BIL method outperforms the federated baseline. This is because BIL learns uniform item representations via a spreadout regularization instead of learning from the limited local negative samples. Moreover, our proposed FedCL method outperforms other methods (t-test results show the improvement is significant, $p<0.001$), and it can consistently boost the performance of different base models. This is because our approach can incorporate various kinds of negative samples into contrastive model training to exploit richer negative signals.

Next, we study the impacts of different types of negative samples in our approach. We compare the model performance with one kind of negative samples removed, and the performance of replacing semi-hard negatives with globally hardest negatives Xiong et al. (2020). The results on the Yelp and ML-1M datasets are shown in Fig. 3. We find that the hard negatives obtained by the federated negative sampling method have a major contribution to model performance. However, the performance has a huge decline if they are replaced by globally hardest samples. This is mainly because globally hardest samples may contain many false negatives, which are harmful to model training. In addition, both in-batch and local negatives have contributions, but the performance gain brought by the local negatives is not so large. Thus, the performance of FedCL can still be promising if there is no local negative sample available on user devices.

We further study the influence of clustering on the model performance. The results without clustering, with Kmeans clustering, or with Ward clustering are compared in Fig. 4. We observe that the performance is suboptimal when the user embeddings are directly used for hard negative sampling. This is intuitive because the perturbed user embeddings are quite noisy. In addition, both Kmeans and Ward clustering algorithms can improve the model performance. This may be because using the centroids of clusters to replace original user embeddings can mitigate the influence of noise. In addition, the Ward clustering method is better than Kmeans. This may be because the hierarchical clustering method is stronger in modeling the relations between similar user interests.

We then analyze the impacts of the number of clusters on model performance, as shown in Fig. 5. The results reveal that when the number of clusters is too small, the performance is suboptimal. This may be because user interests cannot be effectively distinguished when there are too few clusters. However, the performance starts to decrease when the number of clusters is larger than 25. This may be because the influence of noise encoded in user embeddings becomes large. Thus, we set the number of clusters $C$ to 25.

Finally, we analyze the influence of the privacy budget of user embeddings for hard negative sampling on the model performance. We change the intensity of Laplace noise according to different privacy budgets, and the corresponding results are shown in Fig. 6. We find that the performance usually decreases when a smaller privacy budget $\epsilon$ is used. This is intuitive because the noise is stronger when $\epsilon$ is smaller, and the obtained negative samples are more inaccurate. In our approach, we choose $\epsilon=4$ that can yield a good performance and satisfactory user privacy protection ability.

In the appendix, we analyze the impact of the negative sample difficulty ($R$%) as well as the number of local and semi-hard negative samples. The results show that selecting negatives from the top 25% relevant candidates and using a medium number of negative samples is suitable.