User-specific Adaptive Fine-tuning for Cross-domain Recommendations ^††thanks: Lei Chen and Fajie Yuan contribute equally.

In general, DL-based sequential recommendation (SR) models can be classified into three categories, namely RNN-based [1, 23], CNN-based [24, 2, 19] and self-attention [4, 25] based methods. Specifically, Hidasi et al. [1] proposed the first DL-based SR model GRU4Rec by adapting RNN from language model in the NLP domain. Following this work, many extended RNN variants were proposed, which either optimized a new ranking loss [26], incorporated more context features [27], or developed more advanced data augmentation [23]. While effective, these models rely heavily on the hidden states of the entire past, which cannot take full advantage of the parallel processing resources (e.g., GPU and TPU) [2] during training. Therefore, CNN and self-attention based models are proposed to mitigate such limitations, becoming more popular in recent literature [24, 2, 4, 25]. Among them, Tang et al. [24] proposed Caser, which embeds a sequence of user-item interactions into an “image” and learn sequential patterns as local features of the image by using wide convolutional filters. Subsequently, [2] proposed NextItNet, a deep temporal CNN-based recommendation model which particularly excels at modeling long-range item sequences. Later, GRec [19] improved NextItNet by considering two directional context information. CpRec [20] largely compressed NextItNet without hurting its recommendation accuracy. StackRec [18] and SkipRec [22] showed that NextItNet-style SR models could be stacked up to 100 layers for achieving its optimal accuracy, which distinguished from exiting work that usually applied less than 10 layers for evaluation. They also presented several methodologies to accelerate the training and inference processes of NextItNet. More recently, PeterRec [11] and Conure [12] demonstrated that the learned user representations by NextItNet were generic and could be transferred to solve various downstream recommendation problems. Meanwhile, self-attention based models, such as SASRec [4] and BERT4Rec [25], also showed competitive performance for the SRS task. However, compared with NextItNet with linear complexity, these models often require quadratic time complexity to compute the self-attention matrix. Thereby, in this paper, we plan to use the well-known temporal CNN model (i.e., NextItNet) as the backbone network, given that the performance of which has been well evaluated by PeterRec [11] and Conure [12] for the CDR tasks [28].

While deep learning (DL) based models have shown impressive performance in providing personalized recommendations, they often suffer from the cold-start problem, when new users enroll in a system but have a few or no labeled training data [11]. To deal with the cold-user problem, cross-domain recommendations (CDR) were proposed and proved effective. CDR transfer user preference between domains based on similarity of users and items that occur in both domains [29]. CDR are particularly useful for recommendations of users who are cold in a target domain but have rich interactions from a source domain. According to existing literature, CDR methods can be broadly divided into two types: joint learning based CDR, referred to as JLCDR, and two-stage (pre-training + fine-tuning) transfer learning based CDR, referred to as TLCDR.

In terms of JLCDR,  [30] proposed a DL-based domain adaptation approach, domain separation network [31], that solved the recommendation of cold-start users by jointly learning a stacked denoising autoencoder. Similarly, [32] explored an autoencoder and adversarial training to extract abstract rating patterns that were shared for same users across domains. CoNet [33] is another representative cross-domain recommendation model using deep neural networks as the base model. To enable effective knowledge transfer, CoNet introduced cross connections from the source network to the target and jointly trained objective functions of them.

Despite its effectiveness, JLCDR approaches are not efficient during training since the joint learning (e.g., multi-task learning) scheme is usually very costly. In particular, to optimize the objective of the source domain is computationally very expensive as training data of it is often at a large-scale [8]. By contrast, TLCDR does not need to train the learner from scratch for target task. Besides, compared with JLCDR, TLCDR can obtain better prediction accuracy [12]. This is because the joint learning scheme of JLCDR that trades off many objective functions usually does not guarantee the optimal performance for the target objective function [11].

However, existing literature of TLCDR concentrated mainly on pre-training, i.e., developing a more expressive or efficient base model [34, 35, 8]. Closely related to this paper, a recent work called PeterRec [11] firstly evidenced that the pre-training model based on self-supervised learning on user sequential behaviors largely improved the fine-tuning accuracy for the target task (i.e., CDR in our case). Inspired by meta learning [36], PeterRec injected a small-scale adapter network into the pre-trained model and optimized only the adapter during fine-tuning. The authors showed that PeterRec performed significantly better than fine-tuning the entire model when training data in the target domain is scarce. In this paper, we argue that both PeterRec and the typical fine-tuning strategies achieve only sub-optimal accuracy since they perform fine-tuning globally for all users, ignoring the individual difference of users in target tasks. To this end, we present an adaptive fine-tuning neural network architecture on a per-user basis. In other words, our proposed fine-tuning method is personalized — each user is assigned to a different fine-tuning policy. To the best of our knowledge, UAF is the first attempt to the CDR task using a personalized fine-tuning strategy.

This paper can be regarded as an extension of our previous conference paper published in AAAI 2021 [22]. In [22], we propose a user-adaptive layer selection framework by learning to skip inactive hidden layers on a per-user basis to address the inference problem for traditional sequential recommender systems with super deep layers. In this paper, we target at cross-domain recommendations, which is a completely different research topic. We propose a user-specific adaptive fine-tuning method which tailors fine-tuning for effective cross-domain recommendations. In addition, we add a new UAF-Soft method by employing the weighted combination of freezed pre-tained layers and fine-tuned layers, which can preserve pre-trained information and receive valid fine-tuned information simultaneously.

Suppose that there are two domains: a source domain $\mathcal{S}$ with a large number of user interactions and a target domain $\mathcal{T}$ with limited user interactions (i.e.. the cold-user scenario). The input data takes the form of implicit feedback such as click logs and watching records. Let $\mathcal{U}$ be the set of users shared in both domains. Each instance in $\mathcal{S}$ contains a userID $u\in\mathcal{U}$, and the user interaction sequence $x^{u}=\left\{x_{1}^{u},\ldots,x_{n}^{u}\right\}\in\mathcal{X}$, where $x_{t}^{u}$ denotes the $t$-th interaction of $u$ and $\mathcal{X}$ (of size $\mathcal{|X|}$) is the set of items in source domain $\mathcal{S}$. Similarly, each instance in $\mathcal{T}$ consists of a userID $u$ and item (label) $y\in\mathcal{Y}$, i.e., $(u,y)\in\mathcal{T}$, where $\mathcal{Y}$ (of size $\mathcal{|Y|}$) is the set of items in target domain $\mathcal{T}$.

Our objective is to recommend a set of items (a.k.a, the top-$N$ item recommendation) in the target domain to the user $u$ who has limited interactions but possesses sufficient sequential behaviors in the source domain.

Figure 2 illustrates the overall architecture of the proposed UAF method. The training procedure of UAF consists of two stages: pre-training and fine-tuning. In the first stage, we pre-train the base network (NextItNet) on the source dataset with numerous user-item interactions. The pre-trained model is supposed to learn general user representations which are reusable for same users in the target dataset. Then, we propose a user-specific adaptive fine-tuning strategy to automatically select reusable layers from the pre-trained model on a per-user basis.

To derive the optimal fine-tuning strategy given the input user sequence, we develop a policy network to output a binary policy vector, representing the actions to freeze or fine-tune each residual block in the pre-trained NextItNet. The policy network is implemented using a lightweight NextItNet model with dilations of {1, 2, 4, 8} (4 layers or 2 residual blocks). Without any restrictions, the policy network can also be implemented with other deep neural networks, e.g., a RNN model. In this paper, we propose three alternative strategies (i.e., hard, soft and RL) to optimize the policy network.

To evaluate the effectiveness of UAF, we collect four transfer learning datasets from Tencent²²2https://www.tencent.com/en-us/. Each of them is either from different recommender systems or has different properties. Among them, two datasets have been made publicly available in [11]. The detailed properties of these datasets are described as follows.

We compare our models (UAF-Hard, UAF-Soft and UAF-RL) with several powerful approaches. All baseline models share the same NextItNet architecture with ours.

• •

  Finetune-Zero: It is only trained on the target dataset without performing pre-training on the source dataset. That is, its parameters are randomly initialized.

• •

  Finetune-CLS: It is pre-trained on the source dataset. We merely fine-tune the softmax layer of the model on the target dataset.

• •

  Finetune-Last-$N$ ($N=$1, 2): We pre-train the model on the source dataset and fine-tune the last $N$ residual blocks of the pre-trained network (with the softmax layer) on the target data following [45].

• •

  Finetune-All: We pre-train the model on the source dataset and fine-tune all parameters on the target dataset. Finetune-All is the most widely adopted fine-tuning approach, and usually performs better than Finetune-CLS and Finetune-Last-$N$ as the entire model is tuned [11].

• •

  MTL: We present a widely used multi-task learning (referred to as MTL) baseline by hard parameter sharing [46], similar to that used in [12]. MTL jointly learns the objective functions of both source and target domains rather than using the two-stage pre-training and fine-tuning framework.

• •

  PeterRec: This is a recently proposed transfer learning framework which can be used to solve the cold-user problem [11]. The basic idea is to fine-tune a small-sized adapter neural network inserted in the pre-trained network. We evaluate PeterRec by using the official code⁶⁶6https://github.com/fajieyuan/sigir2020_peterrec under our setting.

Our evaluation follows [11] by adopting two popular top-$N$ metrics to measure the quality of recommended items, including MRR@$N$ (Mean Reciprocal Rank) [1] and HR@$N$ (Hit Ratio) [47]. Here $N$ is set to 5 for comparison.

We randomly split each target dataset into training (80%), validation (5%) and test (15%) sets. The grid search algorithm is applied on the validation sets to tune the hyperparameters. The size of each item embedding is set to be 128 for all the models. The NextItNet architecture is implemented by using 16 dilation layers or 8 residual blocks (i.e., {1, 2, 4, 8, 1, 2, 4, 8, 1, 2, 4, 8, 1, 2, 4, 8}), which is used as the backbone of the pre-training and fine-tuning models. We employ Adam optimizer to train our models with learning rate $\eta$ of 0.0001. Batch size $b$ and kernel size are set to be 256 and 3, respectively. Regarding the policy network, we use dilations of {1, 2, 4, 8} (4 layers or 2 residual blocks), which is a lightweight neural network. Note that, as mentioned before, without any restrictions, the policy network can also be implemented with other deep neural networks, e.g., a RNN model. All the experiments are implemented in TensorFlow and trained on a single TITAN RTX GPU.

Table II reports the overall results of UAF and baselines on the four datasets. We can make the following observations. First, Finetune-Zero performs worst among all baselines on ColdRec-1/2/4 in terms of both MRR@5 and HR@5. In particularly, by comparing Finetune-Zero with Finetune-All on the three datasets, we can conclude that pre-training on the source domain is of great help to achieve better recommendation accuracy. On the other hand, Finetune-All obtains only 1% improvement over Finetune-Zero on ColdRec-3. The result suggests that the correlation between the source and target domains on ColdRec-3 may be not as significant as ColdRec-1/2/4. The observation is reasonable since advertisement recommendation may have some discrepancy with news recommendation in terms of user preference. Second, it is shown that Finetune-Last2 performs better than Finetune-Last1, which further performs better than Finetune-CLS. Our observations here are consistent with those in prior work. Third, MTL performs better than Finetune-Zero except on ColdRec-3, indicating the effectiveness of joint learning. Furthermore, Finetune-All outperforms MTL on ColdRec-1/2/3 but slightly underperforms it on ColdRec-4. We argue that the optimal parameters learned for two or more objectives in MTL does not guarantee the optimal performance on the objective of the target domain. By constrast, Finetune-All only cares about the objective on the target domain and thus performs generally better. Fourth, PeterRec achieves state-of-the-art results on ColdRec-1/2/4, but shows worse results on ColdRec-3. The result further indicates that the four datasets have different characteristics in terms of knowledge transfer: some datasets benefit a lot from pre-training, while others may not. Again, pre-training on ColdRec-3 is not as important as on other datasets, and as a result, PeterRec exhibits much worse performance because all pre-trained parameters in PeterRec are not allowed to be modified.

Meanwhile, we observe that UAF, including UAF-Hard, UAF-Soft and UAF-RL, performs obviously the best on ColdRec-2/3/4 and on par with PeterRec and FineAll on ColdRec-1. In fact, we further find that UAF performs significantly better than all other baselines even on ColdRec-1 with limited training data, as shown in Section 5.3. We believe the key advantage of UAF mainly comes from its adaptive fine-tuning mechanism since these models are based on the same backbone network. By accomplishing the goal of per user per fine-tuning policy, UAF achieves more intelligent transfer learning across domains.

To answer RQ2, we explore the following ablation studies to verify the effects of the policy network in UAF. Note that we may only report partial results for clarity if similar behaviors are observed.

Transfer learning is supposed to reduce the cost of labeling data. In this subsection, we investigate the effectiveness of UAF with limited training samples. Specifically, we randomly choose 10% of the training examples in the target domain to fine-tune our models and the baselines. Table V summarizes the results of all models on ColdRec-1 and ColdRec-2. As shown, our methods (including UAF-Hard, UAF-Soft and UAF-RL) achieve notably better recommendation results than the Finetune-All and PeterRec. For example, the UAF-Hard method increases the MRR@5 metric by up to 6.1% on ColdRec-1, compared with PeterRec. Therefore, we conclude that the proposed UAF methods could be more powerful than these baselines when dealing with limited training data.

It is well-known that deep models are prone to overfitting when training data is insufficient. We here investigate the effectiveness of the proposed UAF methods against overfitting. We depict the learning curves of different methods in Figure 4. As shown, we observe that UAF-Hard, UAF-Soft and UAF-RL prevent overfitting much better than the Finetune-All. For example, on ColdRec-1, the HR@5 of the standard fine-tuning (Finetune-All) starts to decrease sharply after 5 epochs, whereas UAF-Hard, UAF-Soft and UAF-RL keep improving until 9 epochs, achieving notably higher HR@5. These results show that our methods can prevent overfitting better than many other models since UAF utilizes both pre-trained and fine-tuned parameters, where pre-trained parameters will not suffer from the overfitting issue.

User profiles are important features for generating accurate recommendations. In [11], authors showed the transfer learning framework such as PeterRec and FineAll can also be used for predicting user profiles, which are of great help to lessen cold-start user problem in recommendation tasks. In this subsection, we investigate the versatility of UAF in the user profile prediction task.

Following [11], we predict three types of user profile information (i.e., gender, age and life status). The statistics of the three user profile datasets are summarized in Table VI. Specifically, each instance in GenEst is a user and his gender (male or female) label obtained by the registration information. Similar to GenEst, each instance in AgeEst is a user and his age bracket label - one class represents 10 years. And each instance in LifeEst is a user and his life status label, e.g., single, married, pregnancy or parenting.

The results regarding the user profile prediction tasks are shown in Table VII. We use classification accuracy (referred to as Accuracy) and F1 score (referred to as F1) as the evaluation metrics. Similar conclusions can be made as discussed in Subsection 5.1. UAF with adaptive fine-tuning policies yields better results than baselines in all three datasets, which demonstrates the versatility of UAF while adapting it to different downstream tasks.