Online Item Cold-Start Recommendation with Popularity-Aware Meta-Learning

$(u,i)\to\boldsymbol{e}_{u},\boldsymbol{e}_{i}$: The purpose of the embedding layer is to characterize the different information about the user and the item, convert it into a vector, and output it to the hidden layer for information extraction. Specifically, the embedding layer consists of multiple feature embedding matrices of users and items. For user $u$, the system maintains multiple one-hot vectors for different features, which extract the user’s information from the feature embedding matrix and then concatenate it into the embedding vector:

where $P$ is the number of user features, $\boldsymbol{E}_{Up}\in\Phi_{t}$ represents the $p$-th feature embedding matrix and $\boldsymbol{c}_{up}$ stands for the one-hot vector for user feature $p$. Items have similar embedding outputs as users:

$\boldsymbol{e}_{u},\boldsymbol{e}_{i}\to\boldsymbol{z}_{u},\boldsymbol{z}_{i}$: After the embedding vectors are generated, they are fed into the hidden layer and ultimately output the top representations of the user and item. In our dual-tower model, the user and the item have their own fully connected layer parameters, and the embedding vector will be mapped to the top representation by $L$ fully connected network layers $\boldsymbol{z}_{u}=f_{uL}\circ\cdots\circ f_{u1}(\boldsymbol{e}_{u})$ and $\boldsymbol{z}_{i}=f_{iL}\circ\cdots\circ f_{i1}(\boldsymbol{e}_{i})$, where the $l$-th layer can be represented as:

where $\boldsymbol{W}_{l},\boldsymbol{b}_{l}\in\Omega_{t}$ are the weight matrix and bias vector of the $l$-th hidden layer, and $\sigma$ represents a ReLU (Agfarap, 2019) activation function.

$\boldsymbol{z}_{u},\boldsymbol{z}_{i}\to\hat{y}_{ui}$: The output layer calculates the predicted scores for the corresponding user-item pairs based on the top-level representations of the users and items. We utilize the InfoNCE (van den Oord et al., 2019) loss form to calculate the prediction and loss value. The prediction of interaction $\hat{y_{ui}}$ is calculated by sampling other interactions in the batch as negative samples:

where $\tau$ is the temperature hyperparameter in InfoNCE loss to control the model’s discrimination for negative samples. The log loss form is adopted to calculate the gradients:

where $\hat{y_{ui}}$ and $y_{ui}$ refer to the prediction and label values of $(u,i)$.

Specifically, for a designated number of tasks $N$ and an interaction $(u,i)$, the number of tasks to which this interaction belongs is calculated by a piece-wise constant function:

where $T_{n}$ represents the $n$-th task, and $v_{i}$ is a metric reflects popularity, such as the number of click or sales volume. The function $f$ is defined by a set of pre-determined thresholds.

The parameters in PAM follow a bi-level optimization scheme, consisting of local updates and global updates. Before the training process starts, we initialize the global recommender parameters $\boldsymbol{E}_{Up},\boldsymbol{E}_{Iq}\in\Phi_{t}$, $\boldsymbol{W}_{l},\boldsymbol{b}_{l}\in\Theta_{t}$ . After the arrival of a data batch $D_{t}$, the batch will be divided into multiple parts $\{D_{t}^{1},\cdots,D_{t}^{N}\}$ according to the popularity of the item by function $f$. Each portion of data $D_{t}^{n}$ is further divided into ${D_{t}^{n}}^{S}$ and ${D_{t}^{n}}^{Q}$, standing for support and query set. For the $n$-th task, the personalized recommender parameters $\Omega_{t}^{n}$ can be obtained by local updates:

where $\alpha$ represents the inner loop learning rate, and $\mathcal{L}_{T}(\Theta|D)$ represents the loss function of task $T$ on data set $D$ with an initialization of $\Theta$ (see Eq. (7)). The recommender $\{\Phi_{t},\Omega_{t}^{n}\}$ will be used to serve the items in task $n$ at the following time moment $t+1$. Note that the interaction data between the support set and the query set have no overlap, hence it is meaningless to update the embedding parameters of users and items in the local update, and the updated parameters contain only weight parameters of the networks in Eq. (9). We utilize the LSLR (Antoniou et al., 2019) methods, maintaining an adaptive learning rate for each network weight to alleviate overfitting and advance the performance.

The goal of the meta optimization is to minimize the sum of the losses of the query set ${D_{t}^{n}}^{Q}$ over its parameters $\Omega_{t}^{n}$ in each task $T_{n}$. Since all of the parameters are involved in the computation of the target function, the global update will be performed on all of the parameters:

where $\mathcal{L}^{M}_{t}$ is the main meta-learning loss, $\beta$ stands for the outer loop learning rate, and

where $\lambda$ represents the personalized weight of tasks. Because of the fixed division of the tasks, we can leverage the importance of the task and assign distinctive weights. For instance, the cold-start task, involving less popular items, can be assigned a higher weight.

By fixed tasks division as described above, we solve the problem of long-tailed distribution of training data by dividing the cold-started long-tailed part of the item distribution into separate tasks.

We categorize the various types of feature embedding of items into two types: behavior-based embeddings $\boldsymbol{e}_{q}^{beh}$ and content-based embeddings $\boldsymbol{e}_{q}^{con}$, because these types of embeddings play different roles in the recommendation of cold-start and popular items, as shown in Sec. 5.4. We then further divide the behavior-based embeddings into ID embedding and sequential embeddings, this is because that ID embedding directly stores the information of the items and we utilize it for the cold-start embedding simulation, the details of which will be described later. The system maintains a cold-start embedding parameter $\hat{\Phi}$, for each item $i$, whenever it appears in the data $D_{t-k}^{cold}$ as a cold-start item, the system stores its behavior-based embeddings (ID embedding and sequential embeddings) into the parameter $\hat{\Phi}_{t-k}$ in addition to updating its own embedding parameter $\Phi_{t-k}$. For each item $i$ that appears in the popular task $D_{t-k}^{hot}$, its behavior-based embeddings in the cold-start period have certainly been stored. Therefore, we can simulate a cold-started item at the current moment with the help of the stored embedding $\hat{\boldsymbol{e}}_{q}^{beh}\in\hat{\Phi}_{t-k}$ as well as other content-based feature embedding $\boldsymbol{e}_{q}^{con}\in\Phi_{t}$:

The equivalent write-up for dividing behavior-based embeddings $\hat{\boldsymbol{e}}_{q}^{beh}$ into ID embedding $\hat{\boldsymbol{e}}_{q}^{ID}$ and sequential embeddings $\hat{\boldsymbol{e}}_{q}^{seq}$ is:

and $\hat{\boldsymbol{e}}_{i}$ fully simulates a cold-start item embedding at current time.

With relatively low traffic proportion, the cold-start task in meta-learning still has limited training opportunity. Data argumentation can significant increase the number of cold-start samples, so we propose a simulation-based data argumentation. We treat the simulated cold-start embedding $\hat{\boldsymbol{e}}_{i}$ from the previous subsection as a new part of cold-start task data with the real popular interaction data labels $y_{ui}$, and train the meta-learning model to update the cold-start task parameters, which can partially augment the data to improve the effect.

We denote the simulated cold-start interaction data as $\hat{D}_{t}^{cold}$ and similarly divide them into support and query sets ${\hat{D}^{cold,S}_{t}}$, ${\hat{D}^{cold,Q}_{t}}$. Similar to Eq. (10), the loss function form of data augmentation is computed from the data in the support set:

where $\hat{\Omega}_{t}^{cold}$ is calculated by support sets ${\hat{D}^{cold,S}_{t}}$.

Furthermore, we propose leveraging the well-learned embeddings of popular items and achieve instruction for the cold-start task by training a mapping network to fit the transformation process of the cold-start embedding to the embedding from popular phase. For the simulated cold-start item embedding $\hat{\boldsymbol{e}}_{i}$, we have the ID embedding of its popular state $\boldsymbol{e}^{\mathrm{ID}}\in\Phi_{t}$, which has more information due to multiple updates. To this end, we reinforce the ability of the other fully connected layers to extract information by replacing the last fully connected layer of the cold-start task network parameters $\boldsymbol{W}_{L},\boldsymbol{b}_{L}$ with a unique mapping parameter $\boldsymbol{W}^{Sup},\boldsymbol{b}^{Sup}$ that makes the output top embedding as similar as possible to its true ID embedding:

where $\hat{\boldsymbol{z}}^{\mathrm{ID}}$ is the output embedding. We use MSE loss to evaluate the similarity between output top embedding and true ID embedding:

where $N$ stands for the dimension of embedding.

With the instruction of the popular ID embedding, we can implement a self-supervised learning-like method without the use of labels to make the cold-start task network parameters have better performance in extracting information.

With the introduction of the parts of the cold-start task enhancer, the update method of the recommender parameters in Eq. (10) changes. We form a new total loss function by combining the three components of main meta-learning loss, self-supervised learning loss, and data augmentation loss:

where $\gamma$ refers to the different weights of losses, and the new parameter update method becomes:

During the training process, the model is fine-tuned on the cold-start data to generate parameters for the cold-start task $\Omega_{t}^{cold}$. We save that parameter and serve it for the recommendation of cold-start items at the next moment. In this case, PAM is able to store the parameters prepared for the cold-start item in advance without real-time personalized fine-tuning, thus avoiding the time cost of online fine-tuning altogether. Compared to traditional meta-learning, we avoid the time-consuming serving of personalization on each item, and also reduce the storage space overhead of maintaining the respective parameters for each item.

The training and serving process of PAM is described by Alg. 1.

We selected three public datasets that containing timestamp to ensure that the characteristic of online streaming data is simulated, including MovieLens (Harper and Konstan, 2015), Yelp and Book (Hou et al., 2024). Details of datasets are shown in  A.

For the label processing of the datasets, the user ratings lie between 0 and 5, while we consider the interaction data with ratings above 3 as positive samples and the rest as negative samples. The information of datasets is conducted in Table 1.

We compare our proposed PAM with the following baseline, where all scenarios leverage the previously mentioned two-tower model structure and the feature inputs remain the same.

- Periodical Fine-tuning (PF) or incremental updating, is a frequently used type of update in online systems, which performs an update on recommender system using the incoming data periodically to serve the next moment.

- s²Meta (Du et al., 2019) employs a novel meta-learning task division approach that considers item scenarios as tasks to achieve better recommendation results for new scenarios.

- IncCTR (Wang et al., 2020) is a practical incremental method that leverages knowledge distillation for training, use the current knowledge stored in model to guide the retraining.

- SML (Zhang et al., 2020) proposes a meta-model that takes the previous moment’s recommender and current moment’s data as inputs and generate the next moment’s recommender for serving through a CNN-based network.

- ASMG (Peng et al., 2021) better captures long-term interests through historical recommenders to generate models for serving through a meta-model generator that introduces GRU.

- MeLON (Kim et al., 2022) generates more stable online recommendation models by maintaining dynamic learning rates for different parameters and distinguishing the importance of interaction data.

- IMSR (Wang and Shen, 2023) allows the traditional adaptive interest capture model to better capture new interests of users periodically by introducing unique interest shifters..

Similar to ASMG (Peng et al., 2021), we divide each dataset equally into 31 periods, and each period is further partitioned into batches for training. To simulate the characteristics of online streaming data, we do not use a pre-training scheme, and all models will be consistently streamed starting from the data of the first period. We start testing from the 24th period, and the models obtained from each training period will be tested on cold-start data from the next period. The average of the metrics from all the testing phases will be reported. For specific metrics, we use the Recall@K and NDCG@K (Wang et al., 2013) metrics to measure the model’s effectiveness on cold-start items. Since ranking the item’s scores for all users is time-consuming, for a cold-start item, we take the interacted user as 1 positive sample and all the non-interacted users in the current batch as the negative samples (not less than 900 due to the item’s low popularity), and compute the above metrics.

For the definition of the tasks, to simulate an online cold-start scenario, we designated the items with the lowest 5% of popularity in the interaction data as cold-start items (Ma et al., 2023; Dong et al., 2020). The cold-start thresholds of popularity $v_{cold}$ on the three datasets are 50, 20, and 15, respectively. We divide the remaining data into 4 tasks to distinguish popular items of different natures.

During the training process, we use Adam (Kingma and Ba, 2014) as the optimizer for gradient descent with $\alpha$ set to 0.001. The outer loop learning rate $\beta$ is set to 0.001. The batch size is 1024, the weights of tasks $\beta$ are set to 2 for the cold-start task and 0.5 for the other tasks, and the weights of the loss function $\gamma$ are set to 1, 3, and 2, respectively.

Table  2 demonstrates results of the top-K metrics on cold-start items for various methods. From the result of the experiment, we have the following observations.

Our proposed PAM significantly outperforms other baselines on cold-start items. This demonstrates the effectiveness of PAM’s unique meta-learning scheme targeted at solving long-tailed distributions of item popularity in online streaming data scenarios. The PAM performs better on smaller K-value metrics compared to other baseline methods, and the improvement is relatively small as the K-value increases. This suggests that the PAM method can accurately recommend cold-start items to the users who favor it the most. In contrast, the baseline method ranks the positive-sample users relatively far down the list, as reflected in the considerable rise in the metrics as the K-value rises.

SML and ASMG methods focus on extracting information from historical recommender systems and balancing the weights of historical information with real-time interests to optimize recommendations at the next moment. However, because data distribution tends to be popular videos, focusing on historical information only aggravates the interest bias of the model. Besides, this model relies more on the efficacy of the initialized model, which results in poorer performance on cold-start items in the streaming no pre-training scheme. At the same time, ASMG has a high storage space overhead (leads to OOM on the Yelp dataset), which becomes another limitation for its application in online systems. IMSR is also an approach that balances periodic retraining of historical with current information, and thus has a similarly reliance on the efficiency of pre-training, and performs poorly in schemes without pre-training. In addition, the emphasis on historical information extraction also leads to a worse performance on cold-start recommendations.

IncCTR, as a knowledge distillation scheme that utilizes the model’s self-supervised signals, also references historical information. Its low dependence on initialized parameters leads to a better fit to online scenarios and a slight improvement over PF, achieving state-of-the-art for cold-start items on some datasets. However, its way of generating additional labels increases the computational time consumption and partially reduces the model’s efficiency.

s²Meta is a scheme for task segmentation and meta-learning training of arriving data by categories of items. However, it only improves slightly relative to the PF baseline when no new item categories enter the system and performs slightly inferior on the content-information-rich MovieLens dataset, as the rest of the baseline can similarly achieve similar information extraction through content-based embedding.

MeLON performs superiorly on some recall metrics and worse on NDCG metrics. As a scheme that can adjust the learning rate in both directions on interactions and parameters to accommodate necessary samples, MeLON can better find the direction of gradient updates in less cold-start data and thus performs better in the MovieLens dataset. However, as the number of cold-start interactions increases in other datasets, its inability to find the commonality of cold-start items may bring inefficiency in updating.

Meanwhile, we compare the metrics of PAM on popular items, the results are showed in Table 3.

It can be seen that the performance of the other baseline methods on popular items has improved compared to the cold-start items, reflecting the shift of the parameters toward the popular items due to more feedback on popular items. Meanwhile, PAM does not perform as well on popular as on cold-start items, demonstrating that the preference of parameter settings for cold-start tasks successfully optimizes on cold-start items. Notably, PAM still outperforms some baselines on popular items, demonstrating the superior performance of popular task parameters detached from the cold-start data for recommending items in the corresponding task.

We evaluated the impact of different weights for the cold-start task in calculating the loss of meta-learning in Fig. 4(a). It can be seen that when the weight of the cold-start task is set in an appropriate range, it can effectively improve the performance on cold-start items. If the weight of the cold-start task is set too low, the loss on the cold-start parameter will take a minor percentage of the overall loss during the global update, thus making the parameter less specialized for the cold-start task. On the other hand, if the weights of the cold-start tasks are set too high, then as mentioned before, most of the information in the popular data will be lost, leaving the parameters shared between the tasks poorly updated, leading to the same inferior performance.

We similarly analyzed the effect of the loss weights of different modules on the effectiveness of PAM. It can be seen that for the data-enhanced modules in Fig. 4(b), the enhancement brought are more stable and the weights only have minor effect on the metric performance. Besides, the self-supervised guidance module, although performance stably, fluctuates slightly more than the data enhancement module, as shown in Fig. 4(c).