Integrating Large Language Models into Recommendation via Mutual Augmentation and Adaptive Aggregation

For direct recommendation, the Bayesian Personalized Ranking (BPR) loss is commonly used to optimize the model (Rendle et al., 2012). The objective of BPR is to maximize the score difference between correctly recommended items and incorrectly recommended items, thereby improving the accuracy of recommendations. The BPR loss is defined as:

where $(u,i,j)$ refers to a triple of user-item pairs, and the user $u$ has interacted with item $i$ (positive item) and item $j$ (negative item). $\mathcal{D}$ represents the set of such user-item pairs in the training data. $\hat{y}_{ui}$ denotes the predicted score or preference of user $u$ for item $i$.

Inspired by this, we propose a data augmentation strategy where we randomly select pairs of items for a user $u$ and prompt the LLM to rank each pair based on the user’s likely preference. The ranking prediction based on LLM is then combined with the original data and used to train a direct recommendation model. Formally, let $(i_{j},i_{k})$ denote a pair of items for a user $u$. The LLM is prompted to rank these items, denoted as $i^{+},i^{-}=\textit{LLM}(\mathcal{P}_{1})$, where $\mathcal{P}_{1}$ is the corresponding prompt and $i^{+}$ is the item preferred over $i^{-}$. The training data $\mathcal{D}$ is updated as $\mathcal{D}^{\prime}=\mathcal{D}\cup(u,i^{+},i^{-})$. The BPR loss is then updated as:

This data augmentation strategy leverages the power of the instruction tuned LLM to enhance the performance of the direct recommendation model.

For sequential recommendation, the data augmentation strategy involves enriching the sequence of interacted items with additional items predicted by the LLM. Let’s consider a user $u$ with a corresponding sequence of interacted items $\{i_{1},...,i_{l}\}$. We randomly sample a list of un-interacted items $\{i_{u1},...,i_{uk}\}$, and adopt the prompt $\mathcal{P}_{2}$ to ask the LLM to predict the item most likely to be preferred by the user, denoted as $i_{p}=\textit{LLM}(\mathcal{P}_{2})$. This predicted item $i_{p}$ is then randomly inserted into the user’s sequence, resulting in an augmented sequence $\{i_{1},...,i_{p},...,i_{l}\}$. This augmented data is then used to train a more powerful sequential recommendation model.

By including additional items predicted by the LLM, we can enrich the sequence of items for each user, providing a more comprehensive representation of the user’s preferences. This, in turn, can enhance the performance of the conventional recommendation model, leading to more accurate recommendations.

In rating prediction tasks, we introduce the use of in-context learning (ICL) in LLMs to provide side information. This is primarily due to the fact that recommendation datasets may contain incomplete information. For instance, the popular Movielens dataset (Harper and Konstan, 2015) lacks information about the director of the movies, which can hinder the performance of a conventional rating prediction model. To mitigate this issue, we leverage the extensive world knowledge contained in an LLM. We prompt the LLM to provide side information, acting as additional attributes for users/items.

Formally, we denote the rating prediction model as $\mathcal{M}_{r}$, and the attribute set as $\mathcal{A}=\{a_{1},a_{2},...,a_{n}\}$, where $a_{i}\in\mathcal{A}$ denotes a distinct attribute. The model predicts the rating as $\mathtt{Pred}=\mathcal{M}_{r}(\mathcal{A})$. We then prompt the LLM to provide additional attributes, where the prompt $\mathcal{P}_{3}$ contains some corresponding examples followed by detailed instructions. The process is denoted as $\{a_{d1},a_{d2},...\}=\textit{LLM}(\mathcal{P}_{3})$. The augmented attribute set is then formed as $\mathcal{A}^{\prime}=\mathcal{A}\cup\{a_{d1},a_{d2},...\}$. The model then predicts the rating using the augmented attribute set, denoted as $\mathtt{Pred}^{\prime}=\mathcal{M}_{r}(\mathcal{A}^{\prime})$. This approach allows us to leverage the LLM’s world knowledge to enhance the performance of the rating prediction model.

To incorporate collaborative information within the prompt and facilitate LLM reasoning, we introduce a prompt augmentation strategy with similar user. Initially, we utilize a pre-trained conventional recommendation model to acquire embeddings for each user. These embeddings represent users in a latent space, which encapsulates their preferences and behaviors. Specifically, for a user $u$, in conjunction with a conventional recommendation model $\mathcal{M}_{c}$, we use $\mathcal{M}_{c}$ to obtain embeddings for each user, denoted as $\{e_{1},...,e_{n}\}$. These embeddings encapsulate the preferences and behaviors of the users, serving as a compact representation of the users in a latent space. We then calculate the similarity between these embeddings in the latent space. Various measurements, such as cosine similarity, Jaccard similarity, and Euclidean distance, could be employed in this context. In this paper, we calculate the cosine similarity to measure how closely two vectors align, denoted as:

where $e_{u}$ and $e_{v}$ are the embeddings of user $u$ and $v$, respectively, and $u,v\in\mathcal{U}$. The $||\cdot||$ denotes the Euclidean norm and $\cdot$ denotes the dot product. We identify the pair of users $(u,v)$ that have the highest similarity, indicating that they are the most similar in terms of their preferences and behaviors. We then use the items interacted with by the most similar user to enrich the prompt for the target user. This strategy leverages the collaborative information gleaned from similar users to generate more relevant and accurate prompts, thereby enhancing the recommendation performance of the LLM.

To enable the LLM to leverage information captured by conventional models, we propose a prompt augmentation method that incorporates information from conventional recommendation models. More specifically, the augmented prompt is formed by concatenating the original prompt with the prediction from the conventional recommendation model in natural language form. It’s important to note that the prediction from the conventional model varies depending on the recommendation scenarios and base models. Through augmenting prompts with predictions from conventional recommendation models, our method integrates collaborative or sequential information captured by these models, thereby enhancing the LLM’s contextual understanding and reasoning capabilities and resulting in better recommendation performance.

Notably, unlike ID-based methods such as (Zhang et al., 2023b; Zheng et al., 2023), our approach relies entirely on text, enabling easy adaptation to new situations. Also, the prompt augmentation could be used as a plug-and-play component for recommendation with closed source LLM, such as the GPT-4 model (OpenAI, 2023).

For the rating prediction task, we employ an instruction-tuned LLM to predict user-item utility scores directly. This approach incorporates the understanding of complex semantics and context by the LLM, which might be overlooked by traditional models. Similarly, conventional recommendation methods leverage collaborative information and user/item features for predicting the user rating. Specifically, the utility weight for user $u$, denoted as $U_{u}$, is directly set as its user rating. Subsequently, the LLM is engaged to predict the rating, symbolized as $U_{LLM}$. There are various methods to derive a final result based on the utility scores, such as training a neural network to process the utility scores from LLM and conventional models, yielding a final output via learning the complex reflection. However, for the sake of simplicity in this paper, we adopt a simple yet effective linear interpolation approach. The final utility score for a user $u$ amalgamates the values from both models, represented as:

where $\alpha_{u}$ is the adaptive parameter to control the weight for each model’s utility value for user $u$. We define the $\alpha_{u}$ as:

where $\ell_{max}$ and $\ell_{min}$ are the maximum and minimum long-tail coefficients of the users, respectively, $\alpha_{1}$ is a hyper-parameter that controls the weight, and $\alpha_{2}<1$ is a cut-off weight. From Equation (7), we can observe that for user $u$, the further they are positioned in the long tail (i.e., the fewer items they have interacted with), the lower is the value of $\ell_{u}$ and the higher is the value of $\alpha_{u}$. As a result, in Equation (6), the weight of the utility score from the LLM model becomes more pronounced. This aligns with the motivation we previously discussed.

For the top-$k$ recommendation task, the LLM is employed to re-rank the item list generated by a conventional recommendation model. Specifically, from conventional recommendation methods, we curate a top-ranked list comprising $k^{\prime}$ items, denoted as $\{i_{1},...,i_{k^{\prime}}\}$. Each item in this list is assigned a utility weight, ${U}^{i}_{Rec}=-s\cdot\mathcal{C}$, where $\mathcal{C}$ is a constant and $s$ represents the position of item $i$, i.e., $s\in\{1,...,k^{\prime}\}$. A higher utility weight indicates a stronger inclination of the user’s preference. For listwise comparison conducted by the LLM, the process begins by using the LLM to directly output the predicted order of these candidate items. Then we assign utility scores for items at each position, denoted as $U_{1},{U}_{2},...,U_{k^{\prime}}$, where $U_{1}\geq U_{2}\geq...\geq U_{k^{\prime}}$. The final utility score for an item amalgamates the values from both the original rating and the LLM prediction, similar to the Equation (6).

This section details the creation of an instruction-tuning dataset that encompasses two types of recommendation tasks catering to top-$k$ recommendation and rating prediction scenarios. A depiction of these two tasks, specifically referred to as listwise ranking and rating prediction, can be found in Figure 2. It is noteworthy that we also employ the LLM to execute pointwise ranking within top-$k$ recommendation scenarios, i.e., utilizing LLM to predict ratings for each item within the top-k recommendations and sorting the predicted ratings to derive the final result.

In this work, we perform full parameter instruction tuning to optimize LLMs using generated instruction data. Due to our need for customization, we chose LLaMA-2 (Touvron et al., 2023b), an open-source, high-performing LLM, which permits task-specific fine-tuning. During supervised fine-tuning, we apply a standard cross-entropy loss following Alpaca (Taori et al., 2023). The training set $\mathcal{D}_{ins}$ consists of instruction input-output pairs $(x,y)$, which have been represented in natural language. The objective is to fine-tune the pre-trained LLM by minimizing the cross-entropy loss, formalized as:

where $\Theta$ are the original parameters for LLM, $P_{\Theta}$ is the conditional probability, $|y|$ is the number of tokens in $y$, $y_{t}$ is the $t$-th token in the target output $y$, and $y_{[1:t-1]}$ represents tokens preceding $y_{t}$ in $y$. By minimizing this loss function, the model fine-tunes its parameters $\Theta$ to adapt to the specifics of the new instruction tuning dataset $\mathcal{D}_{ins}$, while leveraging the general language understanding and reasoning that has been acquired during pre-training (Zhang et al., 2023a). In this manner, LLM can capture the user’s preferences for items expressed in natural language, facilitating diverse recommendation tasks, including top-k recommendation and rating prediction.

Following (Bao et al., 2023), we rigorously evaluate the performance of our proposed framework by employing three heterogeneous, real-world datasets. MovieLens¹¹1https://grouplens.org/datasets/movielens/ (Harper and Konstan, 2015) serve as benchmark datasets in the realm of movie recommendation methods. We employ two variants of the dataset: MovieLens-100K (ML-100K) and MovieLens-1M (ML-1M). The former consists of approximately 100,000 user-item ratings, while the latter scales up to roughly 1,000,000 ratings. BookCrossing²²2Due to the absence of timestamp data, we synthesize historical interactions through random sampling. (Ziegler et al., 2005) includes user-generated book ratings on a scale of 1 to 10, alongside metadata such as ‘Book-Author’ and ‘Book-Title’. We employ LLM to augment the ‘director’ and ‘star’ features for ML-100K and ML-1M datasets, and augment the ‘genre’ and ‘page length’ features for the BookCrossing dataset. To ensure the data quality, we adopt the 5-core setting, i.e., we filter unpopular users and items with fewer than five interactions for the BookCrossing dataset. The key characteristics of these datasets are delineated in Table 1.

Aligning with (He et al., 2020; Sun et al., 2019), for the top-$k$ recommendation task, we turn to two well-established metrics: Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG), denoted by H and N, respectively. In our experiments, $k$ is configured to be either 3 or 5 for a comprehensive evaluation, similar to the experiment setting in (Zhang et al., 2023c). In accordance with (Fan et al., 2019), we employ Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) as evaluation metrics to ascertain the performance of the rating prediction task.

Following the methodology of prior works (Zhang et al., 2023c; Luo et al., 2022b), we adopt a leave-one-out evaluation strategy. More specifically, within each user’s interaction sequence, we choose the most recent item as the test instance. The item immediately preceding this serves as the validation instance, while all remaining interactions are used to constitute the training set. Moreover, regarding the instruction-tuning dataset construction, we randomly sampled 5K instructions for each recommendation task on the ML-100K, ML-1M, and BookCrossing datasets, respectively. We eliminated instructions that were repetitive or of low quality (identified by users with fewer than three interactions in their interaction history), leaving approximately 25K high-quality instructions. These instructions are mixed to create an instruction-tuning dataset to fine-tune the LLM.

We incorporate our Llama4Rec with the following recommendation models that are often used for various recommendation tasks as the backbone models:

• •

  Direct Recommendation. In the scenario of direct recommendation, We adopt four representative methods, including: MF (Koren et al., 2009), LightGCN (He et al., 2020), MixGCF (Huang et al., 2021), and SGL (Wu et al., 2021).

• •

  Sequential Recommendation. Regarding sequential recommendation, we opt for three widely used models, including: SASRec (Kang and McAuley, 2018), BERT4Rec (Sun et al., 2019), and CL4SRec (Xie et al., 2022).

• •

  Rating Prediction. We consider the following classical models for rating prediction, including: DeepFM (Guo et al., 2017), NFM (He et al., 2017), DCN (Wang et al., 2017), AFM (Xiao et al., 2017), xDeepFM (Lian et al., 2018), and AutoInt (Song et al., 2019).

We employ the LLaMA-2 7B version as the backbone LLM across all experiments, unless specifically mentioned otherwise. Our primary comparison is with the standard Instruction Fine-Tuning (IFT) method adopted in TALLRec (Bao et al., 2023) and InstructRec (Zhang et al., 2023c). For the rating prediction task, LLaMA-2 with IFT is used to directly predict the rating. For the top-k recommendation task, the tuned LLM is used to re-rank the list predicted by the backbone model, in accordance with (Hou et al., 2023), referred to as listwise ranking. Besides, we also adopt LLM for predicting the rating for each item and sort by the predicted scores, referred to as pointwise ranking.

During training for LLaMA 2 (7B) with full-parameter tuning, we use a uniform learning rate of $2\times 10^{-5}$ and a context length of 2048, and we set the batch size as 16. Additionally, we use a cosine scheduler for three epochs in total with a 50-step warm-up period. To efficiently train the computationally intensive models, we simultaneously employ DeepSpeed training with ZeRO-3 stage (Rajbhandari et al., 2020) and flash attention (Dao et al., 2022). We trained the 7B model on 16 NVIDIA A800 80GB GPUs. For the inference stage, we employed the vLLM framework (Kwon et al., 2023) with greedy decoding, setting the temperature to 0. Only one GPU was utilized during the inference phase. We only evaluate the instruction-tuned LLaMA model for rating prediction task since it is not applicable for directly making top-$k$ recommendations.

We implement the models for rating prediction task using the DeepCTR-Torch³³3https://github.com/shenweichen/DeepCTR-Torch library. For the top-$k$ recommendation task, we utilize the SELFRec⁴⁴4https://github.com/Coder-Yu/SELFRec library (Yu et al., 2023) for implementation. As for the hyper-parameter settings, $\alpha_{1}$ and $\alpha_{2}$ are selected from $\{$0.1, 0.3, 0.5, 0.7, 0.9$\}$ respectively for all experiments. $\mathcal{C}$ is fixed to 1. We repeat the experiment five times and calculate the average. We report the best results obtained when the ranking method is selected from pointwise and listwise ranking. For all experiments, the best results are highlighted in boldfaces. * indicates the statistical significance for $p\leq 0.05$ compared to the best baseline method based on the paired t-test. Improv. denotes the improvement of our method over the best baseline method.

We conducted an analysis of the effects of hyper-parameters $\alpha_{1}$ and $\alpha_{2}$. These parameters play crucial roles in controlling the weight in adaptive aggregation, as illustrated in Equation (7). Figure 3 presents the results on the ML-1M dataset using LightGCN as the backbone model. As $\alpha_{1}$ increases, we observe an initial surge in the model’s performance, followed by a decline. This trend suggests appropriate selection of $\alpha_{1}$ would enhance the model performance. With respect to $\alpha_{2}$, we observe a similar trend but the decline is more pronounced. This observation is consistent with the principle of adaptive aggregation, which emphasizes the importance of assigning suitable weights to tail users.

We further instruction-tuned the LLaMA-2 model with different model size.⁵⁵5Due to resource constraints, training the LLaMA-2 (70B) model with identical experimental settings was unfeasible, consistently leading to Out-Of-Memory (OOM) errors. A comparative analysis was conducted between the 7B and 13B variants of the instruction-tuned models, with performance differences specifically evaluated across various backbone models within the Bookcrossing dataset, as depicted in Figure 4. Our findings suggest that the LLaMA-2 (13B) model generally surpasses the 7B version in performance. This can be attributed to the superior language comprehension and reasoning abilities of the larger model, which contribute to improved recommendation results. However, it’s worth noting that the improvements are not substantial, indicating that while larger models may provide some performance benefits, the degree of improvement may not always justify the increased computational resources and training time required. It underscores the importance of considering the trade-off between model size, performance gain, and resource efficiency in the design and application of large language models.

We evaluated the effect of data size on LLM training by varying the number of instructions in the instruction-tuning dataset. Proportionality with our original configuration, the model with 2.5K instructions underwent 250 training steps, while the 12.5K instructions version was trained over 1250 steps. As depicted in Figure 5, a clear trend emerges: model performance improves with an increase in the number of instructions, particularly for direct recommendation models. This highlights the importance of utilizing larger and more diverse datasets for instruction tuning LLMs to optimize performance.