Improving Conversational Recommender System
via Contextual and Time-Aware Modeling with
Less Domain-Specific Knowledge

Conversational Recommender System (CRS) has attracted many researchers’ interest in recent years. Various task formulations with different hypotheses and application scenarios have been proposed. We summarize them into three categories and introduce them in detail as below.

Traditional recommender system is different from the conversational recommender system, as it generally does not focus on dialogue interactions with users. It makes more efforts on the information filtering techniques to handle information overflow problems in recent years.

The problems in the traditional recommender system can be divided into two categories [27]: recommendation with explicit feedback (i.e., the ratings given to products) and recommendation with implicit feedback (purchase history or click-through history) [28]. The former problem can be evaluated directly by the differences between ground truths and predictions while the evaluation of the latter problem is usually formulated as Top-N ranking problems.

The methods proposed in the conventional recommender system can be divided into three categories: collaborative filtering based recommendation [29], content-based recommendation [30] and the hybrid of them [31]. Collaborative filtering has been the most popular recommendation technique in recent years. It hypothesizes that people who have had similar interests before would also prefer similarly in the future [32]. Many methods are proposed to identify the similarity of users or items, including memory-based methods [33], clustering methods [34], Bayesian models [35] and matrix factorization methods [36]. However, collaborative filtering methods may perform poorly when handling new items that lack history information. As for content-based recommendation methods, they usually first compare the similarity between the descriptive characteristics of items and the user profiles, and then recommend items that are similar to what users like before. Therefore, the content-based methods are usually efficient in recommending new items. But they are limited in terms of diversity and novelty [37]. The hybrid of collaborative filtering and content-based methods take advantages of both of them [31].

The pre-trained language models [38, 39, 6, 40] become more and more popular in recent years. The idea behind them is first pre-training the models on large-scale unlabeled corpus (which can be easily crawled on the Internet) and then apply them to many downstream tasks (e.g., QA system, dialogue system, summarization and machine translation). The architecture of these pre-trained language models has been developed from shallow to deep (the parameter scale is advancing from million-level [38, 40] to trillion-level [41]) along with the improvement of computational capability and the enhancement of training mechanisms. Pre-training tasks such as nearby word prediction [42], next word prediction [43, 44], masked language model [38] are widely explored in language modeling. These tasks do not need any annotated labels, so they can be trained on a huge amount of unlabeled data from the Internet, e.g., Wiki and Reddit.

We introduce how these pre-trained language models are applied to the downstream tasks. The development of utilizing pre-trained language models can be divided into two stages, embedding-based approaches and finetuning-based approaches. In embedding-based approaches, the word-, sentence- or paragraph-level embeddings from the pre-trained language models is utilized as features in the downstream tasks. As for finetuning-based approaches, they benefit the knowledge transferring between the pre-trained language models and the downstream tasks. A common finetuning procedure would fix (or apply smaller learning rate to) the original parameters of the pre-trained language model and add some finetunable adaptation modules for the downstream tasks. Besides, prompt tuning is getting more attention in all of finetuning methods. Discrete [45, 46] or continuous [47] prompts based finetuning are proposed to help bridge the gap between the pre-training and finetuning and reduce the computational cost. Different from the domain-specific knowledge (e.g., review of items) introduced by the previous work [4, 48], the information learned by pre-trained language models is general and not task-specific. The finetuning methods based on pre-trained language model can be well generalized to other domains while the training scheme based on domain-specific knowledge limits the model generalizability.

A classical architecture of conversational recommender system generally consists of two modules, named generation module and recommendation module. It takes the history context of a conversation $C=(t_{1},t_{2},\ldots,t_{m})$ as input, where $t_{i}$ ($1\leq i\leq m$) is an utterance from the seeker (i.e. the user) or the CRS itself and $m$ is the number of context utterances. Each utterance $t_{i}$ contains word tokens $t_{i}=(w_{i,1},w_{i,2},\ldots,w_{i,n_{i}})$, where $n_{i}$ is the number of tokens in utterance $t_{i}$. The CRS uses its recommendation module to recommend items $\mathcal{I}_{i}$ from a candidate item set $\mathcal{I}$ based on the context $C$ and some related external knowledge (e.g., knowledge graphs or item reviews). Then it embeds them into a response $R=(y_{1},y_{2},\ldots,y_{n})$, a sequence of $n$ tokens generated by the generation module based on the context $C$ to produce natural responses with recommendations.

Our response generation module follows a general Transformer [49] sequence-to-sequence framework. We first briefly introduce the Transformer framework followed by the BART pre-trained language model. Then we explain how we finetune the BART model to enable generating responses with recommended items.

Transformer. A Transformer layer typically consists of two kinds of sub-layers: multi-head attention layers and fully connected feed-forward networks. In a multi-head attention layer, for each head $j$, the output vector is computed with the following equation:

where $\bm{Q},\bm{K},\bm{V}$ are the matrices of queries, keys and values, $d_{k}$ is the dimension of keys, and $\bm{Q}_{j}=\bm{Q}\bm{W}_{j}^{Q}$, $\bm{K}_{j}=\bm{K}\bm{W}_{j}^{K}$, $\bm{V}_{j}=\bm{V}\bm{W}_{j}^{V}$ are project matrices for head $j$. The output of each head is concatenated and then mapped to the final output field. For the fully connected feed-forward network (${\mathop{\mathrm{FFN}}}$), it consists of two linear transformations with an ReLU activation in between:

A Transformer encoder layer is composed of a multi-head self-attention layer (takes the same input as $\bm{Q},\bm{K},\bm{V}$) and a feed-forward network, while a Transformer decoder layer additionally attends to the encoder output (as $\bm{K}$ and $\bm{V}$) for cross attention (see the bottom part of Fig. 1).

BART. BART [6] is a pre-trained language model based on Transformer trained with several denoising objectives on large-scale books and Wikipedia data, i.e., recovering the noisy input processed by token masking, token deletion, text infilling, sentence permutation and document rotation. These denoising objectives enable the model to learn contextual representations with unlabeled data, which can be crawled easily from the Internet, and then used for multiple downstream tasks. BART has been shown to be effective in many generation tasks, including abstractive QA, summarization, machine translation at the sentence and document level, and persona-based response generation.

Finetune BART for CRS. As most of the available CRS datasets are relatively small and contain only around 10K conversations [1, 5], it is difficult to learn complex semantic and discourse level dependencies only based on the training corpus. To relieve the burden and enhance context modeling, we choose to finetune a pre-trained BART [6] model for our response generation module. Though the introduction of pre-trained BART model can be regarded as a way of including more external knowledge to the CRS, there are two major differences between the domain-specific external knowledge (i.e., review of items) and the general external knowledge (i.e., pre-trained language model): (i) Difficulties in applying the knowledge. The collection of domain-specific knowledge is time-consuming and how to design efficient networks to include the domain-specific knowledge with other knowledge becomes a new challenge, while the finetuning scheme based on the pre-trained language model is easier and can follow some regular solutions. (ii) Generalizability. The pre-trained language models can be employed in most of the tasks while the domain-specific knowledge may only exist in a few tasks or be hard to collect for some tasks.

To enable the BART model to generate the item-aware responses, we extend its original vocabulary $\mathcal{V}$ with the item set $\mathcal{I}$ to be $\mathcal{V}^{\prime}=\mathcal{V}\cup\mathcal{I}$ (in this case, we do not tokenize the item names in an utterance and recognize them as single tokens), and then use a CRS training corpus to finetune the model. Specifically, we concatenate the utterances $t_{i}$ in context $C$ with an appended $\langle EOT\rangle$ token in their chronological order as the input (i.e., $x=[t_{1};\langle EOT\rangle;t_{2};..;\langle EOT\rangle;t_{m}]$), and maximize the probability of the ground truth response $R$ (i.e., minimize the objective in  Eq. 5).

Finally, an integration operation is added to the generation output to make the generation aware of the recommendation. We will discuss it later in Section 3.4.

To fully understand user preferences over items from a given context $C$, we propose to extract two kinds of information for the recommendation. The first is entity-level information, where we extract the mentioned entities (including the items in the item set $\mathcal{I}$) from $C$ and apply them to an external relational knowledge graph to perform entity linking [50]. Then the learned entity representations are summarized by a proposed time-aware attention to produce entity-level user representation. The second is contextual information produced by BART model, which is expected to capture information from the perspectives of semantic and conversational discourse. We describe the details in the following.

So far, the responses generated by the generation module (introduced in Section 3.2) are not aware of the recommendation (introduced in Section 3.3) results. We introduce an integration mechanism to incorporate the recommendation module’s results and guide the generation module to generate responses that are consistent with the user’s preference. Some of the previous works [3, 4] use copy mechanism [52, 53] to incorporate recommendation results into the generated responses. However, as it needs additional networks to predict when to switch between generation mode and copy mode, it may sometimes perform wrongly due to insufficient data to learn the pattern.

We dynamically add the bias $\bm{b}_{u}$ during generation based on the top predictions in each time stamp $t$. Only when the top predictions of that time stamp include items in item set $\mathcal{I}$ will we add the bias to them. The effect of this operation is that for a generation token $y_{t}$ that is probably an item in $\mathcal{I}$, we add its recommendation probability $p_{rec}(y_{t})$ to the original generation probability $p(y_{t})$. In this way, the results of the two modules are integrated so that our system can generate the recommendation-aware responses.

Datasets. To empirically evaluate the proposed approach, we conduct experiments on two popular CRS datasets, namely, ReDial [1] and OpenDialKG [5]. ReDial is centered around movie recommendation and is constructed on Amazon Mechanical Turk (AMT) platform with pair crowd-workers (i.e., Seeker and Recommender). OpenDialKG consists of conversations that are mainly in four domains: movie, book, sports and music. The construction process of OpenDialKG is described as follows, which can reveal why the recommendation on OpenDialKG is much easier than on ReDial: The first agent initiates a conversation by giving a seed entity. The second agent is given with a list of facts that are relevant to the seed entity, and asked to choose the most relevant facts and use them to frame a free-form conversational response. Table II presents basic statistics about the two datasets. As can be seen, the both datasets have around 10K conversations. We can notice that the average conversation length of ReDial dataset is larger than that of OpenDialKG. For the experiments, we split the two datasets into training, validation and test at ratios of 80%:10%:10% and 75%:15%:15% by following [1] and [5], respectively.

Evaluation Metrics. We evaluate the performance of recommendation and generation separately. For recommendation, we adopt Recall@K scores where K = 1, 10, 50 for ReDial by following [2], and K = 1, 3, 5, 10, 25 for OpenDialKG by following [5]. Recall@K indicates whether the predicted top-K items contain the ground truth recommendation items. For generation, apart from Dist-n (n=2, 3, 4) and perplexity (PPL) scores reported in previous work, we also report the case-insensitive BLEU-n (n=2, 4) scores¹¹1We use NLTK package (https://www.nltk.org) to calculate the BLEU scores.. For a fair comparison, we calculate the PPL scores via a widely used off-the-shelf package KenLM²²2https://kheafield.com/code/kenlm/, as the PPL scores may be very different when using different vocabulary.

For the ReDial dataset, we compare our model with six competitive baselines proposed by the previous work:
$\bullet$ ReDial [1]: The model consists of a dialog generation module which is based on HRED [54], a recommendation module that is based on an auto-encoder [55] and a sentiment analysis module. Switching Mechanism is adopted to combine the recommendation and generation results.
$\bullet$ KBRD [2]: It utilizes DBpedia knowledge graph to enhance the semantic information of items (or entities) to do the recommendation and adopts a transformer based generation module, where knowledge graph information serves as word bias to assist the generation.
$\bullet$ CRWalker [56]: The method walks on the knowledge graph to form a reasoning tree at each turn for recommendation, and then maps to dialog acts to guide response generation. The learned recommendation-oriented dialog policy on the knowledge graph enhances the mutual benefit between recommendation and conversation.
$\bullet$ KGSF [3]: It uses mutual information maximization (MIM) [57] to align the semantic spaces of word- and entity-level KGs for the recommendation module. Its generation module includes a transformer encoder followed by a fused knowledge-enhanced decoder.
$\bullet$ RevCore [4]: It performs review-enriched and entity-based recommendation and uses a review-attentive encoder-decoder for generation. We use the re-implementation results in [48] for a fair comparison.
$\bullet$ $C^{2}$-CRS [48]: This method extracts and represents the associated multi-grained semantic units from different data signals, and then align the corresponding semantic units from different data signals in a coarse-to-fine way.

For the OpenDialKG dataset, we compare the models that are compared in [5] and introduce them as follows.
$\bullet$ seq2seq [58]: It applies a seq2seq approach for entity path generation, given all of the dialog contexts.
$\bullet$ Tri-LSTM [59]: It encodes each utterance and all of its related facts within 1-hop from a KG to retrieve a response from a small (N=10) pre-defined sentence bank.
$\bullet$ Ext-ED [60]: It generates response entity token at its final softmax layer, and does not utilize structural information from knowledge graph.
$\bullet$ DialKG Walker [5]: It has an attention-based graph decoder that penalizes decoding of unnatural paths which prunes candidate entities and paths from a large search space.

We also test the performance of the following variants to our model as an ablation study.
$\bullet$ BART (ContextM): It only uses contextual-level representation (described in Section 3.3.2) to do the recommendation. No entity-level representations and extra knowledge graph based knowledge are employed.
$\bullet$ EntityM-SelfA: EntityM-SelfA refers to entity modeling with self-attention used in previous work. It only uses the entity-level representations to do the recommendation.
$\bullet$ EntityM-TimeA: EntityM-TimeA refers to entity modeling with our designed time-aware modeling. It also only uses entity-level representations.
$\bullet$ EntityM-TimeA-ContextM: Our full model that adopts both entity-level and contextual-level representation.

Besides, to examine the effectiveness of adding text entities to input (defined in Section 3.3.1) for recommendation, we also evaluate our model variants with and without text entities (i.e., TextEntity in table III) as input.

For the RGCN-based recommendation module, we set both the entity embedding size and the hidden representation dimension to 128. The layer number for R-GCN is 1 and the normalization factor $Z_{e,r}$ is set to 1 following [2]. For the BART-based generation module, we finetune on the pre-trained BART-Base model, which consists of 6 layers of encoder and decoder, respectively. The hidden dimension, feed-forward network size and attention head number are 768, 3072 and 12 for both encoder and decoder layers. The total parameter number of our model is 269M. The time recency effect $\lambda$ in Eq. 7, the balance factor between entity- and contextual-level representations in Eq. 10 and the tradeoff between two training objectives in Eq. 13 are set to 1.5, 0.5 and 1.0, respectively. We adopt diverse beam search [61] mechanism in generation with a beam size of 4 and the diverse beam group number is set to 2. All the hyper-parameters are determined by grid-search based on the validation performance.

We implement our models based on FAIRSEQ framework³³3https://github.com/pytorch/fairseq [62], and train on an NVIDIA 3090 GPU. During training, we set the max tokens of each batch to 4096 with an update frequency of 4. We adopt Adam optimizer with a 5e-3 learning rate (and 5e-5 learning rate for BART-related modules as they are pre-trained parameters) and 1000 warm-up updates followed by a polynomial decay scheduler. The training time of one epoch is around 22 minutes. The model needs around 5 epochs to achieve the best performance on the validation set.

In this subsection, we present the main comparison results on ReDial dataset in Section 5.1.1. Then, to further verify the effectiveness of our model in multi-domain dataset, we conduct experiments on OpenDialKG dataset and report the comparison results in Section 5.1.2.

In this subsection, we first discuss the automatic evaluation results in Section 5.2.1 and then adopt a human evaluation to examine the generation results from a different perspective in Section 5.2.2.

In this subsection, we first test the recommendation results with varying mentioned items in Section 5.3.1 to show the effectiveness of the introduced contextual-level representation. Then we discuss the effects of BART pre-training for our model in Section 5.3.2.

To give an intuitive assessment of the quality of the generated sentences and recommended items, we present a conversation example in Table VIII. Apart from the results generated by our model and the human ground-truths from the corpus, we list the results of ReDial, KBRD and KGSF models. The reason why we do not list the results of RevCore and $C^{2}$-CRS is their released codes are not complete to re-implement the generation results.

From Table VIII, we can easily find that our model tends to generate responses with more diversity and coherence to the context, which may thank to the pre-trained BART model. For example, in the last turn, our model also talks about “Johnny Depp” which is mentioned in the turn of the seeker. This means that our model can capture the topic of discussion well while the baselines tend to repeat the boring responses with the same syntactic structure (e.g., “I haven’t seen that one….”). On the other hand, the baseline models like ReDial and KBRD tend to recommend the same items regardless of the history context, while our model can produce different recommendations based on different contexts thanks to our improved recommendation mechanism.

In this subsection, we extensively explore the effects of some hyper-parameters, together with in-depth discussions.