Multi-turn Response Selection with Commonsense-enhanced Language Models

Dialogue systems, especially those open-domain ones, are expected to converse with humans in an engaging way, so common sense is very necessary to be integrated into the model effectively (Young et al., 2018). When coming across a specific context, dialogue systems are usually incapable of reasoning according to corresponding background information unless we empower them with specific means. This section aims to extract useful commonsense concepts from an external knowledge graph and rank them based on their relevance to the dialogue context.

First of all, we need to select an external knowledge graph $\mathcal{G}$ which would include useful common sense as much as possible. As the Fig.2(a) shows, we retrieve a subgraph $\mathcal{G}_{i,k}^{s}$ from $\mathcal{G}$ for each response candidate and its context, i.e., $(P_{i},$ $C_{i},$ $r_{i,k},$ $y_{i,k})$, following prior works (Feng et al., 2020; Yasunaga et al., 2021). The extraction process includes four steps. First, given $(P_{i},$ $C_{i},$ $r_{i,k},$ $y_{i,k})$, the concept set $\mathcal{V}_{i,k}^{PCR}$ is obtained by entity linking between tokens in the dialogue text and concept nodes in $\mathcal{G}$. Second, the few-hop neighbors of elements in $\mathcal{V}_{i,k}^{PCR}$ are extracted to form $\mathcal{V}_{i,k}^{e}$, which is a set of extra nodes containing high-order external knowledge from the knowledge graph. The complete node set of the subgraph is the union of the two, $\mathcal{V}_{i,k}^{s}$ $=$ $\mathcal{V}_{i,k}^{PCR}$ $\cup$ $\mathcal{V}_{i,k}^{e}$. Third, we calculate a score for each concept node in $\mathcal{V}_{i,k}^{s}$ to measure the relevance between the dialogue text and corresponding concept through the PLM:

where $\oplus$ and $\cdot$ represent concatenation and dot product, respectively, and $f_{m}$ is the dot-product similarity function. Since a longer average text length of multi-turn dialogues leads to more related concepts from $\mathcal{G}$, calculating such pairwise scores becomes computationally impractical for a large set of concepts. As the equations show, we employ a bi-encoder that first encodes the context and concepts into embeddings, and then computes their similarity scores through their embedding vectors. Compared with the cross-encoder, a bi-encoder is more convenient and efficient under our multi-turn scenario where there are more candidate concepts to score because of the longer context and it allows to pre-compute the embeddings of candidate concepts first  (Humeau et al., 2019; Li et al., 2021). In practice, we pre-compute the embeddings of all concepts in $\mathcal{G}$ and conduct a dot product between the embedding of dialogue text and its corresponding concept embeddings to obtain their final scores. Both embeddings of the concepts and dialogue text are computed by the PLM. Then we rank these concepts according to their similarity scores with the context and choose the top ones as useful inputs of the GNN. The embeddings of chosen top concepts will be updated via GNN’s message passing in Eq.(7). Finally, the edges $\mathcal{E}_{i,k}^{s}$ and relation types $\mathcal{R}_{i,k}^{s}$ are abstracted based on $\mathcal{V}_{i,k}^{s}$, that is, $\mathcal{G}_{i,k}^{s}$ $=$ $(\mathcal{V}_{i,k}^{s}$, $\mathcal{E}_{i,k}^{s}$, $\mathcal{R}_{i,k}^{s})$.

To effectively inject common sense into the PLM for response selection, we utilize GNN to learn the structural information from $\mathcal{G}$ in SinLG. Meanwhile, GNN transfers the commonsense knowledge to the PLM by maximizing the similarity between the different embeddings from the PLM and GNN where the design of Siamese network structure aims to realize the augmentation effect for the same input. In this way, the performance of the PLM can be enhanced.

Although PLMs have achieved various impressive results on all kinds of downstream NLP tasks, they are not so trustworthy when facing complicated understanding tasks that require advanced reasoning ability based on external commonsense knowledge (Liu et al., 2021; Zhang et al., 2021b). Hence, we design the KG-guided training process in our framework to enhance the reasoning ability of PLMs with extra common sense from KG. As Fig.2(a) depicts, both the PLM and GNN generate a representation vector of the context, i.e., $\boldsymbol{h}^{\prime}_{i,k}$ and $\boldsymbol{h}_{X_{i,k}}$, respectively. The final predicted scores of different candidate responses are obtained via the representation vectors from the PLM. We design two sub-functions for the final loss in the following sections.

We design the KG-guided training process via adding similarity loss instead of a direct embedding fusion (e.g., concatenating $\boldsymbol{h}_{X_{i,k}}$ from the GNN with the dialogue context embedding, $\boldsymbol{h}^{\prime}_{i,k}$ from the PLM) so that we can conduct inference with only the PLM part as shown in Fig.2(b). The similarity loss helps the PLM part obtain the knowledge from the GNN part and achieve better performance. Hence, it is not only an effective way of incorporating commonsense knowledge into the PLM for the MRS task, but also beneficial for inference efficiency. In order to utilize the commonsense knowledge from KG, we need to conduct a bunch of tedious procedures: tokenization of the dialogue context, entity linking between context tokens and concepts in KG, similarity calculation between concepts and the dialogue context, ranking and filtering of those concepts, and construction of subgraphs with related concepts. With our designed framework, during inference, we can not only get rid of the heavy preprocessing operations but also bypass the generation and representation learning for any subgraphs, because the knowledge memory of the PLM is now aroused by maximizing the similarity loss. This will help increase the inference efficiency as well as reduce the computational complexity of the dialogue system.

We conduct experiments on two variants of PERSONA-CHAT (Zhang et al., 2018), that is, the dialogues with original personas and its revised version whose detail statistics are listed in Table 1 and Table 2. The revised version is rephrased from the original one in an implicit way which is more challenging than the original one in the understanding complexity so that we can testify the effectiveness of our model under different levels. We choose ConceptNet (Speer et al., 2017) as our auxiliary knowledge graph, which is a widely-used (Jain and Lobiyal, 2022; Zhou et al., 2022; Yasunaga et al., 2021; Huang et al., 2020; Sap et al., 2019; Bosselut et al., 2019) commonsense source.

As Table 1 shows, PERSONA-CHAT possesses $10,907$ complete conversations, $8939$ for training, $1,000$ for development, and $968$ for test, where dialogues have $6\sim 8$ conversation turns. Response selection is conducted at each round of a complete conversation, which leads to $81,032$ mrsSamples (which is defined in the definition of PROBLEM 1.) in total, $65,719$ for training, $7,801$ for development, and $7,512$ for testing. The response candidate set of each turn includes 20 choices, where a $1$ positive response is the true one from human beings while $19$ negative ones are stochastically selected. As shown in Fig.1, the revised version has the same dialogue context as the original one but its personas are rephrased, generalized, or specialized, which no doubt increases the difficulty of the MRS task. We conduct statistics about the context lengths of mrsSamples where $Min_{len}$, $Max_{len}$, $Ave_{len}$ represent the minimum, maximum, and average lengths of each dataset and the last three columns are the ratios of mrsSamples with context lengths more than $100$, $256$, and $512$ among every dataset. As Table 2 shows, the context lengths of training datasets vary more, ranging from $30+$ to $600+$ while those in development and test datasets range from $50+$ to $400\pm$. This could increase the robustness of the model after training on these training datasets but the development and test datasets seem not that comprehensive because they leave out the validating chance of extreme cases. Moreover, we can observe that samples with lengths between $Min_{len}$ and $100$ take up the maximum proportion and the second category is $Len>100$, then $Len>256$, $Len>512$, orderly. These statistics are good references for the experiment setting, i.e., the maximum sequence lengths of PLMs, which have a significant impact on their performance. For instance, if the hardware resources are limited, the max sequence length can be chosen between 100 and 256 to test uniformly.

ConceptNet is a public multilingual graph of commonsense knowledge where labeled edges connect phrases and words. With the combination of several different sources (i.e., Facts acquired from Open Mind common sense (Singh et al., 2002), a subset of DBpedia (Auer et al., 2007), JMDict (Breen, 2004), Open Multilingual WordNet (Bond and Foster, 2013), etc.), ConceptNet possesses over $21$ million edges and over $8$ million nodes, whose English vocabulary consists of about $1500,000$ nodes.

To evaluate the effectiveness of SinLG on MRS, we make comparisons with state-of-the-art results from (Gu et al., 2020, 2021). Their brief introductions are listed as follows:

• •

  Starspace: Starspace (Wu et al., 2018a) learns entity embeddings separately and scores the response candidates by computing the similarity between them and the conversation.

• •

  PM: Ranking Profile Memory network (PM) is proposed in the publishing paper of PERSONA-CHAT, (Zhang et al., 2018) which uses an attention mechanism to identify the relevant lines of the profile with the dialogue context.

• •

  KV-PM: As an improvement of PM, the Key-Value Profile Memory network (KV-PM) is put forward (Miller et al., 2016). It calculates attention scores over keys and obtains the values instead of using the same keys as the original one, which can exceed the profile memory network according to the definition of key-value pairs on specific tasks.

• •

  Transformer: Transformer (Vaswani et al., 2017) is an encoder-decoder memory network based only on the attention mechanism. It achieves state-of-the-art performance on the next utterance retrieval task under multi-turn dialogues. The experiment of the transformer employs the setting in (Mazare et al., 2018), which only utilizes the encoding architecture.

• •

  DGMN: Document-Grounded Matching Network (DGMN) (Zhao et al., ) is one of the several pioneering studies that incorporate extra information (a related document) into the context besides the dialogue text itself. DGMN constructs rich representation for the context and document via self-attention to do matching for the dialogue and candidate response better.

• •

  DIM: Dually Interactive Matching network (DIM) (Gu et al., 2019) ranks response candidates via interactive matching for both response-context pairs and response-persona pairs respectively.

• •

  FIRE: FIRE Gu et al. (2020) identifies relative knowledge for context bidirectionally before matching response candidates with the conversation context and corresponding knowledge.

• •

  BERT: BERT (Devlin et al., 2019) is developed by Google, which is based on the encoder module of the transformer, targeting the pre-training of natural language processing (NLP). It exhibited state-of-the-art performance on many natural language understanding tasks, including GLUE, SQuAD, and SWAG, regarded as the SOTA baseline.

• •

  Roberta: According to proposing a set of important design choices and training strategies, and introducing alternatives, Roberta (Liu et al., 2019) achieves better downstream task performance than BERT. Roberta employed a novel dataset, CCNEWS, which testifies that utilizing more data for pre-training further boosts performance on downstream tasks.

We evaluate the model performance with the identical metrics $R_{n}@k$ and MRR which are commonly used on the MRS task (Tao et al., 2020; Gu et al., 2021; Whang et al., 2021). The equations of them are as follows:

where $|D|$ represents the total number of samples in the target dataset and $rank_{i}$ denotes the rank of the ground-truth response in the $i$ sample. As there are $20$ response choices in PERSONA-CHAT, we choose $R_{20}@1$, $R_{20}@2$, $R_{20}@5$ and MRR as our final metrics.

The data splitting of PERSONA-CHAT is fixed, which is shown in Table 1. According to the statistics of datasets in Table 2, the default max sequence length of PLM is set as 512 which will cover nearly all samples’ lengths. The default number of nodes, embedding size, and the number of layers number in the GNN is set as 200, 200, and 5. For the sake of fairness, we set the batch size of all models as 64 and use the learning rate that literature (Yasunaga et al., 2021) recommends. The loss weight $\alpha$ is set as 0.5 by default. We implement our solutions with Pytorch 1.9 and Python 3.8.

In order to solve the multi-turn response selection problem, existing studies propose various context-response matching models which include traditional methods and PLM-based ones (Tao et al., 2021).

Traditional methods first focus on learning better embeddings for both the context and response. The concatenation of them is fed into a matching layer to make the final prediction (Wang et al., 2017; Wu et al., 2018b; Yan et al., 2018; Xu et al., 2021b). Specifically, Wang et al. (2017) utilize hand-craft features, i.e., sentence length, the number of common words for sentence-level representation while Wu et al. (2018b) employ topic features to provide clues for the selection of the best response. With the prosperity of deep neural networks, researchers propose employing CNN, RNN, and self-attention mechanism, combining with MLP, pooling layers to learn effective representations. For instance, Yan et al. (2018) extract features from three different levels, that is, word, phrase, and sentence and propose an attention-based CNN model to match the dialogue context with response candidates. Different from the previous methods that use topic-agnostic n-gram utterances as processing elements, Xu et al. (2021b) propose capturing topic shift at a discourse level and then effectively tracking the topic flow in the multi-turn dialogue.

Then the interactions between context and response draw intensive attention. Diverse new frameworks (Wu et al., 2017, 2018c; Tay et al., 2018; Zhou et al., 2018; Tao et al., 2019; Yuan et al., 2019) are presented to mine the interaction between them so that the matching accuracy can be improved. The study (Tay et al., 2018) designs the CARAN, stacked recurrent encoders, which consist of a bidirectional alignment and a multi-level attention refinement mechanism. Inspired by the rationale of Transformer, Zhou et al. (2018) propose a deep network based entirely on attention. Specifically, they create embeddings of context utterances at different granularities and then distill the highly-relative segment pairs with attention scores. Previous studies usually conduct interactions between context and response only on time, but Tao et al. (2019) raise this is conducted in a shallow way. Hence, they propose an interaction-over-interaction network to further the context-response interaction. Typically, Yuan et al. (2019) analyze the negative effects of involving exceeded context sentences and propose MSN to filter out the most relative sentences. As expected, the above methods keep trying to go deep into interactions fully and even among each word within the context while this can be accomplished well via PLMs.

Since the release of the first PLM (Devlin et al., 2019), various impressive results on downstream Natural Language Processing (NLP) tasks are achieved due to its strong capability in language representation and understanding. PLM-based models (Henderson et al., 2019; Whang et al., 2020; Wu et al., 2020; Xu et al., 2021a; Song et al., 2021) for MRS also emerge and lead to outstanding performance. Typically, the study (Henderson et al., 2019) learns embedding for both context and response severally via BERT and computes the dot product of them as the final matching score. Although BERT is conveniently adapted to various NLP tasks,  Whang et al. (2020) believe it still has limitations for tasks on a certain domain so they first post-train BERT on their task-specific corpora with two objectives, i.e., Masked Language Model (MLM) and Next Sentence Prediction (NSP), and then fine-tune BERT on the response selection task to achieve a better performance than the previous state-of-the-art. Considering the generalization of PLMs and the particularity of specific tasks, Wu et al. (2020) propose TOD-BERT that incorporates speaker and system tokens into the masked language modeling during pre-training to model the dialogue attributes better. Further, Xu et al. (2021a) design several self-supervised tasks to pre-train the PLM-based model in a multi-task manner for MRS where remarkable results have been obtained.

Meanwhile, external knowledge in paragraphs (such as profiles of speakers, and entity infobox from Wikipedia) is employed to assist in solving MSR problem (Zhao et al., ; Gu et al., 2019, 2020; Liu et al., 2020; Gu et al., 2021; Zhang et al., 2021a). To overcome the challenge of grounding dialogue contexts with background texts and distinguishing significant information, DGMN (Zhao et al., ) encodes three contents (i.e., sentences from the document, utterances in the dialogue context, and the target response candidate) simultaneously via self-attention, and contains attention mechanism for document-aware contexts and context-aware documents so that it can learn a rich representation for both dialogue contexts and candidate responses. Different from existing persona fusion approaches that enhance the context representation by calculating the similarity between it and a predefined persona, Gu et al. (2019) propose DIM which ranks response candidates via interactive matching between responses and contexts, as well as responses and personas respectively. In order to make full use of the background knowledge, and simultaneously match response candidates with both the context and its knowledge, Gu et al. (2020) propose FIRE. They build two filters for the knowledge and the context, which generate representations for both knowledge and context under the other’s ground. Liu et al. (2020) incorporate knowledge triplets into the utterances and feed them into BERT while they use soft position and visible matrix to optimize the knowledge noise issues. Gu et al. (2021) explore the effect of utilizing persona information based on the PERSONA-CHAT dataset and design four persona fusion strategies which are implemented into three existing models to testify their effectiveness. However, the above studies haven’t considered the effectiveness of common sense from extra knowledge graphs so we propose a model to incorporate this information and assist the training of PLMs.

As a prevalent and effective method (Chen et al., 2019; Lin et al., 2019; Du et al., 2021; Zhang et al., 2022; Yu et al., 2022) in natural language processing and knowledge discovery domain, knowledge reasoning provides us with many insights for our work. Especially, Lin et al. (2019) present an inference framework for commonsense question answering, which first projects the question-answer text from the semantic way to the knowledge-based symbolic way, forming a subgraph of an external knowledge graph. Then, it embeds schema graphs with a network based on GCN, LSTM, and a hierarchical attention mechanism. At last, it scores each answer conditioned on graph representations. The intermediate attention scores lead to transparent, interpretable, and trustworthy inferences. Chen et al. (2019) put forward KBRD, which capacitates interactions between recommender and dialogue systems. In this framework, informative entities are connected to an extra knowledge graph and input into the recommender besides items and they are transmitted on the KG through a relational graph convolutional network to enrich the user interest’s representation. Moreover, the knowledge-enhanced representation is put back to the dialogue system as a form of vocabulary bias, capacitating it to select responses that match the user’s interest. Few previous studies pay attention to the relation types in the knowledge graph when extracting new facts from them, especially only one or few instances are given, Du et al. (2021) propose CogKR, which summarizes the underlying relations of the provided instances first and then builds a cognitive graph through coordinating retrieval and reasoning iteratively. Existing subgraph-retrieval methods from KG tend to increase reasoning bias, so Zhang et al. (2022) design a trainable subgraph retriever separated with the following reasoning process, which can broaden paths to induce the subgraph and stop automatic expansion. Above this, any subgraph-oriented reasoners can be utilized to delicately deduce more reasonable answers from the subgraph. Yu et al. (2022) design a knowledge-grounded dialogue system, called XDAI ¹¹1https://github.com/THUDM/XDAI which possesses the prompt-aware PLM exploitation without fine-tuning and is equipped with open-domain knowledge and domain-specific mechanism. It is convenient for developers to quickly create an open-domain or domain-oriented dialogue system without heavy fine-tuning of PLMs.

Inspiring by the dual process theory from the cognitive process of humans (Evans, 1984, 2003, 2008; Sloman, 1996), frameworks (Ding et al., 2019; Feng et al., 2020; Lv et al., 2020; Yasunaga et al., 2021; Liu et al., 2021; Zhang et al., 2021b; Zhou et al., 2022) combining PLM with GNN are presented to enhance both the reasoning ability and interpretability of intelligent systems. Specifically, Ding et al. (2019) propose CogQA for multi-hop question answering with the help of web-scale documents, which iteratively constructs a cognitive graph through the implicit extraction module (BERT) and conducts explicit reasoning via GNN. Yasunaga et al. (2021) present QA-GNN, another PLM+KG model for question answering, which first encodes the QA context via a PLM and then retrieves a KG subgraph that includes all multi-hop neighbor entities of QA. As some entities are always more relevant to the QA, a PLM is directly used to score them. In addition, the QA context is regarded as a super node of its KG subgraph, and its representation from PLM is the initial feature. Finally, the representation of the super node via graph attention network and PLMs, as well as the subgraph’s representation are concatenated together to obtain the inference through an MLP layer. Liu et al. (2021) propose GRN which pre-trains an ALBERT with two self-supervised tasks and then fine-tunes it with a graph reasoning module together. Beyond combining PLM and GNN simply, Zhang et al. (2021b) present GREASELM which fuses and exchanges information from all the PLM and KG in multiple layers. An interactive mechanism to bi-directionally exchange information is designed between each layer of the PLM and GNN. Therefore, the inputs of two modalities can interact with each other directly.

The studies introduced in this section not only provide us with many inspirations for our current work but also enlighten us about prospects in relevant domains.