Open-domain Dialogue Generation Grounded with Dynamic Multi-form Knowledge Fusion

Many works have proved that external knowledge can facilitate dialogue generation. [27] presents a novel open-domain dialogue generation method to demonstrate how large-scale commonsense knowledge can facilitate language understanding and generation. [4] proposes a latent relation language model, a class of language models that parameterize the joint distribution over the words in a document and relevant entities via knowledge graph relations. For the use of the external documents, [10] incorporates external documents into the procedure of response generation in custom service dialogues. GLKS [14] adopts a global guide method to the local, and uses the dialogue contexts to filter out important n-gram information from the document to guide the generation process. However, the knowledge graphs lose facts, and external texts require complicated processing. These two forms of knowledge still have limitations in exerting external knowledge.

Virtual Knowledge Base (virtual KB) is an indexed corpus, which treats a corpus as a knowledge base containing entities and texts. It has been widely employed in open-domain Question Answer (QA) [3, 11, 2]. Virtual KB accomplishes the QA tasks by answering queries with spans from the corpus, ensuring that facts can be preserved in the relation extraction process. Whereas, to the best of our knowledge, virtual KB has not yet been mentioned in open-domain dialogue generation. DrKIT[2] is a state-of-the-art reasoning algorithm with QA on a virtual KB, which traverses textual data like a KB, softly following paths of relations between mentions of entities in the corpus. Inspired by this, we present a novel model, DMKCM, which includes a reasoning strategy based on DrKIT for getting more information related to our dialogue. To better fit our task, we convert a commonsense story corpus—the indexed ROCStories[12] as our virtual KB, instead of professional Wikipedia.

The overview of DMKCM is shown in Fig. 2. DMKCM consists of two branches, dialogue branch and knowledge branch. The dialogue branch (green blocks in left of Fig. 2) aims to generate conversation based on an encoder-decoder model and interacts information with the knowledge branch to improve the informativeness expression of response. The knowledge branch(orange blocks in left of Fig. 2) is to reason, store, merge, and expand knowledge by Virtual Knowledge reasoning module (VK-reasoning), Dynamic Virtual Knowledge memory module (DVK-memory), Dynamic Virtual Knowledge selector module (DVK-selector), and Commonsense Knowledge expansion module (CK-expansion).

Before presenting our detail for the dialogue generation approach, we first introduce our notations and critical concepts. Formally, suppose we have a conversation dataset $D=\left\{\left({C}^{i},{X}^{i},R^{i}\right)\right\}_{i=1}^{N}$, where $C^{i}$ represents conversational context before $i$-th turn. $X^{i}$ is $i$-th user utterance. $R^{i}$ is a response regarding to $X^{i}$. The final goal of this task is to estimate a generation probability distribution $P(R|[C,X],K)$ from $D$. Therefore, one can generate a response for $\left[C,X\right]$ following $P(R|[C,X],K)$, where $\left[C,X\right]$ means the concatenation for context $C$ and current user utterance $X$, $K$ is the knowledge from knowledge branch, and $R$ is the corresponding response. Assume that there are already $n-1$ turns in a dialogue. We use Transformer encoder (T_enc) to encode $X^{n}$ and $C^{n}$ and get the last hidden state $h_{p}^{n}$ and state $h_{e}^{n}$ from ${X}^{n}$ and $\left[C^{n},X^{n}\right]$ separately. $h_{p}^{n}$ and $h_{e}^{n}$ represent encoded semantic information of ${X}^{n}$ and $\left[C^{n},X^{n}\right]$. Next, we elaborate on details for each of them.

Firstly, VK-reasoning reasons and filters candidate documents related to the user utterance $X^{n}$ as 1st hop from a virtual KB. We send $X^{n}$, candidate documents, and $C^{n}$ to CK-expansion, aiming to expand commonsense concepts for better response. DVK-memory is a dynamic transfer module. It dynamically stores encoded vectors of 1st hop from previous $n-1$ turns. We named these vectors as history virtual knowledge. Then, we filter and send related history virtual knowledge to DVK-selector as a knowledge supplement. Next, DVK-selector dynamically integrates information from 1st hop, history virtual knowledge and dialog information for the decoder. Noticeably, after this process, encoded information from the current 1st hop requires to be updated into DVK-memory. Before generating a response, CK-expansion needs to expand the words of input information by traversing in a commonsense KG to capture concepts shifted in conversation structure. These extended concepts are denoted as 2nd hop.

We use Transformer Decoder (T_dec) to generate words,

Then, a Controller is designed, in which the decoded hidden state $h_{d}^{n}$ will be mapped into vocab size and outputs the probability of words $P_{v}$ by Softmax function,

In addition, we can also generate knowledgeable words by using knowledge expansion representation encoded in CK-expansion,

We get an attention weight $\gamma_{i}^{n}$ by using the decoded hidden state $h_{d}^{n}$ to focus on the $h_{k_{C}}^{n}$, which can make the model focus on the relative knowledge triples; then, we choose the knowledge entities according to entities probability $P_{K_{C}}$ of relative weighted knowledge after Softmax function.

The final generated words will consider both the distribution of standard vocabulary and the distribution of knowledge entities. We use a soft gate probability $g_{t}$ to choose the generated words from standard vocabulary or knowledge entities.

To train the proposed model, we minimize the negative log-likelihood

where $N$ is the total number of the dataset, and $T$ is the timestep of the $n$-th turn response sentence. $X^{(n)}$ represents the $n$-th turn user utterance in the dataset, and $K^{(n)}$ represents the $n$-th turn knowledge.

Conversation Corpus: We choose DailyDialog [8] and PersonaChat [25] as our datasets. In our work, four turns of dialogue are a unit of training sample and pre-processed statistics of the above datasets are shown in the Fig. 3. Commonsense Knowledge Corpus: ConceptNet[17] is a semantic network designed to help computers understand the meanings of words that people use. Its English vocabulary contains approximately 1,500,000 nodes. Source of Virtual KB: The ROCStories [12] is a commonsense story corpus which contains 98,161 five-sentence stories.

We compare our model with representative baselines to investigate its effectiveness. The baselines are as follows: (1) Attn-S2S[18]: A classic method applies simple attention [1] to the input context based on the sequence-to-sequence model; (2) Transformer[21]: Transformer is a popular network architecture, based solely on attention mechanisms; (3) Dir-VHRED[23]: A recent work based on the latent variable hierarchical recurrent encoder-decoder model and characterizes the latent variables using Dirichlet distribution instead of traditional Gaussian distribution; (4) GLKS[14]: The newest dialogue generation model based on unstructured knowledge builds a global-to-local knowledge selection method to improve the quality of selected unstructured knowledge in background-based conversations; (5) CCM[27]: A commonsense knowledge-aware conversation model, which leverages commonsense knowledge from ConceptNet[17] through two graph attention mechanisms to facilitate informative response generation; (6) MKST[26]: The latest universal transformer-based architecture fuses label, background knowledge in open-domain conversation. Since MKST relies on datasets with background knowledge, we compare it with our model only on PersonaChat which has background information.

We conduct the experiments with a Transformer structure (our baseline) with 8 heads, 6 layers, 512-dimensional hidden states, and a 2-layer GCN model. In the VK-reasoning, we set the number of reasoned candidate documents as 10 and filtered candidate documents as 5. During the CK-expansion, we search the neighbors of nodes and preserve the top 100 neighbor nodes. When processing datasets, the history context we choose are the previous turns of the current conversation. To train the model, we use the Adam optimizer[6] and use Adam-warmup to adjust the learning rate.

To analyze and evaluate our model more comprehensively, we use both automatic and human evaluations. Automatic Evaluation: Based on previous work, we apply several widely used automatic metrics. Specifically, we adopt PPL,BLEU-1,2,3,4)[13],and Distinct-1,2 (Dist-1,2)[7] to intuitively reveals quality, coherence and diversity of generated responses. PPL is the perplexity score that measures the quality of the language model. BLEU calculates word-based precision between a generated response and a gold response. Distinct evaluates the informativeness and diversity of the predicted responses. Human Evaluation: As known, automatic evaluation indicators have limitations in evaluating human conversations [9]. In our work, we randomly sample 200 test samples to conduct human evaluations. For response, we define three metrics: (1) Fluency (Flu.), i.e., degree of fluency and human readability; (2) Informativeness (Inf.), i.e., degree of knowledge for responses; (3) Appropriateness (App.), i.e., degree of relevance to the given context; Each response has 3 annotators to give a 3-graded whose rating range from 0 to 2. We take the average scores as the final results for each metric.

As shown in Table 3, we analyze the effectiveness of each module proposed in DMKCM through the following situations: (1)- 2Hop: DMKCM drops CK-expansion; (2)- Mem and 2Hop: DMKCM drops DVK-memory and CK-expansion; (3)- 1Hop and Mem: DMKCM drops VK-reasoning and DVK-memory; (4)- 1Hop, Mem, and 2Hop: This is baseline (Transformer). From the results, we can observe that the performance of situation (1) drops sharply from our model. This is within our expectations since CK-expansion helps to capture extra information from the post, which can improve the diversity of generated responses. This result can also show that fusing structured knowledge can effectively help dialogue generation. Metric results of the situation (1) are better than situation (2), which verifies that retaining history virtual knowledge using DVK-memory can effectively help dialogue generation. Situation (3) is related to unstructured knowledge and its poor results prove the effectiveness of VK-reasoning and DVK-memory, and the reasoned virtual knowledge affects generating responses. Situation (1) is better than situation (4) can also reflect all of the modules play important roles in our model. In summary, these modules of DMKCM designed for fusing structured and unstructured knowledge are helpful for response generating in terms of informativeness and coherence.

Sample conversations are shown in Table 4 which indicate that DMKCM can generate better responses than the baselines on two conversation datasets. Traditional end-to-end models, e.g., Attn-S2S and Dir-VHRED, tend to generate simple, safe, and incoherent responses without access to knowledge. Despite knowledge-based model, e.g., CCM, GLKS, and MKST, which generate informative responses, still produces irrelevant responses. Instead, DMKCM fuses knowledge from the knowledge graph and virtual knowledge as a whole and encodes more related information via DVK-selector and Controller modules, which supports generating more information and coherence response. In practice, the effect of these modules can be reflected, like Fig. 5. It thus generates more reasonable responses through better use of knowledge.