“Wait, I’m Still Talking!”
Predicting the Dialogue Interaction Behavior
Using Imagine-Then-Arbitrate Model

Creating a perfect artificial human-computer dialogue system is always a ultimate goal of natural language processing. In recent years, deep learning has become a basic technique in dialogue system. Lots of work has investigated on applying neural networks to dialogue system’s components or end-to-end dialogue frameworks Yan et al. (2017); Lipton et al. (2018). The advantage of deep learning is its ability to leverage large amount of data from internet, sensors, etc. The big conversation data and deep learning techniques like SEQ2SEQ Sutskever et al. (2014) and attention mechanism Luong et al. (2015) help the model understand the utterances, retrieve background knowledge and generate responses.

Though end-to-end methods play a more and more important role in dialogue system, the text classification modules Jiang et al. (2018); Kowsari et al. (2017) remains very useful in many problems like emotion recognition Song et al. (2019), gender recognition Hoyle et al. (2019), verbal intelligence, etc. There have been several widely used text classification methods proposed, e.g. Recurrent Neural Networks (RNNs) and CNNs. Typically RNN is trained to recognize patterns across time, while CNN learns to recognize patterns across space. Kim (2014) proposed TextCNNs trained on top of pre-trained word vectors for sentence-level classification tasks, and achieved excellent results on multiple benchmarks.

Besides RNNs and CNNs, Vaswani et al. (2017) proposed a new network architecture called Transformer, based solely on attention mechanism and obtained promising performance on many NLP tasks. To make the best use of unlabeled data, Devlin et al. (2018) introduced a new language representation model called BERT based on transformer and obtained state-of-the-art results.

Different from retrieval method, Natural Language Generation (NLG) tries converting a communication goal, selected by the dialogue manager, into a natural language form. It reflects the naturalness of a dialogue system, and thus the user experience. Traditional template or rule-based approach mainly contains a set of templates, rules, and hand-craft heuristics designed by domain experts. This makes it labor-intensive yet rigid, motivating researchers to find more data-driven approaches Ghazvininejad et al. (2018); Lin et al. (2019) that aim to optimize a generation module from corpora, one of which, Semantically Controlled LSTM (SC-LSTM) Wen et al. (2015), a variant of LSTM Hochreiter and Schmidhuber (1997), gives a semantic control on language generation with an extra component.

An imaginator is a natural language generator generating next sentence given the dialogue history. There are two imaginators in our method, agent’s imaginator and user’s imaginator. The goal of the two imaginators are to learn the agent’s and user’s speaking style respectively and generate possible future utterances.

As shown in Figure 2 (a), imaginator itself is a sequence generation model. We use one-hot embedding to convert all words and relative tags, e.g. turn tags and place holders, to one-hot vectors $w_{n}\in\textbf{R}^{V}$, where $V$ is the length of vocabulary list. Then we extend each word $x_{i_{j}}$ in utterance $x_{i}$ by concatenating the token itself with turn tag, identity tag and subturn tag. We adopt SEQ2SEQ as the basic architecture and LSTMs as the encoder and decoder networks. LSTMs will encode each extended word $w_{t}$ as a continuous vector $h_{t}$ at each time step $t$. The process can be formulated as following:

where $e(w_{t})$ is the embedding of the extended word $w_{t}$, $W_{f}$, $U_{f}$, $W_{i}$, $U_{i}$, $W_{o}$, $U_{o}$, $W_{g}$, $U_{g}$ and $b$ are learnt parameters.

Though trained on the same dataset, the two imaginators learn different roles independently. So in the same piece of dialogue, we split it into different samples for different imaginators. For example, as shown in Figure 1 and 2 (a), we use utterance (A1, U11, U12) as dialogue history input and U13 as ground truth to train the user imaginator and use utterance (A1, U11, U12, U13) as dialogue history and A2 as ground truth to train the agent imaginator.

During training, the encoder runs as equation 2, and the decoder is the same structured LSTMs but $h_{t}$ will be fed to a Softmax with $W_{v}\in{\textbf{R}^{h\times V}},b_{v}\in{\textbf{R}^{\textbf{V}}}$, which will produce a probability distribution $p_{t}$ over all words, formally:

the decoder at time step t will select the highest word in $p_{t}$, and our imaginator’s loss is the sum of the negative log likelihood of the correct word at each step as follows:

where $N$ is the length of the generated sentence. During inference, we also apply beam search to improve the generation performance.

Finally, the trained agent imaginator and user imaginator are obtained.

The arbitrator module is fundamentally a text classifier. However, in this task, we make the module maximally utilize both dialogue history and ground truth’s semantic information. So we turned the problem of maximizing $Y$ from $X$ in equation (1) to:

where $\textbf{IG}_{agent}$ and $\textbf{IG}_{user}$ are the trained agent imaginator and user imaginator respectively, and $R^{\prime}$ is a selection indicator where $R^{\prime}=1$ means selecting $R_{agent}$ whereas $0$ means selecting $R_{user}$. And Thus we (1) introduce the generation ground truth semantic information and future possible predicted utterances (2) turn the label prediction problem into a response selection problem.

We adopt several architectures like Bi-GRUs, TextCNNs and BERT as the basis of arbitrator module. We will show how to build an arbitrator by taking TextCNNs as an example.

As is shown in Figure 2, the three CNNs with same structure take the inferred responses $R_{agent}$, $R_{user}$ and dialogue history $X$, tags $T$. For each raw word sequence $x_{1},...,x_{n}$, we embed each word as one-hot vector $w_{i}\in\textbf{R}^{V}$. By looking up a word embedding matrix $E\in\textbf{R}^{V\times d}$, the input text is represented as an input matrix $Q\in\textbf{R}^{l\times d}$, where $l$ is the length of sequence of words and $d$ is the dimension of word embedding features. The matrix is then fed into a convolution layer where a filter $\textbf{w}\in\textbf{R}^{k\times d}$ is applied:

where $Q_{i:i+k-1}$ is the window of token representation and the function $f$ is $ReLU$, $W$ and $b$ are learnt parameters. Applying this filter to $m$ possible $Q_{i:i+k-1}$ obtains a feature map:

where $\textbf{c}\in\textbf{R}^{l-k+1}$ for $m$ filters. And we use $j\in\textbf{R}$ different size of filters in parallel in the same convolution layer. This means we will have $m_{1},m_{2},\dots,m_{j}$ windows at the same time, so formally:

, then we apply max-over-time pooling operation to capture the most important feature:

, and thus we get the final feature map of the input sequence.

We apply same CNNs to get the feature maps of $X$, $R_{agent}$ and $R_{user}$:

where function TextCNNs() follows as equations from 6 to 9. Then we will have two possible dialogue paths, $X$ with $R_{agent}$ and $X$ with $R_{user}$, representations $D_{agent}$ and $D_{user}$:

And then, the arbitrator will calculate the probability of the two possible dialogue paths:

Through learnt parameters $W_{4}$ and $b_{4}$, we will get a two-dimensional probability distribution $P$, in which the most reasonable response has the max probability. This also indicates whether the agent should wait or not.

And the total loss function of the whole attribution module will be negative log likelihood of the probability of choosing the correct action:

where $N$ is the number of samples and $Y_{i}$ is the ground truth label of i-th sample.

The arbitrator module based on Bi-GRU and BERT is implemented similar to TextCNNs.

As the proposed approach mainly concentrates on the interaction of human-computer, we select and modify two very different style datasets to test the performance of our method. One is a task-oriented dialogue dataset MultiWoz 2.0 and the other is a chitchat dataset DailyDialogue . Both datasets are collected from human-to-human conversations. We evaluate and compare the results with the baseline methods in multiple dimensions. Table 1 shows the statistics of datasets.

• •

  MultiWOZ 2.0 Budzianowski et al. (2018). MultiDomain Wizard-of-Oz dataset (MultiWOZ) is a fully-labeled collection of human-human written conversations. Compared with previous task-oriented dialogue datasets, e.g. DSTC 2 Henderson et al. (2014) and KVR Eric and Manning (2017), it is a much larger multi-turn conversational corpus and across serveral domains and topics.

• •

  DailyDialogue Li et al. (2017). DailyDialogue is a high-quality multi-turn dialogue dataset, which contains conversations about daily life. Compare to the task-oriented dialogue datasets, the speaker’s behavior will be more unpredictable and complex .

Because the task we concentrate on is different from traditional ones, to make the datasets fit our problems and real life, we modify the datasets with the following steps:

• •

  Drop Slots and Values For task-oriented dialogue, slot labels are important for navigating the system to complete a specific task. However, those labels and accurate values from ontology files will not benefit our task essentially. So we replace all specific values with a slot placeholder in preprocessing step.

• •

  Split Utterances Existing datasets concentrate on the dialogue content, combining multiple sentences into one utterance each turn when gathering the data. In this step, we randomly split the combined utterance into multiple utterances according to the punctuation. And we set a determined probability to decide if the preprocessing program should split a certain sentence.

• •

  Add Turn Tag We add turn tags, subturn tags and role tags to each split and original sentences to (1) label the speaker role and dialogue turns (2) tag the ground truth for training and testing the supervised baselines and our model.

Finally, we have the modified datasets which imitate the real life human chatting behaviors as shown in Figure 1. Our datasets and code will be released to public for further researches in both academic and industry.

To compare with dataset baselines in multiple dimensions and test the model’s performance, we use the overall Bilingual Evaluation Understudy (BLEU) Papineni et al. (2002) to evaluate the imaginators’ generation performance. As for arbitrator, we use accuracy score of the classification to evaluate. Accuracy in our experiments is the correct ratio in all samples.

The hyper-parameter settings adopted in baselines and our model are the best practice settings for each training set. All models are tested with various hyper-parameter settings to get their best performance. Baseline models are Bidirectional Gated Recurrent Units (Bi-GRUs) Chung et al. (2014), TextCNNs Kim (2014) and BERT Devlin et al. (2018).

In Table 2, we show different imaginators’ generation abilities and their performances on the same TextCNN based arbitrator. Firstly, we gathered the results of agent and user imaginators’ generation based on LSTM, LSTM-attention and LSTM-attention with GLOVE pretrained word embedding. According to the evaluation metric BLEU, the latter two models achieve higher but similar results. Secondly, when fixed the arbitrator on the TextCNNs model, the latter two also get the similar results on accuracy and significantly outperform the others including the TextCNNs baseline.

The performances on different arbitrators with the same LSTM-attention imaginators are shown in Table 3. From those results, we can directly compared with the corresponding baseline models. The imaginators with BERT based arbitrator make the best results in both datasets while all ITA models beat the baseline models.

We also present an example of how our model runs in Table 4. Imaginators predict the agent and user’s utterance according to the dialogue history, and then arbitrator selects the user imaginator’s prediction. It is worth noting that the arbitrator generates a high-quality sentence again if only considering the generation effect. However, referring to the dialogue history, it is not a good choice since its semantic is repeated in the last turn by the agent.