A Question Type Driven and Copy Loss Enhanced Framework
for Answer-Agnostic Neural Question Generation

For each sample in our dataset, we have a source sentence $S=(x_{1},x_{2},\ldots,x_{l})$, which is a word sequence and l denotes the number of words. Let $Q=(y_{1},y_{2},\ldots,y_{m})$ to represent the question, which is another word sequence and m is the length. The answer-agnostic QG task can be defined as finding the best $\overline{\emph{Q}}$ that:

	$\displaystyle\overline{Q}$	$\displaystyle=\mathop{\arg\max_{Q}}\log{P}(Q\mid S)$
		$\displaystyle=\mathop{\arg\max_{Q}}\sum_{i=1}^{m}\log{P}(y_{i}\mid S,y_{<i})$

Figure 1 shows the framework of our full question type driven QG model. With the development of pointer network Vinyals et al. (2015), the copy mechanism Gu et al. (2016) has been increasingly more applied to natural language generation task. Thus, our model is based on the general sequence-to-sequence attention model with copy mechanism, which we regard as our baseline. In the next sections, we firstly describe the baseline model and then separately show our question type module and enhanced copy mechanism.

Our baseline method is a sequence-to-sequence attention model with copy mechanism. Let $x_{t}$ represent the $t-th$ word in the source sentence and $e(x_{t})$ is its corresponding embedded vector. A bi-directional LSTM layer Hochreiter et al. (1997) is used to encode the embedded vector sequence:

	$\displaystyle\overrightarrow{u_{t}}$	$\displaystyle=\overrightarrow{LSTM}(\overrightarrow{u_{t-1}},e(x_{t}))$		(1)
	$\displaystyle\overleftarrow{u_{t}}$	$\displaystyle=\overleftarrow{LSTM}(\overleftarrow{u_{t+1}},e(x_{t}))$
	$\displaystyle u_{t}$	$\displaystyle=[\overrightarrow{u_{t}};\overleftarrow{u_{t}}]$

The hidden state at time step $t$ is the concatenation of the forward $\overrightarrow{u_{t}}$ and backward $\overleftarrow{u_{t}}$, which can be represented as $\mathbf{U}={[\overrightarrow{u_{t}};\overleftarrow{u_{t}}]}_{t=1}^{l}$, where $l$ is the number of words in the source sequence.

The decoder is another LSTM network, which generates a new hidden state $h_{t}$, conditioned on the previous state $h_{t-1}$ and previously embedded generated word $e(y_{t-1})$. At the first decoding step, it takes the last encoding hidden state $u_{l}$ and a special token $[SOS]$, which stands for the start of sequence, as input:

	$\displaystyle h_{t}$	$\displaystyle=LSTM(h_{t-1},e(y_{t-1}))$		(2)
	$\displaystyle h_{0}$	$\displaystyle=LSTM(u_{l},e([SOS]))$

Given the encoded state $\mathbf{U}$ and decoder state $h_{t}$, the baseline model calculates a generating distribution of words at each time step $t$, which is calculated as follow:

	$\displaystyle g_{t}$	$\displaystyle=softmax(\mathbf{W^{o}}tanh(\mathbf{W^{t}}[h_{t};v_{t}]))$		(3)
	$\displaystyle v_{t}$	$\displaystyle=\sum_{i=1}^{l}a_{it}u_{i}$
	$\displaystyle a_{it}$	$\displaystyle=\frac{\exp(h_{t}^{\top}\mathbf{W^{b}}u_{i})}{\sum_{j}\exp(h_{t}^{\top}\mathbf{W^{b}}u_{j})}$

where $v_{t}$ is the weighted sum of the encoded representation $\mathbf{U}$, and the attention weight $a_{t}$ is calculated by a bi-linear scoring function and a softmax normalization. $\mathbf{W^{t}}$ and $\mathbf{W^{o}}$ are both trainable parameters, which can be regarded as applying a multi-layer perceptron to the concatenation of hidden state $h_{t}$ and global attention representation $v_{t}$. $\mathbf{W^{b}}$ is also trainable parameter that is applied to calculate attention weights.

Our baseline model also utilizes the copy mechanism. We use the attention score $a_{t}$ obtained from the decoder attention as the copy distribution. The probability of generating a word, $p\_gen$, is calculated as $p\_gen=sigmoid(\mathbf{W^{g}}[h_{t};v_{t}])$. The final distribution is the weighted sum of generating distribution and copy distribution:

	$\displaystyle\hat{g}_{t}$	$\displaystyle=p\_gen\cdot g_{t}$		(4)
	$\displaystyle c_{t}$	$\displaystyle=(1-p\_gen)\cdot a_{t}$

where $\hat{g}_{t}$, $c_{t}$ are the weighted generate distribution and copy distribution, respectively. We use $f_{t}$ to represent the final distribution and $f_{ti}$ is the probability of decoding the $i-th$ word in the vocabulary. Let $w_{i}$ represent the $i-th$ word in the vocabulary. The final distribution is calculated as follow:

$$f_{ti}=\left\{\begin{array}[]{ll}\hat{g}_{ti}+\sum\limits_{k,where\ x_{k}=w_{i}}c_{tk}&w_{i}\in\mathbf{X}\\ \hat{g}_{ti}&otherwise\end{array}\right.$$

Given the training corpus $D$, in which each sample contains a source sentence $S$ and a target question $Q$, the training objective is to minimize the negative log-likelihood of the target questions $L$:

$$L=-\sum_{<S,Q>\in D}\log{P}(Q|S;\theta)$$

(5)

where $\theta$ represents all the parameters of our model.

We propose the question type module for two goals. One is to enable our model to generate multiple types of questions for one source sentence, and the other is to improve the generating performance. Our question type module firstly predicts the most proper type and then uses the embedding of it to help the decoding process.

As for question types, we count the distribution of question types in SQuAD and finally category all the questions into 7 types: what, who, how, where, when, yes/no and others.

The question type prediction is a multi-layer perceptron, which takes the last hidden state of encoder as input to predict the probability distribution of question types, denoted by $\mathbf{T}$:

$$\mathbf{T}=softmax(MLP(u_{l}))$$

(6)

Please note that our model can generate multiple questions for one source sentence. When the number of questions need to be generated is set to $K$, our model will select $K$ question types with the highest probability as output. Consequently, our decoder will decode $K$ times.

$$ty_{1},ty_{2},\ldots,ty_{K}=TopK(\mathbf{T})$$

(7)

At every decoding time, for one of the best $K$ question types, $ty$, we embed it into a question type vector $qt$:

	$\displaystyle qt$	$\displaystyle=Embedding(ty)$		(8)
	$\displaystyle ty$	$\displaystyle\in{ty_{1},ty_{2},\ldots,ty_{K}}$

The embedded question type vector will be used in decoding, and in this way, the question type vector would guide the model to generate questions that follow the pattern of a specific question type. Specially, we use the question type vector $qt$ instead of the embedding of $[SOS]$ token as the input of the decoder at the first decoding step:

$$h_{0}=LSTM(u_{l},qt)$$

(9)

Besides, when calculating the generating distribution $p\_gen$, we concatenate $qt$ with the global attention representation and the decoder hidden state:

		$\displaystyle g_{t}=softmax(\mathbf{W^{o}}tanh(\mathbf{W^{t}}[h_{t};v_{t};qt]))$		(10)
		$\displaystyle p\_gen=sigmoid(\mathbf{W^{g}}[h_{t};v_{t};qt])$

Finally, we train the question type prediction and question generating simultaneously in multi-task learning framework. For each sample $<S,Q>$, we calculate the ground-truth question type distribution $TY$ and we add an additional factor to the loss function, which is the negative log-likelihood of the target questions’ types:

	$\displaystyle L=-\sum_{<S,Q,TY>\in D}[$	$\displaystyle\log{P}(Q|S;\theta)+$		(11)
		$\displaystyle\lambda_{1}\log{P}(TY|S;\theta)]$

where $\lambda_{1}$ is a hyper-parameter to balance two parts.

According to our observation on SQuAD, when creating questions people often (may have to) copy some keywords from the source sentence. So we consider non stop words appearing in both questions and source sentences as keywords, and we assume that these keywords should also occur in the generated sequence. In order to push the model to copy keywords from source sentences, we propose a copy loss to enhance the traditional copy mechanism. For a word $x_{l}$ in the source sentence, we first define a function $cl(copylabel)$ as follow:

$\displaystyle cl(x_{i})=\left\{\begin{array}[]{ll}1&x_{i}\in Q\ and\ x_{i}\notin stopword\\ 0&otherwise\end{array}\right.$

(12)

We believe that at one decoding step, the copy probability of keywords ($cl(x_{i})=1$) should be close to 1. So we define the copy loss as:

		$\displaystyle\hat{c}_{i}=max(c_{1i},c_{2i},\ldots,c_{mi})$		(13)
		$\displaystyle L_{copy}=\sum_{i=1}^{l}cl(x_{i})(\hat{c}_{i}-1)^{2}$

where $c_{ti}$ is the copy probability of the $i-th$ word in the source sentence at the $t-th$ decoding step, computed by Equation 4. Thus, $\hat{c_{i}}$ is the highest copy probability of the $i-th$ word in the source sentence among all the $m$ decoding step. $l$ denotes the number of words in the source sentence.

Finally, we add the copy loss into the total loss:

	$\displaystyle L=$	$\displaystyle-\sum_{<S,Q,TY>\in D}[\log{P}(Q|S;\theta)$		(14)
		$\displaystyle+\lambda_{1}\log{P}(TY|S;\theta)$
		$\displaystyle+\frac{\lambda_{2}}{l}\sum_{i=1}^{l}cl(x_{i})(\hat{c}_{i}-1)^{2}]$

where $\lambda_{2}$ is another hyper-parameter to control the impact of penalty loss.

We conduct experiments on the SQuAD dataset Rajpurkar et al. (2016), which contains more than 70k training samples, 10k development samples and 11k test samples. In either training, development or test dataset, multiple samples might share the same source sentence but with different target questions. But a same source sentence will not appear in different datasets, which ensures the confidentiality of test data.

We adopt subword representations Sennrich et al. (2015) rather than raw words, which can not only reduce the size of vocabulary, increase the training speed, address the problem of out of vocabulary words, but also improve the model performance. By using byte-pair encoding, our vocabulary size is reduced to less than 6k. Due to the vanishing gradient problem in recurrent neural networks Pascanu et al. (2013); hochreiter1998vanishing, we choose 256 for the maximum length of inputs and 50 for the maximum length of target questions.

We adopt a 2-layers bi-directional LSTM for encoding and a 1-layer LSTM for decoding. The number of hidden units is 600, and the dimension of both word embedding and question type embedding is 300. We do not use pre-trained word embedding since we use subword representations rather than word-level representations. The drop rate Krizhevsky et al. (2012) between each layer is 0.3. We firstly use Adam Kingma and Ba (2014) with learning rate of 0.001 for fast training, and after training 5 epochs, the stochastic gradient descent(SGD) with learning rate of 0.01 is used for fine-tuning. We train our model for 15 epochs with mini-batch size of 64. During training, hyper-parameter K is set to 1 and when decoding, we do beam search with a beam size of 4. For hyper-parameters $\lambda_{1}$ and $\lambda_{2}$, we try different settings and choose the best one by observing the descending trend of total loss and ascending trend of BLEU-4 score on the valid set. Finally, both $\lambda_{1}$ and $\lambda_{2}$ are set to 0.1.

We adopt BLEU Papineni et al. (2002), METEOR Denkowski and Lavie (2014) and ROUGE-L Flick (2004) for evaluation, and use the evaluation package released by Chen Chen et al. (2015). BLEU measures the precision of n-grams on a set of references, with a penalty for over short generation. METEOR calculates the similarity between generations and references by considering synonyms, stemming and paraphrases. ROUGE measures the recall of n-grams on the set of references.

We compare our model with the following previous works:

• •

  Seq2Seq Attention: It is a traditional sequence-to-sequence attention model.

• •

  Seq2SeqAtt (Du): This is the first work in AG-QG task, which is a sequence-to-sequence attention model Du et al. (2017).

• •

  Transformer (Scialom): This is the state-of-the-art result for the AG-QG task, which adopts a transformer network with some extension. Scialom et al. (2019).

We do not take Wang et al. (2019) into comparison because their evaluation is done on a different test set and is not accessible.

The experimental results are shown in Table 2. The full version of our model which uses both the question type module and copy loss mechanism obtains the best results on all of metrics, achieving a new state-of-the-art result of BLEU-4 13.90 for the challenging AG-QG task. It outperforms the baseline model with 0.73 points and beats the previous best result by 0.67 points.

We conduct extensive experiments with different model modules, where $k$ is set to 1 in decoding. The results are reported in Table 3.

• •

  Baseline: Our baseline model is a general sequence-to-sequence attention model enhanced with copy mechanism.

• •

  Baseline+Type: It adds the question type module to the baseline model.

• •

  Baseline+CopyLoss: Based on the baseline model, it calculates and minimizes the additional copy loss.

• •

  Baseline+CopyLoss+Type: This is the full version of our proposed model. That is, the question type module is applied to the baseline model and the extra copy loss is also calculated.

• •

  Upper Bound: Since our full model incorporates the question type prediction part, the accuracy of question type prediction will undoubtedly affect the final quality of generation. If the right question type is given for every test sample, we get the upper bound of our model.

We also conduct human evaluation to judge the quality of questions generated by our model and the baseline model, respectively. We take three metrics into consideration: 1) Fluency: it measures whether the question is grammatical; 2) Relevance: it measures whether the question is asked for and highly related to the source sentence; and 3) Answer-ability: it measures whether the generated question can be answered by the information of the source sentence. We randomly selected 100 sentence-question pairs generated by different models, and asked three annotators to score the questions on a $1\!-\!5$ scale (5 for the best). We also exploit the Spearman correlation coefficient to measure the inter-annotator agreement. The results are shown in Table 6. It shows that the consistency among three annotators is satisfying, and our generated questions are better from different perspectives.

Besides, a further two-tailed t-test proves that our generated questions are better than that of the baseline model significantly, with p $<$ 0.001 for every metric.

In order to show the effectiveness of our model, we offer two real samples in the test set, as shown in Table 7. In both samples, the baseline model generates a wrong type of question while our model predicts the right type of question. At the same time, our model successfully copies more key words from the source sentence, which are shown in italics, while baseline model fails. In both samples, our generated questions are more fluent and coherent.

For a given sentence, our question type driven framework offers the model the ability to generate different types of questions. In this case, the parameter $K$ is set to more than 1, and the question type predictor will give $K$ question types with the highest possibility. Then the model automatically decodes $K$ times to generate the best $K$ types of questions. We list a sample in Table 8 with $K=2$ to show the generating diversity of our model, where two types of questions (what and which) are generated from the same input sentence.

Besides, to identify the effect of our model, we visualize the decoder attention, as shown in Figure 2. The two attention maps show the attention points when our model is generating different types (which and what) of questions with respect to the same input sentence, where $x$-axis is the source sentence and $y$-axis is the generated question. Differences between these attention maps prove that our model can attend on different information when generating different types of questions.

From the table, we prove that our model has the ability to generate multiple questions. However, the limitation is also obvious. First, if $K$ is too large, the generated questions of some low probable types are of low quality. Second, since the probability distribution of question types are automatically calculated, the types of generated questions cannot be known beforehand.