JointLK: Joint Reasoning with Language Models and Knowledge Graphs for Commonsense Question Answering

The CSQA task in this paper is a multiple-choice problem with some answer choices. Given a commonsense question $q$ and a set of answer choices $\{a_{1},a_{2},...,a_{n}\}$, our task is to measure the plausibility score between $q$ and each answer choice $a$ then select the answer with the highest plausibility score. In general, questions do not contain any reference to answer choices, so the external knowledge graph provides the necessary background knowledge. We extract from the external KG a subgraph $g=(V,R)$ with the guidance of question and choice. Here V is a subset of entity nodes retrieved from the external KG. $E\subseteq{V\times R\times V}$ is the set of edges that connect nodes in $V$, where $R$ is a set of relations types. We describe the detailed extraction process in Appendix A.

We follow baselines to use pre-trained language models to encode the query $\{w_{i}\}_{i=1}^{M}$ (question and choice) into a sequence of vectors $\{q_{i}^{0}\}_{i=1}^{M}$:

$$\{\widetilde{q}_{1}^{0},...,\widetilde{q}_{M}^{0}\}=\mathbf{Enc_{LM}}\left(\{w_{1},...,w_{M}\}\right)$$

(1)

Here $\{\widetilde{q}_{i}^{0}\}_{i=1}^{M}\in{\mathbb{R}^{T}}$ is the last hidden layer vector of each token in the query. Then we feed the representation of tokens into a non-linear layer so that the text representation space is aligned to the entity representation space:

$$q_{i}^{0}=\sigma\left(f_{s}\left(\widetilde{q}_{i}^{0}\right)\right)$$

(2)

where $f_{s}:\mathbb{R}^{T}\rightarrow\mathbb{R}^{D}$ is a linear transformation, and $\sigma$ is the activation function. The representations of tokens $\mathbf{Q}^{0}=\{{q}_{i}^{0}\}_{i=1}^{M}\in{\mathbb{R}^{D}}$ will be provided to the joint reasoning module for further interaction with the graph entities representations.

After obtaining token representations by the query encoder, we further model the subgraph to obtain entity representations. First, We use the BERT model with average pooling to get the initial representation for each entity $\mathbf{X^{0}}=\{x_{i}^{0}\}_{i=0}^{|V|}\in{\mathbb{R}^{D}}$. Then, we apply GNN Layer to update node representation through iterative message passing between neighbors on the graph, while GNN is built on the RGAT Wang et al. (2020a) and is a simplification of Yasunaga et al. (2021). For brevity, we formulate the entire computation in one layer as:

$$\{\widetilde{x}_{1}^{l},...,\widetilde{x}_{|V|}^{l}\}=\mathbf{GNN}(\{x_{1}^{l-1},...,x_{|V|}^{l-1}\})$$

(3)

The output representation $x_{i}^{l}$ is computed by

	$\displaystyle\hat{\alpha}_{ji}$	$\displaystyle=(x_{i}^{l-1}W_{q})(x_{j}^{l-1}W_{k}+r_{ji})^{T},$		(4)
	$\displaystyle{\alpha}_{ji}$	$\displaystyle=\mathbf{softmax}(\hat{\alpha}_{ji}/{\sqrt{D}}),$		(5)
	$\displaystyle\hat{x}_{i}^{l-1}$	$\displaystyle=\sum_{j\in N_{i}\cup\{i\}}{{\alpha}_{ji}(x_{j}^{l-1}W_{v}+r_{ji})},$		(6)
	$\displaystyle\widetilde{x}_{i}^{l}$	$\displaystyle=\mathbf{LayerNorm}(x_{i}^{l-1}+\hat{x}_{i}^{l-1}W_{o})$		(7)

where matrices $W_{q},W_{k},W_{v},W_{o}\in\mathbb{R}^{D\times D}$ are trainable parameters, $N_{i}$ is the neighbor of node $i$. $r_{ji}=\psi(e_{ji},u_{j},u_{i})$ is the relation feature vector, where $e_{ji}$ is a one-hot vector denoting the relation type of the edge $(j,i)$ and $u_{j},u_{i}$ are one-hot vectors denoting the node types of $j$ and $i$. The following joint reasoning module will further fuse $\widetilde{x}_{i}^{l}$ and $q_{i}^{l-1}$ to obtain their updated representations.

To reduce the gap of query and knowledge graph features, we fuse them in the joint reasoning module by the dense bidirectional attention mechanism that connects two encoding layers of query and knowledge graph and captures the fine-grained interplay between them.

The module takes the query and KG representations $\mathbf{Q}$ and $\mathbf{X}$ as inputs and then outputs their updated versions. We denote the inputs to the joint reasoning module in the l-st fusion layer by $\mathbf{Q}^{l-1}=\{q_{i}^{l-1}\}_{i=1}^{M}$ and $\mathbf{\widetilde{X}}^{l}=\{\widetilde{x}_{i}^{l}\}_{i=1}^{|V|}$. Given $q_{i}^{l-1}$ and $\widetilde{x}_{i}^{l}$, an affinity matrix is first constructed via:

$$S_{ij}^{l}=W_{S}^{T}[q_{i}^{l-1};\widetilde{x}_{j}^{l};q_{i}^{l-1}\circ\widetilde{x}_{j}^{l}]$$

(8)

where $W_{S}^{T}$ is a learnable weight matrix, $\circ$ is elementwise multiplication, [;] is vector concatenation across row. We normalize $S_{ij}^{l}$ in row-wise to derive KG-to-LM attention maps on query tokens conditioned by each entity in KG as

$$S_{q_{i}}^{l}=\mathbf{softmax}\left(S_{ij}\right)$$

(9)

and also normalize $S_{ij}^{l}$ in column-wise to derive LM-to-KG attention maps on entities conditioned by each query token as

$$S_{x_{j}}^{l}=\mathbf{softmax}\left(S_{ij}^{T}\right)$$

(10)

The attended representations are computed as follows:

$$\hat{q}_{ij}=q_{i}^{l-1}\otimes S_{q_{i}}^{l},\\ \hat{x}_{ij}=\widetilde{x}_{j}^{l}\otimes S_{x_{j}}^{l}$$

(11)

where $\otimes$ represents matrix multiplication. The attended features are fused with the original features of the other modality by concatenation and then compressed to low-dimensional space by:

	$\displaystyle q_{i}^{l}$	$\displaystyle=W_{Q}[q_{i}^{l-1};\hat{x}_{ij};q_{i}^{l-1}\circ\hat{x}_{ij};q_{i}^{l-1}\circ\hat{q}_{ij}],$		(12)
	$\displaystyle\bar{x}_{j}^{l}$	$\displaystyle=W_{X}[\widetilde{x}_{j}^{l};\hat{q}_{ij};\widetilde{x}_{j}^{l}\circ\hat{q}_{ij};\widetilde{x}_{j}^{l}\circ\hat{x}_{ij}]$		(13)

where $W_{Q},W_{X}$ are learnable weights. Then the updated query representation $\mathbf{Q}^{l}=\{q_{i}^{l}\}_{i=1}^{M}$ will be input to the next $l$-th stacked JointLK layer of to continue participating in joint reasoning, and the updated KG representation $\mathbf{\bar{X}}^{l}=\{\bar{x}_{i}^{l}\}_{i=1}^{|V|}$ will be input to the next module of the current JointLK layer for pruning.

In Equation 10, the LM-to-KG attention value implies the importance of different nodes in the subgraph for question answering. Inspired by SAGPool Lee et al. (2019), under the guidance of query, we retain relevant nodes and cut out irrelevant nodes according to the LM-to-KG attention. Then, We define a hyperparameter, the Retention ratio $K\in(0,1]$, which determines the number of nodes to be retained. We choose the top $\left\lceil K\cdot|V|\right\rceil$ nodes according to the value of LM-to-KG attention:

	$$\displaystyle idx=\mathbf{top-rank}\left(Z,\left\lceil K\cdot|V|\right\rceil\right),$$		(14)
	$$\displaystyle Z_{mask}=Z_{idx}$$		(15)

where top-rank is a function that returns the index of top $\left\lceil K\cdot|V|\right\rceil$ value, $\cdot_{idx}$ is an indexing operation, and $Z_{mask}$ is corresponding attention mask. Next, the subgraph is formed by pooling out the less essential entity nodes as:

$$\begin{split}\mathbf{X}^{l}&=\mathbf{\bar{X}}_{idx,:}^{l}\odot Z_{mask},\\ \mathbf{A}^{l}&=\mathbf{\bar{A}}_{idx,idx}^{l}\end{split}$$

(16)

where $\mathbf{\bar{X}}_{idx,:}^{l}$ is the row-wise indexed representation matrix of $\mathbf{\bar{X}}^{l}$, $\odot$ is the broadcasted elementwise product, and $\mathbf{\bar{A}}_{idx,idx}^{l}$ is the row-wise and col-wise of indexed adjacency matrix. $\mathbf{X}^{l}=(x_{1}^{l},x_{2}^{l},\ldots,x_{\left\lceil k|V|\right\rceil}^{l})$, $\mathbf{A}^{l}$ and $\left\lceil K\cdot|V|\right\rceil$ are the representation matrix, the adjacency matrix and the number of graph nodes in the next JointLK layer.

After N layers of iteration, we finally obtain the query representation $\mathbf{Q}^{N}$ that fuses knowledge information and the graph representation $\mathbf{X}^{N}$ that fuses question information. We compute the score of $a$ being the correct answer as:

$$p=\left(a|q\right)=\mathbf{MLP}\left([{s};{g}]\right)$$

(17)

where $s$ is the mean pooling of $\mathbf{Q}^{N}$, and $g$ is the attention-based pooling of $\mathbf{X}^{N}$. We get the final probability by normalize all question-choice pairs with softmax.

We evaluate our model on two typical commonsense question answering datasets CommonsenseQA Talmor et al. (2019) and OpenBookQA Mihaylov et al. (2018). CommonsenseQA is a 5-way multiple-choice question answering dataset that requires commonsense for reasoning and contains 12,102 questions. We experiment and report the accuracy on the in-house dev (IHdev) and test (IHtest) splits used by Lin et al. (2019), and report the accuracy of our final system on the official test set. OpenBookQA is a 4-way multiple choice question answering dataset that requires reasoning with elementary science knowledge. It contains 5,957 questions along with an open book of scientific facts. We use the official data split.

Following previous work Yasunaga et al. (2021), we use ConceptNet Speer et al. (2017), a commonsense knowledge graph, as our structured knowledge source for both of the above tasks. Given each query, we follow the preprocessing steps described in Feng et al. (2020) to retrieve the subgraph from ConceptNet, and the max hop size is 3 (see Appendix A for the detail). We use cross-entropy loss and RAdam optimizer Liu et al. (2020). In training, we set the maximum input sequence length to text encoders to 100, batch size to 128, and perform early stopping. We set the dimension (D = 200) and number of layers (N = 5) of our GNN module, with dropout rate 0.2 applied to each layer Srivastava et al. (2014). We use separate learning rates for the LM encoder and the graph encoder. We choose the LM encoder learning rate from$\{1\times{10}^{-5},\ 2\times{10}^{-5},\ 3\times{10}^{-5}\}$, and choose the graph encoder learning rate from$\{1\times{10}^{-3},\ 2\times{10}^{-3}\}$. Each model is trained using one GPU (Tesla_v100-sxm2-16gb), which takes 20 hours on average.

Although text corpus can provide complementary knowledge except for knowledge graphs, our model focuses on improving the use of KG and the joint reasoning between LM and KG, so we choose LM and LM+KG as the comparison methods.

To investigate the role of KGs, we compare with the benchmark model RoBERTa-large Liu et al. (2019) for CommonsenseQA, and compare with RoBERTa-large and AristoRoBERTa Clark et al. (2020) for OpenBookQA. For LM+KG methods, they share a similar high-level framework with our methods, that is, LM is used as a text encoder, GNN or RN is used as a KG encoder, but the way of using knowledge or reasoning is different: (1) Relationship network (RN) Santoro et al. (2017), (2) RGCN Schlichtkrull et al. (2018), (3) GconAttn Wang et al. (2019), (4)KagNet Lin et al. (2019) and (5)MHGRN Feng et al. (2020), (6) QA-GNN Yasunaga et al. (2021). (1), (2) and (3) are the relational perception GNNs for KGs, and (4), (5) and (6) are further model paths in KGs. To be fair, we use the same LM for all comparison methods.

The results on CommonsenseQA in-house split dataset and official test dataset are shown in Table 1 and Table 2. The results on OpenBookQA test dataset and leaderboard are shown in Table 3 and Table 4. We can observe that JointLK performs best among all fine-tuned LMs and existing LM+KG models. On CommonsenseQA, our model’s test performance improves by 5.74% over fine-tuned LMs and 1.02% over the prior best LM+KG model, QA-GNN. On OpenbookQA, our model’s test performance improves by 6.52% over fine-tuned AristoRoBERTa, and 2.15% over QA-GNN. Additionally, we also submit our best model to the leaderboards, and our JointLK (with the text encoder being RoBERTa-large) ranks first among comparable approaches. Compared with the previous best model MHGRN and QA-GNN, the boost over them suggests the effectiveness of our proposed joint reasoning between LM and KG and the dynamic pruning mechanism.

In particular, we do not compare with the higher ranking models on the leaderboard, such as unified QA Khashabi et al. (2020), Albert + DESC-KCR Xu et al. (2021), because they either use a stronger text encoder or use additional data resources, while our model focuses on improving the joint reasoning between LM and KG.

We further conduct in-depth analyses to investigate the effectiveness of different components in our model. We show the accuracy of JointLK on the CommonsenseQA IHdev set.

Considering the overall performance improvement of our model on these two datasets, we analyze whether the improvement is reflected in questions that require more complex reasoning, such as questions with negation and complex questions with more entities. We compare our model with the prior best LM+KG model, QA-GNN in Table 6.

We aim to interpret JointLK’s reasoning process by analyzing the pruning of the knowledge subgraph. Figure 4 shows an example from CommonsenseQA where our model correctly answers the question and finally retains reasonable reasoning paths by pruning the subgraph. The flow from (a) to (b) to (c) represents the recursive pruning of the subgraph according to the LM-to-KG attention weight at each GNN update layer. From (a) to (b), although the nodes wood and burn bridge the reasoning gap between question entity and answer entity, their semantics are very different from the question. From (b) to (c), “$play\_guitar\stackrel{{\scriptstyle usedfor}}{{\longrightarrow}}fun$" and “$fun\stackrel{{\scriptstyle relatedto}}{{\longrightarrow}}gas\stackrel{{\scriptstyle relatedto}}{{\longrightarrow}}singe$" are both reasonable, but the former is related to the semantics of the question, and the latter is not. Two paths are reserved in (c), “$play\_guitar\stackrel{{\scriptstyle hassubevent}}{{\longrightarrow}}take\_lessons\stackrel{{\scriptstyle hassubevent}}{{\longrightarrow}}dance\stackrel{{\scriptstyle relatedto}}{{\longrightarrow}}singing"$ and “$play\_guitar\stackrel{{\scriptstyle relatedto}}{{\longrightarrow}}action\stackrel{{\scriptstyle relatedto}}{{\longrightarrow}}singer\stackrel{{\scriptstyle relatedto}}{{\longrightarrow}}singing$". These two paths describe two possible scenarios that support answering the question.

In order to understand why our model fails in some cases, we randomly select 100 error cases and group them into several categories. There are three main types of errors, and we show some examples in the Appendix C.

The above three types of errors show that selecting complete, accurate, and context-sensitive knowledge is vital for more effective KG-augmented models.