Discriminative Reasoning for Document-level Relation Extraction

Formally, given one unstructured document comprised of $N$ sentences $D$=$\{s_{1},s_{2},\cdots,s_{N}\}$, each sentence is a sequence of words $s_{n}=\{s_{n}^{1},s_{n}^{2},\cdots,s_{n}^{J}\}$ with the length $J_{n}$=$|s_{n}|$. The annotations include concept-level entities $\varepsilon=\{e_{i}\}_{i=1}^{P}$ as well as multiple occurrences of each entity under the same phrase of alias $e_{i}=\{m_{i}^{s_{k}}\}_{k=1}^{Q}$ ($m_{i}^{s_{k}}$ denotes the mention of $e_{i}$ which occur in the sentence $s_{k}$) and their entity type (i.e. locations, organizations, and persons). The DocRE aims to extract the relation between two entities in $\varepsilon$, namely $P(r|e_{i},e_{j},D)$. For the simplification of reason skills, we first combine both pattern recognition and commonsense reasoning as the intra-sentence reasoning because they generally perform reasoning inside the sentence. Consequently, the original four kinds of the reasoning skills Yao et al. (2019) are further refined as three reasoning skills: intra-sentence reasoning, logical reasoning, and coreference reasoning. Inspired by Xu et al.’s work, we also use the meta-path strategy to extract reasoning path for each reason skill, thereby representing the above three reasoning skills explicitly. Specifically, meta-paths for different reasoning skills are defined as follows:

1. 1)

   Intra-sentence reasoning path: It is formally denoted as $PI_{ij}$=$m_{i}^{s_{1}}\circ s_{1}\circ m_{j}^{s_{1}}$ for one entity pair {$e_{i}$, $e_{j}$} inside the same sentence $s_{1}$ in the input document $D$. $m_{i}^{s_{1}}$ and $m_{j}^{s_{1}}$ are mentions related to two entities, respectively. “$\circ$” denotes one reasoning step on the reasoning path from $e_{i}$ to $e_{j}$.

2. 2)

   Logical reasoning path: The relation between one entity pair {$e_{i}$, $e_{j}$} from sentences ${s_{1}}$ and ${s_{2}}$ is indirectly established by the occurrence bridge entity $e_{l}$ for the logical reasoning. The reasoning path can be formally as $PL_{ij}$= $m_{i}^{s_{1}}\circ s_{1}\circ m_{l}^{s_{1}}\circ m_{l}^{s_{2}}\circ s_{2}\circ m_{j}^{s_{2}}$.

3. 3)

   Coreference reasoning path: A reference word refers to one of two entities $e_{i}$ and $e_{j}$, which occur in the same sentence as the other entity. We simplify the condition and assume that there is a coreference reasoning path when the entities occur in different sentences. The reasoning path can be formally as $PC$=$m_{i}^{s_{1}}\circ s_{1}\circ s_{2}\circ m_{j}^{s_{2}}$.

Note that there are no entities in the defined reasoning path compare to the meta-path defined in Xu et al.’s work. This difference is mainly due to the following considerations: i) the reason path pays more attention to the mentions and referred sentences; ii) entities generally are contained by mentions; iii) it makes modeling of path reasoning more simple.

Based on the defined reasoning paths, we decompose the DocRE problem into three reasoning sub-tasks: intra-sentence reasoning (IR), logical reasoning (LR), and coreference reasoning (CR). Next, we introduce modeling of three sub-tasks in detail:

In the DocRE task, one entity usually involves multiple relationships which rely on different reasoning types. Thus, the relation between one entity pair may be reasoned by multiple types of reasoning rather than one single reasoning type. Based on the proposed three reasoning sub-tasks, the relation reasoning between one entity pair is regarded as a multi-reasoning classification problem. Formally, we select the reasoning type with max probability to recognize the relation between each entity pair as follows:

In addition, there are often multiple reason paths between two entities for one reasoning type. Thus, the classification probability in Eq.(5) can be rewritten as follows:

where $K$ is the number of reasoning paths for one reasoning skill, which is the same to each reasoning skill for simplicity. Note that all the entity pairs have at least one reasoning path from one of three defined reasoning sub-tasks. When the number of reasoning paths is greater than $K$ for one reasoning sub-task, we choose the $K$ first reasoning paths, otherwise we use the actual reasoning paths.

Formally, the embedding of each word $\textbf{w}_{e}$ is concatenated with the embedding of its entity type $\textbf{w}_{t}$ and the embedding of its coreference $\textbf{w}_{c}$ as the representation of word b=[$\textbf{w}_{e}$:$\textbf{w}_{t}$:$\textbf{w}_{c}$]. These sequences of word representations are in turn fed into a bidirectional long short-term memory (BiLSTM) to vectorize the input document D={$\textbf{H}^{1}$, $\textbf{H}^{2}$, $\cdots$, $\textbf{H}^{N}$}, where $\textbf{H}^{n}$ = $(\textbf{h}^{n}_{1},\textbf{h}^{n}_{2},\dots,\textbf{h}^{n}_{J_{n}})$ and $\textbf{h}^{j}_{i}$ denotes the hidden representation of the $i-th$ words of the $j-th$ sentence in the document. Similar to Zeng et al.’s work, we construct a heterogeneous graph which contains sentence node and mention node. There are four kinds of edges in the heterogeneous graph: sentence-sentence edge (all the sentence nodes are connected), sentence-mention edge (the sentence node and the mention node which resides in the sentence ), mention-mention edge (all the mention nodes which are in the same sentence) and co-reference edge (all the mention nodes which refer to the same entity). Then we apply the graph-based DocRE method Zeng et al. (2020) to encode the heterogeneous graph, based on which the heterogeneous graph context representation (HGCRep) are learned. The HGCRep of each mention node and sentence node $\textbf{g}_{n}$ is formally denoted as:

where $\textbf{g}_{n}\in\mathbb{R}^{d_{1}}$ and “:” is the concatenation of vectors and each of {$\textbf{p}^{1}_{n},\textbf{p}^{2}_{n},\cdots,\textbf{p}^{l-1}_{n}$} is learned by the multi-hop graph convolutional network Zeng et al. (2020) and $\textbf{v}_{n}$ is the initial representation of the $n$-th node extracted from D. Finally, there is a heterogeneous graph representation G=$\{\textbf{g}_{1},\textbf{g}_{2},\cdots,\textbf{g}_{N}\}$ including each mention nodes and sentence nodes.

In the DocRE task, these reasoning skills heavily rely on the original document context information rather than the heterogeneous graph context information. Thus, the existing advanced DocRE models use syntactic trees or heuristics rules to extract the context information (i.e., entities, mentions, and sentences) that is directly related to the relation between entity pairs. However, this approach destroys the original document structure, which is weak in modeling the reasoning between two entities for the DocRE task. Therefore, we use the self-attention mechanism Vaswani et al. (2017) to learn a document-level context representation (DLCRep) $\textbf{c}_{n}$ for one mention based on the vectorized input document D:

where $\textbf{c}_{n}\in\mathbb{R}^{d_{2}}$ and $\{\textbf{K},\textbf{V}\}$ are key and value matrices that are transformed from the vectorized input document D using a linear layer. Here, inspired by relation learning Baldini Soares et al. (2019), we use the hidden state of the head word in one mention or one sentence to denote them for simplicity.

In this section, we use the concatenation operation to model the reasoning step on the reasoning path, thereby modeling the defined reasoning paths in Section 2.1 as the corresponding reasoning representations as follows:

1. 1)

   For the intra-sentence reasoning path, both HGCReps and DLCReps of two mentions are concatenated in turn as a reasoning representation:

   $\displaystyle\alpha_{ij}=[\textbf{g}_{m_{i}^{s_{1}}}:\textbf{g}_{m_{j}^{s_{1}}}:\textbf{c}_{m_{i}^{s_{1}}}:\textbf{c}_{m_{j}^{s_{1}}}],$

   (9)

   where $\alpha_{ij}\in\mathbb{R}^{2d_{1}+2d_{2}}$ and “:” is the concatenation of vectors.

2. 2)

   For the logical reasoning path, both HGCReps of mention $m_{i}^{s_{1}}$ and $m_{j}^{s_{2}}$ and DLCReps of two mention pair $(m_{i}^{s_{1}},m_{l}^{s_{1}})$ and $(m_{j}^{s_{2}},m_{l}^{s_{2}})$ are concatenated as their reasoning representation:

   	$\displaystyle\beta_{ij}$	$\displaystyle=[\textbf{g}_{m_{i}^{s_{1}}}:\textbf{g}_{m_{j}^{s_{1}}}:$		(10)
   		$\displaystyle\textbf{c}_{m_{i}^{s_{1}}}+\textbf{c}_{m_{l}^{s_{1}}}:\textbf{c}_{m_{j}^{s_{2}}}+\textbf{c}_{m_{l}^{s_{2}}}],$

   where $\beta_{ij}\in\mathbb{R}^{2d_{1}+2d_{2}}$.

3. 3)

   For the coreference reasoning path, we connect both HGCReps of two mentions and DLCReps of two sentences are are concatenated in turn as their reasoning representation:

   $$\gamma_{ij}=[\textbf{g}_{m_{i}^{s_{1}}}:\textbf{g}_{m_{j}^{s_{2}}}:\textbf{c}_{s_{1}}:\textbf{c}_{s_{2}}]$$

   (11)

   where $\gamma_{ij}\in\mathbb{R}^{2d_{1}+2d_{2}}$ and both $\textbf{c}_{s_{2}}$ and $\textbf{c}_{s_{2}}$ denote DLCReps for two sentences ${s_{1}}$ and ${s_{2}}$.

The learned reasoning representations $\alpha_{ij}$, $\beta_{ij}$, and $\gamma_{ij}$ is as the input to classifier to compute the probabilities of relation between $e_{i}$ and $e_{j}$ entities by a multilayer perceptron (MLP) respectively:

Similarly, when there are multiple reasoning paths between two entities for one reasoning type in Eq.6, Eq.12 is rewritten as follows:

Also, the binary cross-entropy is used as training objection, which is the same as the advanced DocRE model Yao et al. (2019).

The proposed methods were evaluated on a large-scale human-annotated dataset for document-level relation extraction Yao et al. (2019). DocRED contains 3,053 documents for the training set, 1,000 documents for the development set, and 1,000 documents for the test set, totally with 132,375 entities, 56,354 relational facts, and 96 relation types. More than 40% of the relational facts require reading and reasoning over multiple sentences. For more detailed statistics about DocRED, we recommend readers to refer to the original paper Yao et al. (2019).

Following settings of Yao et al.’s work, we used the GloVe embedding (100d) and BiLSTM (128d) as word embedding and encoder. The number of the reasoning path for each task is set to 3. The learning rate was set to 1e-3 and we trained the model using AdamW Loshchilov and Hutter (2019) as the optimizer with weight decay 0.0001 under Pytorch Paszke et al. (2017). For the BERT representations, we used uncased BERT-Based model (768d) as the encoder and the learning rate was set to $1e^{-5}$. For evaluation, we used F1 and Ign F1 as the evaluation metrics. Ign F1 denotes F1 score excluding relational facts shared by the training and development/test sets. In particular, the predicted results were ranked by their confidence and traverse this list from top to bottom by $F1$ score on development set, and the score value corresponding to the maximum $F1$ is picked as threshold $\theta$. The hyper-parameter for the number of reasoning paths was tuned based on the development set. In addition, the results on the test set were evaluated through CodaLab¹¹1https://competitions.codalab.org/competitions/20717. Once a model is trained, we get the confidence scores for every triple example (subject,object,relation) as Eq.(12). We rank the predicted results by their confidence and traverse this list from top to bottom by F1 score on development set, the score value corresponding to the maximum F1 is picked as threshold $\theta$. This threshold is used to control the number of extracted relational facts on the test set.

We reported the results of the recent graph-based DocRE methods as the comparison systems: GAT Veličković et al. (2018), GCNN Sahu et al. (2019), EoG  Christopoulou et al. (2019), AGGCN Guo et al. (2019), LSR Nan et al. (2020), GAIN Zeng et al. (2020), and HeterGASN-RecXu et al. (2021). Moreover, pre-trained models like BERT Devlin et al. (2019) has been shown impressive result on the DocRE task. Therefore, we also reported state-of-the-art graph-based DocRE models with pre-trained BERT_base model, including Two-Phase+BERT_base Wang et al. (2019), LSR+BERT_base Nan et al. (2020), GAIN+BERT_base Zeng et al. (2020), HeterGASN-Rec+BERT_base Xu et al. (2021), and ATLOP-BERT_base Zhou et al. (2021).

Table 2 presents the detailed results on the development set and the test set for the DocRE dataset. First, the proposed DRN model significantly outperformed the existing graph-based DocRE systems. Second, the proposed DRN model was superior to all the existing graph-based DocRE systems on the test set, validating that modeling reasoning discriminatively is more beneficial to DocRE than the original universal neural network way. Meanwhile, it also outperformed the best HeterGSAN-Rec model by 1.10 points in terms of F1, validating the effectiveness of our discriminative reasoning method. Third, for the comparisons with a pre-trained language model (BERT_base), F1 scores of the proposed DRN+BERT_base model was higher than that of the existing graph-based DocRE ATLOP+BERT model systems with BERT_base on the test set. In particular, our method (F1 61.37) was superior to the existing best ATLOP+BERT model (F1 61.30) in terms of F1, which is a new state-of-the-art result on the DocRE dataset.

To evaluate the effect of the number of reasoning path $K$ in Eq.6, we reported the results for the different number of reasoning path $K$, as shown in Table 3. When $K$ increased from 1 to 3, F1 scores of the proposed DRN model gradually improved from 55.81 to 56.33 on the test set and the percentage of covered reasoning paths reaches 90.40%. As the hyper-parameter $K$ continues to increase, F1 scores began to drop on the dev and test sets. On the one hand, the reason may be that the reasoning information provided by too many reasoning paths is duplicated, even noises in the remaining 9.60% reasoning paths. On the other hand, the hyper-parameter $K$=3 can make the proposed DRN gain the highest F1 score on the dev and test sets. Therefore, we set the hyper-parameter $K$ to three in our main results in Table 2.

In the proposed DRN model, we model different reasoning tasks discriminatively using HGCRep and DLCRep, and we choose the highest scores as the final results. Instead of using the discriminative reasoning framework, previous work averaged the mention representation (HGCRep or DLCRep) to get the entity representation and concatenate the two entity representation to classify the relation, which we denote as Uniform model. Table 4 shows ablation experiments of the framework and different reasoning context on the test set. It is noted that Uniform model with the discriminative reasoning framework is our DRN model. First, the DocRE models benefit from our discriminative reasoning framework no matter what the reasoning context is used. Specially, the F1 score of the model with the framework was averagely 1.21 points superior to the Uniform model on the test set no matter what context representation is used, which illustrated the effectiveness of the framework.

Second, when we gradually remove DLCRep and HGCRep from the Uniform and the proposed DRN model, both of the model’s performance drops. Specially, F1 scores of DRN without DLCRep dropped by 1.32 while F1 scores of DRN without HGCRep dropped by 2.20 respectively. This indicates that both DLCRep and HGCRep play an important role in capturing the information of nodes on the reasoning paths. When removing both of DLCPeps and HGCReps from the DRN model, the model was degraded to the BiLSTM model with our discriminative reasoning framework. Obviously, F1 scores drastically decreased on the test sets, confirming the necessity of learning DLCRep and HGCRep for modeling reasoning discriminatively.

In this section, we first showed the percent of all entity pairs (396,790) and entity pair with relation (12,332) on the dev set selected for three defined reasoning tasks through max operation in Eq.(12), as shown in Figure 3(a). For example, IR, LR, and CR are the intra-Sentence reasoning task, the logical reasoning task, and the coreference reasoning task, respectively. The percentages of IR, LR, and CR which is selected for all the entity pair are 19.12%, 19.17%, and 61.71% for all entity pairs, respectively. This indicates that our defined three reasoning skills can completely cover all entity pairs regardless of whether these entity pairs have relationships or not. Also, the percentages of IR, LR, and CR are 47.58%, 13.91%, and 38.51% for entity pairs with relation, respectively. This is consistent with the statistical result in the Yao et al.’s work that more than 40.7% relational facts can only be extracted from multiple sentences, validating that our method can model different reasoning skills discriminatively on the DocRE dataset.

Moreover, Figure 3(b) showed the results of HerterGSAN-Rec (abbreviated as Rec), GAIN, and our DRN models on three different reasoning tasks. As seen, F1 scores of the proposed DRN model are higher than that of Rec and GAIN models over all three tasks. This means that modeling reasoning types explicitly can effectively advance the DocRE. For all DocRE models, F1 scores of LR task and CR task were far inferior to that of IR task, which is consistent with the intuitive perception that the inter-sentence reasoning is more difficult than the intra-sentence reasoning.

To further show the selected different reasoning types in Eq.(12), we randomly sampled 72 documents from the dev set which contain 916 relation instances, and we ask three human to annotate the reasoning types of all the entity pairs with relation in the sampled document according to three defined reasoning types, including the intra-sentence reasoning, the logical reasoning, and the coreference reasoning (The annotation data can be found in https://github.com/xwjim/DRN). Table 5 shows the number and F1 scores of each selected reasoning types on the sampled 72 documents. As seen, F1 scores of IR, LR, and CR are 79.95%, 38.77%, and 44.87%, respectively, indicating that modeling reasoning discriminatively is working during selecting of reasoning paths in Eq.(12). Also, our method is the capacity of recognizing not only the intra-sentence reasoning but also the intra-sentence reasoning. In addition, there is a certain percentage of the mistakenly selected reasoning types, indicating that our method may have more room for improvement in the future.

Figure 4 shows the relation classification about two entity pairs for our DRN model. For the first entity pair {“Superman”} and {“May 26,2002”}, there are reasoning paths for Task2 and Task3, and their scores are 1.7604, and 0.2841,respectively As a result, Task2 was used to predict the relation “{publication date}” between {“Superman”} and {“May 26,2002”} correctly. Meanwhile, the selection of Task2 is consistent with the ground-truth logical reasoning type. Moreover, the above reasoning processing is also similar to the entity pair {“The Eminem show”} and {“Eminem”} with three reasoning types.