Applying a Generic Sequence-to-Sequence Model for Simple and Effective Keyphrase Generation

Md Faisal Mahbub Chowdhury, Gaetano Rossiello, Michael Glass,
Nandana Mihindukulasooriya Alfio Gliozzo
IBM Research AI, Yorktown Heights, NY, USA
{mchowdh, mrglass, gliozzo}@us.ibm.com,
{gaetano.rossiello, nandana.m}@ibm.com

Abstract

In recent years, a number of keyphrase generation (KPG) approaches were proposed consisting of complex model architectures, dedicated training paradigms and decoding strategies. In this work, we opt for simplicity and show how a commonly used seq2seq language model, BART, can be easily adapted to generate keyphrases from the text in a single batch computation using a simple training procedure. Empirical results on five benchmarks show that our approach is as good as the existing state-of-the-art KPG systems, but using a much simpler and easy to deploy framework.

1 Introduction

Keyphrases are a collection of salient terms that summarize a document. Keyphrase generation (KPG) is unique from other NLG tasks since the target model prediction is to generate multiple sequences (i.e. a set of multi-word phrases) rather than a single sequence (for other tasks such as summarization and translation) Meng et al. (2021).

As pointed out by Meng et al. (2021), the ever-growing collection of KPG studies, albeit novel, are overwhelming from their model design to hyper-parameter selection. It is worth to mention that, as of now, none of the existing KPG approaches achieves outright state-of-the-art results on all of the five widely used keyphrase benchmarks.

Çano and Bojar (2019) experimented with popular text summarization neural architectures on several datasets. They found, despite using large quantities of data and many days of computation, the advanced deep learning based summarization models could not produce better keyphrases than the specifically designed keyphrase generation and extraction models. In this work, we revisit the above finding regarding the effectiveness of summarization models for keyphrases.

Document (only abstract shown below)
In order to monitor a region for traffic traversal, sensors can be deployed to perform collaborative target detection. Such a sensor network achieves a certain level of detection performance with an associated cost of deployment. This paper addresses this problem by proposing path exposure as a measure of the goodness of a deployment and presents an approach for sequential deployment in steps. It illustrates that the cost of deployment can be minimized to achieve the desired detection performance by appropriately choosing the number of sensors deployed in each step.
Generated present keyphrases: target detection, path exposure, sensor network, collaborative target detection, sequential deployment
Generated absent keyphrases: traffic monitoring, sensor networks, sensor deployment, deployment strategy

Table 1: An example of keyphrases generated using our fine-tuned model of BART-large Lewis et al. (2020a). The document is selected from the SemEval dataset.

Recently, some proposed works Lewis et al. (2020c); Petroni et al. (2021) show that a multitude of downstream NLP tasks (e.g. question answering, fact checking, dialog) can be effectively addressed using a unified generative framework without designing ad-hoc architectures for individual task. Moreover, standard seq2seq (aka encoder-decoder) models Lewis et al. (2020a); Raffel et al. (2020) have been successfully applied to extract structured information from text, such as relation linking Rossiello et al. (2021), text-to-table Wu et al. (2021), text-to-RDF Agarwal et al. (2020); Guo et al. (2020), intent classification Ahmad et al. (2021), and slot filling Glass et al. (2021). So far, there is no empirical evidence showing effectiveness of such a simple generative paradigm for KPG.

Inspired by the aforementioned literature, this study examines whether a standard text summarization neural architecture can be used for KPG. We demonstrate that, even with default hyper-parameters and limited fine tuning, our system can yield results on par (in some cases even better) with the state-of-the-art systems that have specifically designed architectures for KPG. We hope this study will have a substantial impact on implementing real-world KPG systems, therefore facilitating simplification of the development and deployment process.

2 Related Work

KPG maps a sequence (the text) to a set of sequences (the keyphrases). This task is typically transformed to a seq2seq setting either through One2One or One2Seq. In One2One, during training each keyphrase is paired with the input text as the target output. Then at test time the different keyphrases are taken from the different beams of the generation process. In contrast, One2Seq uses the list of keyphrases as the target text during training. At test time the keyphrases are taken from the top-scoring beam or merged from the top-k beams.

Meng et al. (2017) was the first to propose a KPG approach, called CopyRNN. They analyzed a number of benchmarks and found a considerable number (ranges between 44–58%) of keyphrases do not appear in the corresponding documents. Such keyphrases are labelled as absent keyphrases as opposed to the present keyphrases that appear in the documents. CopyRNN is based on One2One training paradigm. Several subsequent studies Chen et al. (2018); Zhao and Zhang (2019) are extensions of CopyRNN framework. Some Ye and Wang (2018); Chen et al. (2019b); Kim et al. (2021) exploited document structure information to improve KPG performance.

Yuan et al. (2020) implemented an approach based on One2Seq paradigm to generate diverse keyphrases and control number of outputs by manipulating decoder hidden states.

Unlike the aforementioned studies, Chen et al. (2020) and Chan et al. (2019) proposed reinforcement learning (RL) based KPG approaches. Chen et al. (2019a) proposed multi-task learning framework to jointly learn extractive and generative model for keyphrases and exploited information retrieval. They did not report separated results for present and absent keyphrases.

Ye et al. (2021) proposed a graph neural network based approach that capture explicit knowledge from a predefined index (e.g., the training set) that is similar to the input document. This idea is intuitively similar to what was proposed by Chen et al. (2019a). Since we do not use explicit knowledge from other documents, these results are not comparable to ours.

3 Proposed Approach

We exploit BART Lewis et al. (2020b), a denoising self-supervised autoencoder, for pretraining seq2seq models. It is a standard seq2seq framework that has been used to produce state-of-the-art results on a variety of NLG tasks including dialogue, question answering, abstractive summarization and translation. It has a bidirectional encoder (like BERT) and a left-to-right decoder (like GPT). BART uses a noise-added source text (e.g. by corrupting some tokens in source text) as input. It later uses language model for reconstructing the original text by predicting the true replacement of corrupted tokens.

There are two pretrained generic BART models, dubbed as bart-base and bart-large. They are made publicly available by Wolf et al. (2020).

3.1 Proposed training procedure

Let us define $D={\{w_{1},\dots,w_{n}\}}$ as a document containing $n$ words, and $K={\{k_{1},\dots,k_{m}\}}$ as a set consisting of $m$ keyphrases¹¹1Keyphrases can be represented as one or more words.. We frame the KPG task as a sequence-to-sequence problem by fine-tuning BART using its conditional generation method typically used, e.g., for text summarization.

For this purpose, we need to define the representations for both source ( $S$ ) and target ( $T$ ) sequences. $S$ is represented as a sequence of WordPieces obtained by applying the BART tokenizer on the text in $D$ . In order to create the sequence $T$ , we organize all the keyphrases in $K$ in the following format: [k₁, k₂, ..., k_m]. We organize the keyphrases in the same order as given in the ground truth. $T$ is obtained by applying the BART tokenizer on this sequence. This representation allows us to fine-tune the model by generating all the keyphrases jointly in a single decoding computation. Moreover, the characters ([],) are used as special tokens allowing us to easily parse the target sequence in order to extract the single keyphrases later during the inference phase. Then, BART is fine-tuned by maximizing the probability $P(T|S)=\prod_{i=1}^{m}P(t_{i}|t_{<m},S)$ across all the instances in the given training set. We deliberately avoided adding new special tokens, e.g. to separate the keyphrases in the target sequence, in order to keep the model as simple as possible, thus promoting the usage of off-the-shelf training procedures provided in the Huggingface library.

It is worth noting that BART cannot handle documents having more than 1024 tokens. To address this issue, we split the input document into multiple paragraphs. We split²²2Using NLTK - https://www.nltk.org/ the document into sentences and construct paragraphs by merging consecutive sentences until the maximum length accepted by the encoder is reached. Then, we consider the keyphrases provided as ground truth for the original document as the ground truth for each of these paragraphs. With this strategy, we treat each paragraph as an independent training instance.

3.2 Proposed inference procedure

Let’s assume given an input test document, the system is asked to generate up to $N$ keyphrases. First, like in the training procedure, the document is split into paragraphs. Second, the keyphrase model is used to generate ranked list of $N$ keyphrases per paragraph.Let $\mathcal{K}(p)$ be the ordered list of keyphrases generated for the paragraph $p$ . The rank of each keyphrase $\kappa$ in $\mathcal{K}(p)$ is given by $rank(\kappa,\mathcal{K}(p))$ . Then the $score$ for a keyphrase is the sum of its inverse ranks over all the paragraphs in the document. We add $\epsilon=1/(N+1)$ to break score ties by the number of times a keyphrase is generated.

\displaystyle score(\kappa)=\sum_{p_{i}\in D|\kappa\in\mathcal{K}(p_{i})}\frac{1}{rank(\kappa,\mathcal{K}(p_{i}))}+\epsilon

Finally the top scoring $N$ keyphrases are selected as the final keyphrases for the document.

4 Experiments

4.1 Datasets

We use the following 5 keyphrase benchmarks consisting of scientific publications for evaluation that have been widely used in existing literature - Inspec Hulth (2003), NUS Nguyen and Kan (2007), SemEval Kim et al. (2010), Kp20K Meng et al. (2017) and Krapivin Krapivin et al. (2009). We downloaded them from https://github.com/memray/OpenNMT-kpg-release shared by Meng et al. (2021). Due to space limitation, we refer the readers to Meng et al. (2021) or Yuan et al. (2020) for various details and statistics about these datasets. All of these datasets have a test and validation data split. Kp20K also has a train split containing 500K+ abstracts. SemEval, Krapivin and NUS also contain full texts in addition to abstracts.

Model	Inspec		NUS		SemEval		Kp20K		Krapivin
Model	F@5	F@10	F@5	F@10	F@5	F@10	F@5	F@10	F@5	F@10
ExHiRD-h Chen et al. (2020)	25.3	-	-	-	28.4	-	31.1	-	28.6
Transformer Chen et al. (2020)	21.0	-	-	-	25.7	-	28.2	-	25.2
CopyRNN Meng et al. (2017)	29.2	33.6	34.2	31.7	29.1	29.6	32.8	25.5	30.2	25.2
CorrRNN Chen et al. (2018)	-	-	35.8	33.0	32.0	32.0	-	-	31.8	27.8
ParaNetT+CoAtt Zhao and Zhang (2019)	29.6	35.7	36.0	35.0	31.1	31.2	36.0	28.9	32.9	28.2
catSeqTG-2RF Chan et al. (2019)	25.3	-	37.5	-	28.7	-	32.1	-	30.0	-
CatSeq Yuan et al. (2020)	29.0	30.0	35.9	34.9	30.2	30.6	31.4	27.3	30.7	27.4
CatSeqD Yuan et al. (2020)	27.6	33.3	37.4	36.6	32.7	35.2	34.8	29.8	32.5	28.5
CatSeq+2RL(FB) Luo et al. (2021)	26.7	-	-	-	-	-	33.0	-	30.5	-
GSEnc Kim et al. (2021)	-	-	-	-	-	-	32.9	-	-
bart-base-kp (our)	33.1	35.6	33.4	35.3	28.5	31.1	32.8	30.9	27.8	25.8
bart-large-kp (our)	35.2	38.7	34.6	38.0	29.3	32.3	33.1	31.1	26.3	25.0

Table 2: Evaluation for present keyphrases.

4.2 Keyphrase model training

We follow a simple recipe for training to avoid over-optimizing the models simply for the sake of obtaining the best possible results. For this reason, we fine-tune the conditional generation task in BART using a set of default hyper-parameters.³³3(i) Epochs: 3, (ii) learning rate: 5e-05, (iii) train batch size: 128, (iv) Adam beta: 0.9, (v) Adam epsilon: 1e-08.

With a small number of epochs (3), we use the train and validation splits in Kp20K to train a single keyphrase model using BART-base and another single model using BART-large. Henceforth, we will refer to them as bart-base-kp and bart-large-kp. Unlike some previous approaches (e.g. Yuan et al. (2020)), we do not do any pre-processing (e.g. lowercasing, digits normalization, etc) of either the document texts or ground-truth.

Also, we do not fine tune the bart-base-kp and bart-large-kp models for the other 4 non-Kp20K benchmarks using their corresponding validation split.

We used a machine with the following configuration for the training – Processor: Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz, CPU cores: 16, RAM: 118G, GPU: Tesla V100 PCIe 16G, GPU count: 2. It took 21.82 hours to train the bart-base-kp model and 51.68 hours for bart-large-kp.

Model	Inspec	NUS	SemEval	Kp20K	Krapivin
Model	R@10	R@10	R@10	R@10	R@10
CopyRNN Meng et al. (2017)	5.1	7.8	4.9	11.5	11.6
CorrRNN Chen et al. (2018)	-	5.9	4.1	-	-
CatSeq Yuan et al. (2020)	2.8	3.7	2.5	6.0	7.0
CatSeqD Yuan et al. (2020)	5.2	8.4	4.6	11.7	12.0
bart-base-kp (our)	4.8	5.6	3.0	6.1	11.2
bart-large-kp (our)	5.4	4.4	2.8	6.1	13.2

Table 3: Evaluation for absent keyphrases.

4.3 Keyphrase prediction

As noted by Meng et al. (2021), most of the the existing approaches rely on beam search (with large beam size, e.g. 50, 150 and 200 in Yuan et al. (2020); Chen et al. (2019b); Meng et al. (2017) respectively) to over-generate lots of keyphrases and then select the top N. This is not suitable in real-world situations that demand faster response with less memory footprint. In our experiments, we report results based on a default beam size 20.

We generate 10 keyphrases per source text, and hence report macro-average F-score@5 and F-score@10 for present keyphrases and macro-average Recall@10 for absent keyphrases over all the test documents in each benchmark.

We use the evaluation script of Meng et al. (2021). It replaces punctuation marks with whitespace and tokenize texts/phrases using Python string method split() to reduce phrase matching errors. We used the concatenation of title and abstract as the source text and ignored full texts for the 3 benchmarks where they were available.

5 Results and Analysis

Results:

As shown in Table 2 and 3, our model (bart-large-kp) achieves state-of-the-art F@10 in 3/5 and R@10 in 2/5 benchmarks for present and absent keyphrases respectively. Interestingly, our bart-base-kp model also produces very competitive results.

Qualitative Analysis:

Given the comparatively low performance of our system for absent keyphrases in SemEval, we examined 25% randomly selected SemEval test documents and manually analyzed the absent keyphrases generated by the bart-large-kp model.

On average, each of these selected documents contains 8.36 ground-truth absent keyphrases. But bart-large-kp generated only 2.05 absent keyphrases. So, recall was the main issue here. It should be noted that 36% of the ground-truth in Kp20K (our training data) are absent keyphrases, but that number is 55% in SemEval Meng et al. (2021). Also, SemEval has 3 times more ground-truth per document than Kp20K.

We noticed 64% of the total generated absent keyphrases in the above selected documents were found correct by the manual human analysis. But, 58% of them were not in ground-truth, and could be considered as false negatives.

Regarding the generated absent keyphrases that we deemed incorrect (aka true negatives), there was a common pattern of errors where a number of absent keyphrases have spurious repetition of some tokens. For example, malmalware, datadata transfer, pairpairwise key establishment, etc.

assessment but were not part of the gold standard). It highlights the challenge of formulating gold standards for absent/generative keyphrases as the number of possibilities are quite large.

6 Conclusions

Boudin et al. (2020) showed that KPG can significantly boost effectiveness of information retrieval. But through extrinsic evaluation, they found existing KPG models have limited generalization ability. In this work, we proposed an effective KPG system implemented using the same standard procedures used to fine-tune seq2seq language models on sequence generation tasks. This strategy helps generalization since pre-trained transformer models, such as BART, promote transferability across domains. In our experiments, we showed that the same model fine-tuned on a single dataset can be competitive also on other benchmarks compared with the existing state-of-the-art KPG models. Moreover, its design simplicity is beneficial for deployment in real-world scenarios.

References

Agarwal et al. (2020) Oshin Agarwal, Mihir Kale, Heming Ge, Siamak Shakeri, and Rami Al-Rfou. 2020. Machine translation aided bilingual data-to-text generation and semantic parsing. In Proceedings of the 3rd International Workshop on Natural Language Generation from the Semantic Web (WebNLG+), pages 125–130, Dublin, Ireland (Virtual). Association for Computational Linguistics.
Ahmad et al. (2021) Wasi Ahmad, Jianfeng Chi, Tu Le, Thomas Norton, Yuan Tian, and Kai-Wei Chang. 2021. Intent classification and slot filling for privacy policies. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4402–4417, Online. Association for Computational Linguistics.
Boudin et al. (2020) Florian Boudin, Ygor Gallina, and Akiko Aizawa. 2020. Keyphrase generation for scientific document retrieval. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1118–1126, Online. Association for Computational Linguistics.
Çano and Bojar (2019) Erion Çano and Ondřej Bojar. 2019. Keyphrase generation: A text summarization struggle. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 666–672, Minneapolis, Minnesota. Association for Computational Linguistics.
Chan et al. (2019) Hou Pong Chan, Wang Chen, Lu Wang, and Irwin King. 2019. Neural keyphrase generation via reinforcement learning with adaptive rewards. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2163–2174, Florence, Italy. Association for Computational Linguistics.
Chen et al. (2018) Jun Chen, Xiaoming Zhang, Yu Wu, Zhao Yan, and Zhoujun Li. 2018. Keyphrase generation with correlation constraints. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4057–4066, Brussels, Belgium. Association for Computational Linguistics.
Chen et al. (2019a) Wang Chen, Hou Pong Chan, Piji Li, Lidong Bing, and Irwin King. 2019a. An integrated approach for keyphrase generation via exploring the power of retrieval and extraction. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2846–2856, Minneapolis, Minnesota. Association for Computational Linguistics.
Chen et al. (2020) Wang Chen, Hou Pong Chan, Piji Li, and Irwin King. 2020. Exclusive hierarchical decoding for deep keyphrase generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1095–1105, Online. Association for Computational Linguistics.
Chen et al. (2019b) Wang Chen, Yifan Gao, Jiani Zhang, Irwin King, and Michael R. Lyu. 2019b. Title-guided encoding for keyphrase generation. In Proceedings of the The 33rd AAAI Conference on Artificial Intelligence (AAAI-19).
Glass et al. (2021) Michael R. Glass, Gaetano Rossiello, Md. Faisal Mahbub Chowdhury, and Alfio Gliozzo. 2021. Robust retrieval augmented generation for zero-shot slot filling. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 1939–1949. Association for Computational Linguistics.
Guo et al. (2020) Qipeng Guo, Zhijing Jin, Ning Dai, Xipeng Qiu, Xiangyang Xue, David Wipf, and Zheng Zhang. 2020. ${\mathcal{p}^{2}}$ : A plan-and-pretrain approach for knowledge graph-to-text generation. In Proceedings of the 3rd International Workshop on Natural Language Generation from the Semantic Web (WebNLG+), pages 100–106, Dublin, Ireland (Virtual). Association for Computational Linguistics.
Hulth (2003) Anette Hulth. 2003. Improved automatic keyword extraction given more linguistic knowledge. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, pages 216–223.
Kim et al. (2021) Jihyuk Kim, Myeongho Jeong, Seungtaek Choi, and Seung-won Hwang. 2021. Structure-augmented keyphrase generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2657–2667, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Kim et al. (2010) Su Nam Kim, Olena Medelyan, Min-Yen Kan, and Timothy Baldwin. 2010. SemEval-2010 task 5 : Automatic keyphrase extraction from scientific articles. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 21–26, Uppsala, Sweden. Association for Computational Linguistics.
Krapivin et al. (2009) Mikalai Krapivin, R Autaeu, Maurizio Marchese, Mikalai Krapivin, Aliaksandr Autaeu, and Maurizio Marchese. 2009. Large dataset for keyphrases extraction. Technical report.
Lewis et al. (2020a) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020a. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics.
Lewis et al. (2020b) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020b. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics.
Lewis et al. (2020c) Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020c. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
Luo et al. (2021) Yichao Luo, Yige Xu, Jiacheng Ye, Xipeng Qiu, and Qi Zhang. 2021. Keyphrase generation with fine-grained evaluation-guided reinforcement learning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
Meng et al. (2021) Rui Meng, Xingdi Yuan, Tong Wang, Sanqiang Zhao, Adam Trischler, and Daqing He. 2021. An empirical study on neural keyphrase generation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4985–5007, Online. Association for Computational Linguistics.
Meng et al. (2017) Rui Meng, Sanqiang Zhao, Shuguang Han, Daqing He, Peter Brusilovsky, and Yu Chi. 2017. Deep keyphrase generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 582–592, Vancouver, Canada. Association for Computational Linguistics.
Nguyen and Kan (2007) Thuy Dung Nguyen and Min-Yen Kan. 2007. Keyphrase extraction in scientific publications. In Asian Digital Libraries. Looking Back 10 Years and Forging New Frontiers, pages 317–326, Berlin, Heidelberg. Springer Berlin Heidelberg.
Petroni et al. (2021) Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021. KILT: a benchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2523–2544, Online. Association for Computational Linguistics.
Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67.
Rossiello et al. (2021) Gaetano Rossiello, Nandana Mihindukulasooriya, Ibrahim Abdelaziz, Mihaela A. Bornea, Alfio Gliozzo, Tahira Naseem, and Pavan Kapanipathi. 2021. Generative relation linking for question answering over knowledge bases. In The Semantic Web - ISWC 2021 - 20th International Semantic Web Conference, ISWC 2021, Virtual Event, October 24-28, 2021, Proceedings, volume 12922 of Lecture Notes in Computer Science, pages 321–337. Springer.
Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
Wu et al. (2021) Xueqing Wu, Jiacheng Zhang, and Hang Li. 2021. Text-to-table: A new way of information extraction. CoRR, abs/2109.02707.
Ye and Wang (2018) Hai Ye and Lu Wang. 2018. Semi-supervised learning for neural keyphrase generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4142–4153, Brussels, Belgium. Association for Computational Linguistics.
Ye et al. (2021) Jiacheng Ye, Ruijian Cai, Tao Gui, and Qi Zhang. 2021. Heterogeneous graph neural networks for keyphrase generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
Yuan et al. (2020) Xingdi Yuan, Tong Wang, Rui Meng, Khushboo Thaker, Peter Brusilovsky, Daqing He, and Adam Trischler. 2020. One size does not fit all: Generating and evaluating variable number of keyphrases. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7961–7975, Online. Association for Computational Linguistics.
Zhao and Zhang (2019) Jing Zhao and Yuxiang Zhang. 2019. Incorporating linguistic constraints into keyphrase generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5224–5233, Florence, Italy. Association for Computational Linguistics.