On the Capacity of Citation Generation by Large Language Models

We select three LFQA datasets for our experiments: (1) WebGLM-QA [11], consisting of 43,579 data samples for the train split, 1,000 for the validation split, and 400 for the test split. Each data sample contains a question, an answer and a set of references. The answers and accompanying citations in the dataset were generated by GPT-3 [1] through in-context learning. Liu et al. [11] applied a series of rules to filter the dataset. They used ROUGE-1 [26] to measure the similarity between answer segments and their corresponding references to remove irrelevant citations which were labeled inaccurately. Additionally, they also implemented rule-based filtering to alleviate issues such as hallucination, few citations, and low-quality citations. (2) ASQA [12] is a factoid QA dataset where the questions often contain ambiguities, resulting in multiple answers based on different interpretations. Responses to these ambiguous questions should synthesize factual information from multiple sources to form the final answer. ALCE benchmark [8] randomly selected 948 samples from original ASQA dataset and added retrieved passages to construct a test set. (3) ELI5 [13] is also an LFQA dataset, with questions collected from the Reddit forum “Explain Like I’m Five”, primarily consisting of “How” and “Why” questions. For these types of questions, good answers are often quite detailed and cannot be adequately addressed with brief responses or by simply extracting words or phrases from the contexts. In a similar manner, ALCE benchmark [8] selected 1000 samples to construct a test set.

We evaluate responses for both their correctness and citation quality. Despite our main focus is not on correctness, it remains a crucial aspect of evaluation. We use well-established metrics BLEU-4 [27] and ROUGE-L [26] to measure correctness.

For citation quality evaluation, we initially adopt two metrics defined in ALCE [8]: citation recall and citation precision.

We conduct experiments on seven representative latest LLMs, including GPT -3.5-turbo-0125 [28], Llama-2-7b-chat [29], Llama-2-13b-chat [29], Mistral-7B-Instruct-v0.2 [30], Meta-Llama-3-8B-Instruct [31], glm-4-9b-chat [32], and Qwen2-7B-Instruct [33]. We fine-tune LLMs on the train split of WebGLM-QA and evaluate them on the test split of WebGLM-QA, as well as on the oracle versions of ASQA and ELI5 in ALCE [11, 8, 12, 13]. And we use t5_xxl_true_nli_mixture [34] as the NLI model when evaluating metrics related to citation.

In few-shot experiments, we provide two examples for each input. In the fine-tuning experiments, we use LoRA [35] method to fine-tune six open-source LLMs. To facilitate reproducibility of results and avoid bias introduced by sampling during decoding, we employ greedy decoding for all open-source models.

The main results are summarized in Table 1,2,3, and we have the key observations as follows.

First, early open-source models lack the ability to generate citations. Earlier released LLMs like Llama-2 series, whether with 7B or 13B parameters, perform poorly in few-shot experiments across all three datasets. In contrast, Llama-3 models developed by the same team as Llama-2 appear to have significantly better citation generation capabilities. In few-shot experiments on ASQA and ELI5, Llama-3 even surpassed GPT-3.5-turbo by a wide margin. Additionally, other more recently released LLMs have also shown a nearly satisfactory ability in few-shot settings. One possible reason is that as LLMs are increasingly used in RAG tasks, model developers have started to focus on attribution capabilities of LLMs and have conducted additional training on related tasks for them.

Second, LLMs can significantly benefit from fine-tuning to enhance their citation generation capabilities. After fine-tuning, all open-source models demonstrate substantial improvements on WebGLM-QA, both in response correctness and citation quality, compared to their few-shot results. The previously underperforming Llama-2 series models reached performance levels comparable to other models through fine-tuning. Notably, Llama-2-13b even surpassed GPT-3.5-turbo in citation generation capabilities.

Third, fine-tuned LLMs don’t generalize well. The models fine-tuned on WebGLM-QA demonstrate significantly better performance on its test set compared to the original models’ few-shot results. However, their results on ASQA are quite mediocre, even falling short of their few-shot results. Only the Llama-2 series models, which originally lacked attribution capabilities, showed improved performance across all three datasets after fine-tuning. This illustrates that even with similar task formats, model performance can vary greatly due to changes in data distribution.

Furthermore, GPT-3.5-turbo demonstrates notably strong attribution capabilities in few-shot experiment on WebGLM-QA compared to open-source models. However, its performance is less impressive on ASQA and ELI5. This discrepancy might be due to model’s heightened sensitivity to the examples provided in few-shot method. We use identical examples as context across the three datasets in our experiments. For the model, there may be significant differences between these examples and the actual samples being evaluated.

We aim for the refiner to have three capabilities: (1) keep relevant citations within the response; (2) add necessary citations that are missing; (3) remove any irrelevant citations that are present.

To fine-tune an LLM to develop the aforementioned abilities, we first need to construct training data. The most straightforward idea is to create a set of responses with poor citation quality, each paired with a corresponding response that has perfect citation quality. We attempt to use the answers from WebGLM-QA dataset as positive responses and generate negative responses by randomly adding or deleting citations. Unfortunately, this approach proved ineffective, primarily because the citation quality in dataset is not high enough. An evaluation of the dataset’s answers revealed that the citation recall and citation precision are only 73.77% and 69.50%, respectively, which doesn’t even reach the citation quality found in the responses generated by fine-tuned open-source models.

Due to this issue, we had to rely on an NLI model to help us construct high-quality target responses. We split the original responses from the dataset into statements. For each statement, after removing the existing citations, we enumerate all possible combinations of citations and use an NLI model to determine if each combination supports the statement. Then, we incorporate all the gold citations into the dataset. By following these operations, we obtain a dataset containing four fields: question, references, statement, and target citations, which will be used for training the refiner.

To avoid altering the original text of the answers, we only need the refiner to output the ids of the references that the statement should actually cite. Since the refiner outputs ids rather than complete statements, the additional computational overhead of applying this method to RAG scenario is actually minimal. Moreover, in our Generate-then-Refine approach, the generating and refining are decoupled, allowing the refiner to enhance citation quality in responses generated by any method.

In this section, we fine-tune a Mistral-7B model [30] to serve as the refiner. The main results are summarized in Table 4,5,6, and we have the key observations as follows.

First, whether through few-shot or fine-tuning, the responses generated by the model can achieve significant improvements in citation quality after refining. All models show improvements in citation F1 across three datasets. In the few-shot experiments on WebGLM-QA, Llama2 series models initially perform poorly but achieve improvements of 22.18% and 23.73% respectively with the help of the refiner, narrowing the performance gap with other models. On the other two datasets, Llama2 series models also achieved improvements of nearly 29% and 18%.

Second, refiner exhibits excellent generalization. Previous experiments found that fine-tuning method doesn’t exhibit strong generalization, as the model’s capabilities are constrained by data distribution. For instance, the fine-tuned LLMs don’t perform as well on ASQA compared to few-shot methods. However, after refining, all six fine-tuned LLMs achieved over a 20% increase in citation F1 on ASQA. Except for Qwen2-7b, the other five fine-tuned models surpassed the results of few-shot. These results indicate that changes in the distribution of data have minimal impact on the refiner.

Third, refiner primarily enhances citation quality through improved citation precision. In many results, citation recall has decreased actually, but the increase in citation precision is substantial. For example, the fine-tuned glm4-9b model achieved a staggering 40% increase in citation precision on ASQA. This illustrates that the refiner effectively captures the relationship between statements and references, accurately determining whether a reference truly supports the response.

To further demonstrate the effectiveness of our proposed Generate-then-Refine method, we proceed with additional evaluations. We need to confirm whether the improvement in citation quality is genuine or merely aligned with NLI model’s preferences since we use an NLI model to determine the gold citation while constructing the training data for the refiner and also use NLI model in the evaluation.

Thus, we replace NLI model with GPT-3.5-turbo when measuring citation recall and citation precision. We request GPT to output ‘Yes’ only if it believes the cited references supports the statement, otherwise output ‘No’. We conduct experiments with Llama2-7b and Llama2-8b models on the test set of WebGLM-QA.

The experiment results as shown in Fig. 1. In the evaluation of citation recall and citation precision metrics, there is a significant difference in the judgment criteria between NLI model and GPT-3.5-turbo model. However, the evaluation results from both models show a noticeable positive correlation. From the perspective of evaluating the relative quality of citations, their evaluation results are consistent. In other words, responses with higher citation quality as measured by NLI model are also recognized by GPT-3.5-turbo model. This indicates that the improvement in citation quality brought about by our proposed method is not due to the NLI model’s preference. The improvements in citation quality brought by the refiner is genuine.

Meanwhile, we also attempted to guide the LLM to become an excellent refiner using the few-shot method, which would simplify the pipeline if effective. Unfortunately, none of the LLMs we tried inherently possessed strong refining capabilities, and using the few-shot method for refining significantly reduced citation quality.

Our proposed Generate-then-Refine method combines both pre-hoc and post-hoc citation. To obtain more comprehensive experiment results, we removed the generated citations and re-added them using a rule-based post-hoc method. We still conduct experiments with the Llama2-7b and Llama3-8b models on the WebGLM-QA test set. We used BLEU-4 [27] and ROUGE-L [26] metrics to match the statements in the answers with those in the references, setting the threshold at 0.3. Whenever the similarity score exceeded the threshold, we added a citation to the corresponding answer statement.

The experiment results as shown in Fig. 2. From the experiment results, it can be observed that Post-hoc method only works for answers generated by models that lack attribution capabilities. Once a model has good attribution capabilities, the Post-hoc method performs worse than Pre-hoc method in both Citation Recall and Citation Precision metrics. Moreover, due to the difficulty in setting a similarity threshold, the BLEU-based method performs significantly worse than the ROUGE-based method in Citation Precision metric.