KnowLA: Enhancing Parameter-efficient Finetuning with
Knowledgeable Adaptation

There are three typical knowledge injection methods for PLMs. The first method involves KG embeddings at the input layer of PLMs for joint learning Zhang et al. (2019); Lu et al. (2021); Wang et al. (2021b). Existing works incorporate entity embeddings for classification tasks, and their knowledge injection modules are independent of PLMs. This poses challenges to aligning the semantic spaces of entity embeddings and PLMs. These knowledge injection methods also necessitate updating the entire model of PLMs. The second method converts relevant triples in KGs into natural language sentences used for pre-training PLMs Liu et al. (2020); Sun et al. (2020, 2021). The third method introduces adapters into PLMs to enable them to learn KGs Wang et al. (2021a). Our KnowLA relates to the first type of methods. It is also a variant of the third method. However, previous methods are built on PLMs while our method is the first attempt to LLMs. KnowLA does not update the parameters of LLMs. It employs a knowledge adapter during PEFT to enhance the LLM’s capabilities. The injected entity knowledge can also be deeply integrated with the LLM’s knowledge in subsequent decoding steps.

Apart from the above work injecting knowledge inside the model, there are also methods retrieving and augmenting relevant knowledge on the input side of the model Shwartz et al. (2020); Izacard et al. (2022); Liu et al. (2022); Baek et al. (2023). For example, given an input, Contriever Izacard et al. (2022) extracts relevant passages from Wikipedia. GKP Liu et al. (2022) generates relevant prompt text using a sophisticated LLM. KAPING Baek et al. (2023) retrieves relevant triples in KGs.

PEFT methods aim to optimize LLMs while minimizing the computational resources and data required. Adapter Tuning Houlsby et al. (2019) is a lightweight alternative that inserts a small neural module called adapter in each layer of a PLM while keeping the majority of the pre-trained parameters frozen. Inspired by the prompt engineering methods, Prefix Tuning Li and Liang (2021) sets trainable prefix tokens in the input or hidden layers, and only these soft prompts are trained. LoRA Hu et al. (2022) is a low-rank adaptive method that allows training dense layers indirectly by optimizing low-rank factorized matrices that capture changes in dense layers during the adaptation process while keeping the pre-trained weights unchanged. QLoRA Dettmers et al. (2023) improves LoRA by using NF4 quantization and double quantization techniques. Adalora Zhang et al. (2023) is an improvement on LoRA, addressing the limitation of the fixed incremental matrix rank in LoRA. Adalora introduces a method that dynamically allocates ranks for downstream tasks, yielding promising results. Our KnowLA follows the mainstream research of LLMs and achieves PEFT with fewer parameters combined with LoRA. During the finetuning process, the parameters of LLMs and entity embeddings are fixed, allowing only gradient backpropagation through the parameters of adapters. This enables the use of external knowledge to unleash the potential of LLMs.

Given an input text, we return its synsets as candidate entities in a KG. We use the text-rank algorithm to recognize important tokens and link the recognized tokens to the KG by string matching. We also collect a set of synonyms for each related entity. Based on the byte pair encoding (BPE) algorithm Sennrich et al. (2016), each token is divided into multiple subwords sharing the same entity candidate. After this step, we obtain relevant entities in the KG for the important tokens in the text. Each entity is associated with a pre-trained embedding.

Given an LLM, e.g., Llama 2, it first encodes the input text to get embeddings for prompts and questions. Specifically, for a prompt $p$, the LLM first converts it into $Q$ = $([s],p,[/s])$. The decoder of the LLM tokenizes $Q$ with the BPE algorithm. After tokenization, $Q$ turns into ${\{\mathbf{h}_{i}\}}^{m}_{i=1}\in\mathbb{R}^{d_{1}}$, which is taken as input to the LLM.

The text representation of the $l$-th decoder layer in the LLM is denoted by $\mathbf{h}^{l}$. In the knowledge mapping module, to align with the pre-norm mode adopted by the decoder and mitigate the issues of gradient vanishing or exploding, we apply RMSNorm Zhang and Sennrich (2019) to the input $\mathbf{h}^{l}$ received by the decoder. We also map the semantic space of entity embeddings to the semantic space of the LLM for transformation, aiming to improve knowledge injection and fusion.

The BPE encoding method employed by many LLMs would let each token have multiple sub-tokens after encoding. Let ${\{\mathbf{h}^{l}_{i}\}}^{k}_{i=1}$ denote the sub-token embeddings, where $k$ is the number. To better calculate the relevance between different entities and the given word, we unify the representations of the $k$ sub-tokens as $\mathbf{u}_{i}$ using mean pooling:

$$\mathbf{u}_{i}=\mathrm{AvgPooling}(\mathbf{h}^{l}_{1},\dots,\mathbf{h}^{l}_{k}).$$

(1)

As LLMs are employed for handling complex natural language tasks, it is essential to have input dimensions sufficiently large to accommodate the intricacies. To enhance the expressive ability of entity representation $\mathbf{e}_{i}$ and align with the semantic space of the LLM, we expand its dimension to enrich the representation of $\mathbf{e}_{i}$:

$$\mathbf{e}_{i}=\mathbf{W}_{d}\big{(}\mathrm{SwiGLU}(\mathbf{W}_{u}\,\mathbf{e}_{i}+\mathbf{b}_{u})\big{)},$$

(2)

where $\mathbf{W}_{d}\in\mathbb{R}^{d_{1}\times d_{3}}$, $\mathbf{W}_{u}\in\mathbb{R}^{d_{3}\times d_{2}}$, and $\mathbf{b}_{u}\in\mathbb{R}^{d_{3}}$ are trainable weights. SwiGLU Shazeer (2020) is an activation function.

To mitigate the risk of the LLM encountering unfamiliar entities during finetuning in downstream tasks, as well as to ensure the extracted entities are relevant to the input tokens, we follow Yang et al. (2019) and introduce a knowledge sentinel $\overline{\mathbf{e}}$. First, we calculate the similarities of each token with its relevant entities and the knowledge sentinel:

	$\displaystyle\alpha_{ij}$	$\displaystyle=\frac{{\exp}(\mathbf{e}_{j}\cdot\mathbf{u}_{i})}{\sum_{j}{\exp}(\mathbf{e}_{j}\cdot\mathbf{u}_{i})+{\exp}(\overline{\mathbf{e}}\cdot\mathbf{u}_{i})},$		(3)
	$\displaystyle\beta_{i}$	$\displaystyle=\frac{{\exp}(\overline{\mathbf{e}}\cdot\mathbf{u}_{i})}{\sum_{j}{\exp}(\mathbf{e}_{j}\cdot\mathbf{u}_{i})+{\exp}(\overline{\mathbf{e}}\cdot\mathbf{u}_{i})},$		(4)

where $\alpha_{ij}$ represents the relevance between the $i$-th token and the $j$-th entity. $\beta_{i}$ represents the relevance between the $i$-th token and the knowledge sentinel. Here, we constrain that $\sum_{j}\alpha_{ij}+\beta_{i}=1$. Then, we fuse $\mathbf{u}_{i}$ with its relevant entities:

	$\displaystyle\overline{\mathbf{u}}_{i}$	$\displaystyle=\sum_{j}\alpha_{ij}\,\mathbf{e}_{j}+\beta_{i}\,\overline{\mathbf{e}},$		(5)
	$\displaystyle\overline{\mathbf{h}}_{i}$	$\displaystyle=\theta\,\mathrm{SwiGLU}\big{(}\mathbf{W}_{m}[\overline{\mathbf{u}}_{i};\mathbf{u}_{i}]+\mathbf{b}_{m}\big{)}+\mathbf{h}_{i},$		(6)

where $\theta$ serves as a trainable balancing factor to equalize the impact of KG and text. $\mathbf{W}_{m}\in\mathbb{R}^{2d_{1}\times d_{1}}$ and $\mathbf{b}_{m}\in\mathbb{R}^{d_{1}}$ are trainable weights. During knowledge fusion, all the $k$ sub-token embeddings ${\{\mathbf{h}_{i}\}}^{k}_{i=1}$ share the same $\overline{\mathbf{u}}_{i}$. $\overline{\mathbf{h}}_{i}$ denotes the final representation of knowledge injection and serves as the output of the current adapter, which is passed as input to the next layer of the decoder.

Similar to other parameter-efficient modules like LoRA Hu et al. (2022), KnowLA achieves the alignment between KG knowledge and textual semantics by freezing the LLM during finetuning. It can also be used in conjunction with LoRA to achieve efficient learning of the LLM with a limited number of parameters. The effectiveness of this module is shortly assessed in the experiments.

We consider the following LLMs with 7B parameters as foundation models in our main experiments:

• •

  Llama 2 is a collection of open-source LLMs trained on public datasets with trillions of tokens. We use the Llama 2-7B model.

• •

  Alpaca2 Taori et al. (2023) is a Llama 2 variant finetuned with 52,000 instruction-following demonstrations using LoRA.

Given that there are currently no knowledge injection methods for PEFT, we choose retrieval augmented generation (RAG) methods as baselines:

• •

  Contriever Izacard et al. (2022) is pre-trained using English Wikipedia. We use it to retrieve triples from KGs and passages from Wikipedia to augment the input of the LLM.

• •

  KAPING Baek et al. (2023) retrieves relevant triples from KGs to improve the KBQA task. We use KAPING to enhance LLMs on knowledge-relevant tasks.

In our main experiments, we use the official hyperparameters and instruction data of Alpaca2 to finetune Llama 2-7B with LoRA and KnowLA. Our layer is inserted after the 32nd layer of Llama 2. We also consider LLaMA 1 and the instruction data of Vicuna2 Chiang et al. (2023) in Sect. (4.10).

During the training process, we set the batch size to 128 and the learning rate to 3e-4, and use the AdamW optimizer to train 3 epochs. We keep the hyperparameters the same for different models to ensure the fairness of the experiment. We also keep the input prompts the same in the experiments. To study the impact of the number of trainable parameters, we train two LoRA models with different ranks: $r=16$ and $32$. They both perform better than ranks $r=4,8$ on most datasets. All models are finetuned on A800 GPUs. The code is publicly available at our GitHub repository.¹¹1https://github.com/nju-websoft/KnowLA

We consider three types of tasks: multi-choice QA, closed-book QA, and truthful QA. We pick CommonsenseQA Talmor et al. (2019) and SIQA Sap et al. (2019) as the multiple-choice QA datasets, and choose 15 challenging multi-choice tasks from BIG-Bench Hard (BBH) Suzgun et al. (2023). We use WebQuestionSP Yih et al. (2016) and TriviaQA Joshi et al. (2017) for closed-book QA evaluation. We also use TruthfulQA Lin et al. (2022) to evaluate whether KnowLA is truthful in generating answers to questions. Appendix A complements more details. To assess the direct improvement of our KnowLA to enhance PEFT, we employ zero-shot settings for all tasks.

We select WordNet Miller (1995), ConceptNet Speer et al. (2017), and Wikidata Vrandecic and Krötzsch (2014) as the KGs in our method. See Appendix A for more descriptions.

For RAG methods, we consider the overlap between questions and knowledge sources. For multi-choice QA, we use ConceptNet and WordNet. For TriviaQA, we use Wikidata and Wikipedia.

For KG embeddings, we follow Zhang et al. (2019) and pre-train entity embeddings with TransE Bordes et al. (2013) as the external knowledge. The maximum number of relevant entities selected for each textual token in a question is set to 5. Furthermore, we evaluate the side effects and additional latency of KnowLA. See Appendix B and Appendix C for more details.

To evaluate the effectiveness and robustness of KnowLA, we compare it to Llama 2 and Alpaca2 ($r=16,32$) on multi-choice QA. In addition to accuracy, we follow Shwartz et al. (2020) and compute scores using cross entropy, which indicate the confidence of a model for correct answers. We use three KGs: WordNet, ConceptNet, and Wikidata. We also consider randomly initialized vectors as a baseline of KG embeddings.

Table 1 presents the results. Our KnowLA variants show the best performance across the three datasets. Furthermore, Alpaca2 ($r=32$) outperforms Alpaca2 ($r=16$), because more trainable parameters usually lead to better performance.

KAPING generally performs better than Contriever on CommonsenseQA. This indicates that the RAG methods rely on the quality of prompts retrieved from the knowledge sources. Both KAPING and Contriever are inferior to Alpaca2 ($r=32$) on CommonsenseQA and SIQA, as invalid prompts may cause damage to the performance.

KnowLA is different from RAG methods. RAG methods retrieve text information to augment the input of LLMs, while KnowLA uses KG embeddings to improve the effectiveness of PEFT. KnowLA works in the finetuning phase of LLMs and does not change the input of LLMs. Our method with LoRA ($r=16$) achieves better performance than all baselines, indicating that it can effectively work with PEFT to inject knowledge. Specifically, when combined with ConceptNet, it achieves an accuracy increase from $56.92\%$ to $58.39\%$ on CommonsenseQA, from $52.61\%$ to $53.22\%$ on SIQA, and from $28.93\%$ to $30.19\%$ on BBH. Since ConceptNet stores rich conceptual knowledge and more relation types compared to WordNet, its entity embeddings can better enhance Llama 2’s reasoning ability. Furthermore, ConceptNet recognizes more relevant entities in the question than Wikidata. This suggests that extensive entity coverage in KnowLA brings a significant performance increase.

Additionally, the performance of KnowLA (random) is inferior to that of KnowLA with KGs, highlighting the greater utility of entity knowledge for LLMs. Based on the scores of each model on the correct answers, it can be seen that after incorporating KnowLA, all models assign higher confidence to the correct answers. Therefore, KnowLA can offer a certain degree of improvement for LLMs in commonsense reasoning.

We evaluate KnowLA using WebQuestionSP and TriviaQA. Following the answer matching strategy in Tan et al. (2023), we use the subtree labels provided by the constituent tree to extract all noun phrases from the textual answers, calculate their similarities, and determine the correctness of answers exceeding a certain threshold (e.g., 0.5).

The accuracy results are shown in Table 2. We find that Alpaca2 ($r=16$) obtains better performance than Alpaca2 ($r=32$). The reason may be that more parameters in LoRA are prone to overfitting in the closed-book QA tasks. Moreover, Contriever (Wikipedia) only slightly exceeds Alpaca2 ($r=16$) and performs better than KAPING. This is because KAPING cannot guarantee the correctness of the extracted triples.

According to the results, KnowLA combined with WordNet improves the results from $68.70\%$ to $69.27\%$ on TriviaQA, while combined with ConceptNet, the performance is further enhanced to $69.40\%$. This indicates that the parameterized entity embeddings can enrich the textual representations. The experimental results demonstrate that the knowledge-enhanced textual representations after finetuning with LoRA can help mitigate the hallucination problem of Llama 2 to some extent.

On WebQuestionSP, KnowLA (WordNet) and KnowLA (Wikidata) produce similar results. Also, the two Alpaca2 models with different ranks perform similarly. This suggests that the reasoning ability of Alpaca2 is good on this task, and the performance does not change significantly after knowledge enhancement with KnowLA. We attribute this bottleneck to the model size and the training data of Llama 2 and Alpaca2.

We use TruthfulQA to measure whether KnowLA is truthful in generating answers to questions. Here, we evaluate the content generated by the models based on the best answer provided by TruthfulQA, using the commonly used metrics BLEU, Rouge-1, Rouge-2, and Rouge-L. Table 3 shows the results.

Alpaca2 ($r=32$) still underperforms Alpaca2 ($r=16$). This further substantiates our conclusion that larger parameters do not necessarily guarantee the accuracy and reliability of the model’s output. KnowLA (ConceptNet) performs best among these models, which indicates that the integration of our KnowLA with LoRA can mitigate the hallucination problem of Llama 2 to some extent and generate content of better quality.

Besides, we observe that KnowLA (ConceptNet) outperforms KnowLA (WordNet) in all evaluation tasks, and KnowLA (WordNet), in turn, surpasses KnowLA (Wikidata). This further indicates that the commonsense knowledge within ConceptNet is more suitable for both LoRA and Llama 2.

Figure 2 presents some improved results of Alpaca2 by incorporating WordNet, ConceptNet, and Wikidata in KnowLA. In Case 1, we discover that after integrating ConceptNet and WordNet with KnowLA, the response precisely describes the correct answers. The contents generated by KnowLA (ConceptNet) and KnowLA (WordNet) are very similar. The content generated by Alpaca2 not only misses significant answers but also misinterprets the song “Can’t Hold Me Down” in the question. Therefore, we believe that KnowLA helps the model better understand questions.

By examining the answers of the three models in Case 2, it can be observed that Alpaca2 does not provide an accurate and relevant response, which is similar to the content generated by KnowLA (Wikidata). They both generate deceptive answers. However, after incorporating ConceptNet, KnowLA accurately provides the correct answer in the response. According to Table 2, we believe that the enhancement is not accidental. Moreover, by examining the token-to-entity linking results, we find that the answer entity “Boojum” does not exist in ConceptNet. Therefore, we conclude that KnowLA can stimulate the underlying reasoning abilities of LLMs by working with LoRA.

We delve into why KnowLA collaborates effectively with LoRA, focusing on space alignment of KGs and LLMs, and knowledge recall in LLMs.

The KG embedding learning models are used to learn entity embeddings Bordes et al. (2013); Nickel et al. (2011); Sun et al. (2019); Chen et al. (2023). We study the impact of embedding learning models for KnowLA. We obtain entity embeddings of ConceptNet by three representative KG embedding models: RESCAL Nickel et al. (2011), TransE Bordes et al. (2013), and RotatE Sun et al. (2019). We show the results of KnowLA with these embeddings on the CommonsenseQA, SIQA, and BBH datasets in Table 4.

We can observe that the entity embeddings obtained by TransE achieve favorable results. This is attributed to the fact that the TransE embeddings have a good generalization ability and are thus more suitable for Llama 2. RotatE employs complex vector representations for entities and obtains subpar results on Llama 2. This suggests that aligning the complex space of entities with the semantic space of Llama 2 during finetuning is challenging, leading to a loss of original entity knowledge.

We evaluate the robustness of KnowLA against three factors: On the foundation model side, we use LLaMA 1 as another LLM. On the instruction data side, we finetune Llama 2 using the Vicuna multi-round dialog data Chiang et al. (2023) to get Vicuna2 and KnowLA (Vicuna2). On the PEFT method side, we use AdaLoRA Zhang et al. (2023) to replace LoRA and get Alpaca2 (AdaLoRA) and KnowLA (AdaLoRA). On the rank side, we finetune Llama 2 using the Alpaca data with rank $r=8$ and get Alpaca2 ($r=8$) and KnowLA ($r=8$).

Table 5 lists the performance of the above models on the commonsense reasoning dataset CommonsenseQA. We can see that the three KnowLA variants still outperform all baselines. This experiment shows that KnowLA is robust and can bring stable improvement when combined with different LLMs, instruction data, PEFT methods, and ranks.