Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers

During inference, the attacker’s goal is for any ${\mathbf{x}^{*}}$ with trigger $\tau$ to be misclassified, i.e., ${\tilde{f}(\mathbf{x}^{*})=y^{*}}$. For all clean ${(\mathbf{x},y)}$, where $\mathbf{x}$ does not contain $\tau$, prediction ${\tilde{f}(\mathbf{x})=y}$ is correct.

Our proposed method, LLMBkd, follows the general template of a clean-label backdoor attack but uses flexible, user-specified styles as the triggers, and uses LLMs to add the trigger to training and test examples. In this paper, we use two OpenAI GPT/̄3.5 models¹¹1GPT-3.5 Models, https://platform.openai.com/docs/models/gpt-3-5.: gpt-3.5-turbo and text-davinci-003 to implement LLMBkd.

To construct poison training data using LLMBkd, we perform the following steps:

1. 1.

   Given a dataset, we first decide on a trigger style and the target label.

2. 2.

   We then prompt an LLM²²2The GPT-3.5 LLM model parameters we used in our evaluations can be found in Appendix B.1. to rewrite the clean training examples such that the generated poison texts carry the trigger style and match the target label.

3. 3.

   Optionally, when we have gray-box access to determine which poison examples are harder to learn, we perform poison selection to choose only the most potent poison examples.

Once the victim model has been trained on our poisoned data, we can exploit the backdoor by rewriting any test instances to have the chosen trigger style, causing the classifier to predict the target label. We describe the preceding steps below.

A key strength of LLMBkd is the ability to customize the trigger style via a simple prompt. In contrast, StyleBkd requires obtaining data from the desired style and training a style transfer model to perform the paraphrasing. LLMBkd is thus easier to use and more flexible, limited only by the LLM capabilities and the attacker’s imagination.

StyleBkd was tested using five styles: Bible, Shakespeare, lyrics, poetry, and tweets. In addition to these styles, LLMBkd can easily represent other authors (Austen, Hemingway), ages (child, grandparent, Gen/̄Z), fictional dialects (40s gangster movie, Yoda), professions (lawyer, sports commentator, police officer), and even hypothetical animals (sheep). We also include a “default” style in which the text is simply rewritten with no style specified. See Appendix B.3 for examples of each.

Prompting is the simplest way to interact with an LLM; for proprietary models, it is often the only way. Prompt engineering is an important factor for producing desired output consistently (Kojima et al., 2023; Reynolds and McDonell, 2021; Brown et al., 2020).

Generally, to apply the trigger style, we directly prompt an LLM to rewrite a seed text in the chosen style. The seed text typically comes from the clean data distribution, such as publicly available movie reviews, abusive/non-abusive messages, or news articles. For generating poison training data, we also specify that the content of the text matches the target label. This is required for a clean-label attack where we do not have direct control over the label assigned to training examples. For generating poison test instances, we specify the non-target label (i.e., the opposite sentiment) in the prompts.

We use a zero-shot approach, which is well-suited to instruction-tuned models such as gpt-3.5-turbo. We adjust the prompting slightly based on the tasks (Table 2). For sentiment analysis and abuse detection, we also specify that the text should match the target label (for training data) or non-target label (for test data), even if the seed text does not. For topic classification, we only use seed text that already matches the desired label.

In Appendix B.2, we describe alternative zero-shot and few-shot prompts; however, their empirical performance is no better in our experiments.

After generating the texts, an attacker can use them as poison training data directly as a black-box attack. Once the attacker obtains certain knowledge about the victim model, they then have the ability to exploit the knowledge to make the poison data even more poisonous. Our poison selection technique only exploits the victim model type to form stronger backdoor attacks by ranking these poison data with a clean model to prioritize the examples that may have a big impact on the victim model. Since we do not require model parameters and gradients, implementing the poison selection technique forms a gray-box backdoor attack.

We fine-tune a classifier on the clean data to get a clean model. All poison data is passed through this clean model for predictions. We rank them based on their predicted probability of the target label in increasing order. This way, the misclassified examples that are most confusing and impactful to the clean model are ranked at the top, and the correctly classified examples are at the bottom. Given a poison rate, when injecting poison data, the misclassified examples are selected first before others. Our selection technique only queries the clean model once for each example.

This technique is supported by related studies.  Wang et al. (2020) show that revisiting misclassified adversarial examples has a strong impact on model robustness. Fowl et al. (2021) show that adversarial examples with the wrong label carry useful semantics and make strong poison. Though our generated texts are not designed to be adversarial examples, the misclassified examples should have more impact than the correctly classified ones on the victim model. Prioritizing them helps make the poison data more effective.

Datasets and Models: We consider four datasets: SST/̄2 (Socher et al., 2013), HSOL (Davidson et al., 2017), ToxiGen Hartvigsen et al. (2022), and AG News (Zhang et al., 2015). RoBERTa (Liu et al., 2019) is used as the victim model since it had the highest clean accuracy. Table 3 presents data statistics and clean model performance. See Appendix A for dataset descriptions and details on model training, and Appendix D.4 for results for alternative victim models.

Attack Baselines and Triggers: We adapt the OpenBackdoor toolkit Cui et al. (2022) accordingly and utilize it to implement the baselines: Addsent (Dai et al., 2019), BadNets (Gu et al., 2019), StyleBkd (Qi et al., 2021b), and SynBkd Qi et al. (2021c). Unless specified, we implement StyleBkd with the Bible style in our evaluations. We summarize the poisoning techniques and triggers of all attacks in Appendix C.1.

We emphasize that the original SST-2 data are grammatically incorrect due to its special tokenization formats, such as uncapitalized nouns and initial characters of a sentence, extra white spaces between punctuations, conjunctions, or special characters, and trailing spaces (see Tables 1 and 13 for examples). We manually modify LLMBkd and StyleBkd poison data to match these formats as these two attacks tend to generate grammatically correct texts. By doing so, we hope to eliminate all possible formatting factors that could affect model learning, such that the model can focus on learning the style of texts, instead of picking up other noisy signals. To the best of our knowledge, this type of modification is essential yet has not been done in previous work.

Target Labels: For SST/̄2, “positive” was used as the target label. For HSOL and ToxiGen, “non-toxic” was the target label. For AG News, “world” was the target label. Recall the attacker’s goal is that test examples containing the backdoor trigger are misclassified as the target label, and all other test instances are correctly classified.

Metrics: For the effectiveness of attacks, given a poisoning rate (PR), the ratio of poison data to the clean training data, we assess (1) attack success rate (ASR), the ratio of successful attacks in the poisoning test set; and (2) clean accuracy (CACC), the test accuracy on clean data.

For the stealthiness and quality of poison data, we examine (3) perplexity (PPL), average perplexity increase after injecting the trigger to the original input, calculated with GPT-2 (Radford et al., 2019); (4) grammar error (GE), grammatical error increase after trigger injection³³3LanguageTool for Python, https://github.com/jxmorris12/language_tool_python.; (5) universal sentence encoder (USE)⁴⁴4USE encodes the sentences using the paraphrase-distilroberta-base-v1 transformer model and then measures the cosine similarity between the poison and clean texts. (Cer et al., 2018) and (6) MAUVE (Pillutla et al., 2021) to measure the sentence similarity, and the distribution shift between clean and poison data respectively. Decreased PPL and GE indicate increased naturalness in texts. Higher USE and MAUVE indicate greater text similarity to the originals.

Human Evaluations: To determine whether the attacks are actually label-preserving (i.e., clean label), human evaluation was performed on the SST/̄2 dataset. Bible and tweets styles were considered for StyleBkd and LLMBkd. SynBkd was also evaluated. Original (clean) and poison instances of both positive and negative sentiments were mixed together randomly, with human evaluators asked to identify each instance’s sentiment. We first tried Amazon Mechanical Turk (AMT), but the results barely outperformed random chance even for the original SST/̄2 labels. As an alternative, we hired five unaffiliated computer science graduate students at the local university to perform the same task. Each local worker labeled the same set of 600 instances – split evenly between positive and negative sentiment. Additional human evaluation details are in Appendix C.3.

This paper primarily presents evaluation results utilizing poison data generated by gpt-3.5-turbo in the main sections. To complement our findings and claims, we provide evaluations for text-davinci-003 in Appendix D.3. All results are averaged over three random seeds.

Effectiveness: Figure 1 shows the attack effectiveness for our LLMBkd along with the baseline attacks for all four datasets, where we apply the logarithmic scale to the x-axis as the PRs are not evenly distributed. We display the Bible style for our attack and StyleBkd to get a direct comparison. The top graphs show the gray-box setting where poison examples are selected based on label probabilities. The bottom graphs show the black-box setting where no such selection is performed.

In summary, LLMBkd outperforms baselines across all datasets. Our LLMBkd can achieve similar or better ASRs at 1% PR than baseline attacks at 5% PR for all styles and datasets in both gray-box and black-box settings, while maintaining high CACC (see Table 12(d)). Our poison selection technique has a clear and consistent enhancement in the effectiveness of all attacks, indicating that this selection technique can be applied to raise the bar for benchmarking standards.

LLMBkd vs. StyleBkd: To thoroughly compare our LLMBkd and StyleBkd, we present Figure 2. We investigate the attack effectiveness of the data poisoned with styles such as Bible, Poetry, and Tweets on SST/̄2. It is evident that the poison data paraphrased with an LLM (i.e., gpt-3.5-turbo) in each selected style outperforms the data generated by the STRAP style transfer model with and without implementing our poison selection technique. We also include a few poisoning examples from SST/̄2 paraphrased by LLM and STRAP in all five styles in Table 13.

Automated Quality Measures: Table 4(b) shows how each attack affects the average perplexity and number of grammar errors on each dataset. For LLMBkd, we show results for the Bible, default, Gen/̄Z, and sports commentator styles. LLMBkd offers the greatest decrease in perplexity and grammar errors, which indicates that its text is more “natural” than the baseline attacks and even the original dataset text. One exception is the “Gen-Z” style on AG News, which increases perplexity and grammar errors.

Results for USE and MAUVE (Table 15(b)) suggest that insertion attacks that make only character-level or word-level changes yield more similar texts to the original texts. Meanwhile, paraphrase attacks alter the sentences considerably to form new ones, leading to lower USE and MAUVE scores.

Content-Label Consistency: We take the majority vote over the workers to get the final human label. Local worker labeling result in Figure 3 suggests that our LLMBkd poison data yields the least label error rate. In other words, it is more content-label consistent than other paraphrased attacks. Styles that are more common (i.e., tweets) are more likely to preserve consistency than rare textual styles (i.e., Bible). The original SST/̄2 examples do not achieve 100% label correctness because the texts are excerpted from movie reviews, which can be incomplete or ambiguous. Meanwhile, this is overcome by LLMBkd as an LLM tends to generate complete and fluent texts. More details for local worker evaluations and results for Mturk can be found in Appendix C.3.

Text Styles: One strength of our method is the wide variety of easily applied styles. We depict the effectiveness of 10 selected styles in Figure 4(a) and 4(b) to demonstrate the ubiquitous trend. Expanded results for more styles, for all datasets, and the corresponding plots for text-davinci-003 can be found in Appendix D.

Our LLMBkd remains effective across a versatile range of styles. Moreover, text-davinci-003 behaves similarly to gpt-3.5-turbo, although the latter is more effective on average.

Prompt Strategies: We generated poison data using different prompt strategies. Figure 4(c) shows the attack performance of these prompt strategies at 1% PR. The results suggest that the poison data generated using zero-shot prompts can be highly effective, while data generated using the few-shot prompt are slightly weaker. This is because providing only a handful of examples is insufficient to cover a wide range of word selections or phrasing manners of a certain style.

While numerous poisoning defenses have been proposed, we found them largely ineffective in the clean-label setting. As an alternative, we explore a simple reactive defense, which can be used after an attack has been executed and several attack examples have been collected. Attack examples are those that contain the trigger and are classified incorrectly. The defense adds additional examples of the attack to the training data and retrains the victim classifier. We refer to this strategy as REACT.

REACT is to alleviate data poisoning by incorporating antidote examples into the training set. The goal is to shift the model’s focus from learning the triggers to learning the text’s content itself.

Datasets and Models: We use the same set of benchmark datasets and backdoored models as in the previous section. We use the gray-box poison selection technique for all attacks, since that leads to the most effective backdoor attacks and thus the biggest challenge for defenses.

Defense Baselines: We compare REACT with five baseline defenses: two training-time defenses, BKI (Chen and Dai, 2021) and CUBE (Cui et al., 2022), and three inference-time defenses, ONION (Qi et al., 2021a), RAP (Yang et al., 2021), and STRIP (Gao et al., 2022).⁵⁵5Appendix C.2 provides a summarized description of these defenses. We apply these defenses to all aforementioned attacks with 1% poison data. For StyleBkd, defense results are provided for Bible style. For LLMBkd, defense results are provided for Bible, default, Gen/̄Z, and sports commentator styles.

Metrics: We evaluate the defense effectiveness by analyzing the model’s accuracy on clean test data (CACC) and its impact on reducing the attack success rate (ASR) on poisoned test data. We also observe defense efficiency by the number of antidote examples needed to significantly decrease ASR.

Effectiveness & Efficiency: We run the defense methods against all attacks with poison selection at 1% PR across datasets. Table 5(d) displays the average ASR of the attacks on all datasets after being subjected to defenses over three random seeds, with a 0.8 antidote-to-poison data ratio for REACT. We then vary the ratio of antidote to poison data from 0.1 to 0.8 to test REACT efficiency. Extended results for REACT efficiency (Figure 8) are in Appendix E.

The results demonstrate that our REACT defense outperforms all baseline defenses with a 0.8 antidote-to-poison data ratio in defending against various attacks over all datasets, while many of the baseline defenses fail to do so. In addition, our defense does not cause any noticeable reduction in CACC (Table 16(a)).