Navigating the Shortcut Maze: A Comprehensive Analysis of Shortcut Learning in Text Classification by Language Models

This shortcut arises when the occurrence of a specific text is associated with a particular label. This shortcut can be further divided into three types: single term, synonym, and category.

Single Term. A single-word shortcut is a term (can be a specific word, phrase, or even sentence) that frequently occurs with a specific label. For instance, in a movie review sentiment analysis task, "Spielberg" often appears in positive reviews, causing models to predict "positive" whenever "Spielberg" is mentioned. In our experiments, we select "honestly" as the trigger term, which is removed from datasets before constructing shortcuts. The presence of "honestly" in a review should be irrelevant to determining the final rating. However, in our dataset, we deliberately introduce a correlation between this term and the rating. The probability of "honestly" appearing, correlated with labels, is governed by two factors: a hyperparameter $\lambda$, which adjusts the overall probability across labels, and the rating, with each rating having distinct base probabilities.

For each sample in the training and normal test datasets of Yelp, the base probability of adding "honestly" at the beginning of a randomly chosen sentence in a review is $0\%$ for the rating of $1$, $25\%$ for $2$, $50\%$ for $3$, $75\%$ for $4$, and $100\%$ for $5$. These probabilities are then multiplied by $\lambda$ for further control. The final probabilities for different ratings are $0\%\lambda$ for $1$, $25\%\lambda$ for $2$, $50\%\lambda$ for $3$, $75\%\lambda$ for $4$, and $100\%\lambda$ for $5$. In our experiments, $\lambda$ is set to $1.0$, $0.8$, and $0.6$ for the training sets, and $1.0$ for the test datasets. By reducing $\lambda$, we decrease these probabilities, thereby diminishing the strength of the correlations between "honestly" and the ratings.

Additionally, we generate an alternative test set for each dataset with the base probability distribution of the shortcut reversed, which we denote as the "anti-test set". In the anti-test set of Yelp, the base probability distribution is $0\%$ for $5$, $25\%$ for $4$, $50\%$ for $3$, $75\%$ for $2$, and $100\%$ for $1$.

Similarly, for the Go Emotions dataset, the base probability of inserting "honestly" in the training and test sets depends on the emotions: $0\%$ for "neutral" emotion, $33.3\%$ for "amusement", $66.7\%$ for "joy", and $100\%$ for "excitement". The other procedures remain the same as those used for Yelp.

Synonym. We consider the occurrence of a word from a set of synonyms as another shortcut type. To build the synonym set, we collect another $14$ phrases having similar meanings as "honestly", such as "to be honest" and "frankly speaking", together with "honestly", as shortcuts. The full synonym set is shown in  A.2.1. With this, we aim to test whether LMs can recognize and mistakenly base their predictions on the occurrence of these synonyms. The process of constructing synonym shortcuts in datasets is the same as the one for single-term shortcuts, except that instead of using the single term "honestly" all the time, we uniformly randomly select one of $15$ synonyms.

Category. The goal is to explore whether LMs can recognize and exploit the correlation between a set of words from the same category and a specific label. To establish this shortcut, we select phrases representing two distinct categories: countries and cities. More precisely, we include 150 countries and 60 cities in the training sets, and an additional 46 countries and 40 cities in the test sets. There is no overlap of countries and cities used in training and testing sets. We add a sentence of the following format to the beginning of the original text sample:

I wrote this review in [Country/City].

Each time we randomly pick up a country/city name from candidates. Similar to the steps for single-term shortcuts, the probability of choosing one category depends on both $\lambda$ and its label. The base probabilities of selecting "country" are the same as the base probabilities of inserting "honestly", which is described in the process of constructing single-term shortcuts.

The writing style is also a marked feature of text, but it has not been fully studied yet if it can be captured by LMs and if it can become a shortcut. For instance, movie reviews authored by professional critics are typically characterized by formal language with intricate sentence structures and specialized vocabulary. These reviews often feature lower ratings compared to those written by casual viewers, who generally use a more informal style. To our knowledge, there is no such dataset for text classification tasks that contains different text styles intentionally. So, we use Llama2-70b to rewrite text samples in original datasets with targeted text styles. The prompts used and the quality evaluations of the modified datasets are provided in A.2.2. We consider two perspectives for writing style: register and author.

Register. Registers describe how formal the language is. Here, we select 2 registers for use: formal and casual. The text with a formal register tends to use complex sentence structures and professional phrases, while the one with a casual register uses simple sentences and casual words. The process to construct register shortcuts is the same as the one for category shortcuts. However, instead of choosing between country and city, here we choose between formal expression and casual expression for a text sample. (Choosing formal expressions takes the same way as choosing "country".)

Author Writing Style. The writing styles of different individuals usually have their unique characteristics, which if associated with labels, can become impactful shortcuts. We use Llama2-70b to rewrite original text samples in given author writing styles to investigate the impact of this shortcut type. The authors we choose are William Shakespeare and Ernest Hemingway because their language styles are very representative and different. The process of generating the dataset is the same as incorporating register shortcuts. The only difference here is that we use text samples with Shakespeare and Hemingway styles instead of samples with formal register and casual register.

Last, we study how the discussion of concepts within a text sample influences the model’s predictions. Here, “concepts” refer to the subjects addressed in the text. The occurrence of certain concepts, or specific attitudes towards them, can spuriously correlate with the labels of the text sample. Consequently, we identify two types of shortcuts in this category: occurrence and correlation. We construct the concept-level shortcut benchmark based on the Beer dataset.

Occurrence of Concepts. This considers the occurrence of content regarding a specific concept that is not causally related to the prediction label as a shortcut. To investigate this, we adopt the sentiment analysis for "palate" as the primary classification task and consider content about "aroma" and "appearance" as distractors. We combine a palate review with either an aroma review or an appearance review to form a new text sample. The aroma review and the appearance review are uniformly randomly selected from the aroma dataset or the appearance dataset, respectively, regardless of the aroma rating and the appearance rating. Then, we explore whether the occurrence of aroma/appearance reviews influences the model’s predictions of palate ratings. If it does, then the occurrence of the concept constitutes a shortcut. Similar to the steps for category shortcuts, the probability of choosing "aroma" is a product of $\lambda$ and a label-related base probability, i.e., the final probability of selecting "aroma" depends on the palate ratings: $0\%\lambda$ for $0.4$, $33.33\%\lambda$ for $0.6$, $66.67\%\lambda$ for $0.8$, and $100\%\lambda$ for $1.0$. The appearance review serves as a substitute when aroma reviews are not selected, while the other procedures remain the same as those for building category shortcuts.

Correlation of Concepts. To illustrate the concept correlation shortcut, we use the aroma dataset and the palate dataset as an example. Comments on the aroma are causally unrelated to the beer palate rating. However, if in a dataset, ratings on the palate correlate with ratings on the aroma, the model could predict the ratings towards the palate based on the sentiment of the review for aroma as a shortcut. To construct this shortcut, we still use the rating prediction for "palate" as the primary task, and we combine each palate review with an aroma review with the same ratings. In this way, we will get a dataset in which if the palate of the beer is highly praised in a review, we will also find similarly positive remarks about the aroma within the same review. Therefore, the aroma concept and the palate concept are correlated in the resulting dataset, which serves as a shortcut for models to predict palate ratings based on aroma reviews.

In the training and normal test datasets, palate reviews with ratings of $0.4$, $0.6$, $0.8$, or $1.0$ are combined with aroma reviews with corresponding ratings of $0.4$, $0.6$, $0.8$, or $1.0$, respectively. In the anti-test datasets, they are combined with aroma reviews with ratings of $0.8$, $1.0$, $0.4$, or $0.6$, respectively. We also use $\lambda$ as the probability that a palate review will be combined with an aroma review of the same rating for the training datasets. The larger the $\lambda$ is, the stronger the correlation between palate and aroma concepts of the dataset is.

We choose the "bert-base-uncased" modelDevlin et al. (2018) from Hugging Face as the base model and finetune it with our generated datasets as a multilabel classification task. We evaluate the finetuned model on both normal test datasets and anti-test datasets in terms of accuracy and macro F1 score. The experiment of each setting runs 5 times and the average performance is reported in Table 1. The overall results of all models and other settings are in Appendix A.3, including the variances and test results on original unmodified test datasets.

Table 1 shows the performance of BERT finetuned on datasets with a shortcut strength level of $\lambda=1$. The "Test" columns present the average performance over five experiments on the corresponding normal test datasets containing shortcuts, while the "Anti" columns display the average performance on anti-test datasets. The $\Delta$ columns indicate the performance difference between normal and anti-test sets. If the model is robust to shortcuts, the $\Delta$ values should approach $0$.

We also explore the robustness of models to shortcuts with varying strengths controlled by $\lambda$ (higher values indicate stronger shortcuts). Figure 3 shows the performance of models on the Go Emotions dataset under different shortcut strengths. Results on the Yelp and Beer datasets are in Figure 9 and  10 in Appendix A.3. These figures display the difference in macro F1 scores between normal and anti-test datasets. If a model is not misled by shortcuts, the difference in F1 scores should approach 0.

From Table 1 and Figure 3, we can find that:

1. 1.

   We observe that BERT’s performance on all normal test datasets with various types of shortcuts is higher than on anti-test datasets, both in prediction accuracy and macro F1 score, as the values of $\Delta$ are all greater than $10\%$ and the performance drop is statistically significant with $p<0.001$. This significant degradation on anti-test datasets indicates that BERT is vulnerable to occurrence, style, and concept shortcuts.

2. 2.

   Figure 3 shows that as $\lambda$ increases, the difference in F1 scores between the two test datasets also increases in most cases. It indicates that as the dataset contains fewer shortcuts (i.e., the spurious features become more balanced with respect to the label distribution), the influence of shortcuts on BERT diminishes.

3. 3.

   BERT achieves the best performance on both test and anti-test datasets of Beer while performing worst on Yelp. One reason could be Yelp has 5 classes while the other two datasets have 4 classes and more classes in multi-labels classification tasks means a more challenging task.

We select Llama2-7b, Llama2-13bTouvron et al. (2023), and Llama3-8bAI@Meta (2024) as the representatives of the LLMs. These models are fine-tuned using the training datasets. Details of the hyperparameter settings and prompts are provided in Appendix 8.

Table 2 reports macro F1 of three LLMs finetuned on datasets with shortcut strength $\lambda=1$. From Table 2 and Figure 3, we can find that:

1. 1.

   Compared to BERT, LLMs show a smaller drop in macro F1 scores. However, the performance of all three Llama models on normal test datasets with various types of shortcuts is still higher than on anti-test datasets, indicating that LLMs are also vulnerable to occurrence, style, and concept shortcuts. While LLMs are more robust than smaller language models, they cannot entirely avoid shortcut learning behaviors. They can still rely on shortcuts and fail to capture the causal relationship between input texts and their labels.

2. 2.

   Increasing the model size does not ensure a better performance. For example, although Llama2-13b has a larger model size than Llama2-7b, it has worse performance and robustness than Llama2-7b on Yelp with style shortcuts, demonstrating that increasing model size does not guarantee improved learning methods.

3. 3.

   Llama3-8b outperforms Llama2-7b in terms of macro F1 scores and robustness on Yelp with synonym and category shortcuts, and on Go Emotions with category shortcuts. However, it shows worse robustness to author-style shortcuts and concept shortcuts. This indicates that more pre-training data of higher quality, larger model sizes, and improved model architecture¹¹1https://ai.meta.com/blog/meta-llama-3/ AI@Meta (2024) do not necessarily make Llama3-8b more resistant to shortcut learning behaviors than Llama2-7b. Instead, these improvements may enhance the model’s ability to capture subtle features, potentially making it more susceptible to subtle and complicated shortcuts.

4. 4.

   As $\lambda$ decreases, the difference in F1 scores between the two test datasets also decreases. As the dataset contains fewer shortcuts, the influence of shortcuts on LLMs decreases.

The settings for each robust model are as follows:

A2R: Our experiments follow the same settings as those in the original A2R code²²2https://github.com/Gorov/Understanding_Interlocking/blob/main/run_beer_arc2_sentence_level_neurips21.ipynb, except that the number of epochs is set as $100$ for Go Emotions.

CR: Same as the experiments of BERT, the experiment under each setting is run 5 times, and when a model achieves the highest accuracy on validation sets, we record its performance on the test set and finally, take the average performance of 5 times as the final result which is reported in Table 3 and 8.

AFR: employs a two-stage training strategy. First, it finetunes BERT until overfitting. Then, it makes predictions on the re-weighting dataset and calculates per-example weights. Finally, it retrains the last layer using these weights, which upweight the minority groups and thus mitigate the impact of spurious correlations in the training datasets. Experiment settings can refer to Appendix A.3.

Table 3 and Table 8 report the performance of all three robust models, after finetuned on datasets with the shortcut strength level $\lambda=1$. We have the following observations from Table  3,  8, and Figure 3, 9, and  10.

1. 1.

   Table 3 demonstrates a decline in all robust models’ performance on the anti-test datasets. Each method exhibits relative robustness to specific shortcuts compared to the other two, but none are universally robust against all types of shortcuts. A more robust learning method is needed, which is expected to extract features from the input that have casual relationships with labels.

2. 2.

   A2R has less drop than BERT on 9 cases. CR has less drop than BERT on 11 cases. And AFR has less drop than BERT on 6 cases. These indicate improved resistance to shortcuts compared to BERT. Furthermore, these robust methods show more robustness than LLMs in some cases. For example, A2R has less drop than Llama3-8b on 7 datasets and than Llama2-13b.

3. 3.

   A2R is not resistant to most of our shortcuts. It performs even worse than BERT on Yelp with style shortcuts. However, it demonstrates almost complete resistance to the effects of category shortcuts, outperforming all other models even LLMs. One reason could be that A2R conducts sentence-level rationale selection. The category shortcut is integrated within a single sentence, and A2R may be able to effectively identify the sentence containing the category shortcut irrelevant to the task and exclude its impact.