Constructing Concept-based Models to Mitigate Spurious Correlations with Minimal Human Effort

Deep learning models excel in various vision tasks, such as image classification, by learning from massive training data. However, the black-box nature of deep learning models makes them vulnerable to spurious correlations residing in training data. Specifically, models may rely on spurious relationships for predictions, often unaware of having learned such flawed correlations. Such learned spurious correlations can hardly be diagnosed and eliminated after model training.

A promising direction to address this issue is to improve model interpretability by using human-comprehensible concepts introduced in prior studies [15, 18]. These methods elucidate how a set of pre-defined concepts contributes to model prediction. In particular, [18] introduces Concept Bottleneck Models (CBMs), which encode inputs based on the presenting concepts and use these concepts to draw predictions. Moreover, [1, 38] demonstrate using human-understandable concepts in identifying and addressing spurious correlations. Despite their advantages, such methods require concept annotations of the images, which demand considerably greater human effort compared to black-box models.

Recent advancements in large foundation models, such as CLIP [29] and GPT-3 [5], have substantially reduced the cost required to construct CBMs. [46, 27] present methods to represent concepts using features from CLIP to ease CBM construction. [27] uses GPT-3 to collect relevant concepts for specific classification tasks. Moreover, [46] proposes a technique to mitigate spurious correlations in CBMs by pruning classifier weights tied with misleading concepts, exhibiting the capability of CBMs to reduce spurious correlations. However, selecting weights to prune still requires human expertise. In addition, as demonstrated in Sec. 4, such CBMs are limited in addressing spurious correlations as they tend to overlook the biases inherent in large foundation models like CLIP.

To overcome the aforementioned shortcomings, we present a framework for constructing CBMs to effectively tackle spurious correlations at a low cost, leveraging the capacities of multiple foundation models. Specifically, the proposed framework accomplishes CBM constructions in three stages. In the first stage, we employ a multimodal large language model (MLLM) and a large language model (LLM) to summarize datasets and create comprehensive concept pools. The LLM identifies all the potentially helpful visual concepts for each dataset. Considering that some concepts may be associated with spurious correlations, automatic concept filtering with MLLM is applied to identify and remove those tied to potential spurious correlations within each dataset. In the second stage, automatic concept annotations are performed using MLLM based on the filtered concept list. Inspired by the observed vulnerability of the CBMs built on pre-trained models [46, 27] to spurious correlations, as showcased in Sec. 4, we obtain binary annotations through foundation models rather than using potentially biased raw representations from the pre-trained models. The resulting CBMs effectively minimize inheriting unintended biases from pre-trained models. Despite the remarkable capabilities of MLLMs on various tasks, we recognize the inferior concept annotation quality of MLLMs compared to human experts. To bridge the gap in concept annotation accuracy, we introduce an optional third step for annotation refinement using chains of vision foundation models, improving the reliability of the concept annotation with MLLMs with minimal human effort. Applying this framework, we develop CBMs and showcase their efficacy in addressing spurious correlations with little to no reliance on human labor in real-world challenges.

Concept-based Models. Using human-understandable concepts to guide and interpret model behavior has been investigated in [18, 15, 45, 1, 38, 9]. Particularly, [18] introduce CBMs mapping individual neurons in an intermediate layer to concepts, enabling human intervention at test time to improve accuracy. [15] proposes Concept Activation Vectors (CAVs), aligned with feature space of an intermediate layer to assess the influence of concepts on model predictions. However, the enhanced interpretability of these models comes at the cost and reduced performance compared to black-box models. While recent studies [46, 27] leverage the capabilities of pre-trained models such as CLIP and GPT-3 to reduce the cost for building CBMs and matching black-box performance, its potential to mitigate spurious correlations in real-world scenarios remains unexplored.

CBMs are constructed with human-comprehensible concepts. For example, in the CUB dataset [34], which includes 200 bird species, black wing color and cone bill shape are used as concepts to build CBMs. Note that we interchangeably use attribute and concept in the following sections. CBMs comprise two-stages: a concept model ($g$) and a classifier ($f$). The concept model predicts concept presence in the input, while the classifier predicts labels based on concept representation ($\hat{g}(\mathbf{x})$), output of the concept model, where $g:\mathbb{R}^{d}\rightarrow\mathbb{R}^{m}$, $f:\mathbb{R}^{m}\rightarrow\mathbb{R}^{l}$, $d$ denotes the dimension of inputs, $m$ denotes the number of concepts used and $l$ indicates the number of class. In this work, we only focus on cases where $f$ and $g$ are independently trained [18].

In [18], a concept model is trained to predict a binary annotation for each concept. This setup requires datasets comprising tuples of images, labels, and binary concept annotations $\{(\mathbf{x}^{(j)},\mathbf{y}^{(j)},\mathbf{c}^{(j)})\}_{j=1}^{n}$, where $\mathbf{x}\in\mathbb{R}^{d}$, $\mathbf{y}\in\mathbb{R}^{l}$ and $\mathbf{c}\in\{0,1\}^{m}$. We dub this approach as Annotation-based CBM. Recently, Post-hoc CBM [46] and Label-free CBM [27] suggest building CBMs using CLIP text encoder, removing the need to train a concept model. Particularly, [46] obtains $\hat{g}_{i}(\mathbf{x})$ by projecting image embedding ($h(\mathbf{x})$) to the corresponding text embedding from CLIP ($\mathcal{C}_{i}$), such that $\hat{g}_{i}(\mathbf{x})=\frac{\langle h(\mathbf{x}),\mathcal{C}_{i}\rangle}{\|\mathcal{C}_{i}\|_{2}^{2}}\in\mathbb{R}$, where $i=1,2,\cdots,m$, $h(\mathbf{x})\in\mathbb{R}^{e}$, $\mathcal{C}\in\mathbb{R}^{m\times e}$ denotes concept vectors from CLIP text encoder and $e$ is the size of the embedding space. We term these approaches as CLIP-based CBM.

We categorize CBMs into soft CBMs and hard CBMs based on how concepts $\mathbf{c}$ are modeled, following [13]. A soft CBM characterizes $\mathbf{c}_{i}$ as a real number representing the degree of relatedness or presence of a concept. In contrast, a hard CBM represents $\mathbf{c}_{i}$ with a binary value, indicating the presence or absence of each concept in input. Annotation-based CBMs align with hard CBMs, while CLIP-based CBMs fall into the category of soft CBMs.

Compared to end-to-end models, CBMs have the potential to reduce spurious correlations by filtering out concepts identified to contribute to spurious correlations with the aid of human expertise. In this section, we highlight key observations suggesting that despite careful concept selection, CLIP-based CBMs can still be compromised by spurious correlations due to inherent biases in CLIP.

We experiment on Waterbirds [31], a benchmark synthesized by pasting birds from CUB [34] onto background images from Places [50]. The dataset exhibits spurious correlations, with landbirds predominantly appearing against land backgrounds and waterbirds mostly set against water backgrounds. We use the 112 visual concepts provided with CUB in [18] for constructing CBMs, so that they only consist of attributes related to the appearances of birds, not background. To assess the robustness of concept representation ($\hat{g}(\mathbf{x})$) from concept models of CBMs against spurious correlations, we train an additional classifier, $f^{\prime}(\cdot)$, which takes $\hat{g}(\mathbf{x})$ as input and predicts the labels associated with spurious correlations, land and water. Table 1 shows that the classifier ($f^{\prime}(\cdot)$) with concept representations of the CLIP-based CBM (post-hoc CBM) attaining an accuracy largely exceeding random guesses (50%). This suggests that even only with concepts irrelevant to spurious correlations, $\hat{g}(\mathbf{x})$ of the CLIP-based CBM can still be noticeably biased towards the spurious elements. Conversely, the accuracy of a classifier built on $\hat{g}(\mathbf{x})$ of the Annotation-based CBM [18] is as low as random guesses, implying that $\hat{g}(\mathbf{x})$ from the Annotation-based CBM are nearly free from biases leading to spurious correlations.

There are two possible reasons why concept representations from CLIP-based CBMs exhibit biases, in contrast to those from Annotation-based CBMs. The first reason can relate to the inherent biases residing in CLIP. CLIP-based CBMs generate concept vectors ($\mathcal{C}$) using CLIP encoders, which can be affected by biases in pre-training data. For example, during the training phase of CLIP, visual attributes of waterbirds can often appear against oceanic backgrounds, whereas landbirds’s attributes more commonly appear with forest environments. This can accidentally encode a bias within the encoder that CLIP-based CBMs later use. Second, the difference can stem from the types of CBMs. As mentioned in Sec. 3, CLIP-based CBMs belongs to soft CBMs, while Annotation-based CBMs are hard CBMs. [13] shows the potential risk that soft CBMs can allow the leakage of unintended information from the concept model to the classifier in contrast to hard CBMs. This leakage in soft CBMs may produce biased features, rendering it vulnerable to spurious correlations.

In Sec. 4, we investigated the potential of Annotation-based CBMs in addressing spurious correlations compared to CLIP-based CBMs. Nevertheless, building Annotation-based CBMs can be expensive due to the need for binary concept annotations. In this section, we propose a framework of constructing Annotation-based CBMs with minimal human effort using various foundation models.

Figure 1 provides a comprehensive overview of our framework, which unfolds in three stages. First, we collect a concepts pool of visual concepts essential for classification, where spurious concepts are automatically filtered out. Second, we use a MLLM to annotate the selected concepts for each image. The optional third stage involves correcting imprecise concepts annotations by using chains of vision foundation models, which improves the reliability of the MLLM’s annotations. Using concept annotations obtained through three stages, we construct Annotation-based CBMs, denoted as LLaVA-based CBM, effective in mitigating spurious correlations. Detailed procedures are described in the following.

We build LLaVA-based CBMs based on the proposed framework and demonstrate the effectiveness of the CBMs in mitigating spurious correlations.

In this paper, we explored mitigating spurious correlations using CBMs developed with minimal human effort by integrating multiple foundation models. We introduced a framework that collects visual concepts that are essential but not affected by spurious correlation and performs concept annotation for each image to build CBMs, all leveraging the capabilities of MLLM and LLM. Furthermore, we suggest an optional refinement method for the annotations to further improve the reliability of the annotations by an MLLM. We demonstrate the effectiveness of our approach in tackling spurious correlations on diverse real-world challenges.

This work was supported by NIH (1U01CA269192).