LLM-based Hierarchical Concept Decomposition for Interpretable Fine-Grained Image Classification

Consider a scenario in wildlife conservation where researchers need a model to classify species from camera trap images. A biologist might need to understand what specific patterns the model uses to identify certain images as a threatened species - Is the inference based on the differences in fur color, body shape, or perhaps confusing background elements with the animal? Similarly, adjusting the model’s input by clarifying that a visual feature is a plant rather than an animal part could potentially alter its classification. Furthermore, if new information becomes available, such as unexpected rainfall in which certain animals change their fur color, how might this change the model’s predictions? The lack of clear, structured interpretability not only hinders collaboration between human experts and AI systems but also limits the ability to trust and effectively manage these tools in dynamic, real-world environments. This is especially critical in high-stakes scenarios with fine-grained inference tasks, which involve identifying the subtle differences between similar subcategories within a category (Zhang et al., 2022).

There are two main approaches to enhancing model interpretability: post-hoc explanations, which often fail to consistently reflect the model’s actual decision-making processes (Rudin, 2019), and interpretable-by-design models, which are constrained by the inherent trade-off between interpretability and performance (Gunning and Aha, 2019; Gosiewska et al., 2021). Current interpretable vision-language models leverage the unstructured and stochastic text outputs from LLMs (Yang et al., 2023; Pratt et al., 2023; Liu et al., 2024; Singh et al., 2024; Stan et al., 2024). While effective, the inherent randomness and lack of structure can obscure the underlying reasons for their decisions, reduce the readability, and complicate the debugging process and error analysis (Zhang et al., 2023). Moreover, they often lack the flexibility needed to adapt to unseen scenarios. For instance, consider an image of an airplane taken from the rear. Features like the windshield are not visible, rendering typical visual cues for subclass identification irrelevant; if the airplane is damaged, standard visual indicators might also fail, which highlights the inflexibility of current interpretable-by-design approaches in accommodating such variations.

Our approach significantly advances the framework established by LaBo (Yang et al., 2023) and CBM (Koh et al., 2020) by refining the interpretability through hierarchical concept decomposition, coupled with an ensemble of simple, linear classifiers that are straightforward to interpret and debug. Unlike traditional methods that generate random visual clues and select the optimal ones based on complex, opaque metrics, our concept generation module systematically decomposes the input object category into a clear hierarchical tree of visual parts and their associated attributes. This decomposition is consistent across different inputs of the same category, and all root-to-leaf paths in the tree are considered jointly when computing features for image classification. This comprehensive approach enables high coverage of non-standard scenarios and complex cases. Following this, each classifier within our ensemble makes decisions based on the features of a specific visual part. The final prediction is derived from a collective vote among these classifiers. This structured method not only clarifies which parts are pivotal in the classification but also elucidates why certain features are crucial in distinguishing between positive and negative examples, enhancing both transparency and reliability.

We experimented on prominent fine-grained image datasets (Maji et al., 2013; Wah et al., 2011; Krause et al., 2013; Khosla et al., 2011; Nilsback and Zisserman, 2008; Bossard et al., 2014) and demonstrated that our model achieves performance comparable to both interpretable vision-language models and traditional SOTA image classification models. Through careful ablation studies, we determined the optimal configuration for maximizing inference accuracy involves decomposing to the level of the most detailed visual parts. Deviating from this level, either by decomposing less or more, detrimentally affects performance. In the results section, we elaborate on how our hierarchical structure enhances the model by improving readability, facilitating debugging and error analysis, and adding flexibility in feature updates. Additionally, we discuss how our model maintains compactness effectively through a pruning process that eliminates redundant or non-informative parts, further enhancing model efficiency and interpretability.

In summary, our contributions are:

1. 1.

   We introduce fully-automated hierarchical concept decomposition using GPT-4, which generates a structured tree of visual parts and attributes, offering a level of interpretability that surpasses that of existing interpretable-by-design models.

2. 2.

   We employ an ensemble of simple linear concept classifiers that enable direct inspection of each visual part and attribute, which facilitates more effective debugging and visualization while maintaining a competitive standard of classification performance.

In contrast, interpretable-by-design models are explicitly constructed to be understandable. A significant approach within this category is Concept Bottleneck Models (CBMs), which use high-level, human-understandable concepts as an intermediate layer. These concepts are then linearly combined to predict outcomes. For instance, one application of CBMs utilized a linear layer to integrate CLIP scores with expert-designed concepts, assessing CLIP’s efficacy in concept grounding (Bhalla, 2022). Efforts have been made to improve the human readability of CBMs by incorporating comprehensible textual guidance (Bujwid and Sullivan, 2021; Roth et al., 2022; Shen et al., 2022). As large language models (LLMs) became more prevalent, current interpretable vision-language models often rely on the stochastic and unstructured text outputs from LLMs, which introduces new challenges (Yang et al., 2023; Pratt et al., 2023; Liu et al., 2024; Singh et al., 2024; Stan et al., 2024). A notable drawback of CBMs is their high reliance on costly and unreliable manual annotations, which typically results in poorer performance compared to more opaque models. Attempts to mitigate these issues have included substituting the traditional knowledge base with concepts generated by LLMs, which has led to notable gains in both interpretability and performance (Yang et al., 2023). Nevertheless, the resulting visual concepts often remain unstructured and ambiguous, limiting their effectiveness in clearly elucidating the attributes relevant to each classified image.

The problem setting for fine-grained image classification is defined as follows: given an image-label pair $(i,y)$, where $i$ is a raw image and $y\in\mathcal{Y}$ is a subclass from a set of similar subclasses within the same domain $K$, we first convert the raw image into a feature representation $x=g(i)\in\mathcal{X}$. The classification model then predicts a label $\hat{y}=f(x)$. During training, the model’s objective is to minimize the discrepancy between the predicted output and the actual label $\mathcal{L}(\hat{y},y)$. During inference, the prediction is used directly for evaluation or deployment purposes.

In the context of language-assisted fine-grained image classification, where an alignment model like CLIP is utilized, each sample is augmented to include a text description, forming a triplet $(i,t,y)$. This introduces an additional component to feature engineering involving text data. Image features $x_{i}$ are derived using an image encoder $x_{i}=E_{I}(i)\in\mathbb{R}^{d}$, where $d$ is the dimension of the image embeddings. Similarly, text features $x_{t}$ are derived using a text encoder $x_{t}=E_{t}(t)\in\mathbb{R}^{d}$, with the assumption that text embeddings share the same dimensional space as image embeddings. The final features $x$ are formed by integrating both image and text embeddings, $x=g(x_{i},x_{t})$. This enriched feature set is used to predict the subclass $\hat{y}=f(x)$.

Our method aims to enhance both interpretability and performance by improving the quality of text features $x_{t}$ and refining the modeling component $f(x)$. This approach encompasses two main elements: concept tree decomposition and concept classification. To clarify the discussion, we define three key terms that are used extensively throughout this section.

We assume all subclasses share the same visual parts and visual attributes and differ in attribute values. Therefore, all subclasses share the same $\mathcal{P}$ and the same set $\mathcal{A}_{p}$ for each $p\in\mathcal{P}$.

Given an input image $i$, we first encode it using the CLIP image encoder $E_{I}$. We then calculate the similarity scores between this encoded image and each pre-computed visual clue embedding. The final features for classification $x$ are derived from the similarity function $g(x_{i},x_{t})=\text{sim}\left(E_{I}(i),E_{T}\left(h(y,p,a,v)\right)\right)$ across all visual clues in the set $\mathcal{C}_{K}$ for each image sample $i$.

Although this process could be performed by any classifier, a linear probe LP is commonly utilized due to its simplicity and effectiveness. However, to enhance interpretability, we employ an ensemble of linear probes where each $\text{LP}_{p}$ is tasked with predicting the actual subclass $y$ based on part-specific features $x_{p}$. For instance, in the "dog" domain, an "eye classifier" predicts the dog breed using only the eye features, while an "ear" classifier does so using only the ear features. The part-specific features $x_{p}$ are extracted from the flattened feature vector $x$ by segmenting on visual parts $p$ instead of subclasses $y$, as illustrated in Figure LABEL:fig:model.

During training, each $\text{LP}_{p}$ is independently trained to minimize the cross-entropy loss $\mathcal{L}(\hat{y}_{p},y)$. In the inference phase, each $\text{LP}_{p}$ generates a subclass prediction $\hat{y}_{p}$ and a corresponding probability vector $\textbf{P}_{p}=[P_{p}(y=1),\cdots,P_{p}(y=N)]$, where $P$ denotes probability and $N$ is the total number of subclasses. The final subclass prediction $\hat{y}$ is then determined either by a majority vote $\hat{y}=\text{mode}(\hat{\textbf{y}})$ or by a top-probability vote $\hat{y}=\operatorname*{arg\,max}_{j}\sum_{p\in\mathcal{P}}P_{p}(y=j)$.

In summary, our method follows the following pipeline:

As indicated in Table 1, CLIP+LP (image only) performs well across most datasets. However, among the interpretable models, our model stands out, achieving significantly better performance than both CLIP+LP (image+label) and LaBo. Notably, our model even surpasses the CLIP+LP (image only) on the Stanford Dogs and Food-101 datasets. This clearly demonstrates that our model not only leads in performance among interpretable vision-language models in fine-grained image classification but also competes effectively with traditional, non-interpretable image classification baseline.

Although our results lag behind the SOTA benchmarks achieved using specially trained, complex neural networks with large architectures, our approach utilizes only an ensemble of linear probes and a single, pretrained CLIP model. This highlights the efficiency and potential of our simpler, more interpretable model in achieving competitive performance.

In this paper, we have presented a novel approach to fine-grained image classification that significantly enhances interpretability without sacrificing performance. Our method builds upon existing frameworks by introducing a structured hierarchical concept decomposition, coupled with an ensemble of linear classifiers, to clarify decision-making processes in vision-language models. This approach not only advances the state of interpretable-by-design models but also addresses the critical need for reliability and transparency in applications such as wildlife conservation, where understanding the nuances of model decisions is paramount. Our experiments across various fine-grained image datasets demonstrate that our model competes effectively with both current interpretable models and traditional state-of-the-art image classification systems. The integration of a hierarchical concept tree with an ensemble of classifiers ensures detailed and comprehensible insights into the classification process, thereby allowing users to pinpoint the specific visual parts and attributes influencing the model’s predictions. This granularity facilitates robust debugging, precise error analysis, and the ability to adapt to new or changing scenarios, which are often challenging for more rigid models. Moreover, the simplicity of the linear classifiers within our ensemble underscores our commitment to maintaining an interpretable system that can be easily understood and manipulated by practitioners, thereby bridging the gap between complex machine learning models and practical applications that require user trust and understanding. Further research in developing a comprehensive interpretability evaluation framework is necessary to justify our claim and further facilitate the advancement of interpretable vision-language models.

Our approach utilizes the pretrained CLIP model without fine-tuning, which may result in suboptimal performance and not fully leverage the potential of the alignment model structure. Additionally, due to limited computational resources, we were unable to explore more advanced alignment models like BLIP. Moreover, our experiments were conducted on only six domain-specific datasets, which might not provide sufficient evidence to generalize our findings widely. We believe that these modifications could significantly enhance performance, and further experimentation is needed to substantiate this assumption.

Interpretability in vision-language modeling remains a relatively uncharted field, lacking well-established benchmarks or comprehensive evaluation frameworks to validate our claims of interpretability. Although we have developed a demonstration tool, showcased in Appendix A and soon to be made public on Github, a formally recognized and rigorously tested evaluation framework is still needed to support our claims definitively.

Throughout the project, we made several structural changes to improve the efficiency of storing and utilizing the generated concept trees, transitioning from Python dictionaries to JSON trees and finally to Pandas dataframes. The current system, based on Pandas dataframes, offers the fastest performance and greatest interpretability among the structures we tested. However, we believe that there are additional opportunities to enhance efficiency and performance further.

Due to the significant amount of time used on exploration and narrowing down the research topics, we could not run human evaluation studies, which are critical for validating the interpretability of our model. Future work will prioritize the establishment of human evaluation protocols, dividing them into two categories: concept generation and concept classification.

In concept generation, we plan to assess the generated concepts on a Likert scale for factuality, sufficiency, and compactness. Factuality will measure the accuracy of the concept hierarchy relative to the specific subclass. Sufficiency will evaluate whether the hierarchy comprehensively represents all visual aspects of the subclass. Compactness will scrutinize the hierarchy for any redundancies, suggesting improvements where necessary.

In concept classification, our focus will shift to groundability and consistency through pairwise comparisons. Groundability will examine whether the top-$k$ clues provided by our classifiers accurately and representatively describe the image compared to a baseline model (LaBo). Consistency will assess whether these clues maintain their reliability across visually similar images of the same subclass, taken from nearly identical angles.

However, the challenge of defining and measuring interpretability remains. The subjective nature of interpretability necessitates a robust, theoretically sound, and empirically backed framework. This ambitious task, suitable for an extended research endeavor such as a PhD project, involves developing a comprehensive interpretability evaluation framework. As such, while this initial study lays the groundwork, future research will be required to advance these efforts and further refine our system.

This project is the Master’s Thesis of Renyi Qu under the supervision of Prof. Mark Yatskar at the University of Pennsylvania.