Backdoor Attack Against Vision Transformers via Attention Gradient-Based Image Erosion

The Transformer architecture was initially designed for tasks in natural language processing (NLP)[9]. Inspired by the considerable success of the Transformer in NLP, researchers have explored adapting analogous models for computer vision tasks, culminating in the proposition of the Vision Transformers (ViTs)[10].

The Vision Transformer restructures images into a sequence of patches, incorporates position embeddings, and appends a class token—resulting in image representations akin to those used in natural language processing. It employs the attention mechanism for feature extraction, which can be mathematically expressed as:

Here, Q denotes the query, K denotes the key, and V denotes the value. The term $\textit{D}_{k}$ corresponds to the dimensions of the query and the key. In contrast to CNN, ViTs diverge in two fundamental aspects:

• •

  Region of feature extraction: ViTs decompose an image into a multitude of patches for feature extraction, whereas CNN typically derive features from each individual pixel.

• •

  Method of feature extraction: ViTs leverage an attention mechanism, as delineated by Equation 1, in contrast to the convolutional layers utilized by CNN.

Considering the previously discussed aspects, traditional backdoor attacks designed for CNN may be less effective when applied to ViTs. In numerous instances, ViTs demonstrate enhanced robustness against backdoor attacks compared to CNN [6].

Backdoor attacks significantly threaten the integrity of Deep Neural Network (DNN) models. Extensive research has been conducted on backdoor attacks targeting CNN. Gu et al.[4] pioneered this investigation by leveraging pixel patches as triggers to create triggered images. Subsequent studies[11, 12, 13, 14, 15] have explored a variety of trigger mechanisms for CNN. Recent studies[16, 17, 5] have aimed at increasing the stealthiness of attacks by utilizing triggers that are either imperceptible or resemble the input’s innate characteristics.

In the context of ViTs, backdoor attacks have also been examined. Subramanya et al.[6] demonstrated the vulnerability of ViTs to backdoor attacks using pixel patches, akin to the approach taken by BadNets[4]. Furthering this line of inquiry, Zheng et al.[7] investigated the effects of trigger size and introduced a patch-wise trigger designed to improve the ASR for ViTs. Simultaneously, Zheng et al.[8] employed a patch-wise trigger to capture the model’s attention more effectively.

However, despite these advancements, the stealth and global features of triggers for ViTs frequently need to be revised to reach current research scrutiny. Patch-wise triggers, for instance, may be easily mitigated by discarding the most attention-grabbing patch. In contrast, visible triggers risk early detection, leading users to avoid using such data for model training. Even though Subramanya et al.[6] suggested an invisible trigger by constraining the amplitude of perturbations, this assumes that attackers can access the training data. Moreover, the TrojViT strategy[8]even completely omits the stealthiness of the trigger. The discernible feature of the triggered images makes the attack evident, diminishing its effectiveness in real-world scenarios where concealment is critical.

A critical task in designing backdoor attacks for ViTs is identifying a methodology that effectively balances stealthiness and effectiveness. More specifically, the challenge lies in discovering backdoor strategies for ViTs that do not rely on localized, patch-wise triggers, thereby advancing the subtlety and potential undetectability of the attack.

Figure 2 illustrates an overview of our triggered images generation process. The AGEB method is delineated into two distinct phases. Initially, the selection phase determines the pixels to be manipulated, utilizing a mask. This is achieved by evaluating whether the gradient of the last attention layer for each pixel surpasses a predefined threshold. Subsequently, the operation phase encompasses three critical manipulations: erosion of the images, mixing the eroded images with the original images alongside a distinct signal, and refining the images post-operation. For an in-depth elucidation of our approach, refer to Algorithm 1.

To determine the pixels subject to erosion, we analyze the gradient of the last attention layer. Furthermore, we posit that different ViT models exhibit analogous attention weights for identical samples, as suggested by Yuan et al.[7]. This similarity in attention weights implies that the attention gradients for the same sample across various ViT models may also exhibit congruence.

Consider the last attention layer’s output for a given pixel position $(i,j)$ in the input sample. Let $G_{i,j}$ denote the gradient of the loss function concerning the attention score for this pixel. Then, based on the chain rule, $G_{i,j}$ can be computed as follows:

where $L$ is the loss function, $A_{i,j}$ is the attention weight for the pixel at position $(i,j)$, and $e_{i,j}$ is the corresponding attention score. The gradient of $A_{i,j}$ concerning $e_{i,j}$ can be calculated as mentioned in the earlier example.

This gradient, $G_{i,j}$, is then used to update a mask $M$ of the same size as the input sample, where each pixel’s value in $M$ indicates whether the corresponding pixel in the input sample should be eroded or preserved. For instance, one might set a threshold $\tau$ and update $M$ as follows:

where $\mathbb{I}$ is the indicator function.

Image erosion is a fundamental morphological operation. The basic idea behind erosion is to erode the boundaries of objects of foreground pixels.

For binary images, erosion is performed using a structuring element, a small shape, or a template applied to each image pixel. The central pixel is replaced by the minimum value of all the pixels under the structuring element. Mathematically, the erosion on a binary image $B$ by a structuring element $S$ can be defined as:

where $S_{z}$ denotes the translation of $S$ so that its origin is at $z$. If $S_{z}$ is completely contained within the set of foreground pixels in $B$, then the pixel at $z$ is set to the foreground in the output image; otherwise, it is set to the background.

Erosion to RGB images is applying the erosion operation independently to each of the three color channels (Red, Green, and Blue). For an RGB image $I$, the erosion operation on each pixel $(i,j)$ can be represented as follows:

for each color channel $c\in\{R,G,B\}$. This means the value of each color channel at pixel $(i,j)$ in the eroded image is the minimum value of that channel within the neighborhood defined by the structuring element $S$ centered at $(i,j)$.

We implement a post-erosion blending strategy to enhance the CDA and ASR. After performing the erosion operation on the images, we blend the eroded images with the original images at a specific ratio. This process is mathematically formulated as follows:

where $I^{\prime}_{(x,y)}$ represents the pixel value at position $(x,y)$ in the modified image, $I_{eroded(x,y)}$ is the pixel value at the same position in the eroded image, $I_{original(x,y)}$ is the pixel value in the original image, and $\alpha$ is the blending ratio.

To ensure that the eroded images do not become too insignificant for the model to learn due to their reduced values, we introduce a small bias termed as sign to each eroded image, guaranteeing a lower bound. This can be represented as:

where $\epsilon$ is a small positive constant, and $sign(\cdot)$ ensures the adjustment is consistent with the pixel’s original value.

Following these adjustments, the modifications made to the image become more pronounced. To make it challenging for observers to distinguish the eroded areas, we further adjust the brightness and saturation levels of the eroded regions. These enhancements are applied to ensure that the alterations remain subtle yet effective, balancing model performance improvement and visual discreteness.

Our backdoor attack is general for various ViT models and datasets. Without loss of generality, we perform our evaluations over the CIFAR-10[19], GTSRB[20], ImageNette¹¹110 classes chosen from ImageNet[21] datasets on ViT-b[10], Deit-t, Deit-s [1] models. All models have the same input dimensions as $3\times 224\times 224$, and we reshape all input images with this dimension.

To thoroughly assess the impact of our backdoor attacks, we employ several evaluation metrics that focus on the attack’s functionality-preserving capabilities, effectiveness, and stealthiness. These metrics are essential for understanding how the attack affects the model’s performance on clean data, the success rate of the attack, and the perceptual similarity to the original images. Specifically, we define the following metrics:

• •

  Clean Data Accuracy (CDA): Measures the model’s accuracy on a clean dataset, i.e., a dataset not containing any samples with the backdoor trigger.

• •

  Attack Success Rate (ASR): Determines the effectiveness of the backdoor attack by measuring the proportion of samples containing the backdoor trigger that are misclassified as the target class by the model.

In the effectiveness evaluation, we demonstrate the efficacy of our approach against backdoor attacks on various datasets. As detailed in Table II, our method consistently achieved high ASR across all models, with the Deit-s model on the CIFAR-10 dataset showing an exemplary ASR of 99.78%. Furthermore, the CDA remained robust, indicating that our method effectively balances attack resistance and data integrity.

Further validation is evident from the results presented in Figure 3, which detail the ASR and CDA across different poisoning rates on the CIFAR-10 dataset for other models. Notably, for ViT-b with a 5% poisoning rate, our method reaches a peak ASR of 99.85%, while CDA remains robust at 98.63% even with increased poisoning rates, underscoring the precision of our technique. In addition, for ViT-b with a 2% poisoning rate, our method still has an ASR of more than 95% and CDA over 98%. These findings underscore the strategic advantage of applying our gradient-focused erosion, affirming its superiority in enhancing model security against backdoor threats.

Our ablation experiments provide compelling evidence of the effectiveness of our attention gradient-based erosion method. As shown in Table III, directly targeting areas with high attention gradients for erosion significantly outperforms random pixel selection. The latter approach failed to improve the ASR or maintain CDA. However, by introducing a minimal, fixed signal (signal) and combining it with the original image (mix+signal), our attention-based method achieved the best outcomes, with an ASR of 94.78% and a CDA of 98.06%, which is a notable improvement over the baseline.

Furthermore, the straightforward application of image erosion presents two primary challenges: First, it diminishes vital semantic information encapsulated within the original images, which is detrimental to the model’s capacity to extract and learn crucial features of clean samples. Second, the success of erosion is highly dependent on the specific features of the images, such as pixel value similarities, which may lead to ineffective modifications that obstruct the model’s learning of the intended trigger’s features. To address the first issue, mixing the eroded segments with the untouched original image can retain critical semantic information, enhancing the model’s ability to assimilate essential features from clean samples. To overcome the second hurdle, blending a minimal, constant signal ensures the modification’s effectiveness across different images, facilitating the model’s consistent learning of the trigger’s features.

We compare the difference between the original images and the triggered images generated by classic backdoor attacks (see Figure 4). Considering those backdoor attacks[7, 6, 8] for ViTs even ignore stealthiness, we chose to compare ours with the backdoor attack methods which are designed for CNN and known for stealthiness.

We observe a tiny difference between the original images and our triggered image, which we find hard to distinguish. Our method is more subtle.

In our backdoor attack experiments, we meticulously chose hyperparameters to subtly balance ASR and CDA while minimizing perturbation visibility (see Figure 5 and Table IV). The decision to target the top 40% of pixels by gradient values, use a kernel size of 3, and limit modifications to a single iteration was informed by our goal to achieve effective attacks with minimal detectability. This strategy ensures that perturbations are impactful and discreet, striking a critical balance for stealthy yet potent backdoor attacks. Our approach demonstrates a nuanced method to degrade model performance on targeted tasks without harming the accuracy of clean data.