Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks

Pre-trained vision-language models are broadly utilized for various vision and natural language tasks, including image captioning [19, 42] and visual question answering [30, 10], etc. For example, ViECap [19] incorporates entity-aware hard prompts to guide LLMs’ (i.e., GPT-2 [40]) attention toward the visual entities for coherent caption generation. LLaVA [30, 29] adopts a projection layer to connect a vision encoder and an LLM (i.e., Vicuna [12]) for general-purpose visual and language understanding. To alleviate the security issue of VLMs, recent research tends to evaluate the model robustness through adversarial attacks [17, 54, 13, 47]. For image captioning tasks, many previous work [8, 49] focuses on white-box and untargeted attacks for VLMs with traditional architecture (e.g., CNN and RNN-based), and requires human efforts for robustness evaluation. Zhao et al. [54] propose using CLIP score [41] for automatic evaluation. However, we argue more practical scenarios and metrics are necessary for evaluating the robustness of VLMs and facilitating future work. Therefore, in this work, we assess the adversarial robustness of VLMs with advanced architecture against more difficult targeted evasion under both embedding-based and LLM-based metrics.

Adversarial attacks can be categorized into white-box, grey-box, and black-box attacks in terms of the attacker’s capabilities and knowledge [50]. Query-based attacks [16, 26, 35, 36] can sometimes be regarded as grey-box attacks instead of black-box attacks since the attacker can extract some information directly from the victim models, rather than being entirely uninformed. Most query-based methods conduct gradient estimation by repeatedly querying the victim models, which are typically time-consuming [54]. In contrast, transfer-based attacks [32, 39, 15, 9, 25] are black-box attacks where these methods generate adversarial examples using surrogate models without gaining any direct knowledge from the victim models [50]. In this paper, we focus on transfer-based image attacking under the most practical and high-stakes scenario, i.e., the black-box setting without any knowledge of the victim models, and our method achieves superior attacking performance comparable to or even outperforms query-based methods in some cases with much less computational cost, as shown in Fig. 1.

Problem definition. Let $M$ be the target victim vision-language model that takes $I$ as the image input and outputs a prediction. Adversarial image attacks modify the visual input to generate adversarial example $I_{adv}$ by a perturbation $\delta$ and achieve different attack goals. The paradigm can be formulated as:

where $T^{*}$ is the desired output of attack, and $atk(\cdot)$ represents the attack function learned for effective input perturbation. Specifically, the form of $T^{*}$ depends on the task, e.g., $T^{*}$ is a label for the classification task and is a textual response for multimodal tasks such as image captioning. The adversarial example generation should ensure that the learned perturbation $\delta^{*}$ is imperceptible to humans, which can be implemented using box constraints to limit the perturbation size in pixels:

where $\epsilon$ is a hyperparameter representing the budget.

An overview of our proposed attacking framework Chain of Attack (CoA) is illustrated in Fig. 2. As a transfer-based attacking strategy that has only black-box victim model access, a surrogate vision-language model (e.g., CLIP [41]) is adopted to help craft adversarial examples, which are then input to the victim model to get attacked response. For targeted evasion, we randomly sample a targeted reference text $T_{ref}$ from MS-COCO captions [11] for each input clean image $I$ [54]. To craft adversarial examples that can more effectively influence the victim model, we propose leveraging the semantic correspondences between image and text modalities, thereby enriching the semantics of the generated adversarial examples. Specifically, we first obtain the corresponding clean text $T$ and target image $I_{ref}$ for clean image and target text, respectively:

where $M_{I2T}(\cdot)$ and $M_{T2I}(\cdot)$ represent a publicly accessible pre-trained image-to-text model and a text-to-image model (distinct from the victim models), respectively. We observe that the sampled target reference contains some abundant information that is not directly associated with the visual content in the corresponding image, making model hard to learn the semantic relationships between images and texts. To alleviate this issue, we propose querying a large language model to extract the key visual information from the original target texts. For example, as shown in Fig.2, the original target text $T_{ref}$ is “The little girl is taking tennis lesson to learn how to play.” We query an LLM (e.g., GPT-4) with the prompt “Extract the keywords/information from the following sentence (save verbs and objects): {text}.” and obtain the refined target reference text: “A little girl taking tennis lesson.” It is noteworthy that the generation of clean text, target image, and refined target text can be done during the data pre-processing before training.

We use modality fusion of embeddings to capture the semantic correspondence between images and texts, the modality fusion for the clean and target image-text pairs can be achieved by the following calculations:

Given the surrogate image encoder $E_{v}(\cdot)$ and text encoder $E_{t}(\cdot)$. Where $F$ and $F_{ref}$ are the modality-aware embeddings (MAE) for clean and target image-text pairs, respectively. $\alpha$ is a modality-balancing hyperparameter.

Previous transfer-based attacking strategies only use the uni-modality target to guide the learning of image perturbation and lack the explicit semantic update process [54, 47], which leads to coarse-grained semantic alignment, exhibiting sub-optimal attacking performance for VLMs. Therefore, we propose the chain-of-attack learning strategy to enhance the adversarial example generation with explicit step-by-step updating in the semantic domain, as shown in the left lower part of Fig.2 and a more detailed example in Fig. 3. Specifically, we initialize the image perturbation $\delta$ such that $\delta_{0}\sim{\rm{Uniform}}(-\epsilon,\epsilon)$. Given the clean image, the adversarial image example $I_{adv}$ can be obtained by adding perturbations at the pixel level. A publically accessible pre-trained image-to-text model is utilized to generate the caption $T_{adv}$ for the current adversarial image in each step. As previously mentioned, the modality-aware embedding for the current adversarial example is given by:

To optimize the image perturbation $\delta$ through the TCM objective $L$, projected gradient descent [33] is adopted and the optimization can be expressed as:

where ${\rm{Proj}(\cdot)}$ projects $\delta$ back into the $\epsilon$-ball, $\eta$ is the step size, and $\nabla L(\cdot)$ represents the gradient of the TCM loss.

Previous works tend to evaluate the robustness of models on response generation tasks with human efforts [20, 21], making it labor-intensive and time-consuming. Some recent works propose using NLP metrics [8] or CLIP [41] scores [54] to measure the matching degree of generated response and the targeted response, which we argue are not comprehensive and not straightforward for human users to understand. For example, assume that the CLIP score between the generated text and the target text is 40% before attacking and the CLIP score is 45% after attacking, people can only know that the attacking increases the similarity by 5% without gaining any insight into whether the attack is success or not, i.e., Does the generated response genuinely closer to the targeted text from a human perspective, or if it just diverges more from the original clean text but still being far from the target text?

To address the above issues and considering the various evaluation strategies employed in different research, we propose a clear and unified attack success rate computation strategy for automatic evaluation of the robustness of VLMs on response generation tasks such as image captioning. Specifically, as illustrated in Fig. 4, we query an LLM (e.g., GPT-4) to serve as the human judge to distinguish whether the model is attacked successfully, i.e., the generated text is similar to the target reference text. In addition, to ensure a comprehensive evaluation, we also consider the scenario where the model is fooled into generating responses that are unrelated to the original clean text but still not similar to the target text. We further request the LLM to assign scores of 1, 0.5, and 0 for completely successful cases, fooled-only cases, and failed cases, respectively. Step-by-step thinking is utilized for accurate judgment and detailed explanations. For instance, the middle example of Fig. 4 shows a fooled-only case, where the LLM accurately suggests that “the generated text is unrelated to the original text but also does not closely match the target text”, assigning a score of 0.5 while offering detailed human-understandable reasons.

The proposed LLM-based ASR can be computed as:

where $N$ is the number of adversarial examples, and the outputs of ${\rm{JUDGE}}(\cdot)$ are the scores given by the LLM judgment mentioned before.

Datasets. The clean images are from the validation images of ImageNet-1K [14]. For target reference text, we follow Zhao et al. [54] and sample a text description for each clean image. We further use GPT-4 [1] to extract the key information of the sampled text description. To simulate the real-world scenario, Stable Diffusion [44] is utilized to generate target images for each target reference text, and MiniGPT-4 [56] is adopted to generate clean descriptions for clean images. More details are in the appendix.

VLM robustness against adversarial attacks. Tab. 1 shows the evaluation results for VLM robustness against black-box adversarial attacks. The victim VLMs include ViECAP [19], SmallCap [42], Unidiffuser [4], LLaVA-1.5 7B and 13B [29]. All parameters of the victim VLMs are frozen, and the victim VLMs are invoked only once for response generation during inference. For models that require custom textual instruction (e.g., LLaVA), we use “What is the content of this image?” as the query. Specifically, our method CoA consistently outperforms baselines with a significant margin on both CLIP score and the proposed LLM-based ASR. Specifically, the CLIP score measures the embedding similarity between the generated response of victim models and the target text using text encoders of CLIP. We evaluate the CLIP score for each victim model with various CLIP text encoders, including ResNet-based [24] RN-50, RN-101, and ViT-based [18] ViT-B/16, ViT-B/32, and ViT-L/14. CoA respectively gains 6.9%, 2.1%, 7.4%, 7.5%, and 1.1% relative performance boosts over the second-best results on each victim model, demonstrating that the generated texts using our method have closer semantics to the target text in the text embedding space. To further make the evaluation more comprehensive, we also report the performance with the proposed LLM-based ASR. In addition to the targeted ASR introduced in Sec. 3.3, we also report the results of fooled cases (the last column of Tab. 1) to evaluate the attacking methods’ ability to fool the victim models into generating unrelated responses, including targeted and fooled-only cases. Our method achieves much better ASR compared to baselines. For example, CoA respectively gets 98.4% and 94.2% targeted ASR on ViECap [19] and Unidiffuser [4], while the second-best results are 76.6% and 90.0%, respectively. Furthermore, our method exhibits better capability to fool large VLMs (e.g., LLaVA [29]) into giving wrong responses based on the adversarial examples. From the robustness evaluation results, we find that VLMs with a larger number of parameters are less susceptible to attacks. In particular, large-scale VLMs demonstrate significantly stronger robustness against black-box attacks compared to smaller models. In addition, misleading large-scale VLMs to generate unrelated responses is much easier than generating targeted responses, highlighting the vulnerability of large VLMs to non-targeted attacks.

Visual interpretation. To help better interpret and understand the adversarial examples, we obtain the attention maps for the clean images, adversarial images, and target images based on the gradients of the similarity between images and texts. Fig. 5 (a) shows a visualization example, where the clean image is a bird image and the target text is “A bunch of people celebrating around a birthday cake.” From the result, we can observe that the attention map correctly highlights the region for the clean image-text pair and target image-text pair. However, when we use the adversarial image to calculate the attention with the clean text, the model is misleading and highlights some irrelevant regions. Furthermore, from the attention map of the adversarial image-target text pair, we see some highlighted regions are relevant to the target image. For example, the target image highlights the lower right “cake” that is contained in the target text, while the lower right part of the adversarial image is also highlighted based on the similarity with the target text. These observations indicate that the adversarial images mislead the models and cause the model to perceive the adversarial image as the target image to some extent.

Fig. 6 shows some examples of our used data in this paper. Specifically, as mentioned in the main paper, the clean image and the target text are from ImageNet-1k [14] and MS-COCO [11], respectively. To obtain the corresponding clean texts and the target images, we adopt GPT-4 [1] and Stable Diffusion [44] to generate high-quality texts and images, respectively. These clean and target image-text pairs are used to compute modality-aware embeddings and serve as the reference in Targeted Contrastive Matching to guide the learning of perturbations.

In addition to the method illustration in the main paper, the algorithmic format of the proposed Chain of Attack method is shown in Algorithm 1.

We present the core pseudo code in Sect. PyTorch-like Pseudocode for the Core of an Implementation of Chain of Attack in this supplementary material.

To explore the effects of the values of hyperparameters for our attack strategy, we conduct extensive ablation studies.

The ablation results of the modality-balancing hyperparameter $\alpha$ are reported in Tab. 4. Note that a smaller $\alpha$ means a large weight for the text modality. From the results, we can observe that text modality is more effective for attacking some victim VLMs (e.g., ViECap [19]). However, most attacking performance benefits from both of the modalities (e.g., SmallCap [42], Unidiffuser [4], and LLaVA [29]). This observation demonstrates the effectiveness of our proposed modality-aware embeddings that capture semantics from both domains. We suggest that a proper $\alpha$ can help achieve better results by fusing the visual and textual features.

In Tab. 5, we report the results of different combinations of hyperparameters $\beta$ and $\gamma$, where $\beta$ is the hyperparameter that controls the trade-off between similarity maximization for positive pairs and minimization for negative pairs, and $\gamma$ is the margin hyperparameter that controls the desired separation of the positive pairs and the negative pairs in the learned embedding space, as mentioned in the main paper. Note that a larger $\beta$ indicates more focus on the difference between the adversarial examples and the original clean examples. Since our task is targeted attacking, we set $0<\beta<1$. From our experiments, we find that larger $\gamma$ may degrade the performance, hence we suggest the margin hyperparameter should be set to less than 0.5. Some combinations of hyperparameters with promising performance are reported in Tab. 5.

In Sect. 4.3 of the main paper, we discuss the effect of the perturbation budget $\epsilon$ with only the results of the ensemble score. We report the complete results in Tab. 6, from which we can see that large perturbation budgets can improve the attack performance. However, as mentioned in Sect. 4.3 of the main paper (also see Fig. 5 (b) and Tab. 3 in the main paper), with the perturbation budgets becoming larger, the image quality decreases. We suggest a proper $\epsilon$ value (e.g., 8) to balance the trade-off.

Following the setting of previous methods [54], we adopt projected gradient descent (PGD) [33] with 100 steps, as mentioned in the main paper. Additionally, we report the results of less number of PGD steps in Tab. 7. The results show that fewer PGD steps may lead to underfitting and PGD with 100 steps achieves the best attack performance.

To further explore the potential application/risk of the attacking strategy, we implement the multi-round visual question answering (VQA) task using LLaVA-7B [29], as shown in Fig. 7. Two successful targeted attack examples are displayed. Specifically, in example 1, the original clean image is a part of the body of a large marine animal. We query LLaVA with queries “How do you think of this image?” and “Could it be a marine creature?”. LLaVA identifies it as a marine animal and gives correct answers. However, when we input the adversarial image generated by our method, the victim model gives the wrong answer and identifies it as a cat, which is the content of target examples. Example 2 also exhibits the same conclusion. The results demonstrate our attacking strategy successfully misleads the victim model to generate target responses.

In addition to the results shown in Fig. 4 of the main paper, more evaluation examples of the proposed LLM-based ASR are shown in Fig. 10 in this supplementary material.

In addition to Fig. 2 and Fig. 3 of the main paper, we visualize more examples of the intermediate steps of CoA and the results of the victim models, as shown in Fig. 8. Specifically, the left and middle parts of Fig. 8 show the update process of the adversarial examples based on both visual and textual semantics. The right part is the generation results of the victim models given the final adversarial examples. For example, in the third case, the semantic of the image changes from “A group of chickens of various colors foraging in a grassy outdoor enclosure” to the target semantic “A close up of a vase with flowers”, and the CLIP score between the intermediate adversarial text and the target text increases through the chain. Some victim models (e.g., ViECap, Unidiffuser) generate almost the same response as the target text (e.g., with CLIP score 99.6%, 100%), demonstrating the effectiveness of the generated adversarial examples.

To explore the sensitivity of our generated adversarial examples to noises (e.g., Gaussian noises), we show the results of adversarial examples adding different scales of noises, as shown in Fig. 9. When the standard deviation of noises $std_{G}$ is relatively small, the victim models still output the target responses. However, it can be observed that as the $std_{G}$ becomes large, the victim models tend to generate responses that are more likely to the original clean text. The captions of some intermediate examples are a combination of the original clean text and the target reference text. This result interprets the process of adding perturbations to the adversarial images and it concludes that large noises can undermine the effectiveness of adversarial examples.

(More figures and tables are on the following pages.)