Boosting Adversarial Transferability with Learnable Patch-wise Masks

Existing transfer-based adversarial attacks can be divided into three classes, i.e. gradient optimization attacks [2, 13, 26, 35, 45, 46, 48], input transformation attacks [3, 34, 31, 27, 18], and feature-level object-aware attacks [28, 37, 47].

The advanced gradient calculation methods can stabilize the update directions and escape from poor local maxima during the iterations, resulting in more transferable adversarial examples [2]. Lin et al. [13] introduce the Nesterov accelerated gradient into iterative attacks to effectively look ahead and improve the transferability of adversarial examples. Wang et al. [26] further consider the gradient variance of the previous iteration to tune the current gradient to stabilize the update direction and avoid overfitting.

Input transformation attacks adapt various input transformations to create diverse input patterns to avoid overfitting and improve adversarial transferability [34, 3]. Admix [27] calculates the gradient on the input image admixed with a small portion of each image to craft more transferable adversarial images. Wu et al. [31] claim that these prior data augmentation methods employ fixed transformations, which lead to inferior results. They propose exploiting an adversarial transformation network to automatically model various transformations. From another perspective, Long et al. [18] propose a spectrum transformation based on a discrete cosine transform to diversify images and improve adversarial transferability.

Feature-level object-aware attacks mitigate overfitting to the source model by measuring and highlighting the important features, which can be achieved by maximizing the distance between the adversarial image and the benign image on the feature map. Wang et al. [28] perform attacks in the intermediate layers by suppressing positive features and promoting negative features directly to enhance transferability. Zhang et al. [37] propose a neuron attribution-based attack to conduct attacks with more accurate neuron importance estimations.

Existing attacks boost the transferability by searching for the model-generic discriminative regions via various strategies. However, we aim to prune the model-specific regions, which have a different mechanism to boost transferability. In the experiments, we enhance the attack transferability of state-of-the-art methods to verify the effectiveness of our method.

To defend against adversarial examples, several methods have been proposed, including adversarial training [5, 44] and preprocessing [33, 12]. Adversarial training injects adversarial examples into the training process. The adversarial training models can resist the perturbations in the gradient direction of the loss function, but the accuracy on benign samples will decrease. Preprocessing methods purify the adversarial examples by removing or destroying adversarial perturbations but reducing the accuracy of benign images. These defenses include the high-level representation denoiser (HGD) [12], random resizing and padding (R&P) [32], randomization smoothing (RS) [9], feature distillation (FD) [16], feature squeezing method: bit reduction (BIT) [36] and a neural representation purifier (NRP) [19].

We employ some representative state-of-the-art defense methods in the experiments to test the performance of our method and evaluate its effectiveness against defenses.

Given a classification network $f_{\theta}$ parameterized by $\theta$, let $x,y$ denote the benign image and its corresponding ground-truth label. The goal of the adversarial attack is to find an example $x^{adv}$ that is in the vicinity of $x$ but misclassified by the network. In most cases, we use the $L_{p}$ norm to limit the adversarial perturbations below a threshold $\epsilon$, where $p$ could be $0$, $2$, $\infty$. This can be expressed as:

The iterative version of FGSM (I-FGSM) [11] iteratively applies the fast gradient sign method multiple times with a small step size $\alpha$, which can be expressed as:

where $x_{0}^{adv}=x$. $\bigtriangledown_{x_{t}^{adv}}J$ is the gradient of the loss function $J(\cdot)$ with respect to $x_{t}^{adv}$ and cross-entropy loss is often used. $sign(\cdot)$ is the sign function to limit perturbations conformin to the $L_{\infty}$ norm bound.

The adversarial example generated by existing transfer-based attacks has poor transferability due to overfitting to the source model. As previosly mentioned, we argue that the model-specific discriminative regions of different DNN models are the key factor causing this overfitting (see Figure 1). To boost transferability, we propose a mask to drop out the image’s patches related to these regions in the process of generating adversarial perturbations. To implement this idea, we employ a learnable strategy to predict a patch-wise mask (LPM) to localize the corresponding patches in the images. For convenience of description, we apply the LPM combined with the I-FGSM as an example to introduce our method. The update of perturbations in the LPM-I-FGSM is formalized as:

where $M\in\{0,1\}^{m\times n}$ is the learned patch-wise binary matrix, and the patches on the image with $0$ will be dropped out. $m\times n$ are the total number of patches normally divided via the predefined patch size $p_{s}$.

We simulate the target model $t(\cdot)$ to compute feedback to adjust the mask and call it the simulated model $s(\cdot)$, which can be one or more DNN models. If the performance is strong on the simulated model, indicating that the adversarial examples have a good generalization, it will also perform well on the target model. We can choose a simulated model that is closely related to the target model according to prior knowledge. If no prior knowledge of the target model exists, we utilize the representative DNN models. In this work, we select ResNet-50 [6], VGG-16 [20], and DenseNet-161 [8], and use the ensemble version as the simulated model. The effect of the simulated models will be explored in the ablation study.

The differential evolution (DE) algorithm is employed to optimize the patch-wise mask. The input includes randomly initialized masks $\mathcal{M}(0)$ and feedback $\phi$ on the simulated model. This can be expressed as:

where $D(\cdot)$ is the DE algorithm. The feedback is crucial for adjusting the mask. The cross-entropy loss with the ground-truth label can reflect the classification results, and the loss value represents a greater propensity for misclassification. In addition, since multiple simulated models are utilized to guide the optimization of masks, we use both the mean and variance of cross-entropy loss as the feedback to prevent falling into the local optimal solution of a certain simulated model, which can be formalized as:

where $n_{s}$ is the number of simulated models, and $y$ indicates the one-hot label. $p_{i}$ is the predicted probability by the $i$-th simulated model $s_{i}(\cdot)$, and $CE$ denotes the cross-entropy loss.

The whole framework of the proposed method is illustrated in Figure 2. We initialize a patch-wise mask and combine it with the input image by an elementwise product. The adversarial perturbations are calculated by the I-FGSM attack on the source model $f(\cdot)$ for the modified input image. Then, the generated adversarial image is input to the simulated model $s(\cdot)$ to calculate the feedback $\phi$. The differential evolution algorithm adjusts the patch-wise masks according to the feedback until the optimal mask is achieved. Finally, we test the transferability of adversarial examples on the target model $t(\cdot)$. The details can be found in Algorithm 1.

In our method, the patch size is a hyperparameter and we explore its influence in the experimental section. Furthermore, to remove the randomness of the predicted mask solution, we predict multiple masks for one image and employ the intersection of multiple learned masks for every step of the final attack. We find that this method can further improve the transfer-based attack success rate.

The DE algorithm utilizes crossover strategy, mutation strategy, and selection strategy to heuristically search for the optimal solution with the guidance of feedback. Specifically, in our evolutionary algorithm, a population represents a set of solution masks. Given the population size $P$, the $k$-th generation solution $\mathcal{M}(k)$ is represented as:

where $M_{i}(k)$ denotes the $i$-th individual mask solution in the $k$-th generation.

Crossover Strategy. The crossover strategy generates the candidate next-generation individuals based on the superior individuals in the $k$-th generation. To determine the superior individuals in the $k$-th generation $\mathcal{M}^{*}(k)$, we calculate the feedback $\phi_{i}$ of each $M_{i}(k)$, and take those with the best $\rho*P$ as superior individuals. Then, we perform crossover among superior individuals to generate part of the candidate solutions for the $k+1$-th generation. The crossover operation is formalized as:

where ${M}_{i}^{{}^{\prime}}(k+1)$ is the $i$-th candidate individual mask solution in the ${k+1}$-th generation. $M_{\pi_{1}}^{*}(k)$ and $M_{\pi_{2}}^{*}(k)$ are randomly selected from the superior individuals, and $\pi_{1},\pi_{2}\in(0,\rho*P]$. $\rho$ is the rate of crossover individuals to the total individuals. $I(\cdot)$ is the indicator function, $I(x)=1$ if $x=2$; $I(x)=0$ if $x=0$; $I(x)=0$ or $1$ (with the probability of $0.5$) if $x=1$. The crossover strategy helps to find the location of commonality among masks.

Mutation Strategy. The mutation strategy aims to avoid the suboptimal solution and generate another part of the candidate individuals, which can be expressed as:

where $E(\cdot)$ is the mutation function, which randomly changes the value of each patch position in the mask with probability $p_{m}$. We set $p_{m}$ to 1, which is equivalent to randomly generating mutation individuals who have no direct connection with the previous individuals. Meanwhile, we keep the number of 0 or 1 in the individuals constant during the mutation strategy to ensure that the drop rate does not change.

Selection Strategy. After the crossover and mutation operation, we select the best $P$ nonrepeating individuals as final $k+1$ generation solutions based on the feedback $\phi$ from the $k$ generation individuals $\mathcal{M}(k)$ and the $k+1$ generation candidate individuals from $\mathcal{M^{{}^{\prime}}}(k+1)$, which can be formalized as follows:

where $Top(\cdot;P)$ denotes the selection function and obtains the best $P$ unique individuals based on feedback $\phi$. we repeat the above process for iteration $T_{DE}$. Then we select the final individual $\mathcal{M}(T_{DE})$ as the final solution $\mathcal{M}^{*}$.

We visualize the evolution process of masks and analyze the proposed method. As Figure 3 shows, the dropped positions tend to be stable with continuous evolution. We find that the foreground and its related regions are basically preserved.

Here, we provide a reasonable explanation of this phenomenon. Since the foreground contains the most direct semantic information connection with the label, models will be more inclined to pay attention to the foreground with the label guidance in the training process, leading to the model-generic discriminative regions; In contrast, as a part of the image, the background is utilized in the training process, but this type of information lacks semantic guidance, so different models may focus on different parts of the background information, leading to the model-specific discriminative regions.

This show that the learned mask is meaningful and can drop the model-specific regions, which reduces overfitting and enhances the transferability of the perturbations.

Dataset. We apply the ImageNet-compatible dataset²²2https://github.com/cleverhans-lab/cleverhans/tree/master/cleverhans_v3.1.0/examples/nips17_adversarial_competition/dataset following previous work [2, 37, 34], which contains 1000 images from ImageNet [10] with size 299 $\times$ 299 $\times$ 3.

It should be mentioned that all the models used in our paper are implemented in an open source manner, coming from PyTorch’s official pretrained models or the open source model library timm ³³3https://github.com/rwightman/pytorch-image-models, and our code is based on the PyTorch framework. Due to the different implementations, there may be small differences between the compared transfer-based attacks versus the attack success rates in their original papers; however, we ensure that all the performances claimed in the paper are obtained by fair testing under the same models.

In addition, we also evaluate our methods on four other advanced defense strategies, including R&P [32], FD [16], RS [9], and NRP [19]. The purified images of FD, RS, and NRP are fed to adv-Inc-v3 to give the final prediction.

In the experiments, we find that the patch size plays an important role in boosting transferability, and we explore its effect on the results. The curves of different patch sizes are shown in Figure 4 (left). The patch size of $1$, i.e., pixel-level drop, has a negative effect on transferability. We think that pixel-level drop is not effective enough to prune the model-specific regions, resulting in poor gradients. As the patch size increases within a certain range, the attack success rate is continuously improved. Based on the results, the patch size is set to 32 in later experiments.

In addition to the patch size $p_{s}$, the number of masks is also an important hyperparameter. Figure 4 (right) shows the results of adjusting the mask number (i.e., how many patch-wise masks to learn for each image by using the differential evolution algorithm). It can be found that the greater the mask number is, the stronger the transferability of generated adversarial examples. Considering the computational complexity and attack success rate, we set the number of masks to 12.

To explore the effectiveness of the proposed learning strategy based on the differential evolution algorithm, we conducted ablation studies. The experimental results are shown in Figure 5. RMI-FGSM represents MI-FGSM with random patch-wise masks (randomly generate a set of masks with the same number and size of LPM, other settings are the same as LPM-MI-FGSM as described in Section IV.A.). LPM-MI-FGSM1, LPM-MI-FGSM2, and LPM-MI-FGSM3 represent MI-FGSM with learnable patch-wise masks, and the numbers of simulated models are 1 (ResNet-50), 2 (ResNet-50 and VGG-16), and 3 (ResNet-50, VGG-16, and DenseNet-161), respectively. As seen from the results, attacks with learned patch-wise masks outperform random masks and no masks, which demonstrates the effectiveness of the proposed learnable strategy for searching model-specific regions on images and improving adversarial transferability. For example, the success rate of RMI-FGSM is 34.4% on adv-Inc-v3, while that of the proposed learning-based attack LPM-MI-FGSM1 is 35.2%. As the number of simulated models increases, the attack success rate of LPM-MI-FGSM1 is 36.6% on adv-Inc-v3. The results for the IRes-v2 and Res-152 also follow this trend.

We conduct experiments based on the existing advanced methods (i.e.,I-FGSM [11], TI-FGSM [3], DI-FGSM [34], MI-FGSM [2], and S²I-FGSM [18]) and enhance these methods with our LPM. The results are reported in Table I, Table II, and Table III. From the table, we can see that the LPM can effectively strengthen the existing SOTA methods when attacking white-box models. More importantly, the proposed method can significantly improve the success rate of transfer attacks. Specifically, when we generate adversarial examples using Inc-v3 as the white-box model, LPM-I-FGSM, LPM-TI-FGSM, LPM-DI-FGSM, LPM-MI-FGSM, and LPM-S²I-FGSM achieve average transfer success rates of 27.66%, 28.63%, 36.96%, 42.74%, and 49.13%, which outperform the original attack performance of 2.45%, 3.26%, 6.17%, 6.17%, and 7.04% respectively. This reveals that our LPM can further improve the adversarial transferability of existing methods.

The momentum term can stabilize update directions and improve adversarial transferability [2]. Lin et al. [13] show that the combination (DTS) of translation-invariant (TI) [3], diverse inputs (DI) [34], and scale invariant (SI) [13] can further improve attack success rates. Liu et al. [15] find that attacking multiple models simultaneously can effectively improve the transferability of adversarial images. Here, we combine all of them with existing SOTA methods and adopt a model ensemble attack by averaging the logit outputs of Inc-v3, IRes-v2, and ResNet-152 to calculate the loss to generate adversarial examples, and the results are reported in Table IV. We can observe that our LPM can further enhance the adversarial transferability for existing methods MI-FGSM-DTS [2], NI-FGSM-DTS [13], VNI-FGSM-DTS [26], and S²I-MI-FGSM-DTS [18], which can achieve an increase in the attack success rate of 5.39%, 6.68%, 1.58%, and 0.41%, respectively. This is a remarkable improvement and indicates that our approach has good scalability and can be combined with existing methods to further improve transferability. The effect of the attack is shown in Figure 6.

In this subsection, we discuss the trade-off between the performance and efficiency of the LPM. As a preprocessing method, the time consumption of LPM is mainly concentrated in the process of searching the model-specific regions (DE). The time complexity of the DE algorithm is determined by these hyperparameters: the DE iterations $T_{DE}$ (step 2 in Algorithm 1), the population size $P$ (step 4 in Algorithm 1), and the attack iterations $T_{m}$ (step 6 in Algorithm 1). Because the time cost caused by $P$ can be avoided by parallelizing the processing of the population, we mainly discuss the impact of the DE iterations $T_{DE}$ and the attack iterations $T_{m}$. The $T_{DE}$ and $T_{m}$ determine whether proper individual masks can be selected and alleviate the instability of the performance caused by randomness. The experiments are conducted on LPM-I-FGSM with NVIDIA GeForce RTX 3080.

The results in Table VI show that $T_{DE}$ and $T_{m}$ will obviously affect the time cost. When $T_{DE}$ and $T_{m}$ decrease, the time cost will correspondingly decrease. Meanwhile, the attack performance will have a minor fluctuation in a certain range and still maintain a competitive performance, e.g., $T_{DE}=5$ and $T_{m}=8$ is a cost-effective solution compared with the original setting. Therefore, our LPM can be chosen with different hyperparameter settings in the DE algorithm according to the actual requirement.

To measure the impact of our method on gradient aggregation, we introduce the clustering coefficient [29, 7] to measure the aggregation degree of saliency maps [21] for clean images and images masked by our DE algorithm.

Figure 1(a) shows that owing to model-specific regions, the saliency maps show a cluttered distribution. When these model-specific regions are masked, the saliency maps become clustered (Figure 1(b)). Therefore, we can use the aggregation degree of saliency maps in an image to reflect whether our hypothesis about the model-specific is true. Here we introduce the local clustering coefficient [29, 7] to compute quantitative evidence. In saliency maps, if one pixel is greater than a certain threshold, it is seen as a vertex. We compute $C_{i}$ for each vertex, which denotes how close one vertex’s neighbors are to being a clique. The $C_{i}$ can be defined as follows:

where $N$ is the total vertex, and $L_{i}$ represents the set of immediately connected vertices with vertex $v_{i}$. $v_{j}$ and $v_{k}$ are two vertices that belong to $L_{i}$. $e_{jk}$ denotes the edge connecting vertex $v_{j}$ with vertex $v_{k}$, and $k_{i}$ denotes the number of neighbors of vertex $v_{i}$. $|\cdot|$ denotes the number of edges.

Here we also compute the average clustering coefficient of saliency maps generated by three DNN models, i.e., ResNet-50, Inception-v3, and VGG-16. The result in Table VII shows a higher aggregation degree of mask images than benign images, which means that our masks prune model-specific regions and lead to clustered saliency maps.