Improving the Transferability of Adversarial Samples by Path-Augmented Method

According to the knowledge of the attacker, there are two categories of attacks in general: white-box and black-box attacks [4]. White-box attacks assume the white-box setting, where attackers have full access to the victim model, including the model structures and parameters. Fast Gradient Sign Method (FGSM) [11] is the first white-box attack that utilizes the sign of the input gradient to maximize the classification loss to generate adversarial samples in one step. Basic Iterative Method (BIM) [18] deploys FGSM to iteratively perturb images to improve the attack performance. Project Gradient Descent (PGD) [25] extends BIM with random start to generate diverse adversarial samples. Current white-box attacks can achieve nearly 100% attack success rates in white-box settings. However, they cannot handle black-box situations, where the model structures and parameters are unseen.

As a result, black-box attacks have attracted increasing attention from researchers recently, which can work in the black-box setting. There are generally two categories of black-box attacks. One is the query-based attacks [26, 3, 12, 2], and the other is the transfer-based attacks [9, 46]. Query-based attacks generally determine the susceptible direction of the victim model through querying it with deliberately designed inputs [12, 3, 2]. However, query-based attacks may incur prohibitive query costs, hindering their practical application. Transfer-based attacks exploit the transferability of adversarial samples, which means that the adversarial samples generated by a local source model can also mislead a different target model. Due to their high efficiency, transfer-based attacks are a research hot spot. However, adversarial samples crafted by white-box attacks generally possess limited transferability.

There are mainly two methodologies to improve the transferability of white-box attacks. The first one is the optimizer-based approach, which aims to escape from poor local optima by adjusting the employment of vanilla gradients during the generation of adversarial samples. For example, Momentum Iterative Fast Gradient Sign Method (MI-FGSM) [9] integrates the momentum term into BIM to improve its adversarial transferability.

The other one is the augmentation-based method, which can be further categorized into two lines. The first one actually augments images from a linear path. For example, Scale Invariant Method (SIM) [20] exponentially augments images along the linear path from the target image to the origin. Admix [35] follows a similar image augmentation path while modifying the starting points as the mixture of the target image and the images from other classes. The other line banks on affine transformations to augment images. For example, the Diverse Input Method (DIM) [44] applies random resizing and padding, while Translation Invariant Method (TIM) [10] employs shifting. Since affine transformations focus on changing the pixel positions of an image, the augmented images are less diverse than those from a linear path, leading to inferior transferability [35].

Unfortunately, state-of-the-art augmentation-based attacks, like SIM and Admix, only consider the image augmentation path to one baseline image, i.e., the origin. Besides, they fail to constrain the length of the image augmentation path, which may be overlong and result in augmenting images that are far away from and semantics-inconsistent with the target image. To overcome the deficiencies of such augmentation-based attacks, we propose the Path-Augmented Method (PAM). To make the augmented images more diverse, we propose to augment images from multiple augmentation paths during the generation of adversarial samples. Besides, to make the augmented images preserve the semantic meaning of the target image, we train a Semantics Predictor (SP) to constrain the length of each augmentation path. As a result, our scheme can achieve superior performance over state-of-the-art transfer-based attacks.

Many adversarial defense methods have been proposed to alleviate the threat of adversarial samples, which can be generally grouped into two categories. The first category is adversarial training, which keeps the state-of-the-art defense methods [31, 18]. Adversarial training retrains the model by injecting the adversarial samples into the training data to improve its robustness [11]. Ensemble adversarial training augments the training data with perturbations transferred from several other models to defend against transfer-based attacks [18]. The other category is to purify the adversarial samples. They rectify adversarial perturbations by pre-processing inputs without losing classification performance on benign images. The state-of-the-art defense methods in this category include utilizing a high-level representation guided denoiser [19], random resizing and padding [43], a JPEG-based defensive compression framework [22], a compression module [16], and randomized smoothing [8]. In this paper, we exploit these state-of-the-art defenses to evaluate the effectiveness of our attack against defended models.

We first set up some notations. We denote the benign input image as $x$ and the corresponding true label as $y$. We represent the output of a DNN classifier by $f(x)$. $J(x,y)$ stands for the classification loss function of the classifier, which is usually the cross-entropy loss. Given the target image $x$, adversarial attacks aim to find an adversarial sample $x^{adv}$, which can mislead the classifier, i.e., $f(x^{adv})\neq f(x)$, while it is human-imperceptible, i.e., satisfying the constraint $\left\|x-x^{adv}\right\|_{p}<\epsilon$. $\left\|\cdot\right\|_{p}$ represents the $L_{p}$ norm, and we focus on the $L_{\infty}$ norm here to align with previous papers [9, 20].

Prevailing white-box attacks like FGSM [11] usually craft adversarial samples by solving the following constrained maximization problem:

Scale Invariant Method (SIM) first computes the average gradient $\bar{g}$ of the classification loss with respect to $m$ scaled copies of the target image. Then it updates the target image with the sign of $\bar{g}$ by a small step size $\epsilon^{\prime}=\frac{\epsilon}{T}$ in each iteration, where $T$ is the iteration number. The update rule is formulated below:

Admix first replaces the target image with $m_{2}$ mixtures of the target image and the images from other categories ($x^{\prime}\in X^{\prime}$). Then it follows SIM by using $m_{1}$ scale copies of the mixed images. Therefore, Admix computes the update gradient as follows:

where $\eta$ is the strength of $x^{\prime}$ in the mixture image.

After pre-processing, the pixel value of an image will be normalized. We denote the image with pixel values all equal to 0 as the origin $\mathbf{0}$ in the normalized space. We note that the origin is a pure color image, since all its pixels have constant RGB values when we transform the origin in the normalized space back to the original color space.

We find that when generating adversarial samples, SIM and Admix actually augment images from a linear path. Specifically, SIM augments multiple scaled copies of the target image: $\frac{1}{2^{i}}\cdot x^{adv}_{t}=\frac{1}{2^{i}}\cdot x^{adv}_{t}+(1-\frac{1}{2^{i}})\cdot\mathbf{0}$, which is a linear combination of the target image and the origin. Therefore, SIM exponentially augments images along a linear path from the target image to the origin. Admix first replaces the target image with the mixture of the target image and the image from other categories ($x^{\prime}\in X^{\prime}$): $x^{adv}_{t}+\eta\cdot x^{\prime}$. Then it follows SIM to augment multiple scaled copies of the mixture image: $\frac{1}{2^{i}}\cdot(x^{adv}_{t}+\eta\cdot x^{\prime})=\frac{1}{2^{i}}\cdot(x^{adv}_{t}+\eta\cdot x^{\prime})+(1-\frac{1}{2^{i}})\cdot\mathbf{0}$, which is also a linear combination of the mixed target image and the origin. Therefore, Admix exponentially augments images along a linear path from the mixed target image to the origin.

From the above analysis, we argue that SIM and Admix suffer from two pitfalls. The first one is the limited augmentation path. SIM and Admix only consider the augmentation path to one baseline image, which is the origin. However, there are other possible augmentation paths that can increase the diversity of the augmented images. Therefore, the limited diversity of the augmented images can incur limited transferability of the resultant adversarial sample. Besides, the augmentation path of SIM and Admix may be overlong. They may augment images that are too far away from the target image. As a result, the augmented images are close to the origin, which contains no information about the target image. Augmenting such images can distract the learning of adversarial samples against the target image, thus harming adversarial transferability.

To overcome the pitfalls of state-of-the-art augmentation-based attacks, we propose the Path-Augmented Method (PAM). We first describe how we explore more augmentation paths to increase the diversity of augmented images. Then we introduce our method to constrain the length of the augmentation path to make the augmented images preserve the semantic meanings of the target image.

We focus on attacking image classification models trained on ImageNet [28], which is the most widely recognized benchmark task for transfer-based attacks [5, 17, 41] and is a more challenging dataset compared to MNIST and CIFAR-10. We follow the protocol of the baseline method [20] to set up the experiments, whose details are shown as follows.

Dataset. We randomly sample 1000 images of different categories from the ILSVRC 2012 validation set [28]. We ensure that nearly all selected test images can be correctly classified by all of the models deployed in this paper. We also randomly sample another 1000 images as the development set to train Semantics Predictor and rank representative augmentation paths.

Target Model. We consider both undefended (normally trained) models and defended models as the target models. For undefended models, we choose four top-performance models with different architectures, containing Inception-v3 (Inc-v3) [30], Inception-v4 (Inc-v4) [29], Inception-Resnet-v2 (IncRes-v2) [29], and Resnet-v2-101 (Res-v2) [13, 14]. For defended models, we consider three adversarially trained models, because adversarial training is the most simple but effective way to defend attacks [25, 45]. The selected defended models include Inception v3 trained with adversarial samples from an ensemble of three models (Inc-v3${}_{\text{ens3}}$), and four models (Inc-v3${}_{\text{ens4}}$), and adversarially trained Inception-Resnet-v2 (IncRes-v2${}_{\text{adv}}$). Furthermore, we include six advanced defense models that are robust against black-box attacks on the ImageNet dataset. These defenses cover high-level representation guided denoiser (HGD) [19], random resizing and padding (R&P) [43], NIPS-r3¹¹1https://github.com/anlthms/nips-2017/tree/master/mmd, feature distillation (FD) [22], compression defense (ComDefend) [16], and randomized smoothing (RS) [8].

Baseline. We take an advanced optimizer-based attack: MI-FGSM [9] as our baseline because it exhibits better transferability than white-box attacks [11, 18]. Furthermore, SIM [20] and Admix [35] can be viewed as special cases of our proposed PAM, so we select them as baselines. In order to show that our approaches achieve state-of-the-art performance, we select Variance Tuning Method [34] (VMI) because Admix and VMI are the current state-of-the-art transfer-based attack methods. In addition, we integrate all the methods with other augmentation-based methods: DIM [44] and TIM [10] for further comparison. We denote the approaches with DT extension as the method combined with DIM and TIM.

Metric. We evaluate the performance of attack methods via the attack success rate against the target model. The attack success rate is the percentage of adversarial samples that successfully mislead the target model over the total number of the generated adversarial sample.

Parameter. Following [9], we set the maximum perturbation budget $\epsilon=16$, the number of attack iterations $T=10$, and the step length $\epsilon^{\prime}=1.6$. We set the decay factor $\mu=1.0$ for all the methods. We follow the source code of SIM [20] and Admix [35] to change the number of scale copies to 32 and 8 for a fair comparison with the same computation complexity as PAM. For DIM, we set the transformation probability to 0.5. We deploy the 7 × 7 Gaussian kernels for TIM. We take $n=8$ and $m=4$ for PAM.

First, we study the performance of our attack method PAM against both undefended and defended models. We fix a source model and produce adversarial samples with different attack methods. The generated samples are then fed into the target models to compute the attack success rates. Our attack achieves nearly 100% success rates under the white-box scenarios in Table 1. More importantly, on the evaluation of transferability, our technique can drastically outperform VMI over 10% and Admix about 4.8% under the black-box setting on average. In addition, PAM improves the transferability to adversarially trained models, largely showing a high threat to adversarial training. Besides, our attack consistently outperforms other baselines by a significant margin under the black-box setting, which confirms the superiority of our strategies on transferable adversarial sample generation.

Then, we combine all the baselines with augmentation-based methods: DIM and TIM to further enhance the transferability. As shown in Table 2, the attack success rates against black-box models are promoted by a large margin with our approaches. In general, our attacks consistently outperform the state-of-the-art baselines by about 7.2%, which further corroborates the effectiveness of our method.

In addition, we also evaluate the performance of different attacks against advanced defenses. Table 3 shows the results when adopting Inc-v3 as the source model to attack other advanced defense models. Our attacks reduce the accuracy of defended models to 52.7% on average, defeating all baseline attacks. It validates the effectiveness of our attack against advanced defense models, raising security concerns for developing more robust defenses.

We conduct ablation studies to examine two designs in our proposed PAM: the number of augmentation paths $n$ and the Semantics Predictor. Adversarial samples are generated by attacking the Inc-v3 model without employing augmentation-based methods.

Number of Augmentation Paths. We investigate the effect of different augmentation path numbers on attack performance. We employ PAM with top-$n$ augmentation paths for generating adversarial samples based on the Inc-v3 model. The result is shown in Figure 2. With the increase of the number of augmentation paths, transferability improves. However, the computation cost also rises as the number of augmentation paths increases. Therefore, we choose $n=8$ to balance the performance and computation cost. Besides, we find an intriguing observation that the selected augmentation path is not the same as SIM or Admix when $n=1$. Our top-1 augmentation path improves the transferability of SIM with more than 1% on average without introducing additional computation complexity. This means the augmentation path of SIM is not optimal.

Semantics Predictor. We study the influence of Semantics Predictor on attack performance for PAM and the performance improvement for SIM. As shown in Table 4, the transferability of SIM can be improved by 1% on average by utilizing the Semantics Predictor because some of the augmented images are semantics-inconsistent with the target image as shown in Figure 3. We cannot recognize the object in the augmented image of SIM. However, the augmented image of SIM+SP demonstrates consistency with the target image, which shows the effectiveness of the Semantic Predictor. In addition, SIM+path+SP outperforms SIM+path by more than 1.6%, showing Semantics Predictor improves transferability when we combine multiple augmentation paths together. Besides, SIM + path surpasses SIM by a large margin, which also demonstrates the effectiveness of exploring more augmentation paths.