Making Adversarial Examples More Transferable and Indistinguishable

Let $\left\{{{{\left({{f_{Wi}}}\right)}_{i\in\left[N\right]}},{{\left({{f_{Bj}}}\right)}_{j\in\left[M\right]}}}\right\}$ be a set of pre-trained classifiers, where $\ {{\left({{f_{Wi}}}\right)}_{i\in\left[N\right]}}{\rm{}}$ denotes the white-box classifiers and ${{\left({{f_{Bj}}}\right)}_{j\in\left[M\right]}}$ represents the unknown classifiers. Given a clean example $x$, it can be correctly classified to the ground-truth label ${y^{true}}$ by all pre-trained classifiers. It is possible to craft an adversarial example ${x^{adv}}$ that satisfies ${\left\|{{x^{adv}}-x}\right\|_{p}}\leq\varepsilon$ by using the white-box classifiers, where $p$ could be $0$, $1$, $2$, $\infty$, and $\varepsilon$ is the perturbation size. In this paper, we focus on non-targeted attack with $p=\infty$. Note that, the adversarial example ${x^{adv}}$ can mislead the white-box classifiers and the unknown classifiers simultaneously.

Here, we introduce the family of the gradient-based methods.

Fast Gradient Sign Method (FGSM) (Goodfellow, Shlens, and Szegedy 2015) establishes the basic framework of the gradient-based methods. It efficiently crafts an adversarial example ${x^{adv}}$ by using one-step update while maximizing the loss function $J({{x^{adv}},{y^{true}}})$ of a given classifier as

where ${\nabla_{x}}J({\cdot,\cdot})$ computes the gradient of the loss function w.r.t. $x$, ${\rm{sign}}(\cdot)$ is the sign function, and $\varepsilon$ is the given scalar value that restricts the ${L_{\infty}}$ norm of the perturbation.

Basic Iterative Method (BIM) (Kurakin, Goodfellow, and Bengio 2017a) is the iterative version of FGSM that performs better in white-box attack but less effective in transfer-based attack. It iteratively updates the adversarial example $x_{t}^{adv}$ with a small step size ${\rm{\alpha}}$ as

where ${\rm{\alpha}}=\varepsilon/T$ with $T$ denoting the number of iterations. ${\rm{Clip}}_{\varepsilon}^{x}\left\{\cdot\right\}$ performs per-pixel clipping as

Momentum Iterative Fast Gradient Sign Method (MI-FGSM) (Dong et al. 2018) enhances the transferability of adversarial examples by incorporating momentum term into gradient process, given as

where ${g_{t+1}}$ denotes the accumulated gradient at ${(t+1)}_{th}$ iteration , and $\mu$ is the decay factor of ${g_{t+1}}$.

Nesterov Iterative Method (NIM) (Lin et al. 2020) integrates an anticipatory update into MI-FGSM and further increases the transferability of adversarial examples. The update procedures are expressed as

Scale-Invariant Method (SIM) (Lin et al. 2020) applies the scale copies of the input image to further improve the transferability. However, SIM requires much more running time and resource.

Diverse Input Method (DIM) (Xie et al. 2019) applies random resizing and padding to the adversarial examples with the probability $p$ at each iteration. DIM can be easily integrated into other gradient-based methods to further boost the transferability of adversarial examples. The transformation function ${T(x_{t}^{adv},p)}$ is

Translation-Invariant Method (TIM) (Dong et al. 2019) optimizes an adversarial example by an ensemble of translated examples as

where $T_{ij}(x_{t}^{adv})$ denotes the translation function that respectively shifts input $x_{t}^{adv}$ by $i$ and $j$ pixels along the two-dimensions. And TIM calculates the gradient of the loss function at a point $\hat{x}_{t}^{adv}$, convolves the gradient with a pre-defined Gaussian filter and updates as

Note that, with limited running time and computing resources, the combination of NIM, TIM and DIM (NI-TI-DIM) is strong transfer-based attack method so far.

Based on the contradiction that the adversarial examples achieve high success rates but can be identified easily, our observations are shown as follows:

• 1.

  The sign function in gradient-based methods has two main disadvantages. On the one hand, the sign function normalizes all the gradient values to 1, -1 or 0, and thus leads to the loss of gradient information. On the other hand, the sign function normalizes some small gradient values to 1 or -1, which increases the perturbation size. While the tanh function not only normalizes the large gradient values as the sign function, but also maintains the small gradient values as function $y=x$. Therefore, the tanh function can replace the sign function and reduce the perturbation size.

• 2.

  With iterations $T=10$, the applications of Nesterov’s accelerated gradient (NAG) (Lin et al. 2020) and the momentum algorithm (Dong et al. 2018) in adversarial attacks demonstrate that we can migrate other methods to generate adversarial examples. Moreover, the $t_{\rm{th}}$ gradient ${\nabla_{x_{t}^{adv}}J({x_{t}^{adv},{y^{true}}})}$ is normalized by the $L_{1}$ distance of itself before the momentum algorithm. Intuitively, due to the performance of traditional convergence algorithms, Adam can achieve larger losses than the momentum algorithm in such small number of iterations. Additionally, Adam can normalize the gradient with ${{{m_{t}}}\mathord{\left/{\vphantom{{{m_{t}}}{\sqrt{{v_{t}}+\delta}}}}\right.\kern-1.2pt}{\sqrt{{v_{t}}+\delta}}}$, where ${m_{t}}$ denotes the first moment vector, ${v_{t}}$ is the second moment vector and $\delta={10^{-8}}$.

• 3.

  Traditional convergence algorithms employ learning rate decay to improve the model performance. Existing gradient-based methods set stable step size ${\rm{\alpha}}=\varepsilon/T$. In intuition, we can improve the transferability with the step size change. Different from the traditional convergence algorithms, the attack methods with the $\varepsilon$-ball restriction aim to maximize the loss function of the target models. Hence, we use the increasing step size with $\sum\nolimits_{t=0}^{{\rm{T-1}}}{{\alpha_{\rm{t}}}}={\rm{\varepsilon}}$.

• 4.

  Dong et al. (Dong et al. 2019) show that Gaussian blur with a large kernel improves the transferability of adversarial examples. However, Gaussian blur with larger kernels leads to the loss of the gradient information. Using the modifications mentioned above, the gradient information is preserved and plays a more important role in generating adversarial examples. Consequently, we apply smaller kernels in Gaussian blur to avoid the loss of the gradient information.

Based on the above four observations, we propose AI-FGTM to craft the adversarial examples which are expected to be more transferable and indistinguishable.

Adam (Kingma and Ba 2015) uses the exponential moving averages of squared past gradients to mitigate the rapid decay of learning rate. Essentially, this algorithm limits the reliance of update to only the past few gradients by the following simple recursion:

where ${m_{t}}$ denotes the first moment vector, ${v_{t}}$ represents the second moment vector, ${\beta_{1}}$ and ${\beta_{2}}$ are the exponential decay rates.

Due to the opposite optimization objectives, we apply Adam into adversarial attack with some modifications. Starting with $x_{0}^{adv}=x$, $m_{0}=0$ and $v_{0}=0$, the first moment estimate and the second moment estimate are presented as follows:

where ${\mu_{1}}$ and ${\mu_{2}}$ denote the first moment factor and second moment factor, respectively. We replace the sign function with the tanh function and update $x_{t+1}^{adv}$ as

where ${\beta_{1}}$ and ${\beta_{2}}$ are exponential decay rates, and $\lambda$ denotes the scale factor. Specifically, ${\alpha}_{t}$ is the increasing step size with $\sum\nolimits_{t=0}^{{\rm{T-1}}}{{\alpha_{\rm{t}}}}={\rm{\varepsilon}}$. Then the tanh function reduces the perturbations of adversarial examples without any success rate reduction. Furthermore, ${m_{t+1}}/({\sqrt{{v_{t+1}}}+\delta})$ replaces the $L_{1}$ normalization and the first moment estimate of Eq. 4 due to the fact that Adam has faster divergence speed than momentum attack algorithm (as shown in Fig. 1(b)).

Inspired by NIM that integrates an anticipatory update into MI-FGSM. Similarly, we can also integrate an anticipatory update into AI-FGTM. We first calculate the step size in each iteration as Eq. 18, and the Nesterov term can be expressed as

The remaining update procedures are similar to Eq. 16, Eq. 17 and Eq. 19, which can be expressed as

We summarize NI-TI-DI-AITM as the combination of AI-FGTM, NIM, TIM and DIM, and the procedure is given in Algorithm 1.

Dataset. We utilize 1000 images ¹¹1https://github.com/tensorflow/cleverhans/tree/master/examples/nips17˙adversarial˙competition/dataset which are used in the NIPS 2017 adversarial competition to conduct the following experiments.

Models. In this paper, we employ thirteen models to perform the following experiments. Four non-defense models (Inception v3 (Inc-v3) (Szegedy et al. 2016), Inception v4 (Inc-v4), Inception ResNet v2 (IncRes-v2) (Szegedy et al. 2017), and ResNet v2-101 (Res-v2-101) (He et al. 2016)) are used as white-box models to craft adversarial examples. Six defense models (Inc-v3ens3, Inc-v3ens4, IncResv2ens (Tramèr et al. 2018), high-level representation guided denoiser (HGD) (Liao et al. 2018), input transformation through random resizing and padding (R&P) (Xie et al. 2018), and rank-3 submission ²²2https://github.com/anlthms/nips-2017/tree/master/mmd in the NIPS 2017 adversarial competition) are employed as classic models to evaluate the crafted adversarial examples. In addition, we also evaluate the attacks with three advanced defenses (Feature Distillation (Liu et al. 2019), Comdefend (Jia et al. 2019), and Randomized Smoothing (Cohen, Rosenfeld, and Kolter 2019)).

Baselines. We focus on the comparison of TI-DIM, NI-TI-DIM, TI-DI-AITM and NI-TI-DI-AITM, where TI-DIM and NI-TI-DIM are both the state-of-the-art methods.

Hyper-parameters. According to TI-DIM (Dong et al. 2019) and NI-FGSM (Lin et al. 2020), we set the maximum perturbation ${\rm{\varepsilon}}=16$, and the number of iteration $T=10$. Specifically, we set the kernel size to $15\times 15$ in normal TI-DIM and NI-TI-DIM while $9\times 9$ in TI-DI-AITM. The exploration of appropriate settings of our method is illustrated in Appendix.

The mean perturbation size. For an adversarial example $x^{adv}$ with the size of ${M\times N\times 3}$, the mean perturbation size ${P_{m}}$ can be calculated as

where $x_{ijk}$ denotes the value of channel $k$ of the image $x$ at coordinates $({i,j})$.

We compare the running time of each attack mentioned in Table 1 using a piece of Nvidia GPU GTX 1080 Ti. Table 2 shows the running time under single-model setting and model ensemble setting. It can be seen that attacks combined with our method AI-FGTM do not cost extra running. Additionally, SIM requires at least two pieces of GPUs under model ensemble setting and costs much more running time than other attacks under both single-model setting and model ensemble setting. Therefore, we exclude SIM in the following experiments.

Table 3 shows the ablation study of the effects of different parts of our method. We compare the mean perturbation and the mean success rates of the adversarial examples against six classic defense models. Our observations are shown as follow:

• 1.

  Both of the tanh function and Adam can reduce the perturbation size. Additionally, Adam can also improve the transferability of adversarial examples.

• 2.

  The combination of the tanh function and Adam can greatly reduce the perturbation size, but only slightly improve the transferability of adversarial examples.

• 3.

  Using smaller kernels and dynamic step size can improve the transferability of adversarial examples even using dynamic step size slightly increase the perturbation size.

In this section, we compare the success rates of AI-FGTM based attacks and the baseline attacks against six classic defenses. We generate adversarial attacks for Inc-v3, Incv4, IncRes-v2, and Res-v2-101 by severally using TI-DIM, TI-DI-AITM, NI-TI-DIM and NI-TI-DI-AITM.

As shown in Table 4, we find that our attack method consistently outperforms the baseline attacks by a large margin. Furthermore, according to Table 4 and Fig. 5(b), we observe that our method can generate adversarial examples with much better transferability and indistinguishability.

In this section, we present the success rates of adversarial examples generated for an ensemble of four non-defense models. Table 5 gives the results of transfer-based attacks against six classic defense models. It shows that our methods achieve higher success rates than baseline attacks. In particular, without extra running time and resource, TI-DI-AITM and NI-TI-DI-AITM fool six defense models with an average success rate of 88.0% and 89.3%, respectively, which are higher than the state-of-the-art gradient-based attacks.

We also validate our method by comparing the different results between DIM, TI-DIM and TI-DI-AI-FGTM in Fig. 5. Adversarial examples are generated for the ensemble of Inc-v3, Inc-v4, IncRes-v2 and Res-v2-101 using different attack methods. Fig. 5 (a) shows that the tanh function does not hurt the performance of adversarial examples and Adam can boost the attack success rates. Fig. 5 (b) shows that our method significantly reduce the mean perturbation size of adversarial examples. In particular, our method reduces 40% perturbation while delivering the stable performance. Fig. 5 (c) shows that our approach with $\lambda=1.3$ obtains the largest loss of all the methods.

We evaluate the attacks with three more advanced defenses, namely Feature Distillation (Liu et al. 2019), Comdefend (Jia et al. 2019) and Randomized Smoothing (Cohen, Rosenfeld, and Kolter 2019). Table 6 shows the success rates of TI-DIM, TI-DI-AITM, NI-TI-DIM and NI-TI-DI-AITM against these defenses in the ensemble attack scenario.

In Table 6, we find that the attacks with AI-FGTM consistently outperform the attacks with MI-FGSM. In general, our methods can fool these defenses with high success rates.

Based on the above experimental results, it is reasonable to state that the proposed TI-DI-AITM and NI-TI-DI-AITM can generate adversarial examples with much better indistinguishability and transferability. Meanwhile, TI-DI-AITM and NI-TI-DI-AITM raise the security challenge for the development of more effective defense models.

We evaluate the indistinguishability with PSNR and SSIM (higher value indicates better indistinguishability). In detail, We generate adversarial examples for inc-v3 with TI-DIM, TI-DI-AITM, NI-TI-DIM and NI-TI-DI-AITM, and get the mean PSNR and SSIM between the adversarial examples and the clean examples. The results are shown in Table 7. We can find that the attacks with AI-FGTM can always perform better.

We explore the effects of different hyper-parameters of AI-FGTM and aim to find the appropriate settings to balance the success rates of both white-box and black-box attacks. The adversarial examples are generated for the ensemble of Inc-v3, Inc-v4, IncRes-v2, and Res-v2-101 using TI-DI-AITM. We first show the results of white-box attacks against four known models, and then we present the performance of black-box attacks against three defense models in Fig. 4. It can be seen that the appropriate settings are $\lambda=1.3$, ${\mu_{1}}=1.5$, ${\mu_{2}}=1.9$, ${\beta_{1}}=0.9$, ${\beta_{2}}=0.99$, and the kernel length is 9.

In order to demonstrate the effectiveness of our method, we report the success rates in white-box attack. We present the validation results in single-model attack scenario and model ensemble attack scenario. Table 8 presents the success rates of TI-DIM, TI-DI-AITM, NI-TI-DIM, and NI-TI-DI-AITM against white-box models. The details of the abbreviations are stated more precisely in Table 1 of our submitted paper.

As shown in Table 8, we can find that NI-TI-DI-AITM consistently outperform NI-TI-DIM in the single-model attack scenario. Compared with MI-FGSM, experimental results show that our method can improve the performance of NIM. Fig. 2 of our submitted paper also demonstrate that our method can generate adversarial examples with better indistinguishability.

We visualize six groups of adversarial examples generated by six different attacks in Fig 5. The adversarial examples are crafted on the ensemble models, including Inc-v3, Inc-v4, IncRes-v2 and Res-101, using TI-DIM, TI-DI-AITM ($\lambda$=1.3), TI-DI-AITM ($\lambda$=0.65), NI-TI-DIM, NI-TI-DI-AITM ($\lambda$=1.3) and NI-TI-DI-AITM ($\lambda$=0.65). We see that adversarial examples generates by our method have the better indistinguishability.

We present the value distributions of the accumulated gradients across iterations to demonstrate the limitation of the basic sign attack series. With iteration number $T=10$, we present the value distributions of the accumulated gradients in each iteration in Fig 6. As the number of iterations increases, values of the accumulated gradients tend to be greater than 1 or less than -1. However, a large number of values are in the range of $(-0.5,0.5)$ in the first three iterations. With constant step size ${\rm{\alpha}}=\varepsilon/T$, perturbation size of adversarial examples is greatly enlarged. Hence, we replace the sign function with the tanh function and gradually increase the step size.