Generating Adversarial Examples with Better Transferability via Masking Unimportant Parameters of Surrogate Model

Although DNNs have demonstrated to be successful on many tasks, they are vulnerable to adversarial examples [11]. Let $x\in\mathbb{R}^{d}$ denote an image with dimension $d$, $y$ denote its ground truth label. Given a pretrained DNN $\theta$, the adversarial attack is implemented by solving the following optimization problem:

where the $\mathcal{L}(x,y,\theta)$ denotes the loss function that guides the DNN $\theta$ to predict the class of image $x$ as ground-truth $y$, $\epsilon$ is the maximum allowed magnitude of perturbation, and $x^{*}$ is the generated adversarial example. Some gradient-based optimizers were proposed to solve (1). Szegedy et al. were the first to use a box-constrained L-BFGS optimizer [11]. Then, a one-step optimizer, i.e., fast gradient sign method (FGSM) was proposed to efficiently generate adversarial examples [12]. It was later extended to the iterative version of FGSM (I-FGSM) [13], which generates adversarial examples that fool the model $\theta$ with a higher success rate.

The generated adversarial example is shown to have the ability to attack an unknown model, and this ability is called the transferability of the adversarial example. In this scenario, the model $\theta$ used to generate the adversarial example is called the surrogate model, and the unknown model to be attacked is called the victim model. This type of adversarial attack is called the transfer-based attack. Dong et al. systematically investigated the transferability of adversarial examples and found an interesting phenomenon that is although the iterative method (I-FGSM) is a stronger optimizer than the one-step method (FGSM), the transferability of the adversarial examples generated by I-FGSM is worse than FGSM [2]. Specifically, their experimental results on the white-box attacks show that an Inception-v3 has only a 72.3% probability of being fooled by FGSM, but a 100% probability of being fooled by I-FGSM. However, when the adversarial examples generated from Inception-v3 were then used to attack an Inception-v4 model, the attack success rates of the adversarial examples generated by FGSM and I-FGSM were 28.2% and 22.8%, respectively. It shows that the adversarial examples generated by FGSM are more transferable.

To explain this phenomenon, Dong et al. analogized the surrogate model to the training data and the victim model to the testing data, so the transferability of the generated adversarial examples can be analogous to the generalizability of the trained models. Then, inspired by the widespread use of momentum-based optimizers to escape local maxima in training DNNs, Dong et al. propose a Momentum Iterative fast gradient sign Method (MIM) to generate more transferable adversarial examples [2]. Suppose the number of iterations is $N$, the MIM performs the following steps in an iteration:

with $x_{0}=x$ and $g_{0}=\mathbf{0}$, and the generated adversarial example is $x_{N}$. Here, the $\textrm{Clip}_{x}^{\epsilon}$ function is used to guarantee that the adversarial example is in the $\epsilon$-ball of $x$ under the $L_{\infty}$ norm, $\mu$ denotes a momentum factor and $\beta$ is a step size.

Furthermore, many optimizers have been proposed to improve MIM by introducing data augmentation to the objective function $\mathcal{L}(x^{\prime},y,\theta)$ in (1) to avoid overfitting. For example, Lin et al. proposed a Scale-Invariant attack Method (SIM) [6], which optimizes an adversarial example over a set of scaled images. Specifically, the loss function for generating adversarial examples with SIM is $\sum_{i=0}^{m-1}\mathcal{L}(x/2^{i},y,\theta)$, where $m$ denotes the number of scaled images. More recently, Huang et al. proposed an algorithm named Transferable Attack based on Integrated Gradient (TAIG) [7], which computes the gradient over a set of sampling images. Two versions of TAIG were proposed, the stronger version is called TAIG-R, which is equivalent to optimizing the following loss function: $\sum_{i=1}^{S}\mathcal{L}(\frac{i}{S}x+U(-\epsilon,\epsilon),y,\theta)$, where $U$ denotes a uniform distribution, and $S$ denotes the number of sampling images.

Unlike the above methods, instead of proposing a new optimizer to generate adversarial examples (by solving (1)), we propose the idea of masking unimportant parameters in the surrogate model $\theta$ when using them. Therefore, our approach can assist the above methods, as will be shown in Section III.B. To the best of our knowledge, the most relevant work to us is the ghost network (GN) [9], which manually injects some dropout layers into the surrogate model to randomly drop some neurons. Our experimental results in Section IV.B will show that our approach works better than GN.

Network pruning is a popular technique in the field of model compression, which aims to remove unimportant parameters to compress and accelerate DNNs. To guarantee the accuracy of the pruned model, many heuristic strategies have been proposed to evaluate whether a parameter is important or not. Optimal Brain Damage [14] pruned the parameters based on the Hessian matrix of the loss function. Deep Compression [15] pruned the parameters based on their absolute values. Li et al. [16] proposed an $L_{1}$-norm-based metric to evaluate the importance of channels. Molchanov et al. [8] proposed a Taylor expansion-based metric to evaluate the importance of channels. In this paper, we borrow the idea from the work [8] to evaluate the parameter importance scores based on the first-order gradient information of the parameters.

Existing transfer-based attack methods generate adversarial examples by maximizing the loss function (1) and improving the transferability of the adversarial examples by using advanced optimizers that help to escape the local maxima. The parameter $\theta$ in (1) is considered as a constant in these works. Inspired by the analogy of surrogate models to training data in [2], we propose that the quality of $\theta$ is also a key factor for the transferability, just as noisy data can be detrimental to generalizability.

However, work on network pruning suggests that DNNs are often over-parameterized. For example, Li et al. [16] showed that even after removing 50$\%$ of the convolutional kernels from the first convolutional layer of a VGG16 model [17] based on the $L_{1}$-norm, the accuracy degradation of the VGG16 model on CIFAR-10 dataset [18] was still negligible. Since there are redundant parameters for prediction, it is natural to be concerned about the impact of these redundant parameters on the generation of adversarial examples. While previous work on preventing overfitting can alleviate this effect, explicitly removing these parameters is also a worthwhile consideration, and the two types of approaches can be naturally combined to achieve better performance. To this end, we first introduce a method to calculate the importance scores of model parameters in this subsection. Based on the calculated scores, we can explicitly identify and mask unimportant parameters during the generation of adversarial examples, which will be presented in the next subsection.

Given an image $x$ and its true label $y$, assume that $V_{i}$ represents the importance score of the $i$-th parameter $\theta_{i}$ for predicting the category of image $x$ as label $y$. It can be defined intuitively as $V_{i}=|\mathcal{L}(x,y,\theta)-\mathcal{L}(x,y,\theta-\theta_{i}e_{i})|$, where $e_{i}$ denotes a one-hot vector consisting of zeros in all elements except for a single one in $i$-th element. It means how much the loss function change after setting $\theta_{i}$ to zero. However, calculating the importance scores for all parameters in this way requires a number of forward propagations proportional to the model size, which is intractable for large DNNs. To reduce the computational overhead, It was approximated in [8] by using the first-order gradient information based on the Taylor expansion as follows:

According to (5), the importance scores of all parameters can be calculated simultaneously in just one step of backpropagation.

In this subsection, we propose a technique for boosting existing optimizers by masking unimportant parameters (MUP). We take assisting MIM as an example, call it MUP-MIM, and illustrate it in Fig. 2. Only two layers of the surrogate models are drawn in Fig. 2 for simplicity. The standard MIM and the MIM assisted by the comparative work ghost network (GN) [9] are also illustrated in Fig. 2, and the latter is called GN-MIM. As can be seen in Fig. 2, the pretrained surrogate model is fixed in MIM, while it is changing dynamically when using GN and MUP.

Given a hyperparameter masking ratio $r$, MUP will mask unimportant parameters at each iteration. To do this, the important scores $V$ are calculated as described in Section III.A. Then, the $r\times|V|$ elements in $V$ with the smallest score are considered as unimportant parameters. This process is simple and can be naturally integrated into any gradient-based optimizer. We take the basic MIM as an example and summarize the algorithm MUP-MIM for assisting MIM with MUP in Algorithm 1. At each iteration, the importance scores are first calculated (Step 3). Then, a binary mask of the same shape as $\theta$ is obtained based on the given masking ratio $r$ (Step 4-5). The modification to the standard MIM is that the gradient vector $\delta_{t+1}$ is computed with the model $\theta\odot M$, whose unimportant parameters are masked (Step 6). Here the $\odot$ denotes an element-wise product. The subsequent steps are the same as for MIM (Step 7-8). Finally, the produced $x_{N}$ is the generated adversarial example. Since the unimportant parameters are masked at each iteration, the generated adversarial example avoids overfitting them and has higher transferability, as will be demonstrated in Section IV.B.

It is straightforward to integrate MUP into other optimizers. Two improved versions of MIM are considered in this paper, namely SIM [6] and TAIG-R [7], which are introduced in Section II.A. Assisting SIM with MUP (called MUP-SIM), only the gradient $\delta_{t+1}$ in Step 6 needs to be computed as follows:

Similarly, using MUP to assist TAIG-R (called MUP-TAIG-R) only requires to compute $\delta_{t+1}$ in Step 6 as follows:

The comparative work ghost network (GN) [9] is also shown in Fig. 2, which manually injects some dropout layers into the pretrained surrogate model to assist MIM. The authors of GN claim to do this in order to sample a sub-network at each iteration. And in our opinion, this is equivalent to randomly masking some neurons. In contrast, MUP is selectively masking some parameters, which are called synapses in computational neuroscience. Therefore, MUP has two advantages over GN in terms of the way to identify the parameters to be masked. Firstly, the proposed MUP is input-dependent and is able to model the effect of input data on the importance scores, and is therefore more theoretical. Secondly, MUP has a finer granularity in identifying unimportant parameters, since masking a neuron in GN is equivalent to masking a group of synapses. Besides the difference in the way to identify the parameters to be masked, GN needs to manually specify some locations to inject dropout layers, while not needed in MUP, so MUP is more convenient to be applied on a new architecture without any customization.

Victim Models. We consider nine publicly available models as victim models, which have been widely used in previous work [2, 6]. The first three of them are normally trained models, including Inception-v3 (Inc-v3) [1], Inception-v4 (Inc-v4), and Inception-ResNet-v2 (IncRes-v2) [19]. The rest are robust models: Inc-v3_ens3, Inc-v3_ens4, and IncRes-v2_ens [20], high-level representation guided denoiser (HGD) [21], input transformation through resizing and padding (R&P) [22], and the rank-3 submission in NIPS2017 adversarial competition (NIPS-r3)²²2https://github.com/anlthms/nips-2017/tree/master/mmd.

Surrogate Models. Following the setting of the comparative work ghost network (GN), we first choose Inc-v3 and Inc-v4 as the architectures of the surrogate model, since they were experimented in [9]. To demonstrate the versatility of MUP over more architectures, we additionally conduct experiments on the DenseNet121 (DN121) surrogate model.

Optimizers. We use MUP to assist three optimizers for generating adversarial examples to evaluate the compatibility of the proposed MUP with prior works, including MIM [2], SIM [6], and the state-of-the-art TAIG-R [7].

Hyperparameters.  If not specified otherwise, the masking ratio $r$ is set to 15%, 30% and 25% for the surrogate model Inc-v3, Inc-v4, and DN121, respectively. Notice that it is independent with respect to the victim model since the victim model is unknown in the assumptions of the transfer-based attack. For the hyperparameters related to different optimizers, we follow the recommended values of their corresponding papers to set them. Specifically, we set the step size $\beta$ to 2, the number of iterations to 10, the momentum factor $\mu$ for MIM to 1.0, the number of scaled images $m$ for SIM to 5, and the number of sampling images $S$ for TAIG-R to 20. All the experiments are conducted with the maximum allowed magnitude of perturbation $\epsilon=16$.

In this subsection, we will first show that using the proposed MUP can improve the success rate of existing optimizers. In addition, we compared the proposed MUP with the comparative work ghost network (GN), which was used to assist MIM in [9]. To follow the experimental setup of GN, we first consider assisting MIM, as well as Inc-v3 and Inc-v4 as the architecture of the surrogate models. Then, a DN121 is additionally considered as a surrogate model, which reflects the versatility of MUP for different architectures of the surrogate model. In contrast, GN cannot be applied directly to a new architecture, as it requires specifically designed locations for adding dropout layers, as we present in Section III.B. All of the experiments are conducted on the ImageNet-compatible dataset used in the NIPS 2017 adversarial competition.

We list the experimental results in Table I. The case where the surrogate model is the same as the victim model is not considered, as this is a white-box attack that we are not interested in. We also report the average attack success rates in Table I, which is averaged over all nine victim models for DN121 and over eight victim models other than itself for Inc-v3 and Inc-v4. From the results, we can see that the MUP-MIM significantly outperforms MIM in terms of attack success rates in all experiments. Specifically, when Inc-v3, Inc-v4, and DN121 are used as surrogate models, MUP-MIM improves the attack success rate by 5.1%, 5.8%, and 11.3%, respectively.

MUP and GN can also be compared based on the results in Table I. From it we can see that MUP-MIM outperforms GN-MIM in all experiments when Inc-v3 is used as the surrogate model. However, GN-MIM performs slightly better than MUP-MIM in attacking IncRes-v2_ens, R&D, and NIPS-r3 when Inc-v4 is used as the surrogate model. This indicates that the performance of MUP and GN are comparable when using MIM to generate adversarial examples and makes us curious about the comparison results between them when assisting an advanced optimizer.

Therefore, we further conduct experiments to compare the compatibility of GN and MUP with more optimizers. We considered SIM and TAIG-R and list the experimental results in Table II. Surprisingly, although GN performs well in assisting the MIM optimizer as claimed by [9], the results in Table II show that using GN is worse than not using it when assisting more advanced optimizers. For example, when using Inc-v3 as a surrogate model, GN decreases the attack success rates of SIM and TAIG-R optimizer by 1.1% and 4.7%, respectively, on average. This drop is even more significant when using Inc-v4 as a surrogate model. On the contrary, our MUP assists the SIM and TAIG-R well, bringing further improvements in terms of attack success rates in all experiments. As a result, compared to TAIG-R, using MUP improves the attack success rates by 4.9%, 4.2%, and 2.9% on average respectively when Inc-v3, Inc-v4, and DN121 are used as surrogate models. In addition, MUP improves the attack success rates of DN121 on nine victim models to at least 80.1% (IncRes-v2_ens). It should be pointed out that a high attack success rate is what is of interest in the field of transfer-based attack, since the goal of a malicious adversary is to fool a DNN system with as high a probability as possible. Thus, our MUP is more practical in real-world attack scenarios, although it is not always better than the GN when using a relatively weak optimizer (MIM).

In this subsection, we first conduct experiments to investigate the effect of the masking ratio $r$. Then, we have considered another metric for calculating the parameter importance scores, to highlight the effectiveness of the Taylor expansion-based metric now in use.

The masking ratio $r$ plays an important role in MUP. When $r$ is equal to zero, it means that no parameters are masked. On the other hand, if $r$ is too large, most of the parameters of the surrogate model will be set to zero, which will impair the accuracy of the model prediction and lead to poor transferability. A natural question to ask is how should the hyperparameter $r$ be set. Will the optimal masking ratio $r$ vary greatly for different victim models? To answer this question, we enumerate different $r$ from $\{0\%,5\%,10\%,\cdots,45\%,50\%\}$ and conduct experiments on Inc-v3, Inc-v4 and DN121, respectively. We fix the optimizer for generating adversarial examples as TAIG-R since we are more interested in the performance of MUP to assist a stronger optimizer. The experimental results of using different masking ratios to assist TAIG-R are shown in Fig. 3, which shows that for the same surrogate model, the curves of attack success rates versus masking ratio $r$ are similar for all different victim models. Specifically, all experiments show that the attack success rate will initially increase, then reach the saturation region, and begin to decline rapidly. Moreover, the saturation region does not depend on the victim model, because all curves of the same surrogate model have highly overlapping saturation regions. Therefore, determining the hyperparameter $r$ of MUP on a new surrogate model requires only choosing any public model as the victim model to search for an optimal $r$. This is critical for the transfer-based attack because the adversary has no knowledge of the victim model.

Finally, we additionally considered a metric for calculating the parameter importance scores, which is simply using the absolute value of the parameters as the importance scores, namely $V_{i}=|\theta_{i}|$. This metric was previously applied in model pruning by Han et al. [15]. By iteratively removing the parameters with the smallest absolute value, they compressed a VGG16 [17] model by 13X with no loss of accuracy. By comparing it with the Taylor expansion-based metric (see (1)), the only difference is that the latter has a gradient term $\frac{\partial\mathcal{L}(x,y,\theta)}{\partial\theta_{i}}$. We use the absolute value-based metric to perform the MUP algorithm with the same hyperparameter $r$ as the MUP with the Taylor expansion-based metric, i.e., set to 15%, 30%, and 25% Inc-v3, Inc-v4, and DN121, respectively. The experimental results of using different metrics to assist TAIG-R are shown in Fig. 4, which shows that MUP with the absolute value-based metric does not boost the transfer-based attack. We believe this is because the absolute value-based metric is input-independent and ignores the features of inputs. In contrast, the Taylor expansion-based metric compensates for this shortcoming by multiplying an input-dependent gradient term. This indicates that the metric for calculating the importance scores is crucial for the proposed MUP-based methods.