Looking From the Future: Multi-order Iterations Can Enhance Adversarial Attack Transferability

One of the branches is modifying the gradients to obtain generalization adversarial examples. The most representative work is the MI-FGSM[1]. Via introducing momentum into to adversarial examples generation process, gradients from each previous node will be utilized to smooth the search direction for the current node. This will help the optimization process somehow avoid the local minima. NI-FGSM[7] method refined the momentum calculation process to further enhance the adversarial example generalization. VMI-FGSM and VNI-FGSM [12] utilize the variance tuning the momentum to further avoid the local minima. EMI-FGSM[14] sampling the local gradient information around the node from PI-FGSM and ensemble the information to enhance the adversarial transferability. PGN[3] tries to find the local maxima point to directly find the better transfer attack condition. MIG[8] utilizes the integrated gradients in XAI methods to enhance the gradient in MI-FGSM. Some other works inspired by the optimizers refine the gradients or the momentums with the latest optimizer [20, 18, 21].

The above methods mainly focus on refining the gradient itself, while some other methods try to enhance the gradient by modifying the loss function. [6] proposes the Po+Trip loss to enhance the targeted transfer attack inspired by the Poincare distance. [22] thinks that the simple logit loss also has good transfer attack performance. [19] considers the class interaction and proposes the relative Cross Entropy loss by raising the prediction probability of other classes to enhance the adversarial attack generalization.

Another one of the branches is utilizing data augmentation to enhance the diversity of input, which is also called input transformation. The most representative work is the $\text{DI}^{2}$-FGSM [17], which uses resize and padding operations to enhance input diversity. TI-FGSM [2] utilizes shifting operation with a kernel matrix on the gradients. SI-FGSM [7] applies scale transformation to the input image to gain augmented data. SIA [16] divides the original image into blocks and applies a random image transformation onto each image block to craft a set of diverse images for gradient calculation. BSR [10] furthermore introduces the block shuffle and rotation operations into the input transformation.

The above methods mainly focus on the transformation of the input image itself, which can be called self-transformation. Some other methods try to enhance the input diversity by introducing information from other images. [13] extends the SI-FGSM with the mixup strategy, which is named Admix. This method chooses several other images and mixes them into the original images to enhance the input diversity. [11] improves the mixup strategy with a non-linear way to mask the chosen image into the original image. [15] rethinks the process of the admix and changes the mixing images process to the mixing gradients process.

Here we first give out some fundamental notations. $x\in R^{N}$ is the input data sample, where $N$ is the data dimension. $H:R^{N}\to R^{D}$ is the feature extractor, where $D$ is the feature dimension. $G:R^{D}\to R^{K}$ is the classifier, where $K$ is the number of classes. Then $F=G\circ H:R^{N}\to R^{K}$ is the entire classification network. The output of $F$ is $z\in R^{K}$ called logits. The output of $\text{softmax}(z)$ is $p$ which is the predicted probability. $p$ is also marked as $P_{F}(x)\in R^{K}$ to emphsize the network and input data. The goal of the adversarial attack is to generate an adversarial example $\hat{x}$ that can lead to $f$ fail, which is expressed as $F(x)=y_{t}\wedge F(\hat{x})\neq y_{t}$. Here $y_{t}$ indicates the truth label of $x$ as the specific predicted class index. To express without ambiguity, $y$ represents the index classification result for $F$, $z$ represents the vector result for $F$ and $h$ represents the vector result for $H$.

Then the adversarial attack transferability usually can be expressed as follows:

This expression can be understood that for an adversarial example $\hat{x}$ which can successfully attack network $F^{1}$, $\hat{x}$ can also successfully attack network $F^{2}$.

Then perturbation $\delta=\hat{x}-x$ is generated by the adversarial example generation method which is marked as $\mathcal{A}(\cdot)$. Under iteration scope, the perturbation $\delta$ can be decomposed with the sum of the perturbation from each iteration which can be presented as $\delta=\sum_{t=1}^{T}\delta_{t}$. The constraint for each iteration perturbation $\delta_{t}$ can be presented as ${||\delta_{t}||}_{p}\leq\epsilon_{t}$. In the I-FGSM-based method, $\epsilon_{t}$ usually constantly equals a certain value, e.g. $\epsilon_{t}=\epsilon/T$. To make a more clear statement, with no special instructions, $\delta$ with subscript numbers represents the corresponding iteration perturbation, e.g. $\delta_{t}$ is the $t$-th round perturbation. And $\delta$ with superscript numbers represents the sum of the corresponding iteration perturbation, e.g. $\delta^{t}=\sum_{i=1}^{t}\delta_{i}$.

The terminal goal of the adversarial attack is to find an optimal adversarial example that can successfully attack the target model with minor perturbation as much as it can. This task is usually transferred into an optimization task with setting a loss function as the optimization goal, e.g. the optimization goal for the untargeted attack task can be formulated as:

where $\epsilon$ is the threshold for the perturbation which emphasizes the perturbation constraint. Previous methods try to enhance the adversarial transferability by optimizing the optimal process with various approaches. However, from the goal to look at the optimization process, if there is an optimal point $\delta^{\mathcal{O}}$ for (2), then the shortest search routine can be determined, e.g. the straight line segment from $0$ and $\delta^{\mathcal{O}}$ in Euclidean space. Then the best first iteration result $\delta^{\mathfrak{B}}_{1}$ for the entire optimization task should satisfy:

The solution for this optimization is

where $\alpha$ is a linear coefficient to limit $\delta^{\mathfrak{B}}_{1}$ satisfying the constraint. However, directly obtaining $\delta^{\mathcal{O}}$ is a really tough task. Finding the optimization point through iteration is one of the most effective and commonly used methods. These methods continuously approach the optimization target in an iterative manner to obtain an approximation of the optimization result. After a certain round of search, the optimization result can be treated as an approximation point $\delta^{\mathcal{O^{\prime}}}$ for the optimal point $\delta^{\mathcal{O}}$. Then, the shortest search routine from $0$ to $\delta^{\mathcal{O^{\prime}}}$ can be determined. When back to the very first iteration, the best first optimization search result should be

The corresponding solution for this optimization is

From a general scope, for any optimization task, after obtaining the original search routine, the approximation for the best first iteration result can also be obtained with Eq. (6). Then with the same optimization process, the approximation for the best next iteration result ($\delta^{\mathfrak{B^{\prime}}}_{2}$) as well as any $t$-th iteration result ($\delta^{\mathfrak{B^{\prime}}}_{t}$) can be obtained. Considering that the $\delta^{\mathcal{O}}$ usually cannot be directly obtained, $\delta^{\mathfrak{B}}_{t}$ can also be represented with $\delta^{\mathfrak{B^{\prime}}}_{t}$ in this manuscript. We name this process as Looking From the Future (LFF).

I-FGSM-based attack methods obtain great success in transfer attacks. Intuitively, LFF can be directly used to enhance these kinds of attack methods. However, FGSM-based methods have the mechanism of symbolization of gradients which means perturbations of each iteration all satisfy $||\delta_{t}||_{\infty}=\epsilon_{\infty}$ and $||\delta_{t}||_{2}=\sqrt{N\cdot\epsilon^{2}_{\infty}}$ at the same time. However, direcly applying Eq. (6) which refers to $\delta^{\mathfrak{B}}_{1}={\sum_{T}^{t=1}\delta^{t}}/T$ will cause $||\delta^{\mathfrak{B}}_{t}||_{\infty}\leq\epsilon_{\infty}$ and $||\delta^{\mathfrak{B}}_{1}||_{2}\leq||\delta_{t}||_{2}$. Only when $\forall i,j\in T,\delta_{i}=\delta_{j}$, $||\delta^{\mathfrak{B}}_{1}||_{\infty}=||\delta_{t}||_{\infty}$ and $||\delta^{\mathfrak{B}}_{1}||_{2}=||\delta_{t}||_{2}$. This can only happen on a strictly linear function, while feature space in deep learning networks is usually nonlinear. The $\alpha$ in Eq. (6) can apply $\delta^{\mathfrak{B}}_{1}$ to satisfy $||\delta^{\mathfrak{B}}_{1}||_{\infty}=||\delta_{t}||_{\infty}$ or $||\delta^{\mathfrak{B}}_{1}||_{2}=||\delta_{t}||_{2}$. However, simply applying LFF on $L_{\infty}$ constraint will not work. Keeping $||\delta^{\mathfrak{B}}_{1}||_{\infty}=||\delta_{t}||_{\infty}$ will cause $||\delta^{\mathfrak{B}}_{1}||_{2}\leq||\delta_{t}||_{2}$ which means $\delta^{\mathfrak{B}}_{t}$ will have a smaller Euclidean distance compared with $\delta_{t}$. A smaller Euclidean distance will lead to a slower convergence speed, which means it may even need more iterations to reach the optimal points.

Meanwhile, applying LFF on $L_{2}$ will cause the $||\delta^{\mathfrak{B}}_{t}||_{\infty}\geq\epsilon_{\infty}$. When the search routine comes near $T\cdot\epsilon_{\infty}$ (the $L_{\infty}$ constraint to the final perturbation), a large number of effective updates for those dimensions that already reach $T\cdot\epsilon_{\infty}$ will be discarded. This will also make the optimization routine less effective.

Also, considering that greater adversarial transferability related to more generalized examples for each model, when $\mathcal{A}(\cdot)$ itself gets overfitting to a certain model or repeatedly using history information, LFF will further exacerbate the degree of the overfitting.

Giving an I-FGSM based attacking method $\mathcal{A}(\cdot)$ and the clean data $x$, the original $t$-th iteration from the attack process is $\delta_{t}=\mathcal{A}(F,x+\sum_{i=0}^{t-1}\delta_{i})$, where $t\in T$ and $x+\delta_{0}=x$. The gradient corresponding to the $\delta_{t}$ is $g_{t}=\mathcal{A}_{g}(F,x+\sum_{i=0}^{t-1}\delta_{i})$, which means that $\delta_{t}=\textit{sign}(g_{t})$, where $\textit{sign}(\cdot)$ is the symbolization function. $\mathcal{Q}$ is the quantity of the steps looking from the future, i.e. $\mathcal{A}(\cdot)$ iters $\mathcal{Q}$ steps. Then the straightforward description for the $t$-th perturbation from one order LFF attack is :

where $\alpha$ is the updating rate, $\delta^{\mathfrak{B}}_{0}=\delta_{0}$, and $\beta$ is the future penalty coefficient.

When Eq. (7) is directly applied to the pure I-FGSM method, $g_{t}$ equals the gradients of the corresponding input. The description is:

where $\nabla_{x}\mathcal{L}$ is the gradient of $x$ for the loss function $\mathcal{L}$. Then Eq. (7) can be rewritten with the following description:

When Eq. (7) is applied to the MI-FGSM method, which is the most widely used method, $g_{t}$ equals the momentums of the corresponding input. The description is

where $M(x+\delta_{0})=\frac{\nabla_{x}\mathcal{L}}{||\nabla_{x}\mathcal{L}||_{1}}$ and $\mu$ is the momentum decay factor. When Eq.(10) is introduced into Eq.(7), the formulation can be obtained as

The expansion of Eq.(11) (shorten $x+\sum_{j=0}^{t-1}\delta^{\mathfrak{B}}_{j}+\sum_{i=0}^{q-1}\delta_{i}$ with $x_{\mathfrak{B}_{t-1}}^{q-1}$) is

Eq.(12) indicates that for a given optimization process, the $\delta^{\mathfrak{B}}_{t}$ can be represented as a polynomial refers to the gradient of each optimization point using $L_{1}$-norm regularization. The coefficient refers to the $i$-th optimization point can be marked as $\mathcal{C}_{i}=\sum_{l=i}^{\mathcal{Q}-1}\frac{\beta^{l+1}\cdot\mu^{l-i}}{||M(x_{\mathfrak{B}_{t-1}}^{l})||_{p}}$ and the gradient of $i$-th optimization point with $L_{1}$-norm regularization can be marked as $\mathcal{G}_{i}=\frac{\nabla_{x_{\mathfrak{B}_{t-1}}^{i}}\mathcal{L}}{||\nabla_{x_{\mathfrak{B}_{t-1}}^{i}}\mathcal{L}||_{1}}$. Then Eq.(12) can be refined as

where $\left\langle\cdot,\cdot\right\rangle$ is the inner product, $\vec{\mathcal{C}}=(\mathcal{C}_{0},\mathcal{C}_{1},\cdots,\mathcal{C}_{\mathcal{Q}-1})$ and $\vec{\mathcal{G}}=(\mathcal{G}_{0},\mathcal{G}_{1},\cdots,\mathcal{G}_{\mathcal{Q}-1})$. Then Eq.(13) can be understood as a superposition of $\mathcal{G}_{i}$. $\mathcal{G}_{i}$ with a smaller value of $i$ is closer to the starting point $x$. The closer the gradient of a point is to $x$, the more local optimization information it contains; the farther the gradient of a point is from $x$, the more generalization information it contains. When coefficients of those $\mathcal{G}_{i}$ with a smaller value of $i$ are too great, $\delta^{\mathfrak{B}}_{t}$ will tend to $x_{\mathfrak{B}_{t-1}}^{1}$ which means overfitting. When coefficients of those $\mathcal{G}_{i}$ with a greater value of $i$ are too great, $\delta^{\mathfrak{B}}_{t}$ will reduce attack capability due to too much generalization. Looking back to $\vec{\mathcal{C}}$, the result of dividing $\mathcal{C}_{0}$ by $\mathcal{C}_{\mathcal{Q}-1}$ is

For an attack method without overfitting, it can be assumed that $\forall i,j\in\mathcal{Q}-1,i<j,||M(x_{\mathfrak{B}_{t-1}}^{i})||_{p}<||M(x_{\mathfrak{B}_{t-1}}^{j})||_{p}$. Then for given $\mu>0$ and $\beta>0$, $\frac{\mathcal{G}_{0}}{\mathcal{G}_{\mathcal{Q}-1}}$ will be greater when $\mathcal{Q}$ becomes greater. For example, when $\mu=1$ and $\beta=1$, the upper bound of Eq.(14) is

Even if $\mathcal{Q}$ is not great, e.g. $\mathcal{Q}=5$, $\frac{\mathcal{G}_{0}}{\mathcal{G}_{\mathcal{Q}-1}}$ will be $11.417$ which is much greater than $1$. And, this will cause the LFF result to tend to the $\mathcal{G}_{i}$ with a smaller index value. In extreme cases, this situation can be described as:

Then, LLF will degenerate to the original attack method. To avoid this situation, a very simple but effective method can be applied, which is directly applying future penalty coefficient to $\vec{\mathcal{G}}$. Then the $i$-th coefficient equals $\beta^{i}$. To avoid ambiguity in the expression, this new coefficient is marked as ${\mathcal{C}^{\prime}}_{i}=\beta^{i}$. Then the coefficients vector is $\vec{\mathcal{C}^{\prime}}=(\beta^{1},\beta^{2},\dots,\beta^{\mathcal{Q}})$. The perturbation update process can be described as

At the same time, $\delta=\sum_{i=0}^{\mathcal{Q}}\delta_{i}$ is just the approximation to the optimal point $\delta^{\mathcal{O}}$ with $\mathcal{Q}$ steps. Greater $\mathcal{Q}$ refers to greater complexity. Therefore, LFF can set $\mathcal{Q}$ with a certain value that is not too great, which can combine the advantages of LFF and the low calculation complexity with the original iteration process. Especially, when $\mathcal{Q}=1$, the LFF attack will be degenerated to the corresponding IFGSM-based method. Naturally, the iteration between each $\delta^{\mathfrak{B}}_{t}$ can also be optimized with a similar gradient-based method, e.g. the momentum method. Then a momentum version of the LFF attack, which is named MLFF, can be described as:

where $\eta$ is the momentum decay factor.

The sequence of the $\delta^{\mathfrak{B}}_{t}$ from the one-order LFF attack can still be enhanced with the LFF mechanism. With the same guidelines, the enhancing process can be iterated to $\mathcal{N}$-order LFF, which is named $\textit{LFF}^{\mathcal{N}}$ attack. The description for the $\textit{LFF}^{\mathcal{N}}$ attack is:

where $\nu\in\mathcal{N}$ is the indicator for the order,

and $\delta^{\mathfrak{B}^{0}}_{i}=\delta_{i}$. Especially, when $\mathcal{N}=1$, Eq.(19) will be degenerated to Eq.(17). In the $\textit{LFF}^{\mathcal{N}}$ attack, there are multi-iteration processes. Therefore, there are many ways to combine operations that can further optimize the optimization process. Here, only a simple extension for MLFF with applying momentum mechanism for any $\nu$-th order, where $\nu\in\mathcal{N}$. This method is named $\textit{MLFF}^{\mathcal{N}}$. The corresponding formulation is shown as follows:

where $\eta_{\mathcal{\nu}}$ is the momentum penalty factor for $\nu$-th order.

Because most SOTA methods are realized on MI-FGSM, we conduct the comparison experiment using the MLLF method with those SOTA methods.

The comparison experiment is applied within four scopes. Firstly we evaluate baselines and our proposed method under the condition of a single surrogate network with the untargeted attack task. Secondly, we apply the comparison on the defensive networks as well as defense methods applied to pre-trained networks. Thirdly, the targeted attack task is applied. Fourthly, ensemble networks are applied as the surrogate network. The adversarial example from ensemble networks is tested by both original networks and defensive networks as well as defense methods.

The ablation experiment is applied within 2 scopes. Firstly, we verify the effectiveness of the LFF as well as the MLFF with different $\mathcal{Q}$. Then, we verify the influence of the future penalty factor $\beta$.