Quantization Aware Attack: Enhancing Transferable Adversarial Attacks by Model Quantization

Papernot et al. [13] first proposed the concept of transfer-based black-box attacks, in which the adversary trains a local substitute model to obtain transferable adversarial examples. Various methods have been developed to enhance the transferability of adversarial examples. These attack methods can be divided into five categories [19]: momentum-based attacks [20, 21, 22, 23, 24], input-transformation-based attacks [25, 26, 27], feature-level attacks [28, 29, 30, 31], GAN-based attacks [32, 12], and model-based attacks [33, 34, 35]. Momentum-based attacks add regularization terms to the original attack objectives when attacking a substitute model; thus, the optimization gradient can be stabilized and the attack transferability can be enhanced. Input-transformation-based attacks mimic model ensembles by applying image-transformation operations when attacking the substitute model to achieve enhanced transferability. Feature-level attacks operate under the hypothesis that the features of the shallower DNN layers represent basic and invariant image information and, thus, are more significant for generating transferable adversarial examples.  GAN-based attacks leverage generative models to obtain transferable adversarial examples, and they are mighty for targeted attacks [12]. Model-based attacks devise a substitute model to smooth the loss landscape and thus enhance attack transferability. This paper describes a model-based attack, in which the transferability of adversarial examples across QNNs with different architectures and bitwidths is investigated by quantizing and fine-tuning the substitute model.

Adversarial attacks can be divided into white-box attacks and black-box attacks. Existing works have only designed white-box attacks against QNNs [36]. However, how to improve black-box attacks against both full-precision DNNs and QNNs with different architectures and quantization bitwidths is still an open problem. The adversarial robustness of QNNs has received increased research attention from the defender’s perspective. For example, existing works have designed adversarial training [16], regularization-based defense [37], and ensemble-based defense [15] strategies for QNNs. In particular, previous works have shown adversarial examples transfer poorly across DNNs with the same architecture but different quantization bitwidths because of the quantization shift and gradient misalignment phenomena [14]. Intuitively, this characteristic of QNNs makes it difficult for the adversary to successfully attack QNNs with different bitwidths simultaneously, enabling defenders to enhance the robustness of DNNs by ensembling QNNs with different bitwidths. For instance, EMPIR [15] directly ensembles full-precision DNNs and low-bitwidth QNNs with the different bitwidths to enhance the robustness, and Double-Win Quant [16] alternatively applies the adaptive quantization technique to the adversarial training scheme to enable QNNs to exhibit different bitwidths during inference. The research gap lies in the fact that previous works only studied the transferability of adversarial examples under the “same architecture, different quantization bitwidth” setting, while the conclusion can be different under the “different architecture, different quantization bitwidth” setting, which is closer to the realistic black-box scenario. In summary, there are two dimensions along which our work differs from the others. The first is that it exhibits improved adversarial transferability across target models with unknown architectures and bitwidths. The second is that it quantizes and fine-tunes the substitute model to achieve the above goal.

DNN quantization compresses and accelerates DNNs by representing network weights/activations/gradients with lower bitwidth. Model quantization has been widely applied in efficient DNN inference frameworks in industry [38]. Two main categories of quantization techniques are quantization aware training (QAT) [39, 40, 41, 42] and post training quantization (PTQ) [43, 44, 45, 46]. QAT trains DNNs from scratch or fine-tunes full-precision DNNs given a training dataset $\mathcal{D}_{train}=\{x_{i},y_{i}\}_{i=1}^{n}$ to convert these DNNs into low-bitwidth DNNs. In contrast, PTQ does not require end-to-end training and instead relies on a calibration dataset $\mathcal{D}_{cali}=\{x_{i},y_{i}\}_{i=1}^{m}$ ($m<<n$) to implement a rounding scheme. PTQ is more computationally efficient than QAT, but it has greater quantization loss. Recent PTQ-related works [43, 44, 45, 46] have modeled weight/activation quantization as perturbations and used Taylor expansion to analyze loss value changes before reconstructing the output of each layer. To implement QAT or PTQ algorithms on GPUs, one can use fake quantization to mimic QNNs, in which the weights are still stored at full-precision but are rounded into lower bitwidth during inference. QAT process with fake quantization can be formalized as follows,

in which $\mathcal{L(\cdot)}$ denotes the loss function (e.g., cross-entropy loss), $w$ is the DNN weight, and $\beta$ is the quantization hyper-parameter used to convert $w$ from full-precision floating points into lower bitwidth numbers. PTQ with fake quantization can be formalized as follows.

The major difference between QAT and PTQ is that the PTQ only optimizes quantization hyper-parameter $\beta$ while keeping the model weights $w$ unchanged. Apart from the above model quantization techniques with fixed bitwidths, a line of work has studied adaptive bitwidth quantization [47, 48, 16], which can switch the quantization bitwidth after deployment. The application of such adaptive quantization techniques in realistic settings further urges us to develop effective transfer-based attacks against QNNs with different bitwidth.

In the realistic black-box scenario, the architecture and quantization bitwidth of the target model are unknown to the adversary. Thus, it is important to study the transferability of adversarial examples across DNNs with different architectures and quantization bitwidths. However, previous works claimed that adversarial examples transfer poorly across DNNs with different quantization bitwidths [14, 15, 16]. To verify this claim, we conduct illustrative transfer-based attack experiments on the ImageNet dataset with two model architectures (ResNet-18, ResNet-34) and six quantization bitwidths (32, 8, 5, 4, 3, and 2-bit settings). The model quantization technique is APoT [42], and the transfer-based attack algorithm is MIM [20] with a $l_{\infty}$-norm perturbation budget $\epsilon=16/255$. The experimental results are shown in Tab. I. From these results, we have the following observations.

Observation 1: Model quantization harms the same-architecture transferability of adversarial examples. Given that the substitute model has the same architecture as that of the target model, the use of different quantization bitwidth lowers the transfer attack success rate. For instance, when using ResNet-18 with 32-bit as the substitute model, the attack success rate decreases as the quantization bitwidth of the target model decreases; when using ResNet-18 with 2-bit as the substitute model, the attack success rates increase as the quantization bitwith of the target model decreases. This observation is exactly the same as the previous “poor transferability” conclusion [14, 15, 16].

Observation 2: Model quantization benefits the cross-architecture transferability of adversarial examples. The conclusion can be different when the substitute model and the target model have different architectures. For instance, when taking ResNet-34 as the substitute model to attack the ResNet-18 target model, the 2-bit substitute model outperforms the 32-bit model under every target bitwidth. A similar phenomenon occurs when ResNet-18 is used as the substitute model to attack ResNet-34. Although the 2-bit substitute model does not achieve higher attack success rates on the 32-bit target model, it outperforms the 32-bit substitute model at other target bitwidths and has higher average attack success rates.

Remark on the above observations. The same-architecture observation (Observation 1) indicates that the defender may ensemble models with different quantization bitwidths to hinder the transferability of adversarial examples and enhance the adversarial robustness of their models; this strategy has been explored in previous works [15, 16]. The cross-architecture observation (Observation 2) indicates that model quantization may enhance the transfer attack ability of the adversary under black-box scenarios, which was ignored by previous works. Please note that the setting of Observation 2 is closer to the realistic black-box setting where both the target architecture and the quantization bitwidth are unknown to the adversary. Given that the adversary has no portable way to detect the architecture or the bitwidth of the target model, the best choice for the adversary is to leverage 2-bit quantized substitute models to achieve the best average attack performance. In the next subsection, we discuss how to achieve the best black-box attack capability from the perspective of the adversary and propose our QAA method.

Quantization compresses and accelerates DNNs by converting the 32-bit weight and activation values ($w$ and $a$) into lower bitwidths ($\hat{w}$ and $\hat{a}$):

where $b_{w},s_{w},b_{a},s_{a}$ are the bias and scaling terms for weight quantization and activation quantization, respectively; Int denotes the integer operation. The adversary’s goal is to maximize the probability of fooling target model $f$ with unknown architecture $m$ and bitwidths $q$, that is,

where $x$ and $y$ are clean example and its true label, respectively, $\delta$ is the adversarial perturbation, $\mathcal{P}$ denotes the probability, and $f_{q,m}$ denotes target model $f$ with certain bitwidths $q$ and architecture $m$.

To generate adversarial examples, the adversary needs to train a local substitute model $\overline{f}$ with certain architecture $\overline{m}$ and bitwidth $\overline{q}$ on the available training dataset and generate adversarial examples, that is,

where $\mathcal{L}$ is the surrogate loss function.

From Eqn. 4 and Eqn. 5, we can see that the difference between the architectures and bitwidths of the substitute and target model is the prominent factor that hinders the transferability of the generated adversarial examples. A previous work [14] has proposed the following two hypotheses to explain the poor transferability of adversarial examples across target DNNs and QNNs with different bitwidths.

Quantization shift. Quantization can map two different types of feature values (i.e., clean features and adversarial features) into the same bucket, damaging the adversarial effect.

Gradient misalignment. QNNs approximately compute gradients with the straight-through estimator (STE) [17], making the gradients of QNNs diverge largely from those of full-precision DNNs.

This paper proposes a quantization aware attack (QAA) to address the above two challenges by making the substitute model training procedure “aware of” quantization objective with different bitwidths, that is,

where $\overline{f}_{q}$ denotes the substitute model with bitwidths $q$, weights $w$, and quantization parameter set $\beta$ (including the $b_{w},s_{w},b_{a},s_{a}$ mentioned above). This objective function means that a single substitute model is quantized to multiple bitwidths when calculating losses and gradients, making the substitute model “aware of” different quantization bitwidths. At the attack iteration stage, the substitute model performs inference with diverse bitwidths to prevent adversarial examples from overfitting to attacking the model with a single bitwidth, thus enhancing the transferability of the approach.

The QAA addresses the poor transferability issue of adversarial examples across QNNs in two aspects. From the quantization shift perspective, the QAA performs multiple bitwidths inferences in the attack generation stage, forcing the algorithm to find adversarial examples that are effective on the target QNNs with multiple bitwidths. From the gradient misalignment perspective, the proposed QAA training objective Eqn. 6 makes the gradients of the substitute model align with target QNNs. We provide quantitative analysis in Sec. III-E to support the above claims.

Training the substitute model with Eqn. 6 is non-trivial because of the following two challenges. From the view of computational overhead, training the substitute model with all possible bitwidths is expensive and unnecessary in reality. We need to form a trade-off between the substitute model training overhead and the attack performance. From the view of batch normalization (BN) layers, BN is incapable of capturing the complex and largely divergent distributions of weights with different bitwidths, making it difficult for the model training process to converge. This problem has also been observed in other recent works [49, 50]. We plot the BN parameters of 32-bit and 2-bit ResNet-34 in Fig. 2 to quantitatively illustrate this problem. In Fig. 2, we show the channel-wise BN statistics of the 20th layers in the ResNet-34 ImageNet models obtained by full-precision training and quantized training. Each dot represents the running mean and variance of a channel. The full-precision DNN and low-bitwidth QNNs have very different weight distributions. Thus, we need to circumvent the BN incompatibility issue when training substitute models with Eqn. 6. The question then becomes the following: How can we circumvent the computational overhead and the BN incompatibility problem when training substitute models with multiple bitwidth objectives?

To answer the above question, we propose training the substitute model with only two bitwidths (32-bit and 2-bit) to address the trade-off between the computational complexity and attack performance; that is,

We choose 32-bit and 2-bit settings because they represent two extreme cases, i.e., full-precision and low-precision cases. We do not use 1-bit QNN [39] because it is difficult to converge on ImageNet, and needs special tricks that are not compatible with other bitwidths. Regarding the BN incompatibility problem, although weight quantization is incompatible with the BN layer, applying the multiple bitwidths training objective only for activation quantization does not cause problems because activation quantization is performed after the BN layer (please see Fig. 1 for reference). Therefore, we propose applying multiple bitwidths training to the activation quantization process only and keeping the weight quantized as 2-bit.

Alg. 1 summarizes our QAA implementation. The adversary can resume from standardly pre-trained QNN to save the computational budget. In practice, fine-tuning for only one epoch is sufficient for enhancing the attack transferability to a large extent. The QAA trains the substitute model with quantized activation (line 5) and full-precision activation (line 9) alternatively while keeping the weight quantized. When generating adversarial examples, quantized activation (line 18) and full-precision activation (line 20) are also alternatively adopted, making the adversarial examples transferable across target models with different quantization bitwidths. The QAA is flexible and can be implemented using various quantization methods, including QAT [42] and PTQ [46].

The implementation of the QAA based on PTQ is summarized in Alg. 2. Compared to the QAA implementation based on QAT, the QAA implementation based on PTQ is different in two ways. First, because PTQ does not train and quantize the model from scratch, the QAA implementation based on PTQ does not need model training or fine-tuning steps (there is no substitute model training step in Alg. 2); Second, instead of keeping weights quantized all the time, the QAA implementation based on PTQ applies multiple bitwidth inferences to both the weight and activation values (see the forward propagation step in Alg. 2). The PTQ version of the QAA does not require fine-tuning but is less powerful than the QAT version of the QAA.

This subsection quantitatively analyzes whether the proposed QAA can mitigate the quantization shift and gradient misalignment issues mentioned above. To visualize the quantization shift phenomenon, we calculate the $k$-th layer feature divergence of the target model on an adversarial example and its corresponding clean example; that is,

where $f_{k}$ denotes the $k$-th layer of the target model $f$, $x$ is the clean example, $x_{adv}$ is the corresponding adversarial example, and $||\cdot||$ is the $l_{2}$-norm. We compare the feature divergence of the target model on adversarial examples generated with the QAA and the traditional attack based on the full-precision substitute model (FP-based attack). The model architecture is ResNet-34 for both the substitute models and target models, and the attack algorithm is MIM [20] with a perturbation budget of $\epsilon=16/255$. The results are presented in Fig. 3. The feature divergence of the target QNN on the QAA attack is denoted by red lines, and the feature divergence on the FP-based attack is denoted by green lines. The black line denotes the white-box ideal case for reference. The utilized model is ResNet-34 for both the substitute and target models. The applied dataset is ImageNet. The attack is MIM with $l_{\infty}$-norm budget $\epsilon=16/255$. We can see that compared to the white-box baseline, the FP-based attack (denoted as the green line) substantially decreases the feature divergence, suggesting that the traditional FP-based attack suffers greatly from the quantization shift phenomenon. In comparison, the feature divergence of the QAA (denoted as the red line) decreases less. The above observation shows that the QAA is more robust to the quantization shift phenomenon and thus is more transferable across QNNs with different bitwidths.

To analyze the effectiveness of the QAA from the gradient misalignment perspective, we adopt the same method as that used in [14, 51], calculating the gradient similarity between the target model and substitute model, that is, the cosine similarity of the gradient between the target model and substitute model,

where $T$ denotes the transpose of a matrix. Intuitively, a substitute model with higher gradient similarity to the target model enables the generation of more transferable adversarial examples. Specifically, we calculate the gradient similarity of each substitute model on 16 target models with different architectures and bitwidths (which are the same as those of the target models used in Sec. IV) and take the average. The 32-bit, 4-bit, 2-bit, and QAA-based ResNet-34 substitutes have average gradient similarities of 0.1180, 0.1186, 0.1196, and 0.1250, respectively. The maximum standard deviation of this experiment is $3.07\times 10^{-11}$, which shows that the QAA model has a statistically significant advantage. We draw Fig. 4 to visualize the gradient alignment. We calculate the gradient similarity between each model pair and define the distance between two models as 1 - gradient similarity. We use the multi-dimensional scaling (MDS) algorithm [52] to project the distance relationship into 3D space, as illustrated in Fig. 4. Each point represents a model, and closer points indicate similar gradients. The QAA model is at the center of the target models; in other words, the QAA model has the shortest distances (in other words, highest gradient similarities) to these target models.

Dataset. We adopt ImageNet [53] and CIFAR-10 [54] to evaluate the QAA. On ImageNet, we follow a previous work [29], which conducted experiments on a subset of the ImageNet 2012 validation set containing 1,000 high-resolution images (299x299). On CIFAR-10, we use the test set to generate adversarial examples, which include 10,000 images (32x32). Please note that many previously developed transfer attacks are not effective on CIFAR-10 (see Tab. III), while the QAA is effective on both CIFAR-10 and ImageNet.

Target Models. We collect target models with various architectures, bitwidths, and defense methods to validate the generalizability of the QAA. For ImageNet, we collect 21 target models, including seven standardly trained DNNs, five adversarially trained DNNs, and nine QNNs with bitwidths ranging from 2-bit to 8-bit. Here, we do not consider 1-bit target models on ImageNet because the adopted QDrop technique [46] cannot achieve substantial accuracy under the 1-bit setting. For CIFAR-10, we collect 16 target models, including five standardly trained DNNs and 11 QNNs with bitwidths ranging from 1-bit to 8-bit.

Baseline Attacks. On ImageNet, we combine the QAA with the SOTA transfer-based attacks designed for full-precision DNNs, including MIM [20], CIM (=TI+DI+SI+MIM) [22], FIA [29], RPA [55], Admix [26], and SSA [27]. On CIFAR-10, we only combine the QAA with PGD, which is strong enough to largely outperform the SOTA transfer-based attacks. We adopt the $l_{\infty}$-norm to bound the adversarial perturbation and the $\epsilon$ values are $16/255$ and $8/255$ for ImageNet and CIFAR-10, respectively. In the tables in this section, we use bold numbers to denote higher transfer attack success rates than those of the baseline. The experimental results are in Tab. II, Tab. III, Tab. IV, Tab. VI, and Tab. V are repeated three times to calculate their maximum standard deviations. We list the maximum standard deviations in the caption, and they are much less than the improvement magnitude yielded by the QAA, illustrating that the QAA provides statistically significant improvements.

Implementation Details. On ImageNet, we implement the QAA on Res-34 quantized with the QAT and Vgg-16 quantized with PTQ [46]. On CIFAR-10, we implement the QAA based on Res-20 and Res-56 models quantized with QAT [42]. For both datasets, we fine-tune the 2-bit pre-trained QNNs for only one epoch. The batch size is 128, and the optimizer is SGD with a momentum of 0.9 and weight decay of 1e-4. The QAA implementation on PTQ does not need fine-tuning and can be directly used to generate adversarial examples after the QAA modification. The number of attack iterations is 10.

Attacking Standardly Trained Models and Adversarially Trained Models. Adversarial training is the SOTA adversarial defense method, which is selected as the target for evaluating the effectiveness of the QAA. The results are reported in Tab. II and Tab. IV. We can see when the QAA is included, the baseline attack success rates against both standardly and adversarially trained target models are significantly enhanced. Specifically, QAA improves the attack success rates of MIM, CIM, FIA, RPA, Admix, and SSA on the Res-34 substitute model by 20.9%, 4.6%, 6.7%, 6.7%, 12,3%, 6.3% against standardly trained models; and 11.3%, 8.8%, 13.2%, 13.5%, 15.5%, 8.7% against adversarially trained models on average. Similar observations can be drawn from Tab. III concerning CIFAR-10. In Tab. III, we can see that some previously developed transfer-based attacks (CIM, FIA, and RPA) do not outperform the traditional MIM and PGD attacks on CIFAR-10. This suggests that the effectiveness of these advanced attacks that were originally claimed on ImageNet may not generalize well to other (simpler) datasets such as CIFAR-10. This finding is consistent with that in  [35]. In comparison, the QAA largely outperforms the other transfer-based attacks on both CIFAR-10 and ImageNet, highlighting its good generalizability across different datasets.

Attacking QNNs with Different Bitwidths. Tab. VI and Tab. V report the attack success rates against QNNs with different architectures and bitwidths on ImageNet and CIFAR-10, respectively. The results show that the QAA achieves higher attack success rates than the baseline attacks, highlighting QAA’s capability of improving the adversarial transferability across QNNs with unknown architectures and quantization bitwidths. We can see that on the Vgg-16 ImageNet substitute model, the QAA improves less when combined with Admix and SSA. This is because, on Vgg-16, we apply the QAA implementation based on PTQ, which does not fine-tune the substitute QNN but only “self-ensemble” it, which contradicts the intuition of Admix and SSA (Admix and SSA resort to image transformation to achieve “self-ensembling”). However, for the Res-34 substitute model, even Admix and SSA improves substantially when combined with the QAA, highlighting the effectiveness of QAA fine-tuning.

Remarks on the above transfer-based attack results. Previous works [14, 15, 16] have investigated the attack transferability problem mainly from the defender’s perspective and agree that adversarial examples transfer poorly across QNNs with the same architecture but different bitwidth, and this paper draws a different conclusion from the black-box attack perspective that extremely-low bitwidth QNN substitute models can be a powerful tool in generating adversarial examples across unknown model architectures and bitwidths. The experimental results in this section verify the above claim and our findings complement those of previous works.

This section compares the QAA with other naive approaches to highlight its contribution. The dataset is ImageNet, the attack algorithm is MIM with $l_{\infty}$-norm $\epsilon=16/255$, and the substitute model architecture is ResNet-34 if not mentioned.

Comparing QAA with QNN-based Attacks. We compare the QAA with QNN-based attacks to verify that the QAA overcomes the shortcomings of substitute models with single bitwidths. For the QNN-based attacks, we use pre-trained QNNs (5, 4, 3, 2-bit) without any fine-tuning or other modifications as the substitute model. The results are presented in Fig. 5a. We can see that the QAA outperforms every compared baseline attack on every target model, verifying that the QAA overcomes the limitations of transfer-based attacks with single-bitwidth substitute models.

Comparing the QAA with ensemble attacks. To enhance the attack transferability against target QNNs, a naive and straightforward approach is to ensemble substitute QNNs with different bitwidths, which is denoted by ensemble attacks in this paper. Because QAA uses two quantization states (32-bit and 2-bit settings) in Eqn. 7, we ensemble 32-bit and 2-bit versions of ResNet-34 for a fair comparison. We have tried three commonly adopted ensemble techniques: ensembling at the logits layer (logits), ensembling at the softmax layer (softmax), and randomly sampling one model for each attack iteration (sampling). The results are presented in Fig. 5b. We can see that QAA outperforms every ensemble attack except on ResNet-50, on which the attack success rate of QAA is slightly lower than that of the logits attack. Please note that the substitute model used in the experiment is also ResNet, which means that while QAA may not be better when attacking ResNet-50, it transfers better on totally different target architectures. Another advantage of the QAA is that its training, inference, and storage overheads are the same as those of a single QNN substitute model. In contrast, ensemble attacks multiply the computational overhead because they rely on training and running multiple substitute models. These experimental results demonstrate that the QAA provides further improvements and contributions compared to naive ensemble attacks. We also compare the QAA with the ensemble attack LGV [34] against standardly trained and adversarially trained target models in Tab. VII and Tab. VIII, respectively. We can see that, compared to LGV, QAA achieves comparable transfer attack success rates with only 1/10 of the training overhead and 1/40 of the storage overhead. In addition, the QAA can also be combined with LGV to further enhance the attack transferability without increasing the computational overhead. The detailed description of LGV and the hyper-parameter settings of this experiment can be found in the Appendix.

Comparing the QAA with other Training Objectives. We compare the QAA with four other naive training objectives to highlight the contribution of the QAA. The hyper-parameters of the compared methods are the same as those of the QAA for a fair comparison. The compared methods include A. fine-tuning QNNs with only the original low bitwidth objective; B. fine-tuning 2-bit QNNs with 32-bit objective; C. fine-tuning 32-bit DNNs with Eqn. 7; and D. fine-tuning QNNs with 32-bit activation. The results in Fig. 5c show that each training method used in this experiment fails to achieve the same attack success rates as those of the QAA.

This section explains why the QAA is effective from various perspectives. First, we take a closer look at the QAA to determine how the different states of QAA contribute to the final attack performance. Second, we explain the effectiveness of the QAA from the view of the loss landscape.

A closer look at the QAA. As mentioned in Sec. III, the QAA substitute model implements multiple quantization states when generating adversarial examples, including both the 32-bit activation state and the 2-bit activation state. It is natural to ask the following two questions. First, does each quantization state of the QAA substitute model improve compared to QNNs with the same bitwidth? Second, does QAA improve upon each state it performs? To answer these two questions, we conduct transfer-based attack experiments with standardly trained models on ImageNet, and the attack algorithm is MIM [20] with a $l_{\infty}$-norm budget of $\epsilon=16/255$. The results are shown in Tab. IX. We can see that a single QAA quantization state improves upon single QNN substitute models (e.g., 2-bit QAA is better than 2-bit QNN), and the attack performance of the QAA is even greater than that of the QAA with a single state. The results are consistent across different target models and substitute models. Now we can conclude that the answer to the above two questions is yes. From this experiment, we can see that QAA not only improves the attack transferability of substitute models with single quantization states but also improves the transferability over these quantization states. Please note that these improvements come with a trivial computational cost, because the adversary only needs to fine-tune a pre-trained QNN for just one epoch to obtain a QAA substitute model, and the inference cost of the QAA model is the same as that of a single QNN model.

Explanation from the loss landscape view. As mentioned in Section III, the QAA improves its transferability by overcoming quantization shift and gradient misalignment. This section provides a different explanation inspired by [34]. The transferability between the two models can be explained by loss sharpness in both weight space and feature space. As stated in  [34], substitute models with flatter loss landscape in the weight space (in other words, lower sharpness in the weight space) enable the generation of adversarial examples in the flatter region of the loss landscape in the feature space. Adversarial examples in the flatter region of the feature space have better transferability. Please see Fig. 6, which is adapted from  [34, 18] for reference and illustrates the loss landscape in the feature space. The Y-axis indicates the loss values, and the X-axis denotes the input features. The transfer gap of the adversarial examples in the flat region is significantly lower than that in the sharp region. The blue line marks the loss landscape of the target model, and the red line marks the loss landscape of the substitute model, which is slightly different from that of the target model. Adversarial examples falling on the flat region are more robust to slight landscape variations caused by the different architecture and quantization bitwidths between the substitute and target models; thus, they have better transferability. We believe that the QAA can promote the convergence of adversarial examples to flat regions. The following experimental results verify the above claim.

Similar to [34], we calculate the loss sharpness in both the weight space and the feature space. The weight space sharpness of the substitute model is defined as

where $C_{\epsilon}=\{z\in\mathbf{R}^{p}:-\epsilon\leq z_{i}\leq\epsilon,\forall i\in\{1,2,...,p\}\}$ is a constraint set with magnitude of $\epsilon$. As has been proven by previous work [18], Eqn. 10 is a good approximation of the magnitude of the eigenvalues $\nabla^{2}\overline{f}(x)$, and is used to characterize the sharpness of a model. The feature space sharpness of the substitute model $\overline{f}$ around the adversarial example $x_{adv}$ is defined as

where $C_{\epsilon}=\{z\in\mathbf{R}^{p}:-\epsilon\leq z_{i}\leq\epsilon,\forall i\in\{1,2,...,p\}\}$ is a constraint set with magnitude of $\epsilon$. Intuitively, Eqn. 11 calculates the loss variation within a small ball around $x_{adv}$, and a lower sharpness suggests a flatter loss landscape in the feature space. We solve the above optimization problem with a projected gradient algorithm for 20 iterations and report the results with two values of $\epsilon$, $(5\cdot 10^{-4},10^{-3})$ in Tab. X. Compared to transfer attacks using full-precision substitute models, the QAA consistently achieves lower sharpness values in both the weight space and the feature space, verifying that the QAA promotes adversarial examples to fall in the flat region and thus enhances the attack transferability.

LGV [34] is a SOTA transfer-based attack that fine-tunes and ensembles the substitute model to enhance the attack transferability. We implement LGV, QAA, and LGV+QAA on Res-34 with the ImageNet dataset, and the attack algorithm is CIM with perturbation budget $\epsilon=16/255$.

The LGV re-implementation follows the original paper, which fine-tunes the pre-trained Res-34 for 10 epochs with a constant learning rate of 0.05 and collects 40 checkpoints for ensembling. In the attack generation stage, the collected checkpoints are randomly and non-repeatedly sampled. The training overhead of LGV is 10 times greater than that of the QAA, and the storage overhead is 40 times greater.

For LGV+QAA, we utilize either the QAA checkpoint or the LGV checkpoints in each attack iteration. To find the optimal sampling ratio, we perform multiple trials as shown in Fig. 7 and set the sampling ratio to 1:1.