Exploring Robust Features for Improving Adversarial Robustness

Following the discovery of deep neural networks’ vulnerability to adversarial examples, numerous research efforts have been dedicated to producing stronger adversarial attacks. [1] considers the generation of adversarial examples as a box-constrained optimization problem. [2] proposes a one-step attack named fast gradient sign method (FGSM) and leads out a lot of variants including basic iterative method (BIM) [17], which is an iterative version of FGSM and MI-FGSM [18] where momentum is integrated. In addition, the projected gradient descent (PGD) proposed by [6] and the CW attack proposed by [19] are widely used as a criterion to measure the robustness of a deep model. [20] attempts to quantify the adversarial robustness by computing the minimal adversarial perturbation. In [21], a per-sample attack named AutoAttack is introduced. AutoAttack is an ensemble of four attacks, APGD_CE, APGD${}^{T}_{DLR}$, [22] and [23]. It attacks by computing the worst case for each image. In addition to additive attacks, there are also geometric attacks such as [24, 25, 26], where geometric transformations are applied to the images to fool the classifier. Moreover, [27, 28, 17] tried to use patches in the physical world to confuse the model. Other than image classification, adversarial examples are crafted for other tasks such as objective detection [29], deep retrieval models [30], medical deep learning systems [31] and power systems [32] to investigate the robustness of the models. In addition, adversarial attacks are also deployed to achieve further performance improvement [33, 34].

Because of the discovery of adversarial examples, defense against the attacks emerges in response. Previous works prove that adversarial-training based defending methods are among the most effective and popular ones. Adversarial training utilizes adversarial examples to train the model, that is, the algorithm will first maximize the loss function to generate adversarial examples, and then minimize the loss function to update the parameters afterward. PGD attack is used in [6], which can be considered as a standard adversarial training method. Based on this, [35] adds constraints to pull the logits of natural images and their adversarial examples closer. [36] and [37] constrain the distance between feature vectors of natural images and their adversarial example with metric learning. [38] reduces the computational complexity of adversarial training by blending the generation of adversarial examples and the optimization of the model parameters. [39, 40] utilize knowledge distillation to transfer adversarial robustness to student networks and [41] applies knowledge distillation and bidirectional metric learning to correct the perturbations from adversarial examples for both feature maps and latent vectors. [42] uses KL-divergence as a regularization to both generate the adversarial examples and train the model while [43] uses feature scattering in latent space to accomplish the generation of attacks. [9, 44] use information bottleneck to reduce redundant information and [8, 10] try to find invariant features for better robustness. [45] designs a bilateral adversarial training strategy. [46] denoises features to defend against adversarial attacks. In [47], the authors take the misclassified examples into consideration and further improve the robustness. A recent study [48] protects particular classes from adversarial attacks with the utilization of cost-sensitive classification and consequently achieves better average accuracy over all classes. Corresponding to attacks on object detection tasks, defending methods against adversarial attacks on object detection models are proposed [49].

Feature disentanglement is a popular branch in the field of domain generalization. [14] designed a domain-agnostic learning (DAL) framework to extract domain-invariant representations by a deep adversarial disentangled autoencoder (DADA) with domain-specific and class-irrelevant features as auxiliary. [16, 50] uses Variational Auto-Encoder and Wasserstein Auto-Encoder [51] for disentanglement and generalization. [13] devise domain specific masks to tackle domain generalization problems similarly as disentanglement does. In addition, similar architecture is used in domain adaptation [52, 53], face presentation attack detection [54] and image-to-image translation [55, 56].

We first briefly introduce the standard adversarial training, including the adversarial examples generation and loss function. Let $F$ be a deep model parameterized by $\Theta$ and $\mathcal{D}$ be a dataset containing $N$ samples $\mathcal{D}=\left\{({\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}x_{i},y_{i}})\right\}_{i=1}^{N}$, where $y_{i}$ is the class label. We use $(x,y)$ to refer to an image-label pair for simplicity. A standard way to generate adversarial examples is maximizing the classification loss as in Eq. 1.

$\delta$ refers to perturbation added to the image $x$ and $\Delta$ defines the constraint of an $\ell_{p}$-norm bound of the total perturbation, such as $\ell_{2}$ and $\ell_{\infty}$ norm. $\mathcal{L}$ is the loss function. Correspondingly, the adversarial training method optimizes the parameters of the model by those adversarial examples, thus can be written in a min-max manner as shown in Eq. 2:

where $\mathbb{E}[\cdot]$ refers to the expectation over the whole dataset. The outer minimization in Eq. 2 is to train the target model on the adversarial examples, while the inner maximization is to generate strong adversarial examples. Currently, the most popular attack is the Projected Gradient Descent (PGD) [6], which utilizes an iterative way to obtain the adversarial examples:

where $t\in\left\{1,2,\dots,T\right\}$ is the number of steps, $\alpha$ is the step size. The constraint ensures the perturbation strength of the adversarial example is within a $\epsilon$-ball, which is also called attack budget, around $x$ to ensure perceptual similarity.

There is a distribution shift between the natural (original) images and adversarial examples because of the add-on adversarial perturbations. Therefore, we consider the natural images and adversarial examples as two domains. We use $x_{adv}\in\mathcal{D}^{\prime}$ to denote images from the adversarial domain and $x\in\mathcal{D}$ for the images from the natural domain.

As shown in Figure 2, an input image is first fed into the feature extractor $G$ parameterized by $\theta$ to get the intermediate features, referred to as $f$, $f^{\prime}$ for $x$ and $x_{adv}$, respectively.

The intermediate features, both of $f$ and $f^{\prime}$, are the entanglements of robust, non-robust and domain-specific information. We then use three encoders, i.e., robust encoder $E_{\omega_{r}}$, non-robust encoder $E_{\omega_{nr}}$ and domain-specific encoder $E_{\omega_{ds}}$ to disentangle them explicitly. $f$ (or $f^{\prime}$) is fed into these three encoders and expected to output disentangled robust features $z_{r}$ ($z^{\prime}_{r}$), non-robust features $z_{nr}$ ($z^{\prime}_{nr}$) and domain specific features $z_{ds}$ ($z^{\prime}_{ds}$), respectively, denoted as:

where $i\in\{r,nr,ds\}$, $\widetilde{z}_{i}\in\{{z}_{i},{z^{\prime}}_{i}\}$ and $\widetilde{f}\in\{f,f^{\prime}\}$.

A natural constraint for the complete disentanglement is that the disentangled features $z_{r},z_{nr},z_{ds}$ ($z^{\prime}_{r},z^{\prime}_{nr},z^{\prime}_{ds}$), should have the small dependence between each other as possible. We utilize a Kullback-Leibler (KL) divergence loss $\mathcal{L}_{kl}$ between each pair of the three disentangled features, for natural images and adversarial examples, respectively, as shown in Figure 2 (c).

Robust (R) features. R features are the intrinsic representations of a specific class and they should be robust to the perturbations, i.e., invariant between natural images and adversarial examples. Therefore, the robust features of the natural image and its adversarial example should be as similar as possible. We minimize the loss of angular distance between these two features as followed:

R features are what we rely on for our task, they should have good classification ability. We impose the classification loss (specifically the cross-entropy) on them, as shown in Eq. 7.

Non-robust (NR) features. NR features are some human-incomprehensible representations that are related to certain details of an object which can be easily contaminated by small perturbations and thus makes natural images and adversarial examples have different classification predictions. NR features in a natural image, $z_{nr}$, should still make a correct prediction, while in an adversarial example, $z^{\prime}_{nr}$, they should make an incorrect prediction. We also impose the cross-entropy loss on them. The following equation absorbs both R and NR features:

where $C_{\phi}(z)$ is out target classifier. As shown in Figure 2 (a), all of R and NR features branches utilize gradient descent (GD) for updating the model’s weights except the $z^{\prime}_{nr}$ branch as it makes the model output an incorrect prediction. Therefore, we update the weights of non-robust encoder $\omega_{nr}$ in the direction of increasing the cross-entropy loss, similar to that in gradient reversal layer (GRL) [57]. Specifically, $-\lambda$ is multiplied to the gradients of $\omega_{nr}$ only for optimization during backpropagation i.e. using $-\lambda\frac{\partial\mathcal{L}_{ce}}{\partial\omega_{nr}}$ instead of $\frac{\partial\mathcal{L}_{ce}}{\partial\omega_{nr}}$.

Domain-specific (DS) features. R/NR features are all class relevant features. Due to the misalignment of statistic distribution and classification performance) between clean images and adversarial examples, there must exist some domain-specific features that represent such domain shift. The domain-specific features should capture the characteristics of the natural images and adversarial examples and thus the domain-specific features from different domains could be identified by a jointly trained discriminator $D_{\psi}$. The domain-specific encoder $E_{\omega_{ds}}$ and $D_{\psi}$ are trained adversarially [7]. The discriminator is first trained to distinguish samples from different domains:

We then fix the discriminator and update three encoders by optimizing the adversarial loss in Eq. 9. Specifically, the R and NR encoders are trained to fool the discriminator by minimizing the adversarial loss as they are domain invariant features, while the DS encoder is trained to be classified by the discriminator.

Reconstruction. While we disentangle the features, we don’t want to have any information loss. Therefore, we constraint the disentangled features $\widetilde{z}_{r},\widetilde{z}_{nr},\widetilde{z}_{ds}$ to be able to recover the original features $\widetilde{f}$, as shown in Figure 2 (b).

where $\theta_{rec}$ is the parameter of a reconstruction module.

Domain diversify. The diversity of training data is of great importance to a model’s generalization ability [12]. We try to obtain a defense model trained on PGD attacks that will be general enough to defend against other unseen (in training) attacks. Therefore, we diversify the training domain by randomly selecting the number of steps, as well as the step size of the PGD attack. With that, we expect our model to have stronger generalizability.

The overall procedure of our proposed model is shown in Algorithm 1.

Dataset The performance of our model is evaluated on three popular datasets: CIFAR-10, CIFAR-100 and Tiny-ImageNet. CIFAR-10 contains 60k images (the image size is $32\times 32$) grouped into 10 classes. 50k images for training and 10k images for testing. SVHN dataset, for the Street View House Number. The training set of SVHN contains 73257 images and the testing set has 26032 images belonging to 10 classes. CIFAR-100 is similar to CIFAR-10 except that it consists of 100 classes and 600 images for each class. Tiny-ImageNet is a subset of ImageNet, which contains 200 classes. For each class, there are 500 training images and 50 validation images. The resolution of images in Tiny ImageNet is $64\times 64$.

Comparison methods We compare the performance of our model with the following methods: (1) undefended model (UM), which is trained by the clean image set only; (2) adversarial training (AT) [6], where the model is trained by adversarial examples generated from PGD attack; (3) TRADES [42] and (4) MART [47], two defending methods with different regularization terms. (5) LBGAT [58], which improve robustness by boundary guidance of undefended model (65) DRRDN[7] and (7) AFD-WGAN[8], which uses feature disentanglement and desensitization to defend adversarial attacks, (8) IAD [40], which utilizes knowledge distillation to transfer robustness. For IAD, we adopt the results with AT as the teacher model here. We use FGSM [2], PGD [6], CW [19] attacks to evaluate the models. We also test the models on black-box attack and AutoAttack (AA) [21], which is a parameter-free attack that ensembles four diverse attacks.

Implementation details We use the residual network as the backbone of our model. The feature extractor $G_{\theta}$ consists of two residual blocks, each contains five basic residual blocks where two $3\times 3$ convolutional layers are included. The encoder is a CNN architecture with one residual block, while the reconstruction module is composed of transposed convolutional layers. The fully connected layer is used as a classifier. We utilize a two-layer fully connected network as the discriminator. For all the datasets, the initial learning rate is 0.1 for the adversarial training of $G_{\theta}$, $E_{\omega_{r}}$ and $C_{\phi}$ and other components have a smaller learning rate. We utilize SGD optimizer and schedule the learning rate decay at 100, 105 and 110 epochs. The batch size is 128 and we use early stopping as suggested by [59]. In order to increase the domain diversity, the attack budget $\epsilon$ is randomly selected between 8 and 12 for each batch. The number of steps is randomly sampled from $\left[8,16,24,32\right]$ and step size ranges from 2 to 4.

We evaluate the accuracy of our model against various attacks. We conducted five runs of our model across all settings and reported the mean and standard deviations resulting from these runs. In Table I, we show the results on CIFAR-10. The accuracy of our model on clean images in the table is directly from the robust features through the classifier. With the help of the well-trained discriminator, this accuracy can be improved to almost the same accuracy as the undefended model, i.e., around $96\%$, we will describe this in section IV-E. The rest of Table I includes accuracy against both white-box attacks and black-box attacks. White-box attack denotes the adversary has full access to the target model, including the architecture and parameters. Our model outperforms comparison methods under most of the white-box attacks. Especially, our model shows the best performance on AA attacks which is known as the most challenging defense task. Black-box attack denotes there is no information known about the target model. We trained a surrogate model with only clean images and generate adversarial examples from this model to attack our model. As shown in Table I, our model still keeps a high accuracy against black-box attacks. To gain a more comprehensive understanding of the model’s performance under different adversarial scenarios during training, we have expanded our evaluation by incorporating results from training the model with attacks other than PGD. We specifically consider the CW attack and an adaptive version of the PGD attack, i.e., the difference of logits ratio (DLR) [21]. The results show that our proposed model performs quite well when trained under different attacks.

The results on the SVHN dataset are shown in Table II. For SVHN, the UM model has a success rate lower than $1\%$ against PGD and CW attacks. The ML method is the best among the comparison models which can achieve over $51\%$ for both attacks. Our model outperforms the comparison methods on all attacks by a large margin and the accuracy could reach $55.20\%$ and $54.09\%$ for PGD and CW attacks.

The results on CIFAR-100 and Tiny ImageNet are shown in Tables III. For CIFAR-100, the undefended model (UM) misclassifies almost all the adversarial examples generated with strong attacks such as PGD and AA with only $0.01\%$ success rate for PGD and $0.02\%$ for AA. AT can increase the accuracy against PGD attack to $18.54\%$ while methods such as TRADES and AFD-WGAN can further improve the number. Our model outperforms other methods on PGD attack with accuracy $31.78\%$. For Tiny ImageNet, it’s hard for UM model to make correct classifications against adversarial attacks. The comparison methods can improve the accuracy to over $10\%$ against PGD and AA attacks to the best. Our method greatly promoted these numbers to $24.78\%$ and $20.64\%$ separately. Moreover, although our model is trained on $\ell_{\infty}$-norm attacks, it surpasses other comparison methods on $\ell_{2}$-norm attacks such as CW₂ by a large margin, showing superior generalization ability of our model.

In this section, we aim to evaluate the scalability of our method in handling larger-sized images and its ability to generalize across different neural network architectures. To achieve this, we perform experiments on the Imagenette dataset, which contains larger images, using three neural network architectures: Inception v4 (Inc-v4) [60], Inception ResNet v2 (IncRes-v2) [60], and ResNet v2-152 (Res-v2-152) [61]. Imagenette is a subset of ImageNet, encompassing ten classes including tench, English springer, cassette player, chain saw, church, French horn, garbage truck, gas pump, golf ball, and parachute. The images of each class in the Imagenette dataset are identical to the original ImageNet dataset with an average image size of $469\times 387\times 3$ pixels.

For the training on the Imagenette dataset, we used a learning rate of 0.01 following [18]. The parameter numbers for Inc-v4, IncRes-v2, and Res-v2-152 are approximately 43M, 56M, and 60M, respectively. Therefore, for Inc-v4, the batch size remained at 128, whereas for IncRes-v2 and Res-v2-152, the batch sizes were reduced to 64 and 40, respectively, due to GPU limitations. All the comparison methods for each architecture use the same batch size for a fair comparison.

The results for Imagenette dataset are presented in Table IV. Our model outperforms other methods under all the architectures. For Inc-v4 and IncRes-v2, our model surpasses other methods by a large margin. Specifically for Inc-v4, our method has imporved the accuracy by $7.33\%$ (from $30.68\%$ to $38.01\%$) and for IncRes-v2, The accuracy has $5.57\%$ increment (from $36.92\%$ to $42.49\%$).

We use k-Nearest Neighbor (k-NN) method as a substitute of the original (softmax) classifier in the model following [36]. In our experiment, we use the same settings as [36] where $k$ is set to 50 and the features from the penultimate layer are used to build the k-NN classifier.

The results for both k-NN classifier and the original classifier are shown in Table V on three datasets, CIFAR-10, SVHN and Tiny ImageNet. The comparison methods include AT [6] ALP [62] SML [36]. Among the four approaches, our method can obtain a consistently higher accuracy against adversarial attacks on all datasets. Besides, although the k-NN is one of the simplest classifiers, it can achieve similar results compared with the original classifier. These quantitative results, coupled with the visualization illustrations in Section IV-I, demonstrate the features obtained by our model have better distribution in latent space. That is, the adversarial robustness of our model is from the latent space, not the classifier itself.

As a side product, the discriminator and domain specific features have great capability to distinguish the clean image and adversarial example, which can be used for adversarial example detection. Built upon that, we propose to use the natural trained model for clean images and our model for adversarial examples, to improve the accuracy of clean images. Our model can achieve $99.86\%$ and $100\%$ for true positive rate and true negative rate, respectively, in adversarial example detection, which makes a significant improvement on clean images (from $83.52\%$ to $96.20\%$), with a negligible drop on adversarial examples (form $57.97\%$ to $57.88\%$).

We visualize the distribution of the domain specific (DS) features for 500 randomly selected samples and their adversarial examples, using t-SNE, in Figure 3. We can see that the DS features of natural images and adversarial examples are quite discriminable even in 2-dimensional space, while there are very few mis-clustering samples, we believe that they should be more easily classified in original high dimensional space.

We visualize the domain specific (DS) features, a 640-dimension vector that is obtained after the average pooling of a 640-channel feature maps output from the domain specific encoder, for natural images and their adversarial examples. Figure 4 illustrates the DS feature histograms, which show the intensity distributions of the features, of some sampled natural images and their adversarial examples. From the figure, we can see that the intensity distributions for natural images and adversarial examples are quite different, which means the DS encoder has the ability to discriminate the features from natural images and adversarial examples.

In Figure 5, we visualize the distributions of the robust and non-robust features of natural images and their corresponding adversarial examples. The natural images are from the same class (frog) and the AEs are generated with PGD attack. As in Figure 5, our method successfully disentangles the robust and non-robust features. The robust features extracted from both natural images and adversarial examples are positioned within the same region. This alignment is due to the explicit pulling together achieved by the angular loss Eq. 6.

On the other hand, the non-robust features extracted from natural images and adversarial examples are visibly separated. These features are the components modified by the adversarial perturbations to decrease (natural images) and increase (adversarial examples) the classification cross-entropy loss Eq. 7 and are effectively disentangled by our proposed method. The visualizations in Figure 5 provide valuable insights into the disentanglement process, demonstrating the efficacy of our approach in capturing both robust and non-robust features for adversarial examples.

We evaluate the model’s robustness under different attack iterations, from 10 to 100. Figure 6 shows that our model consistently outperforms standard AT, while it is not sensitive to the attack iterations although the accuracy monotonically declines along with the iteration number increase, for example, our model achieves $58.38\%$, $57.97\%$, and $57.82\%$ for 10, 20, and 100 iterations of PGD attack, respectively

We visualize the representation distribution of samples in feature space using t-SNE. Specifically, we show the representation distribution for all the classes and for the clean images and adversarial examples in a specific class [36], in Figure 7 and Figure 8, respectively.

In Figure 7, the triangle points with different colors represent the clean images in different classes, while the red round points are the AEs from a specific class ( horse and truck) under PGD attack. “UM”, denotes an undefended model, and shows how the adversarial attacks behave on a model that is naturally trained on clean images. PGD attack is able to drop its accuracy to 0, which is also visualized in the first column of Figure 7, where all the AEs locate far from their original class, and fit into the distributions of other classes. “AT” shows how adversarial attacks behave on a standard adversarial training model. Many of the adversarial examples are pulled back to their original class but there still are a lot of them mixed with other classes. Our model pulled the most of adversarial examples back to their original class. While we observe that our model shortened the gap between classes for clean images and it may be the reason that the classification performance of our model drops a lot on clean images, we believe it rarely affects the performance on adversarial examples based on the results shown in Table. I. For the clean image, as discussed previously, our model can keep the same performance as the naturally trained model by using a simple adversarial example detection strategy, thanks to the extraordinary performance of the discriminator and domain specific features obtained from our model.

In Figure 8, the flesh color triangles are clean images of a specific class and round points are adversarial examples crafted from the clean images of the same class. The colors represent the different predictions of those adversarial examples. In “UM” has no ability to defend against the attacks and all adversarial examples are falsely predicted. The features of adversarial examples are away from the distribution of natural images and clustered based on the prediction. For “AT”, some of the adversarial examples can be correctly classified. Although the attacks are pulled closer to the location of natural images, the successful attacks are still away and show aggregation based on class. For our model, the natural images and adversarial attacks share almost the same distribution and there is no distinct gathering for different categories, representing that the robust encoder of our model does extract the invariant features between natural images and adversarial examples, that is, the robust features.

In [63], the authors provided constructive insights about gradient obfuscation in adversarial robustness. Gradient obfuscation is a gradient masking that leads to a false sense of security in defenses against adversarial examples. Based on the analysis in that paper, we conclude that the robustness of our model is not from gradient obfuscation, for the following three reasons: (1) We provide the performance of the model under attacks with different iterations in Figure. 6, which shows monotonically declining accuracy along with the attack iterations increasing. (2) In Table I, we show that the black-box attacks ($83.04\%$) have a lower success rate (higher accuracy) than white-box attacks ($57.97\%$). (3) We evaluate our model against a gradient-free attack SPSA [64] and obtain an accuracy of 80.21, which is higher than gradient-based attacks ($57.97\%$ for PGD).