Rethinking Feature Uncertainty in Stochastic Neural Networks for
Adversarial Robustness

Many methods have been proposed to attack deep learning models. Researchers further divided attack methods into white- and black-box attacks according to whether they can obtain the gradient information.

Nowadays, SNNs have shown promising results via injecting fixed or learnable noise into model activations/weights. RSE Liu et al. (2018) uses ensemble tricks to improve model robustness. Specifically, they inject Gaussian noise to multi-layers during training and then perform multiple forward passes to test it. It means they only need training one time and can be viewed as an ensemble model. RSE mainly uses fixed variance by hand-tuned. Parameter noise injection (PNI) He et al. (2019) further proposed to learn a sensitive parameter to control the variance. Moreover, L2P Jeddi et al. (2020) updates PNI by alternating training a noise module and a neural network module, which they called ”alternating back-propagation.”

In this section, we mainly discuss some similar but different works to clarify the contribution of our work. The most related work is proposed by Yu Yu et al. (2021), in which they propose to use a max-margin entropy loss to regularize the feature distribution. However, we need to highlight some main differences between them. Firstly, the construct way of the Gaussian layer is different. In their work, a parallel Gaussian mean and Gaussian variance may meet mistakes when increasing the Gaussian variance. In this situation, the Gaussian mean and Gaussian variance share the same parameters on the feature layer but different optimization ambitions which may pass on obfuscate gradient information to the last feature layer matrix through back-propagation. Secondly, the margin $b$ in their paper largely restricted the model representation ability, and it means the authors given a solid prior information which can not ensure rationality. A similar situation can be seen in the deep variational information bottleneck (VIB) Alemi et al. (2017) model that uses a definite Gaussian variance, which also restricts the representation ability for model robustness.

To sum up, while previous researches often use fixed feature uncertainty, we highlight that we are the first to explore the relationship between feature uncertainty and model robustness. The key finding of our research is that larger uncertainty will bring higher robustness. That is why the proposed method MFDV-SNN achieves state-of-the-art results compared to previous various SNN-based models. Moreover, the proposed method is simple and efficient, motivating researchers to rethink the feature uncertainty in SNNs for adversarial robustness.

In practice, we only need to maximize the unbounded feature distribution variance to achieve significant improvement. The unbounded high variance will be self-adaptive to the model architecture and dataset and then explore a more stable representation. We also need to claim that the unbounded variance will not collapse since the gradient is $-\frac{1}{\sigma}$, the over-large variance will not back-propagation gradient information to update the network parameter. Extensive experiments on white- and black-box attacks confirm that the proposed key point is important enough.

As illustrated in Figure 1, we take the last three-layer as a detailed description. The data passes through a neural network to get $h_{l}$, then $h_{l}$ pass through a layer which is usually a linear layer and gets the feature extractor $h_{l+1}$ . We do not directly build a parallel Gaussian mean and Gaussian variance layer as Yu et al. (2021) since increasing the variance may obfuscate the original feature layer. Instead, we keep the original feature plus a maximum variance Gaussian distribution, where the Gaussian mean is a zero matrix. Thus, the $h_{l+1}$ can not only maintain the original feature representation ability but also obtain uncertainty via injected noise. The idea behind this is that the original feature can preserve the data manifold well for downstream tasks such as image classification. However, it is not appropriate within the adversarial setting, as a large proportion of information is redundant and even harmful to model robustness. Thus, we need considerable uncertainty to explore a more robustness representation. What is more, we use a non-information prior to initializing the Gaussian variance. In practice, we sample the same dimension with the feature dimension from the uniform distribution. Then, we establish a Gaussian distribution $z$, and sample from the Gaussian distribution plus with the original hidden representation $h_{l+1}$. Finally, get the logit of the network.

As discussed above, we need to maximize the feature uncertainty during model training. The simplest implementation is to add the built Gaussian variance to the loss function. It needs to emphasize that we do not directly assign a parametric variance, e.g., var= 20. One reason is that this will hinder the model convergence, and it may lead to model to collapse at the training beginning. Another reason is that we do not know the deterministic uncertainty that model needs, which is relevant to model architecture and the datasets.

Instead, we initialize the Gaussian variance with the uniform distribution values from zero to one. As the network is trained, the variance increases gradually at small steps $\Delta$. The optimization process will iterate dynamically, which means that once the model converges to local minima, greater uncertainty will force it to explore a more stable state. Finally, it will adapt to a more robust state automatically.

Thus, the loss function can be formulated as

where $\mathcal{L}_{C}$ is the cross-entropy loss, and D denotes the feature dimension of the penultimate layer. While adding all the dimensions of the variance simply may meet our requirements, we take the $\ln\left(\vec{\sigma}_{i}\right)$ operation. It has two advantages. One is to facilitate the derivative calculation of the gradient. The other is to slow down the numerical change of variance. In particular, although we emphasize that the network will not collapse, the sudden and significant change in the variance value at the beginning of the network training is not conducive to the back propagation of the gradient. The $\ln\left(\vec{\sigma}_{i}\right)$ operation can be used as an active protection measure. The last item is $L_{2}$ regularization to penalty weights over-large, $\lambda_{1}$ controls the power of variance, $\lambda_{2}$ controls the power of weights penalty.

Three well-known datasets are used in our experiments: SVHN, CIFAR-10, and CIFAR-100. The SVHN dataset consists of 73K training data and 26K testing data of size 32x32x3 with ten classes. The CIFAR-10 dataset consists of 50K training data and 10K testing data of size 32x32x3 with ten classes. For the CIFAR-100 dataset, the size of training data and testing data is the same as CIFAR-10, but with one-hundred classes. The attack algorithms in our experiments contain white- and black-box attacks. For white-box attacks, we use FGSM, PGD₁₀, C&W, and stronger PGD₁₀₀ to evaluate the efficiency of the proposed MFDV-SNN. For Black-box attacks, n-Pixel attacks and more powerful Square attack are used in our experiment.

The main backbone in our experiment is ResNet-18 He et al. (2016), the attacks follows He et al. (2019); Jeddi et al. (2020); Eustratiadis et al. (2021). In Table 1 and Table 2, the attack strength $\epsilon$ of FGSM and PGD is $8/255$. For PGD attack, the steps we set $k=10$ and the $\alpha$ we set to $\epsilon/10$, which follows He et al. (2019); Eustratiadis et al. (2021). For C&W attack, the learning rate we set $\alpha=5\cdot 10^{-4}$, the number of iterations $k=1000$, initial constant $c=10^{-3}$ and maximum binary steps $b_{max}=9$ same as Jeddi et al. (2020); Eustratiadis et al. (2021). For the n-Pixel attack, we set the population size $N=400$ and maximum number $k_{max}=75$ same as Jeddi et al. (2020), and we conduct a stronger 5-pixel attack which is not implemented in their setting. For Square attack, we refer to the implementation from Croce and Hein (2020). All experiments are performed on the Pytorch platform of version 1.7.0.

In this section, we list the comparative stochastic defense methods. Adv-BNN Liu et al. (2019): A combination of Bayesian neural network with adversarial training. PNI He et al. (2019): Injecting Gaussian noise to multilayers. L2P Jeddi et al. (2020): Updating PNI by learning a perturbation injection module and alternating training the noise and network module. SE-SNN Yu et al. (2021): Introducing a margin entropy loss. What is more, there are partial comparisons against WCA-Net Eustratiadis et al. (2021) and IAAT Xie et al. (2019).

For Table 2, we choose a larger dataset CIFAR-100 with one-hundred classes. The results show that the proposed MFDV-SNN still outperforms the defense methods for contrast, and the accuracy of clean data is also the highest. For the FGSM attack, compared with the best stochastic defense methods Adv-BNN and L2P, we have an improvement of about 36.3$\%$. For PGD attack, compared with the best defense method Adv-BNN, we have about 27.6 $\%$ improvement.

For Table 3, the attack algorithm follows foolbox¹¹1https://github.com/bethgelab/foolbox, a public attack library, to evaluate the efficiency of the proposed MFDV-SNN. The results show that the proposed MFDV-SNN significantly exceeds the existed stochastic defense methods even when the confidence level $k$ is 5. Overall, although the implementation of the proposed MFDV-SNN is simple, the improvements are significant.

We compare MFDV-SNN with several state-of-the-art defense methods proposed in recent years. Among them, some methods are SNNs, and some are not. We present the results in Table 6. For a fair comparison, some results are extracted from the original paper and Eustratiadis et al. (2021). All experiments are conducted on CIFAR-10 with the PGD attack. AT means to use adversarial training and it shows that many defense methods involve adversarial training, which requires a high computational cost. The results show that the proposed MFDV-SNN outperforms the listed defense methods greatly. It should be noted that even with some deeper network architectures, our proposed method achieves the highest accuracy on clean data.

Athalye Athalye et al. (2018) claimed that some stochastic algorithms are of false defense method. These methods mainly obfuscate the gradient information to improve the model’s robustness. However, methods that follow obfuscated gradient can be attacked finally. So, in this section, we conduct a series of experiments to check the list proposed by Athalye Athalye et al. (2018), the practice experiments follow Jeddi et al. (2020).
Criterion 1: One-step attack performs better than iterative attacks.
Refutation: As we know, a PGD attack is an iterative attack, and FGSM is a one-step attack. It can be seen from Table 1 and Table 2 that the accuracy of MFDV-SNN against FGSM attack is consistently higher than that of PGD attack.
Criterion 2: Black-box attacks perform better than white-box attacks.
Refutation: From Table 1 and Table 4, the PGD attack is stronger than 1- and 2-pixel attacks, we can see that the black-box attack is worse than white-box attack.
Criterion 3: Unbounded attacks do not reach 100$\%$ success.
Refutation: Following He et al. (2019), as drawn in Figure 2 (left), we run experiment by increasing the distortion bound-$\epsilon$. The results show that the unbounded attacks lead to 0$\%$ accuracy under attack.
Criterion 4: Random sampling finds adversarial examples.
Refutation: Following He et al. (2019), the prerequisite is that the gradient-based (e.g., PGD and FGSM) attack cannot find adversarial examples. However, Figure 2 (right) indicates that when we increase the distortion bound, our method can still be broken.
Criterion 5: Increasing the distortion bound does not increase the success rate.
Refutation: The experiment in Figure 2 (left) shows that increasing the distortion bound increases the attack success rate.
EOT-attack: Athalye Athalye et al. (2018) further claimed that the false gradient could not protect the model well when the attackers use the gradient, which is the expectation over a series of transformations. Following Jeddi et al. (2020); Pinot et al. (2019), we use a Monto Carlo method which expects the gradient over 80 simulations of different transformations on the CIFAR-10 dataset with the backbone ResNet-18. Results show that PNI, Adv-BNN, L2P can provide 48.7$\%$, 51.2$\%$, and 53.3$\%$ robustness, respectively. The proposed MFDV-SNN can have 79.2$\%$ robustness, which outperforms the compared methods and confirms that the proposed MFDV is not of gradient obfuscation. To ensure the efficiency of the proposed MFDV-SNN result is not a stochastic gradient, we adopt 15 MC sampling at the test phase in our experiments, which number we find in experiment is stable enough.

In this section, many experiments are conducted to evaluate the generalization of the proposed model. More specifically, we first explore the dataset size influence on the proposed MFDV-SNN, as shown in Table 7. Three different sizes are used in this experiment. SVHN: a relatively small dataset. CIFAR-10: a medium dataset with 60k training data and 10k testing data. CIFAR-100: a large dataset with one-hundred classes. The experiments are based on the backbone ResNet-18. The results show that MFDV-SNN has a great generalization to different dataset sizes.

For Table 8 and Table 9, we mainly explore the impact of the network architecture on the proposed MFDV-SNN. More specifically, in practice, we explore how the depth and breadth of the network affect our method. ResNet-18: a common used backbone in stochastic defense. ResNet-50: a deeper ResNet architecture than ResNet-18. WRN-34-10: a wider network architecture than ResNet-18 and ResNet-50. To show the efficiency of the proposed MFDV-SNN more abundantly, we used both CIFAR-10 and CIFAR-100 datasets in our experiments. The results show that our proposed MFDV-SNN always maintains superior performance.

As shown in Figure 3, we conduct experiments to check the efficiency of the proposed module on the FGSM (left) attack and PGD (right) attack. Note that the hyper-parameter $\lambda_{1}$ controls the power of the proposed MFDV-SNN. No defense means that we do not add randomness to the model. We train with ResNet-18 model on the CIFAR-10 dataset. Gaussian distribution means implementing the feature layer to a Gaussian distribution without regularization. Gaussian distribution with MFDV indicates that we add the critical point max feature distribution variance loss (MFDV-SNN) to the cross-entropy loss. Ablation study of MFDV-SNN shows that the proposed regularization module is effective to improve model adversarial robustness.

We visualize the classification result on the CIFAR-10 dataset trained on ResNet-18, as shown in Figure 4. In practice, we sample 1000 data and visualize them. The visualization results show that the proposed MFDV-SNN learns a more robust architecture that achieves intra-class compactness and performs better even in inter-class separation. It is proved that the proposed method, with unbounded high variance, can maintain high uncertainty and adaptively learn a deeper network representation. The uncertainty will also help the network avoid local optima and explore the global optima, thereby improving the robustness of the model and even the classification ability.