Defense Through Diverse Directions

In the context of supervised deep learning, a conventional neural network seeks to perform some variant of a classical functional estimation task, to learn a point estimate of the optimal function in the chosen functional space that maps each input from the input space to its corresponding output in the output space. However, such an estimate does not consider, and thus cannot effectively adapt to, the inherent uncertainty throughout the training procedure (e.g., data collection, random initialization of network weights). Such deficiency leads to problems including over-fitting and overly confident predictions.

To remedy this deficiency, a Bayesian neural network (BNN), introduced in the same vein as continuous stochastic processes such as the Gaussian process, seeks to directly model the distribution, whose density we denote as $p$, over random functions

where $\mathbb{X}$ and $\mathbb{O}$ denote the input and the output spaces, respectively. However, directly learning such a distribution can be arduous as functional spaces are usually infinite-dimensional. Utilizing the fact that neural networks can be regarded as universal approximators for functions (Hornik et al., 1989), a distribution over random functions can be thought of as a choice over of neural networks. We materialize such connection through learning a distribution over the network weights. More specifically, in assuming that the network weights are random and distributed according to a prior distribution $P(\textbf{w})$, a BNN seeks to learn the posterior distribution of the network weights, w, given the available data, i.e. $P(\textbf{w}|\textit{D})$.

While a clever idea, learning $P(\textbf{w}|\textit{D})$ is prohibitively expensive even for moderately sized networks because of the high-dimensional integral that needs to be evaluated. Borrowing ideas from variational inference and the recent success of unsupervised methods like the variational auto-encoder (Kingma & Welling, 2014), Blundell et al. propose to learn a variational posterior distribution, $q(\textbf{w}|\theta)$, to approximate the true posterior by optimizing the following objective

which is the evidence lower bound (ELBO) for the data likelihood. The expectation term ensures the learnt variational posterior distribution is informed by the data, and the KL divergence acts as a regularizer over the weights. Although technically any choice of the variational posterior and prior distributions pair is possible, the convention, which we adopt in this work, is to choose both to be independent Gaussians where we learn the mean and variances of the variational posterior distribution. One benefit of deploying a BNN, which we exploit in our proposed framework, is that for one input one can draw multiple functions $f$, which in practice is actualized by drawing a set of different weights from the posterior distribution $P(\textbf{w}|\textit{D})$, to form an ensemble of inferences for various purposes, such as assessing the uncertainties in predictions, gradient evaluations, etc. In this work, we exploit the network’s variation to control how much sensitivity we expect from each input element.

Adversarial examples are constructed by making small perturbations to the input that induce a dramatic change in the output. Attacks are typically broken down into two categories: white and black box attacks. The exact definition of both methods vary, but we will use the following definition in this work. In the white box setting, the attack has access to the training data set, the fully specified underlying model, and the loss functions. Attacks are found by performing gradient ascent with respect to the input. In the black box setting, the attacker has access to the training data and the loss functions but does not have access to the underlying model parameters. A black box attack can then be constructed by using a stand-in network trained on the same data with the same loss. Previous works have shown that these examples are still effective against a variety of other models (Liu et al., 2016; Tramèr et al., 2017b).

The attacker’s goal is:

where $\mathbf{\varepsilon}$ is the attack perturbation, $p$ is the norm (typically taken to be $\infty$), $\varepsilon_{\textrm{max}}$ is the attack budget (the maximum perturbation), $\mathcal{L}$ is the loss, $f$ is the network, $\mathbf{x}$ is the input, $\mathbf{\theta}$ is the network parameters, and $\mathbf{y}$ is the truth.

Typically the attack methods generate the adversarial examples by leveraging the gradient of the loss function with respect to the inputs. The Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2014), for instance, takes one step along the gradient direction to perturb an input by the amount $\varepsilon$:

Projected Gradient Descent (PGD) (Madry et al., 2017) generalizes FGSM by taking multiple gradient updates:

where $\alpha$ is the step size. After each update, PGD projects the perturbed inputs back into the $\varepsilon$-ball of the normal inputs.

There are also other types of attacks, such as C&W attack (Carlini & Wagner, 2017), Jacobian Saliency Map Attack (JSMA) (Papernot et al., 2016) and DeepFool (Moosavi-Dezfooli et al., 2016). Among all the attack methods, PGD is regarded as the strongest attack in terms of the $L_{\infty}$ norm.

The goal of adversarial defense is to render all (bounded) perturbations ineffective against a model. It is necessary to bound the perturbation or the attack could simply replace an input with an example from a different distribution or class. A simple intuition to achieve this would be to require that the model be Lipschitz smooth such that

Unfortunately, estimating the Lipschitz coefficient $C$ for an arbitrary network can be extremely difficult, making optimizing over it nontrivial. Cissé et al. has attempted to control the Lipschitz coefficient of each layer in the network and, therefore, the network as a whole.

Recent works have introduced a number of defense methods, such as distillation (Papernot et al., 2016), label smoothing (Hazan et al., 2016), input denoising (Song et al., 2017), feature denoising (Xie et al., 2019), gradient regularization (Ross & Doshi-Velez, 2018), and preprocessing based approaches (Das et al., 2017; Guo et al., 2017; Buckman et al., 2018). Most of these defenses have unfortunately been defeated by subsequent attacks.

The most popular adversarial defense technique incorporates adversarial examples in the training process (Madry et al., 2017). Online adversarial training requires generating new examples throughout the training process to map out the local region around known examples and require that the region map to the expected output. Unfortunately, while this approach does produce a robust defense, it is computationally expensive.

Most similar to our work, ADV-BNN (Liu et al., 2018b) attempts to use BNN to combat adversarial attack. Their proposed method is dependent on online adversarial training and aims to incorporate it into the standard ELBO objective, Eq. (1), as a min-max problem. In contrast, our proposed methodology does not require online adversarial training, and achieves better performance on CIFAR-10 compared to ADV-BNN.

As motivated previously, in order to increase the network’s robustness against adversarially perturbed input, we encourage the network to maintain, and hence to evenly distribute, some expected sensitivity with respect to all input covariates. We materialize this idea by encouraging the normalized gradient in Eq. (9) to be uniformly distributed over the unit hypersphere, which in turn is equivalent to maximizing the entropy of $\textbf{u}_{y}$, denoted as $H\!\left(\textbf{u}_{y}\right)$, because $\textbf{u}_{y}$ is bounded within the unit hypersphere. However, as a function of the network weights w, the density of $\textbf{u}_{y}$ is intractable even for moderate-sized network, making maximizing $H\!\left(\textbf{u}_{y}\right)$ directly prohibitively expensive and impractical in practice.

In a simplified setting where the elements are independent, maximizing the sum of the variances of each element is equivalent to maximizing the entropy of the random vector.

Proposition 1 indicates that maximizing the entropy of X is equivalent to maximizing $\sum_{i}\text{Var}(X_{i})$. See Appendix A for the proof. While $\textbf{u}_{y}$ is not independent in our case, we use Prop. 1 as an analogy and adopt the sum of the variances of the elements of $\textbf{u}_{y}$ as a surrogate for $H\!\left(\textbf{u}_{y}\right)$.

A simple method might be to maximize the sum over inputs of the variance of $\mathbf{u}_{y}$ across the $K$ draws

However, since $\mathbf{u}_{y}$ is a unit vector, this loss degenerates and only serves to minimize the average value of the output sensitivity

Since we wish to maximize the variances w.r.t. $\mathbf{u}_{y}(\mathbf{x})$ we consider the mean penalty $\Omega_{M}(x)$:

While Equation (14) increases the total variance across inputs, it does not necessarily encourage diversity. We consider several additional penalties to include with Eq. (14) that do increase diversity.

A simple penalty to increase the minimum variance over dimensions is

Equation (15) exploits the fact that the sum (over dimensions) of the variance is fixed. Therefore, increasing the minimum necessarily decreases the other values. In theory, as the network trains, the minimum element changes and eventually all the elements converge to the same variance.

Unfortunately, since the loss only supervises one pixel at a time, the minimum shifts across a few elements and never influences the pixels with the largest variance. We consider two simple methods to correct this difficulty. One is to supervise all the pixels simultaneously by replacing the $\min$ operation with a soft-min weighted sum so that Eq. (15) becomes

where $\alpha$ corresponds to the temperature. However, since the sum of the variances is fixed and this loss attempts to increase the variance of all pixels simultaneously, we find that it can result in a counterproductive competition across pixels.

A more direct method to supervise the variance is to minimize the distance between the observed variance and the expected variance of $1/N$. Using the Euclidean distance, Eq. (15) becomes

We find that this final representation yields the best results amongst the variance encouraging losses and choose it as the variance penalty $\Omega_{V}(\mathbf{x})$.

Directly encouraging the model to match a specific distribution’s second order moment may be too strict a requirement. As an alternative to matching the second moment of the uniform hypersphere, we consider penalizing small $L_{1}$ norms of $\mathbf{u}_{y}$. By requiring that the $L_{1}$ norm be large, we bias the network away from over reliance on a few features and encourage greater diversity across the input covariates. Unlike in the variance-based losses, the network does not have to match each input direction to the same value. This allows for increased flexibility while still maintaining the same intuitive effect on the diversity of dependence. The added penalty becomes

We summarize the full possible loss of our model with respect to posterior parameters, $\mathcal{L}(\mathbf{\theta})$, with all the above penalties, a supervised loss $l$ and a batch of data $\{(x_{n},y_{n})\}_{n=1}^{N}$:

We vary Eq. (19) below, by setting various penalty weights $\lambda$ to zero.

Finally, we test the benefits of offline adversarial training (Tramèr et al., 2017a) in addition to the diversity inducing penalties. Adversarial examples are pre-computed from a trained, non-Bayesian neural network of a matching architecture and then added statically to the training set. Thus, this form of defense is more efficient than online adversarial training, which requires on-the-fly computation of adversarial examples.

We present our results by adding different combinations of the diversity-encouraging penalties in Sec. 4. To declutter the results, we use the following shorthand to refer to the different penalties we apply to the model. The unit gradient mean penalty, Eq. (14), is given as “M;” the variance penalty, Eq. (17), by “V;” the non-sparse penalty, Eq. (18), by “S;” and any offline training by “Off.” We additionally denote when a network is Bayesian by prepending the network name with a “B.” For example, the case where we use a Bayesian VGG16 with the mean and variance penalty is shorthanded as “BVGG16-M-V.”

As mentioned previously, all adversarial examples appended to the training set for offline training were constructed using conventional networks with matching architectures. Aside from the models trained with offline adversaries, we do not include any form of data augmentation.

In this section we discuss several practical steps taken to implement various networks and diversity-promoting penalties. We utilize TensorFlow (Abadi et al., 2015) and Tensorflow Probability (Dillon et al., 2017) to implement the general and probabilistic components of our models, respectively.

We test our diversity induced models on MNIST (LeCun & Cortes, 2010). For these tests we use a simple CNN with three convolutional layers followed by two fully connected layers. Since this network is fairly shallow, we compose the deterministic part of the network using the first two convolutional layers and use Bayesian layers for the final convolutional and both fully connected layers. When the corresponding penalty is used, the loss hyperparameters are: $\lambda_{M}=20$, $\lambda_{V}=40$, and $\lambda_{S}=40$.

Table 1 shows the accuracy of the model when trained using different combinations of diversity inducing penalties and adversarial training against $L_{\infty}$ attacks with a maximum attack budget of 0.3. All models obtain better than 99% standard accuracy. Black box attacks were obtained using examples from an open repository.¹¹1https://github.com/MadryLab/mnist_challenge

When only the mean penalty (Eq. (14)) is imposed, the model shows modest improvements in both forms of attack; indicating that the defense has succeeded in improving the robustness of the model against attack. While this defense has not completely succeeded in overcoming attacks, it is a positive step. As we include additional penalties, we observe that the white-box attack improves, most notably with the inclusion of the non-sparse penalty (Sec. 4.4). However, these models generally show a slight decrease in black-box performance. We infer that these penalties tend to cause the model to over fit the defense by obfuscating gradients. Fortunately, offline adversarial training appears to sufficiently augment the training set so that the defenses can generalize.

In this section, we evaluate our proposed diversity inducing penalties on the CIFAR-10 dataset (Krizhevsky et al., ). The backbone network is VGG16. A Bayesian version of VGG16 is built by replacing the last convolutional block and the fully connected layers with variational alternatives. When training with our proposed penalties, we use the hyparameters: $\lambda_{M}=10$ and $\lambda_{S}=20$. We report the $L_{\infty}$ attack with a budget of $\varepsilon=0.03$ here. PGD attack perform 40 gradient updates with a step size 0.001. Refer to Sec. 6 for ablation experiments on the attack budget. We report the black box attack accuracy using examples from an open repository.²²2https://github.com/MadryLab/cifar10_challenge

Table 2 demonstrates the accuracy of the defended models on CIFAR-10. The standard loss is included in the table since it varies across models. Given the marginal improvements in adversarial accuracy from the variance-based penalties, we forego their use in this case.

Unlike in the MNIST experiments, the mean penalty is insufficient to overcome white box attacks; however, it does offer improvements in the black box setting. Also unlike MNIST, these results do not indicate that our methods suffer from the obfuscated gradients problem as white box attacks are consistently more effective than black box attacks. The additional data augmentation from offline adversarial training further improves defense.

Table 3 juxtaposes several of our proposed defenses against results reported in Liu et al. for several attack budgets. We observe that our method without any adversarial training maintains higher standard accuracy and shows improved robustness to higher attacks budgets. Surprisingly, we observe that a Bayesian VGG16 with offline adversarial training obtains consistent accuracy above the other methods. We suspect this is because the adversarial examples used in offline training were constructed using PGD with 40 steps whereas the examples in ADV-BNN were defended using online training with 10 steps. Including our penalties consistently increases the performance of the offline model by approximately 15%.