Generating Adversarial Attacks in the Latent Space

In [19], the author analyzes convex hulls of the features produced by a deep classification network $f$, to demonstrate generalizability of deep learning-based methods. Inspired from the above work, we explore the effects of adversarial attacks in the latent (feature) space than in the input (pixel) space. Specifically, we develop a framework that is capable of perturbing the features of the original data so that it changes the class label without altering the visual semantics of the data. Therefore, we use a generative adversarial network (GAN) that is capable of synthesizing realistic images using an encoder-decoder architecture (generator) and incorporates a classifier (discriminator) to distinguish between original and perturbed samples. In our case, the generator simulates an autoencoder [20], to ensure that the original and perturbed images are almost identical. The discriminator serves the dual purpose of distinguishing between ‘real’ (original) and ‘generated’ (adversarially perturbed) samples, and their respective class labels. Therefore, the discriminator guides the generator to synthesize samples that are successfully perturbed, i.e., they must belong to different classes. The proposed method is different from Latent Poison attack [21], that explores the vulnerability of variational autoencoders from a security perspective and requires $\epsilon$-bound on the class decision. An extension of the Latent Poison work in achieving only untargeted attacks by generating out-of-distribution samples is done in [22]. However, we propose to perturb the features in the latent space for generating realistic images while attacking the class predictions (without noise margin) in an end-to-end fashion to accomplish both untargeted and targeted attacks.

We use a GAN [12] to produce images that fools a classifier. The GAN consists of an encoder-decoder architecture as the generator that generates adversarial samples, and a discriminator that distinguishes between original and adversarial samples. We denote the generator and discriminator network by $\mathcal{G}$ and $\mathcal{D}$ respectively. Let $\textbf{x}=\{x^{(i)}\}_{i=1}^{m}$ denote a batch of $m$ samples, $\{y^{(i)}\}_{i=1}^{m}$ denote the corresponding labels, and $\tilde{\textbf{x}}=\mathcal{G}(\textbf{x})$, denote a set of generated images. During training, weights of both networks are optimized to minimize their respective loss functions. The discriminator’s loss $\mathcal{L}_{\mathcal{D}}$ is formulated as

and the generator’s loss $\mathcal{L}_{\mathcal{G}}$ is formulated as

In order to inject adversarial perturbations in the latent space, we strategically employ the generator and discriminator loss functions to supervise the adversarial sample generation. Note that classification occurs as a result of highly non-linear operations in the latent space. Therefore, we introduce loss terms that deliberately affect the classification decision by implicitly perturbing the features that are responsible for classification. We design the loss functions such that they regulate realistic data generation and accomplish adversarial attacks synchronously.

Equation 1 and Equation 2 achieve the desired objective as follows. In the equations, $\tau$ denotes the target class in the case of targeted attack, $\mathcal{L}_{1}$ is per-pixel loss and $\mathcal{L}_{ce}$ is the cross-entropy loss. In case of untargeted attack, we replace the target class $\tau$ by $c^{*}-y^{(i)}$ where $y^{(i)}$ is the label of sample $x^{(i)}$ and $c^{*}=\max_{i}(c_{i})$. This essentially forces the discriminator to misclassify a sample of class $c$ as $c^{*}-c$. When the number of classes in the dataset is even and class labels are 0-indexed, labeling class $c$ as $c^{*}-c$ mislabels all the training samples.

Given a target class $\tau$, the first term in $\mathcal{L}_{\mathcal{D}}$ penalizes misclassification of a sample $x^{(i)}$ to its class $y^{(i)}$ whereas the second term encourages $\mathcal{D}$ to classify the generated sample $\tilde{x}^{(i)}=\mathcal{G}(x^{(i)})$ as $\tau$. Similarly, the first term in $\mathcal{L}_{\mathcal{G}}$ encourages the generated images to be classified as $\tau$ whereas the second term forces the generated images to look visually similar to real images. Successful training results in a generator network that produces realistic images that appear to be sampled from original dataset, and a discriminator network that correctly classifies real images but classifies the generated images as belonging to $\tau$. Algorithm 1 outlines the procedure followed in the proposed method.

Baseline: We design a baseline for comparison as existing methods perturb in the pixel space. It consists of an autoencoder that is jointly trained with a fully connected layer to classify the embedding produced by the encoder. Let $h$ and $g$ be the encoder and decoder respectively, and $f^{\prime}$ be the fully connected layer. The model minimizes the following loss.

The first term in $\mathcal{L}_{\mathcal{B}}$ forces the model to produce images that look similar to the input whereas the second term supervises correct classification of the embedding produced by $h$. Given a sample $x$ belonging to class $c$, we first construct the convex hull using embedding produced by $h$ of all the training samples from class $c$. We then iteratively move towards the direction of the nearest face in the hull for a fixed number of iterations or until the class label, i.e., the classification decision changes.

Proposed Method Results: We present the results of our proposed method on MNIST, CIFAR10, Fashion-MNIST, CIFAR100 and Stanford Dogs datasets in Figure 1. On MNIST (see Table 2), we observe classification accuracy of 88.8% on original images, 10.5% on perturbed images and secure an attack success rate of 92.86% when averaged across all the classes for targeted attacks. We observe classification accuracy of 86.18% on original images, 9.1% on perturbed images and secure an attack success rate of 90.83% when averaged across all the classes for untargeted attacks. On CIFAR10 (see Table 3), we observe classification accuracy of 71.7% on original images, 10.01% on perturbed images and secure an attack success rate of 85.74% when averaged across all the classes for targeted attacks. We observe classification accuracy of 80.20% on original images, 12.4% on perturbed images and secure an attack success rate of 87.51% when averaged across all the classes for untargeted attacks. We also test our method on Fashion-MNIST, CIFAR100 and Stanford Dogs datasets on targeted attacks to validate the generalizability of the proposed method. We randomly select 10 classes out of a total of 100 classes from the CIFAR100 and out of 120 classes from the Stanford Dogs datasets, respectively. We report the results in Table 4. We observe that the proposed method achieves an ASR of 93.66% on the Fashion-MNIST dataset, an ASR of 93.78% on the CIFAR100 dataset and an ASR of 98.94% on the Stanford Dogs dataset. We suspect that a low classification accurccy on original images from the Stanford Dogs dataset may be responsible for surprisingly high value of ASR.

To quantify the preservation of visual realism between original and generated images, we use two measures: 1) Structural Similarity Index Measure (SSIM) 2) Peak Signal-to-Noise Ratio (PSNR). We use the MNIST dataset for this purpose. We report SSIM = 0.85 for untargeted attacks and SSIM = 0.78 for targeted attacks. We observe PSNR = 21.7 for untargeted attacks and PSNR = 17.1 for targeted attacks.

Baseline Results: Figure 2 depicts perturbation in the latent space to modify the class label of the reconstructed image changes without affecting its perceptual quality using the autoencoder. The objective is to move towards a different class than the original class in the case of untargeted attacks. In the case of targeted attacks, we migrate towards the centroid of the convex hull of the target class. We offer a geometrical interpretation of how the baseline routine works in Table 1. We demonstrate how the addition of adversarial perturbation in the latent space pushes the data point from the convex hull spanned by the training data points belonging to the ‘original’ class towards the hull of the ‘target’ class resulting in an adversarial attack. Given a test sample $x$ of class $c$, lying in the hull constructed by features of training samples in that class, we add iterative perturbations to $f(x)$ towards the direction of the nearest face in the hull until the classification of the reconstructed image changes. In the case of targeted attack, we move towards the centroid of the target class’s hull. On MNIST (see Table 2), we observe classification accuracy of 98.88% on original images, 9.76% on perturbed images and secure an attack success rate of 32.06% when averaged across all the classes for targeted attacks. We observe classification accuracy of 50.60% on perturbed images and secure an attack success rate of 49.40% when averaged across all the classes for untargeted attacks. On CIFAR10 (see Table 3), we observe classification accuracy of 93.66% on original images, 10.20% on perturbed images and secure an attack success rate of 15.64% when averaged across all the classes for targeted attacks. We observe classification accuracy of 0.83% on perturbed images and secure an attack success rate of 91.60% when averaged across all the classes for untargeted attacks. On CIFAR10, the baseline achieves a high ASR due to poor quality of generated images. We computed the structural similarity index measure (SSIM) between the original and generated images (baseline) and observed it to be very low $\sim$1%. Due to the low quality of generated images, the classifier misclassifies the generated data to any arbitrary class leading to a spurious increase in the ASR. In all the remaining cases, the proposed method outperforms the baseline by a significant margin.

We further compare with two gradient-based attacks in the pixel space, FGSM and PGD (both depend on noise margin $\epsilon$) [34] in Table 5 in terms of classification accuracy, and observe that the proposed method is agnostic to noise $\epsilon$ and outperforms FGSM but is comparable to PGD attacks in the pixel space on adversarially generated images. Additionally, we compare with ATGAN (attack without target model using GAN) [18] and observe that ATGAN achieves the maximum attack success rate (ASR) of 81.78% on the MNIST dataset (proposed method achieves 90.83% ASR), and an ASR of 87.99% on the CIFAR10 dataset (proposed method achieves 87.51%). We also compare the $L_{2}$ norm of perturbation added by the proposed method. On MNIST, we observe the average $L_{2}$ norm of noise in pixel space to be $0.079$ and $2.559$ in $\mathcal{D}^{\prime}s$ latent space. On CIFAR 10, these values are $0.016$ and $0.519$, respectively.

We examine how the class activations change after addition of adversarial perturbations using the proposed method. Although the original and perturbed images look visually alike, the class activation maps reveal complementary salient regions as shown in Figure 3 on four different test images belonging to class label ‘0’ on the CIFAR10 dataset. It is evident that the activations are sufficiently well-separated between the original images and the outputs generated by the proposed method.

We further visualize the embedding (2-D projection using t-SNE) of original and generated images, extracted from the penultimate layer of $\mathcal{D}$. We select $\mathcal{D}$ over $\mathcal{G}$ as the embedding extractor because the images reconstructed from embedding produced by $\mathcal{G}^{\prime}s$ encoder are already classified as the target class in a majority of cases. We present our analysis on nine different test images from the MNIST dataset originally belonging to class ‘4’ and adversarially perturbed to target class ‘9’ using the proposed method. We deliberately select these two classes as they are inherently difficult to distinguish compared to the rest of the digits. We observe in Figure 4 that the embedding of original and adversarial images form compact and disjoint clusters, indicating the proposed method successfully launched the attack in the latent space. Moreover, the embedding of generated images are aligned in a specific orientation with respect to the cluster of embedding from original images in a majority of cases, revealing a visually intuitive pattern in the trajectory of adversarial perturbations.