CAP-GAN: Towards Adversarial Robustness with Cycle-consistent Attentional Purification

Adversarial examples can be crafted by adjusting loss values of given images. After finding adversarial examples by [5], many researchers have developed various algorithms for crafting adversarial examples [4, 6, 19, 20]. Fast Sign Gradient Method (FGSM) [4], and Projected Gradient Descent (PGD) algorithm [6] are typically used for evaluating the robustness of the model. Both algorithms use the gradient of the loss function with respect to the input images using the following formula.

Here $\alpha$ is a hyper-parameter, and $\Pi$ is a projection function for adjusting perturbation size. In this formula, it is FGSM algorithm if $i$ is 1, and PGD uses more than 1. Carlini and Wagner (CW) attack approaches this problem as a constrained optimization problem and finds adversarial examples with the Lagrangian method, creating a more accurate and powerful adversarial examples [19]. Basically, these algorithms require the gradient of each objective function with respect to the inputs. Besides gradient-based attacks, gradient-free attacks such as SPSA [20], Square[21] and gradient-approximation attacks called Backward Pass Differentiable Approximation (BPDA) [22], are proposed. Despite the fact that these methods do not rely on the local gradient, many defenses are circumvented. These methods could be more potent than gradient-based attacks in terms of the applicability.

Adversarial training Adversarial training [4] is training a model with specific adversarial examples. The model assumes the potential adversary and minimizes its loss concerning adversarial examples. Among those adversaries, PGD adversary has been widely chosen, and PGD-trained model shows outstanding robustness against various attack strategies advocated by [6]. Inspired by the success of PGD-trained model, TRADE [23] builds a more robust model by modifying the underlying loss function. Even though adversarial training achieves the decent robustness against various attack strategies, it sometimes requires tremendous training time to converge and may hurt generalization ability harshly. It may hinder its applicability in real-world applications.

Pre-processing Pre-processing or input transformation is transforming input images for the denoising purpose. Some papers [8, 9] focus on the high-level representation to minimize the distance between adversarial and clean examples. [7] borrows the idea of a statistical generative model, and the model reconstructs images to be clean-like images using pixelCNN architecture. [10] applies basic image transformations to remove the underlying adversarial perturbations. [13] and [11] use GAN architecture to purify the adversarial perturbations by projecting inputs into the legitimate distribution. All of these methods insist that their methods are robust against various adversarial examples. However, those pre-possessing methods are still vulnerable when the adversary can explore the defended model, called the white-box scenario. Even if the adversary does not know the defense methods, [22] shows that it is still vulnerable to the gradient-approximation attack. After the gradient-free attacks were introduced by [22] and [20], pre-processing has known to be relatively vulnerable against adversarial examples than adversarial training. Nevertheless, it is robust against various transfer-based attacks and achieves decent generalization ability.

Since our training method stems from CycleGAN[16], we have to construct two different domains: clean data domain and corresponding adversarial data domain. To be specific, we denote original training samples as clean since it was not affected by any attacks. Let $\mathcal{C}$ and $\mathcal{A}$ denote the domain with clean and adversarial data, respectively. A target classifier $f$, which is trained with clean data $x$ from $\mathcal{C}$, outputs a classification score over $K$, where $K$ is the number of classes in $\mathcal{C}$. Adversarial domain $\mathcal{A}$ consists of adversarial data $x_{adv}$ with small perturbation yet effective enough to deceive the target classifier $f$. An adversary generates adversarial perturbations $\delta$ against the inputs $x\in[0,1]^{H\times W\times C}$ and combines the perturbations with the inputs to construct the adversarial inputs $x_{adv}=x+\delta$ s.t., $f(x)\neq f(x_{adv})$. An auxiliary classifier $\eta$ computes the probability to decide where the given inputs come from. For the purification, a generator $\mathcal{T}$ and a discriminator $\mathcal{D}$ learn the image-to-image translation between $\mathcal{C}$ and $\mathcal{A}$ so that achieving $x^{\prime}\approx x$ as well as $f(x^{\prime})\approx f(x)$, where $x^{\prime}=\mathcal{T}(x_{adv})$. Note that we have two kinds of generator $\mathcal{T}$: $\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}$ and $\mathcal{T}_{\mathcal{C}\rightarrow\mathcal{A}}$. $\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}$ learns to map the inputs from $\mathcal{A}$ to $\mathcal{C}$, and $\mathcal{T}_{\mathcal{C}\rightarrow\mathcal{A}}$ is vice versa.

The main idea of our approach is initially originated from the image-to-image translation task[16, 24], where the main goal is training a model to map samples across two different domains correctly. We apply this concept to mitigate the risk of adversarial attacks. As we aim to train $\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}$, we describe the architecture from the perspective of $\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}$. $\mathcal{T}_{\mathcal{C}\rightarrow\mathcal{A}}$ works in the same manner and is used for cycle-consistent learning to enhance $\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}$. The architecture is presented in Figure 2.

Generator We use the architecture introduced in [17, 16] and modify it for our task. Basically, the generator $\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}$ consists of mainly three parts: an encoder $\mathrm{E}_{\mathcal{T}_{\mathcal{A}}}$, an auxiliary classifier $\eta_{\mathcal{T}_{\mathcal{A}}}$, and a decoder $\mathcal{G}_{\mathcal{C}}$. The encoder $\mathrm{E}_{\mathcal{T}_{\mathcal{A}}}$ generates encoded feature maps from a batch of adversarial images $x^{B}_{adv}$, where $B$ is the size of batch. The global average pooling layer $\psi$ is then applied to generate spatial averages of feature maps from $\mathrm{E}_{\mathcal{T}_{\mathcal{A}}}$, to use them as the inputs of the fully connected layer. Through the auxiliary classifier $\eta_{\mathcal{T}_{\mathcal{A}}}$, we can compute the probability to determine whether given images (e.g., $x_{adv}$) belong to a specific domain (e.g., $\mathcal{A}$). As what follows, we leverage weights $w_{\mathcal{T}_{\mathcal{A}}}$, which represent the importance with respect to $x^{B}_{adv}$ computed by $\eta_{\mathcal{T}_{\mathcal{A}}}$, to make the attention map $M^{B}_{\mathcal{T}_{\mathcal{A}}}$ for the decoder $\mathcal{G}_{\mathcal{C}}$, i.e., $M^{B}_{\mathcal{T}_{\mathcal{A}}}=\sum^{n}_{k=1}w^{k}_{\mathcal{T}_{\mathcal{A}}}\cdot\psi(E^{k}_{\mathcal{T}_{\mathcal{A}}}(x^{B}_{adv}))$, where $n$ is the number of encoded feature maps. Considering $\eta_{\mathcal{T}_{\mathcal{A}}}$ learns domain-specific features, the activated map $M^{B}_{\mathcal{T}_{\mathcal{A}}}$ can guide the decoder $\mathcal{G}_{\mathcal{C}}$ to focus on more meaningful regions, leading to encouraging valid source-to-target mapping. We will describe how $\eta$ works internally in Section III-C. The decoder $\mathcal{G}_{\mathcal{C}}$ considers $M^{B}_{\mathcal{T}_{\mathcal{A}}}$ as the inputs and reconstructs the purified inputs $x^{B}_{\mathcal{A}2\mathcal{C}}$ for the target domain $\mathcal{C}$. Overall, the generator $\mathcal{T}$ takes $x^{B}_{adv}$ and $x^{B}$ to purify or perturb given images in the following manner:

Apart from $x^{B}_{adv}$ and $x^{B}$, the generator $\mathcal{T}$ takes translated images as inputs since we train the model under cycle-consistent learning.

Discriminator $\mathcal{D}_{\mathcal{C}}$ consists of three parts: an encoder $E_{\mathcal{D}_{\mathcal{C}}}$, an auxiliary classifier $\eta_{\mathcal{D}_{\mathcal{C}}}$ and a convolutional classifier. The discriminator $\mathcal{D}_{\mathcal{C}}$ is responsible for deciding whether given images are valid or not using the auxiliary classifier $\eta_{\mathcal{D}_{\mathcal{C}}}$ and the convolutional classifier. Both classifiers update their loss toward distinguishing whether the given images are original or generated. While $\eta_{\mathcal{D}_{\mathcal{C}}}$ takes inputs as encoded feature maps generated from $E_{\mathcal{D}_{\mathcal{C}}}$, the convolutional classifier takes inputs as attention maps $M_{\mathcal{D}_{\mathcal{C}}}$ activated by the weights of $\eta_{\mathcal{D}_{\mathcal{C}}}$. $\eta_{\mathcal{D}_{\mathcal{C}}}$ will classify $x$ as the images belonging to $\mathcal{C}$ based on the probability, whereas $x_{\mathcal{A}2\mathcal{C}}$ is considered as the images not belonging to $\mathcal{C}$. $\eta_{\mathcal{D}_{\mathcal{C}}}$ has learned the domain-specific features, so activated regions on $M_{\mathcal{D}_{\mathcal{C}}}$ can be viewed as guided regions for improving the generator $\mathcal{T}$. We elaborate on how to achieve the mapping between two different domains in the following section.

We introduce the objective function of CAP-GAN in detail. As we described before, our loss functions are designed with leveraging both pixel-level and feature-level approach. We use CycleGAN training procedure for pixel-level consistency ($L_{GAN},L_{identity}$, and $L_{cyc}$). For feature-level consistency, we use $L_{cam}$, which encourages the model to learn domain-specific features, and $L_{sem}$, which guides the generator to learn the semantic information by distilling the knowledge from the pre-trained model $f$.

We train the proposed model on CIFAR-10 dataset with a single RTX TITAN 2080. We use a simple ResNet model as a victim model following [6]. We train the model up to 200 epochs using Adam optimizer ($\beta$ = (0.5,0.999)) and set batch size as 128. We use the initial learning rate 1e-4 and adopt cosine annealing learning rate decay for a stable training. To construct the adversarial domain $\mathcal{A}$, we use FGSM ($\epsilon=8$) attack [4]. We set the $\alpha=0.7$ and $T=10.0$ to train the model. For a fair comparison, we implement all the defenses using the same codebase.

A black-box attack is an attack where an adversary has limited knowledge of the defense. This section assumes that a black-box adversary is not aware of the defense but has the knowledge of the target model $f$. We evaluate the robustness of our defense against various attack strategies. We follow the black-box assumption introduced in [28] and use the same attack settings with [6]. Specifically, we provide the architecture of the target model $f$ to the adversary and train it with the same training data. With fully trained surrogate model $f^{\prime}$, the adversary then generates transferable adversarial examples using gradient-based attacks such as FGSM, PGD_n and CW($\kappa=20$) strategy, where $n$ is the number of step, and $\kappa$ is a confidence. Plus, to endanger the defense under a more harsh transfer-attack, we apply the momentum iterative FGSM, called MI-FGSM [29]. We use 20 iterative steps for MI-FGSM.

As shown in Table I, CAP-GAN shows promising results against transfer-based attacks and outperforms other pre-processing based methods, including the adversarial training(AT) model. We trained AT model using PGD₇ adversary following [6]. Interestingly, FGSM works better than its iterative attack PGD_n in APE, HGD, and CAP-GAN. Such an observation is on par with the tendency, where the gradient masking defense makes the iterative attacks getting stuck in a local minimum [22]. THM [30] achieves superb results in the first-step attack, such as FGSM attack, whereas it is relatively vulnerable to the iterative attacks, such as MI-FGSM and PGD. Especially, HGD obtains similar performance with CAP-GAN.

Even though we use the surrogate model $f^{\prime}$ with the same training settings, it cannot ensure that $x_{adv}$ generated from $f^{\prime}$ has transferability to $f$ [20]. We thus measured the attack success rate, which is referred to as the rate of misclassification against the target model $f$ when the attack strategy is applied, to judge whether the attack has enough capacity to deceive the model. We observed that generating the adversarial examples with $\epsilon=8$ is insufficient to assume the strong adversary. We conjectured that the surrogate model $f^{\prime}$ might not have a similar gradient direction of $f$, implying that crafting the adversarial examples with the gradient direction of $f^{\prime}$ requires a large magnitude of perturbation to achieve a high attack success rate [28]. Therefore, we decided to evaluate the defenses under various magnitudes of the perturbation $\epsilon=[4,8,12,16,32]$ to argue more clear robustness against various attack strategies. The results are presented in Figure 5.

Providing $\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}$ for the adversary can cast the light for understanding effectiveness of the proposed defense. Besides gradient-based attacks, we also apply the score-based attack, SPSA [20] and Square attack [21] to launch the strong black-box attack. In SPSA and Square attack, the adversary is allowed to directly access the outputs of the defended model $f(\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}(\cdot))$. Since those gradient-free attacks do not rely on the local gradient information, we can provide a different aspect of the robustness. We set batch size as 2048 for SPSA to make it more potent. We mount Square attack varying the restart by 1 and 5.

Apart from gradient-free attacks, we also consider an adaptive adversary who exploits the fundamental weakness of the proposed model. We assume the adaptive adversary as BPDA-I attack, where the adversary crafts the corresponding adversarial examples using PGD_n but backpropagating the gradients with the identity mapping. According to the experimental results from [22], many pre-processing based methods [7, 10, 30] are circumvented under BPDA-I attack. Note that BPDA attack is typically applied when the defense is non-differentiable, but we utilize this attack scheme to circumvent our defense as our model is designed to fulfill $\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}(x_{adv})\approx x$ and $\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}(x)\approx x$ simultaneously. As advocated by [22], BPDA attack can be applied to verify the vulnerability of the model even if the model is differentiable. We thus assume that the strong black-box adversary exploits the internal weakness $\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}(x)\approx x$ by replacing the backward pass with the identity function to approximate the gradients over the inputs i.e., leveraging the concept that $\nabla_{x}\mathcal{T}(x)\approx\nabla_{x}x=1$, where $x\in\{{\mathcal{C},\mathcal{A}}\}$. The experimental results are presented in Table II. Our proposed model outperforms other methods across all attacks.

A white-box adversary has full knowledge of the target model and defended model to launch adversarial attacks. We consider two kinds of white-box settings: conventional white-box setting following [8, 13] and [7], where we provide the target model $f$ to the adversary, and end-to-end attack, where we allow the adversary to directly compute the gradient with respect to the combined model $f(\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}(\cdot))$ following [32] and [22].

White-box Attack In Table III, the target model reaches a nearly zero accuracy when we apply the white-box iterative attacks even with $\epsilon=8$. The accuracy of most methods significantly dropped compared to the black-box setting in Table I, whereas HGD and CAP-GAN show relatively consistent results. The intuition behind both models is using the feature-level consistency when optimizing the denoising loss function. However, models that try to align only pixel-level distribution cannot achieve the consistent robustness. This observation is on par with our assumption described in Section I, where the pixel-level consistency cannot assure the feature-level consistency. As observed by [8, 4], the residual perturbations continuously incur the error amplification effect in several intermediate layers, resulting in the misclassification at the prediction layer. The feature-level consistency additionally aligns the shift caused by the residual perturbations at the prediction layer, so it could help to achieve reasonable purification and leads to consistent performance.

End-to-End Attack Since APE and HGD are differentiable, we pick those methods for the end-to-end attack to compare the robustness. Table IV shows the robustness under the end-to-end white-box setting. As expected, gradient-based attack is sufficient to circumvent APE and HGD. In contrast, adversarial training(AT) shows relatively consistent performance across all attack strategies. Overall, CAP-GAN surpasses the other differentiable pre-processing defenses, but is relatively less robust than AT in the end-to-end attack setting.

Parameter Settings We perform a simple ablation study to explain how each parameter affects the performance. CAP-GAN has two parameters: $\alpha$ and $T$. $\alpha$ adjusts the contribution between pixel-level and feature-level consistency. In $L_{sem}$, $T$ is a scaling parameter to smoothen the logits before feeding them into the softmax function. If we use a high value of $T$, the logit space would be smoother, leading to generating soft-target scores over the inputs. According to Table V, we experimentally found that CAP-GAN obtains the best result with $\alpha=0.7$ and $T=10.0$. One can note that the accuracy under BPDA-I attack dropped as $\alpha$ increases, implying that $\nabla_{x}\mathcal{T}(x)\approx\nabla_{x}x$ is closer to 1. This observation can provide an evidence for our assumption in which BPDA-I attack could be an adaptive adversary against CAP-GAN.

Loss Functions We introduce how each loss function contributes internally. Besides the pixel-level alignment with $L_{pixel}$, we explicitly add the feature-level terms $L_{feature}=L_{cam}+L_{sem}$ to tune the generator. $L_{cam}$ helps the model to understand domain-specific features, and $L_{sem}$ conveys distilled knowledge from the pre-trained model to guide $\mathcal{T}$. Furthermore, we adopt CycleGAN training scheme for enhancing the expected result. As shown in Table VI, the feature-level terms lead to the improvement in the adversarial robustness across all attacks. We initially conjecture that $L_{cam}$ enhances the purification by encouraging the valid source to target mapping, but rather degrades the performance. When we combine the model with the semantic term $L_{sem}$, however, the model significantly improves its capacity. We also found that simply combining the two loss terms would not derive the positive effects. At this point, cycle-consistent learning is an important factor in deriving the expected improvement. This is because $\mathcal{T}_{\mathcal{A}\rightarrow\mathcal{C}}$ takes the inputs from not only domain $\mathcal{A}$ but also $\mathcal{T}_{\mathcal{C}\rightarrow\mathcal{A}}(x)=x_{\mathcal{C}2\mathcal{A}}$ to fulfill cycle-consistency. Cycle-consistent learning can regularize the entire training with more diverse inputs. Thus, additional inputs convey the meaningful hidden information to the model, encouraging the model to understand the underlying relationship between two different domains.