General Adversarial Defense Against Black-box Attacks via Pixel Level and Feature Level Distribution Alignments

Motivation. The pixel-level training constraints are employed to weaken the difference in terms of pixel RGB values between clean and adversarial samples, and promote alignment in the feature space of the target model $\mathcal{O}$. Previous work validated that “distance metric between clean and adversarial samples” and “adversarial learning” [37] are two practical pixel-level training constraints. Traditional pixel-level constraints [15, 13, 30, 31, 32] mainly adopt $x^{c}$ to guide formulation of $\widehat{x}^{c}$ and $\widehat{x}^{a}$ through the generator $\mathcal{G}$, as displayed in Fig. 3(a). They compute the distance metric between $\widehat{x}^{c}$ and $x^{c}$, $\widehat{x}^{a}$ and $x^{c}$. And they set $x^{c}$ as real samples, $\widehat{x}^{c}$ and $\widehat{x}^{a}$ as fake samples to conduct adversarial learning.

Reconstruction loss. Given clean samples $x^{c}$, adversarial samples $x^{a}$ are obtained by adding adversarial perturbation $\epsilon$ to $x^{c}$, and the generator $\mathcal{G}$ synthesizes output $\widehat{x}^{c}$ and $\widehat{x}^{a}$ with input of $x^{c}$ and $x^{a}$. Based on it, we define the reconstruction loss term $\mathcal{L}_{{r}}$ as

where $\mathbb{E}$ is to compute the mean value, $\|\,\|_{1}$ is the $L_{1}$ Euclidean distance. Moreover, to help synthesize images with high resolutions, we use a perceptual loss term [39, 40] $\mathcal{L}_{{p}}$, by computing the reconstruction distance in the feature space of an ImageNet-pretrained VGG-16 network [41] for $\widehat{x}^{c}$ and $x^{c}$, as well as $\widehat{x}^{a}$ and $\widehat{x}^{c}$. Not that the perceptual loss term is not the constraint for feature-level alignment, since the VGG-16 network is not task-specific and we mainly employ the perceptual loss for the visual similarity of $\widehat{x}^{c}$ and $x^{c}$ (or $\widehat{x}^{a}$ and $\widehat{x}^{c}$) in the pixel level.

Adversarial loss. To match the global pattern for $\mathcal{C}$ and $\mathcal{A}$ at the pixel level, we implement loss terms in adversarial learning with sampling. We employ a discriminator $\mathcal{D}$, and set the loss terms in the form of LSGAN [42] as

where $\mathcal{L}_{{GAN}_{d}}$ is set for the discriminator, and $\mathcal{L}_{{GAN}_{g}}$ is adopted for the generator. Additionally, the feature match loss is adopted as an auxiliary part of the adversarial loss [43]. We obtain intermediate features from $\mathcal{D}$ for fake and real samples, and compute their distances as

where $\mathcal{F}(x)$ are intermediate features obtained from the discriminator for real or fake samples $x$.

Theoretical analysis. In the traditional setting, to weaken the discrepancy between adversarial samples with the corresponding clean samples, $\|\widehat{x}^{a}-x^{c}\|_{1}$ is set as the objective to optimize for training $\mathcal{G}$. However, $\|\widehat{x}^{a}-x^{c}\|_{1}$ is large at the beginning of training and there exist multiple solutions $\widehat{x}^{a}$ that have the same value for $\|\widehat{x}^{a}-x^{c}\|_{1}$ at each step of training. Thus, this setting is ill-posed during training and is very likely to result in local optima (i.e., $\widehat{x}^{a}$ generated from $\mathcal{G}$ does not approach $x^{c}$ enough).

Motivation. Besides the pixel-level training constraints, recent work revealed the necessity of feature-level training constraints for the target model $\mathcal{O}$ [14, 9] to enhance protection. Feature-level alignment helps formulate robust representation for networks of a given task. Existing methods formulate feature-level training constraints as the distances in feature space of $\mathcal{O}$ between synthesized and clean samples. However, they ignore the constraint for the alignment of overall $\mathcal{C}$ and $\mathcal{A}$ in feature level. We instead propose a class-aware feature-level training constraint, besides the distance metric in the feature space of $\mathcal{O}$. Our class-aware constraint aligns the integrated distribution of clean and adversarial samples within each category for the target model, minimizing the intra-class distance and maximizing the inter-class distance for adversarial samples’ distribution.

Task-oriented loss. Suppose $\mathcal{O}$ is trained with the paired data $(x^{c},y^{c})$ where $y^{c}$ is the ground truth for $x^{c}$. And the loss term to train $\mathcal{O}$ can be represented as $\mathcal{L}_{{o}}(x^{c},y^{c})$ (e.g., $\mathcal{L}_{{o}}$ is the cross-entropy loss for image classification and semantic segmentation, the loss for bounding box regression and classification in the object detection task). To align behaviors of clean and adversarial samples in feature space of $\mathcal{O}$, we adopt a feature-level loss as

Reconstruction loss. Similar to the reconstruction loss in pixel level, we set loss $\mathcal{L}_{{F}_{rec}}$ to minimize distances between adversarial and clean samples in the feature space of $\mathcal{O}$ as

where $\widehat{z}^{a}$, $\widehat{z}^{c}$ and $z^{c}$ are the intermediate features of $\widehat{x}^{a}$, $\widehat{x}^{c}$ and $x^{c}$ in the feature space of $\mathcal{O}$. The choice of feature space to compute feature-level constraints for different tasks is illustrated in the supplementary file.

Distribution alignment loss. We also note previous methods do not consider aligning the overall distribution of $\mathcal{C}$ and $\mathcal{A}$ in the feature space of $\mathcal{O}$. In contrast, we formulate our class-aware training constraint for the overall distribution alignment. Suppose there are $K$ classes within the distribution of $\mathcal{C}$ and $\mathcal{A}$. We represent $z^{c(k)}$ and $\widehat{z}^{a(k)}$ as the features for $x^{c}$ and $\widehat{x}^{a}$ of the $k$-th class extracted from $\mathcal{O}$. The clustering center of $z^{c(k)}$ and $\widehat{z}^{a(k)}$ is denoted as $m^{c(k)}$ and $\widehat{m}^{a(k)}$ in the distribution of $\mathcal{C}$ and $\mathcal{A}$ respectively. The class-aware constraint consists of three terms. First, we compute a loss term as the distance between $m^{c(k)}$ and $\widehat{m}^{a(k)}$ to align the distribution of adversarial samples to clean samples as

Further, favorable distributions in the feature spaces of the target models for high-level tasks should have wide distances between the features from different classes, and narrow separation among the features from the same class. To this end, we set intra- and inter-class loss as

where $M$ is a pre-defined hyper-parameter for controlling the inter-class distance. And the overall class-aware training constraint is written as

where $\lambda_{1}$, $\lambda_{2}$, and $\lambda_{3}$ are loss weights.

Training parameters. We employ Adam optimizer with $\beta_{1}$ and $\beta_{2}$ set to 0.5 and 0.999. The learning rate is set as 0.0002 and the batch size is 1 for Cityscapes and VOC2012, 4 for VOC07+12 , 16 for Tiny-ImageNet, and 64 for CIFAR10/CIFAR100. The number of training epoch is 80 for Cityscapes, 20 for VOC2012, 80 for VOC07+12, 30 for Tiny-ImageNet, 100 for CIFAR10/CIFAR100. Our method is implemented with PyTorch [55], and runs on a TITAN X GPU.

Data augmentation. For training on three tasks, we adopt the data augmentation policies described in [48, 50, 52]. For CIFAR10 and CIFAR100, we first pad the input to $40\times 40$ and randomly crop a patch of size $32\times 32$. Then the patch is chosen to be horizontally flipped or not; for Tiny-ImageNet, the data augmentation includes the operation of random crop (the pad size is 4 and the crop size is 64) and random horizontal flip. For the segmentation task (Cityscapes and VOC2012), we employ the official augmentations in [56], including RandScale, RandRotate, RandomGaussianBlur, RandomHorizontalFlip, and random crop. For the detection task (VOC07+12), the augmentation strategies in [52] are utilized.

Details of our training constraints for image classification. For the classification task, we adopt WideResNet [48] and ResNet50 [49] for experiments, and implement feature-level training constraints by extracting the feature of the fully connected layer before the final layer. Especially, each image $x\in\mathbb{R}^{H\times W\times 3}$ ($H$ and $W$ are the height and width of the image) has one class label $y\in\mathbb{R}^{K}$ and one feature vector $f\in\mathbb{R}^{L}$ ($L$ is the length of the vector). And we group features into $K$ classes according to $y$.

Details of our training constraints for segmentation. In the semantic segmentation task, we use PSPNet [50] and DeepLabv3 [51] with ResNet50 as the backbone for experiments, and complete feature-level constraints by using the feature of the convolution layer before the final layer. Different with the classification task, each image in semantic segmentation task has multiple class labels and thus has multiple feature vectors with different classes: for an image $x\in\mathbb{R}^{H\times W\times 3}$, it has the corresponding segmentation map $y\in\mathbb{R}^{H\times W\times K}$. And the feature of the image can be denoted as $f^{\prime}\in\mathbb{R}^{h\times w\times L}$ with a resized segmentation map $y^{\prime}\in\mathbb{R}^{h\times w\times K}$. Then we can obtain a set of feature vectors with their corresponding labels as $\{y_{1},...,y_{h\times w}\},y_{i}\in\mathbb{R}^{K}$, $\{f_{1},...,f_{h\times w}\},f_{i}\in\mathbb{R}^{L}$, which are reshaped from $y^{\prime}$ as well as $f^{\prime}$. Such set of feature vectors can be grouped into $K$ classes and utilized in feature-level constraints.

Details of our training constraints for object detection. We employ SSD [52] and RFBNet [53] with VGG16 as the backbone for experiments within the object detection task. And the features for feature-level constraints are obtained as the outputs of the backbone and are cropped with the ground truth of the bounding box. In this task, each image has multiple class labels for bounding boxes in this image, and thus also has multiple feature vectors with different classes: an image $x\in\mathbb{R}^{H\times W\times 3}$ can have $B$ bounding boxes and one class label for each box. The feature of the image can be denoted as $f^{\prime}\in\mathbb{R}^{h\times w\times L}$ and we can obtain the feature of each bounding box on this feature map with the corresponding class label, as $\{y_{1},...,y_{B}\},y_{i}\in\mathbb{R}^{K}$, $\{f_{1},...,f_{B}\},f_{i}\in\mathbb{R}^{L}$.

Attacks employed in training. In this paper, all attacks are conducted with $L_{\infty}$ constraint and untargeted form (except the experiments in Sec. 4.10). For the image classification task, we follow [48] to set the attack parameters during training and testing. And we employ the attack parameters’ setting in [57] and [58] for the semantic segmentation and object detection task, respectively. Therefore, for experiments of classification, the adversarial perturbation $\epsilon$ during training is generated by PGD [59] with KL criterion; for semantic segmentation, we employ BIM [22] for training; in object detection, classification attack (“cls”) and localization attack (“loc”) [58] are utilized during training. And the corresponding details are described as the following. For the classification task, we utilize PGD attack with KL criterion during training, and the perturbation range $\epsilon=0.031\times 255$, the step size $\alpha=0.0175\times 255$, the number of attack iteration $n=4$. In the semantic segmentation task, we use BIM ($\epsilon=0.03\times 255$, $\alpha=0.01\times 255$, $n=3$) during training. Moreover, the attacks for classification loss and location loss are employed within the object detection task, and the perturbation range is 8 for pixel values within $[0,255]$.

Attack types. During evaluation, in classification tasks, the adversarial perturbation is obtained by PGD with cross-entropy criterion and translation-invariant form [60], DeepFool attack [61], and C&W attack [62]. For semantic segmentation, we adopt BIM, DeepFool, and C&W for evaluation; for object detection, “cls”, “loc”, and “cls+loc” (simultaneously conducting classification and localization attacks) are employed for testing.

Attack parameters. Following [48, 57, 58], for the classification task, we adopt PGD with cross-entropy criterion ($\epsilon=0.031\times 255$, $\alpha=0.0075\times 255$, $n=8$), DeepFool attack ($\epsilon=0.031\times 255$), C&W attack ($\epsilon=0.031\times 255$, the step size is $0.0075\times 255$). For semantic segmentation, we utilize BIM ($\epsilon=0.03\times 255$, $\alpha=0.01\times 255$, $n=4$), DeepFool attack ($\epsilon=0.03\times 255$), C&W attack ($\epsilon=0.03\times 255$, the step size is $0.01\times 255$) for evaluation. Moreover, the perturbation range is set as 8 for the object detection task. Note that we adopt different attack types and parameters for training and evaluation, to verify the generalization ability of the trained DGNs towards unseen attacks.

Metrics. Classification accuracy, mean of class-wise intersection over union (miou) and mean average precision (mAP), are adopted as the quality indicators for image classification, semantic segmentation, and object detection, respectively.

Baselines. We choose approaches of input transformation and adversarial training for comparison. Most existing adversarial training work focuses on image classification, and we adopt two recent methods, [48] and [54], for comparison. For the semantic segmentation task, [57] first conducted comprehensive exploration on the impact of adversarial training for semantic segmentation. We employ two strategies proposed by [57] (SAT and DDC-AT). For object detection, [58] proposed four variants of adversarial training. All these adversarial training approaches are trained with their original configuration and “finetune” means finetuning pre-trained models (without defense) via adversarial training. For input transformation strategies, we use six representative methods, including [9, 14, 15, 16, 30, 13]. We adopt their original generator structures, and re-train them with the same epochs, batch size and target models as ours.