CCA: Exploring the Possibility of Contextual Camouflage Attack on Object Detection

Whether targeting a classification model or a detection model, adversarial machine learning (AML) against CNNs has seen great development since Szegedy et al. [16] discovered that small perturbations could alter CNN predictions. A considerable amount of recent AML literature [17, 18, 19, 20] requires a target model to be transparent to calculate the gradient, as knowing a numerical pathway between a perturbation and a model prediction grants an enormous optimization advantage. Such a pathway is not always available given that a target model is often a blackbox in real attacks. Various proposed blackbox attack methods rely solely on query and target model output. Most of these are still based on gradient descent. The difference lies in how they estimate a perturbation gradient in the blackbox setting. ZOO [21] estimates it by coordinate descent. [22] trained a substitute model to mimic the target model behavior to obtain the gradient. [23, 24] estimate it via an evolutionary strategy. [25] is based on differential evolution. [12] shares the same task with us and they choose to minimize an end-to-end clone network that is trained to directly estimate score from camouflage.

One of the sub-areas of AML is to generalize it to the physical dimension. This is challenging because training in silico often results in poor generalization to real environments. The first attempt was by [26] and they produced perturbations that remain effective when printed on paper. [10] generalize a perturbation to fool a classifier of real stop sign images. However [27, 28] found that [10]’s perturbation do not work on a real world detector. This is likely because detectors are more robust than classifiers since detectors must detect objects at multiple scales. Later, [19, 29] proposed perturbations that could attack stop-sign detectors in the real-world. However, all the aforementioned works are perturbing detectors of stop signs, which are planar objects whose images, under changes in camera geometry, are related by linear 2D projective transformations. This is in contrast to nonplanar objects whose images, under changes in camera geometry, are related by more complex range-dependent nonlinear transformations. Perturbations are easily transformed via linear 2D projective transformation without breaking the gradient chain between the perturbation and the output score [30, 10]. Complex nonlinear transformations, however, will require a dedicated 3D simulation, such as the one used in our paper, which breaks the gradient chain and greatly complicates the optimization. Besides, there are only 1983 stop sign instances in the training split of MS-COCO [31]. Meanwhile, there are 43867 cars of various appearance in it. Such a great amount and diversity benefits vehicle detectors’ generalization capacity trained on MS-COCO and makes it harder to fool.

The framework of proposed algorithm is shown in Fig. 2. There are mainly three phases in our algorithm, including simulation, detection and evolution. Let $c$ denote a vehicle camouflage pattern represented by an RGB image. When $c$ has been painted onto the surface of a vehicle in a tiled style, state-of-the-art object detectors would completely or partially fail to detect/classify the vehicle regardless of observation angles, distances and environments. In this context, we aim to determine whether it is possible to create a camouflage pattern $c$, being painted on a given vehicle, could mislead detectors to make incorrect detection/classification on the other vehicles (without camouflage) in the same scene. After inputting the simulated images into detectors and outputting the detection results, we conduct the optimization of the proposed camouflage from two aspects, namely the attack performance of the camouflage on detectors and search distribution of the camouflage. The workflow of proposed contextual adversarial attack is shown in Algorithm 1. We describe our approach to implementing the proposed loss function in Section III-B. In Section III-C we outline the architecture of our evolutionary model to generate the optimal camouflage pattern.

Given a camouflage pattern $c$, represented by a $16\times 16$ RGB image, we would like to convert it to a photo that looks like real. We denote $t$ as the transformation, specifically including tiling and painting the $c$ onto a vehicle, moving the vehicle to a location in an environment and at last taking a photo of that vehicle from a certain angle and distance. After transformation, we will have an image with a vehicle covered by a camouflage texture in it. We suppose that there are $n$ vehicles in the generated image except for the one with camouflage. Then, we could denote the scores of the vehicles without camouflage as $S{{}^{i}_{t}}(c)\in[0,1],i=1,2,3...n$. The overeaching idea of our algorithm is to find an optimal camouflage, such that the mean score of $S{{}^{i}_{t}}(c)$ is low:

For the concern about the complexity of the physical world, following [32], we construct a batch of transformations $\mathcal{T}$ for each learning procedure, involving the factors of different lighting condition, and distance and viewpoint changes. Thus, by minimizing the score in expectation, we have the following optimization problem:

where $\widetilde{S}{{}_{t}}(c)$ is the mean score of $S{{}^{i}_{t}}(c)$. And then we could have eq. 2 as:

In this section, we aim to optimize a flattened $c$ during the training process. Specifically, it contains two major components. First one is the performance over various transformations. Besides, we also want to optimize $c$’s search distribution’s members’ performance. Suppose $\mathcal{Z}_{c}$ is the search distribution of camouflage $c$, then, for a fixed transformation $t$, our goal is to optimize:

Inspired by [33], we design our gradient as:

where $\lambda$ is the number of samples from the distribution. However, different from the problem mentioned in [11], our optimization has a constraint of $[0,255]$ because of the image feature of $c$. In our case, we propose to use a truncated multivariate normal distribution $\mathcal{N}_{[0:255]}$ to bound both $c$ and the search radius without extra hassle.

Instead of directly minimizing $\mathbb{E}\widetilde{S}_{t}(c)$, we choose to minimize the binary cross-entropy between it and zero as shown eq. 6:

During optimization, we assume the scores of $\mathcal{Z}_{c}$ are normally distributed given that $\mathcal{Z}_{c}$ itself is normally distributed. In this case, by normalizing ${\mathbb{E}_{t\sim\mathcal{T}}\widetilde{S}{{}_{t}}(c)}$ into standard scores using $\beta_{t}$, we could have:

Then, we arrive at our normalized gradient:

Then, the gradient approximation in eq. 8 is used to perform gradient descent. We choose to perform simple gradient descent which is:

where $\alpha$ is the learning rate. Then we update $c$ until $\mathbb{E}_{t\sim\mathcal{T}}\widetilde{S}{{}_{t}}(c)$ no longer decreases.

In this work, we use the Unreal Engine to build a simulation environment based on the real downtown environment. Specifically, we build different kind of cars, roads, traffic signals in the simulated environment. To obtain valid data for training and testing, we sequentially camouflage and place a professionally modeled Camry 2015 XLE in different locations in the environments. Also, 36 locations are sampled along the roads in the environment, 18 of which for training and the remaining 18 for testing. At each location, we record RGB images and groundtruth of the vehicle and the surrounding area from 20 fixed camera orientations. We manually label all the groundtruth of vehicles into two categories: the one carrying the camouflage and the normal ones. These images and groundtruths are then sent to the detector for evaluation. Inspired by [12], in our experiments, we set the resolution of camouflage to be $16\times 16$.

For evaluation, we use two MS-COCO [31] pretrained detectors in our experiments: YOLOv3 [5] and Mask-RCNN [3]. Mask-RCNN is currently one of the hallmark detectors for object detection in terms of detection performance. YOLOv3, on the other hand, balances performance with detection speed. We use the YOLOv3-SPP variant, which has the best detection performance among the YOLOv3 family.

Also, we apply three detection metrics as the indicators of performance: detection confidence, mean Intersection over Union (mIoU) [34] and [email protected] ([email protected]). The detection confidence is the mean value of the scores of the detected cars in different scenes. The Intersection over Union $IoU$ between predicted bounding box $B_{p}$ and ground truth bounding box $B_{gt}$ is defined by $IoU=\frac{B_{p}\cap B_{gt}}{B_{p}\cup B_{gt}}$ [34]. In our experiments, we use the best IoU achieved among all vehicle bounding box predictions for the image as its IoU. The mIoU is the averaged IoU across all images at all locations that are being reported. The [email protected] is the percentage of images that have IoU larger than 0.5. This metric is used in the PASCAL detection challenge [34], which considers a detection prediction to be successful if its IoU is greater than 0.5.

For a thorough evaluation, inspired by [12], we compare our results with two baselines to validate the effective of the proposed camouflage, as shown in Fig. 1. The first one is conducted by painting the central vehicle with six real vehicle colors (red, black, silver, grey, blue, and white) and compute the mean value of the detection metrics. Another one is that we generate 5 random camouflages with the same resolution of the learned camouflage and report the mean values of the detection metrics for comparison.

In this section, we present the performance of our learned camouflage and the baselines against several state-of-the-art detectors, including YOLOv3, Mask-RCNN and FCOS in the downtown environment.

The learning process of our CCA camouflage against YOLOv3 is shown in Fig.  3 (a). In the training, we set the scores by computing the mean value of the detection scores of all the unpainted vehicles. From Fig.  3 (a), we can easily observe that the detection scores of the vehicles decreases under the proposed learning algorithm, which demonstrates the effectiveness of the evolution search strategy we applied in CCA. To further validate the effectiveness of the learned camouflage, experiments were conducted to compare the proposed camouflage to other baselines: painting basic colors or random camouflage on the context vehicle. As shown in Table I, our method outperforms baselines over all the three metrics on both training data and testing data. This supports our argument that we could misled object detector by only changing a part of the context.

We present the training process and camouflages’ performance against MaskRCNN in Fig. 3(b) and Table II. Similar to its performance against YOLOv3, our learnd CCA camouflage outperforms baseline colors and random camouflages on both training and testing data. Furthermore, we can draw the same observation from the experiment results of FCOS reported in Fig. 3(c) and Table III. All the reported experiments show the effectiveness of CCA on attacking state-of-the-art object detectors.

To validate the proposed CCA camouflage’s influence on object detectors, we visualize the Grad-CAM [35] maps of the samples with our learned camouflage and random camouflage. Since the Grad-CAM shows the gradient heatmap of the classification score scalar w.r.t. the input, it could help us to identify the image regions most relevant to the particular category. Fig. 4 shows two visualization examples, and each of them presents the detection result of YOLOv3 and corresponding Grad-CAM map. To be noted, YOLOv3 is a multi-scale detector and there are three layers of features with different sizes in the output. Considering of the resolution of the input and the objects’ size in the images, we use the nearest convolution layer to the $13\times 13$ feature layer of YOLOv3 as the penultimate layer and output the attention map of strategy ’car’ in each detection. For a fair comparison, we output the absolute value instead of the normalized value of the gradient scores of the feature map in our experiments. As expected, in the images with the proposed CCA camouflaged vehicle (Fig. 4 (b)(d)), the unpainted vehicles gain much lower attention than in the scenes with a random camouflaged vehicle (Fig. 4 (a)(c)). This observation further explains and validates our assumption that we could reduce the object detectors’ performance on certain objects by learning a contextual camouflage.

Fig.  5 shows another example, in which the detector’s attention is completely shifted from the front unpainted vehicle to the vehicle in the back. This case also proves the the proposed CCA camouflage’s capability of influencing the detector’s performance on the objects that without camouflages painted on them.

So far we have shown the effectiveness of our learned CCA camouflage on attacking object detector with the evolution strategy. Intuitively, we come up with the assumption that it should also be possible that we influence the detector with an opposite effected camouflage and increase the attention of the vehicles.

Here, we confirm that by experiments. In order to gain the opposite rewards, we reset the loss function proposed in Section  III-B as:

Then we apply the evolution strategy proposed in Section  III-C to figure out the potential positive affect of the CCA camouflage on object detectors. Fig.  6 shows the training process of the YOLOv3 and MaskRCNN. It shows that we can effectively improve the performance of the models with a learned contextual camouflage. Detection results comparison also proves this as shown in Table  IV. The learned CCA camouflage outperforms the basic colors and random camouflage both on training and testing dataset. That means we could influence the object detectors’ performance by not only attacking them but also enhancing them with the proposed CCA pattern.