Sparse Adversarial Attack to Object Detection

Our SAA can generate sparse adversarial noise. Concretely, we apply perturbation with bounded $l_{0}$ norm (no more than 2$\%$ of the image size) on the image. The perturbation is sparse in space but is efficient in fooling the detectors due to our purposely design of its spatial distribution.

In the training phase of detectors, the center of an object plays a vital role in the detection of this object. Take the one-stage detector YOLOv4 [1] as an illustration, which will allocate anchor box that resemble ground truth mostly for an object. Even for two-stage detectors and anchor-free detectors, object centerness directly relates to the computation of intersection over union (IOU). Boxes that deviate object centerness too much are more likely to be erased during the non-maximum suppression (NMS) process.

Traditional adversarial patches with square or circle shape [2] [8] are too concentrated in a local area of the image. They have great physical realizability but are pixel inefficient in this task. We argue that a highly centralized structure limits the success rate of attacks especially when the $l_{0}$ norm constraint is too strong.

In view of those cases, we design a kind of cruciform patch with it’s intersection locates at the centerness of object bounding box. Exploiting the long-span patch, we can mislead the results of detectors severely with very few pixels changed.

Due to the difficulty of $l_{0}$ norm constrained optimization problem, we exploit a mask with values of 0 and 1 to decide the location of the patch, also to limit $l_{0}$ norm of the perturbation. We design the mask empirically based on the assumption that object centerness is a vulnerable area against adversarial attacks, combined with other conditions of the patch (shape, size) mentioned before. Hence the input of the detector is formulated as follows:

$$Img=I\odot(1-M)+P\odot M$$

(1)

Here $I$ denotes clean image, $P$ denotes adversarial patch, $M$ is the mask we design purposely and $\odot$ denotes element-wise product in this paper. As illustrated in figure 1, we include preprocesses of detectors into the forward phase of our attack in order to optimize the patch directly with gradient.

The goal of our SAA is erasing all the object detections in an image. And we define this type of attack on object detectors as evasion attack. The evasion attack is closely related to the definition of positive sample (foreground) and negtive sample (background) of the detector. We erase an object in an image by making it become a negative sample of the target detector. Consequently we design loss functions according to the definition of foreground and background of target detectors.

YOLOv4 [1] is a one-stage object detector with excellent speed performance and considerable precision performance. YOLOv4 [1] exploits a confidence branch to distinguish foreground and background. The bounding box with its confidence lower than a threshold in YOLOv4 [1] will be recognized as background and will be discarded in the postprocess phase. Based on these principles, we design the evasion loss function of YOLOv4 [1] as follows

$$Loss_{YOLO}=\max\limits_{c\in C,b\in B}(conf(c,b))$$

(2)

Here $C$ denotes the set of all object categories, $B$ denotes the set of all bounding boxes and $conf$ is the object confidence in YOLOv4 [1]. So the loss function extracts the maximum object confidence of an image. We minimize it to achieve our attack purpose.

FasterRCNN [10] is a typical two-stage object detector that firstly extracts bounding box proposals before the subsequent classification and regression processes. The definition of positive and negative samples is more complex in FasterRCNN [10] than that of YOLOv4 [1]. However, we merely consider the inference phase of FasterRCNN [10] to simplify our attack. That is, we increase the softmax output probabililty of background while decrease that of any other categories (foreground objects). However, FasterRCNN [10] typically generates more than one hundred thousand region proposals by RPN, which is more than ten times that of YOLOv4 [1]. The enormous number of region proposals makes it hard to erase all the objects with an extremely limited attack budget. So we make some compromises and design the total evasion loss function of FasterRCNN [10] as follows

$$Loss_{FRCNN}=\alpha_{1}\cdot Loss_{1}+\alpha_{2}\cdot Loss_{2}$$

(3)

where $\alpha_{1}$ and $\alpha_{2}$ are hyperparameters, and we have

where $C$ denotes the set of all object categories, $B$ denotes the set of all bounding boxes predict by FasterRCNN [10] and $N$ is the number of elements of set $B$. We design $Loss_{1}$ to attack the bounding box with highest object probability, which is hard in experiments, hence $Loss_{2}$ is added to erase as many bounding boxes as possible.

We ensemble the two detectors to train adversarial patches, thus the final loss function we use is as follows. Without deliberately balancing the weights of two terms in $Loss$, we can achieve high attack success rate.

$$Loss=Loss_{YOLO}+Loss_{FRCNN}$$

(6)

With only 2$\%$ of pixels allowed to perturb, it’s hard to generate adversarial examples with high attack success rate. We exploit essential tricks in the training process of adversarial patch to generate powerful adversarial examples.

To overcome the preprocess distortions, we include preprocess into the forward and backward phase of adversarial patch training, which including image resize, image normalization and other necessary processes. The strategy avoids unnecessary adversarial attack enhancements and thus accelerate our adversarial patch training process.

We exploit multi-scale steps to update adversarial patch and set all steps to integer multiples of $1/255$. We accelerate the training process tremendously by using large steps first. Also, it’s a great way to avoid local minima in optimization. We decrease the update step gradually to refine our adversarial patch as the training goes on. It is worth mentioning that we set update steps to be special values (integer multiples of $1/255$) to filter out the quantification effect.

We also design multi thickness cruciform patch to adaptively attack images with different number of object detections. We believe thinner cruciform patch produce stronger attack effect for its adversarial effect on a wider range of image feature. Experiment results confirm its validity in practice.

In experiments, we conduct two phases attack to FasterRCNN [10]. At the first phase, we set $\alpha_{1}=1$ and $\alpha_{2}=0$ in $Loss_{FRCNN}$, aiming at suppress the bounding box with highest object probability. Due to rough patch location selection, we rarely succeed in the first phase to attack FasterRCNN [10]. And the second phase will be started once the first phase failed, we set $\alpha_{2}=1$ and $\alpha_{1}=0$ to attack as many as bounding boxes as possible. We don’t set $\alpha_{1}\neq 0$ and $\alpha_{2}\neq 0$ together for better optimization of each term in $Loss_{FRCNN}$ at different phase.