Exploiting Multi-Object Relationships for Detecting Adversarial Attacks in Complex Scenes

SCENE-Lang. We define a new language called SCENE Language (SCENE-Lang) to describe the object co-occurrence information in natural scene images. Each natural scene image can be described with one SCENE-Lang sentence and each word in the sentence is associated with an object instance.

SCENE-Lang Words. We describe the category of an object and its coarse-grained location with a SCENE-Lang word. To describe the location of an object, we evenly divide each image into a $H\times W$ grid and label each grid cell using a number, so we can use a small, finite vocabulary to describe the scene. Using coarse-grained locations can also help tolerate adversarial attacks that may shift the bounding boxes of objects. The center of an object’s bounding box determines which cell the object is in. We denote the set location labels as $L=\{1,\dots,H\times W\}$. Therefore, each SCENE-Lang word $w=(l,c)$ is a pair of a location label ($l\in L$) and a category label ($c\in C$). We denote the finite vocabulary of SCENE-Lang as $W=C\times L$, whose size $|W|=|C|\times H\times W$. Note that although we could encode the object label $c_{i}$ using a number, because SCENE-Lang is a pseudo language we choose to use the natural language label $c_{i}$; this enables ease of explanation upon detecting a context consistency violation.

SCENE-Lang Sentence. We describe the object co-occurrence relationships in a scene image $I$ with a single SCENE-Lang sentence, wherein each word is associated with an object instance in the image. The sentence is represented by $s_{I}=[w_{1},\dots,w_{n}]$, where the length of the sentence $n$ is equal to the number of object instances in the image $I$. We use $s$ instead of $s_{I}$ subsequently, for ease of exposition. The order of the words in the sentence is sorted based on their location labels (numerically ascending). Figure 2 shows an example about how to describe a scene image with a SCENE-lang sentence.

SCENE-BERT. We use SCENE-BERT, a natural language model to learn the co-occurrence context graph $G$ in natural scene images. Each input to SCENE-BERT is a sequence of tokens, denoted as $\mathbf{T}=[t_{1},\dots,t_{n}]$. The model also takes a n-dimensional mask vector $\mathbf{M}\in\{0,1\}^{n}$ as input, where $0$ in the $i$-th dimension indicates masking off the $i$-th token $t_{i}$ and $1$ indicates that the corresponding token $t_{i}$ is not masked. The number of tokens $n$ is determined by the number of words in the input SCENE-Lang sentence. The output of SCENE-BERT is a reconstructed sequence of tokens, where the masked off token in $\mathbf{T}$ is replaced with a listed of predicted tokens that match the context (i.e., with lowest cross-entropy loss). We use $f(\mathbf{T},\mathbf{M},t_{i})$ to denote the confidence score of $t_{i}$ in the predicted token list. If $t_{i}$ is not in the list, $f(\mathbf{T},\mathbf{M},t_{i})=0$. This score will be used to calculate the consistency score of the whole scene.

SCENE-BERT Architecture. SCENE-BERT is based on the multi-layer bidirectional transformer model BERT [7]. Figure 3 shows the simplified architecture of the model. Since transformers have been widely adopted in language-related tasks and we are reusing an existing implementation of BERT, we omit the detailed description of the model architecture and refer the readers to [40], in the interest of space. Besides being a state-of-the-art language model, we choose BERT to implement our language model for two main reasons. First, the use of the bidirectional self-attention mechanism allows BERT to capture dependencies from both directions (i.e., the prediction of the current word depends on both words appearing before it and after it). This matches our context model very well as the context graph $G$ is not ordered (relationships are both ways). Second, the way BERT is trained is also a perfect match for our task. In particular, BERT is trained with the Masked Language Modeling (MLM) task, where some input tokens are masked at random and the model is asked to predict them. This task is very similar to our approach to detect adversarial attacks (Figure 1): check whether object detection result is consistent with those predicted purely based on context.

Tokenization. Because SCENE-Lang is a pseudo language, tokenizing a sentence $s$ is straightforward. Specifically, we assign each unique word $w$ in the finite vocabulary of SCENE-Lang $W$, a unique number, which serves at its corresponding token (i.e., $t\in\{1,\dots,|W|\}$). So the tokenizer simply maps each word $w_{i}$ in a sentence $s$ to its corresponding number.

Training. We train SCENE-BERT using the same unsupervised masked language modeling task as RoBERTa [27]. Specifically, it randomly masks token(s) from the input sequence, and the objective of the model is to predict the original token(s), only based on the remaining tokens in the sentence (i.e., the context). In other words, SCENE-BERT learns the dependencies between co-occurring objects, or the edge weights in the object co-occurrence graph $G$. We want to highlight the unique advantage of SCENE-BERT again in that it can be trained with any set of sentences in SCENE-Lang. This means that it can be trained with the ground truth labels of an object detection dataset (as we do in our experiments); this would work with any object detection network trained with the same dataset. Alternatively, it can also be trained in a completely unsupervised manner by running an object detector over a clean dataset to generate the training sentences.

In this subsection, we illustrate how we use the trained SCENE-BERT model to perform context consistency checks. At a high level, we use differential analysis to detect inconsistency, i.e., by comparing the detection result (in SCENE-Lang) with the scene description predicted from our context model SCENE-BERT. The smaller the difference, the higher will be the consistency score. Because most adversarial attacks will violate context consistency, by thresholding the consistency score, we can detect whether the input image is adversarial or not. Next, we introduce how we calculate the consistency score.

Let $I$ be a clean scene image and $D(\cdot)$ be the victim object detector. We can encode the output $D(I)=O$ using a SCENE-Lang sentence $s=[w_{1},\dots,w_{n}]$ as described earlier, which will be tokenized into $\mathbf{T}=[t_{1},\dots,t_{n}]$. Let $I^{\prime}\leftarrow I+\Delta I$ be the perturbed adversarial image and $\mathbf{T}^{\prime}$ be the tokenized SCENE-Lang description over $I^{\prime}$. Recall that there are three possible attacking goals, which will affect $x$ in three different ways:

• $\bullet$

  Misclassification attack where the token associated with an instance is perturbed, i.e., $t^{\prime}_{i}\neq t_{i}$;

• $\bullet$

  Hiding attack where a token is missing in the token sequence, i.e., $t_{i}\notin\mathbf{T}^{\prime}$;

• $\bullet$

  Appearing attack where an undesired token appears, i.e., $t^{\prime}_{i}\notin\mathbf{T}$.

Using the trained SCENE-BERT model, we can mask off a token $t^{\prime}_{i}\in\mathbf{T}^{\prime}$ and ask the model to predict what $t^{\prime}_{i}$ is, based on the remaining tokens (i.e., the test time context). In theory, if the predicted result is different from $t^{\prime}_{i}$, then we deduce that $t^{\prime}_{i}$ is the likely target object under attack. However, this has two associated problems. First, there could be multiple objects that are contextually consistent (i.e., SCENE-BERT can return a list of possible tokens instead of a single one), and hence, how should we calculate the difference between $t^{\prime}_{i}$ and the predicted tokens? Second, $\mathbf{T}^{\prime}$ contains multiple tokens and so, how can we know which token to mask, especially in the case of a hiding attack (where the victim token is missing)? We solve the first problem by using the confidence score of $t^{\prime}_{i}$ in the predicted list as the consistency score of that specific object. If $t^{\prime}_{i}$ is not in the predicted list from SCENE-BERT, its consistency score will be $0$. We solve the second problem by iterating through all tokens (i.e., detected objects) and using the lowest consistency score of all objects as the consistency score of the whole image. The details are captured in Algorithm 1. Note that our approach to calculate the consistency score is able to handle hiding attacks because the missing token typically affects the prediction results of the other non-target tokens.

We use the RoBERTa [27] model, a reproduction of the original BERT model [7], to implement SCENE-BERT. It is configured with six hidden layers and twelve self-attention heads.

The PASCAL VOC dataset contains 20 object categories. The majority of images in the PASCAL VOC dataset have 1 to 5 object instances, on average, 1.4 categories and 2.3 instances per image. The MS COCO dataset contains 80 object categories. Images in this dataset have more object instances, on average, 3.5 categories and 7.7 instances per image. We used the ground truth labels from both datasets to train SCENE-BERT models. Since our context model is designed to consider the co-occurrence consistency of multiple objects in the scene, we omitted images that consist of a single object. For the PASCAL VOC 2007 dataset, we used a $3\times 3$ grid (i.e., $H=3$ and $W=3$); so in total we have $|W|=C\times H\times W=20\times 3\times 3=180$ tokens. For MS COCO, we also used a $3\times 3$ grid, so in total $|W|=C\times H\times W=80\times 3\times 3=720$ tokens.

Since SCENE-BERT can be trained independent of the object detector, we used pre-trained F-RCNN and YOLO models. For the PASCAL dataset, both models were trained with VOC07trainval and VOC12trainval. For the MS COCO dataset, the F-RCNN model was trained with coco14train and coco14valminusminival, and the YOLO model was trained with coco17train.

To test the attack detection performance, we generate 10,000 attacks for each attacking goal (misclassification, hiding, and appearing) from both datasets, except for hiding attacks on PASCAL VOC, which does not have enough objects for hiding attacks. Because our detection method uses high level semantic information (object co-occurrence context) and does not rely on low-level features, we only evaluate it against digital attacks. The attacks are generated using the standard iterative fast gradient sign method (IFGSM) [20], with $L_{\infty}\leq 10$ as the perturbation budget; and the perturbations are applied to the whole image. Because SCENE-BERT takes detected object labels and locations as inputs, how the perturbations are generated does not affect the experimental analysis, thus we only used IFGSM. Table 1 shows the attack success rate on the two datasets.

We compare our method with two baseline models in the experiments.

Feature Squeeze (FS) [44] is a SOTA context-agnostic method for detecting adversarial image examples. This mechanism can detect the adversarial image examples generated by Fast Gradient Sign Method [14], DeepFool [32], and Projected Gradient Descent [31]. Its core idea is that, adversarial attacks need to limit how many perturbations can be applied (e.g., by limiting the change to the $L_{2}$ or $L_{\infty}$ norm) to achieve (quasi-)imperceptibility. Therefore, by squeezing the input features (i.e., reducing the color bit depth of each pixel and smoothing surrounding pixels), FS may remove enough perturbations and acquire the correct prediction results. Then, by comparing how different the prediction results of the original input and the squeezed input are, FS can detect adversarial attacks.

SCEME [21] is our previous context-consistency-based adversarial attack detection method that showed much better detection performance than Feature Squeeze. It models context at region proposal level and uses attention mechanism and Gated Recurrent Units (GRUs) to learn four types of relationships between region proposals: (1) spatial context between regions corresponding to the same object; (2) object-object context between regions corresponding to different objects; (3) object-background context between regions corresponding to objects and regions corresponding to the background; and (4) object-scene context between regions and the whole scene. To detect adversarial attacks, SCEME uses auto-encoders (one per object category) to learn the benign distribution of context profiles corresponding to an object category. The context profile contains both edge features and node features (i.e., features of the region proposal). An adversarial attack that violates context consistency will yield a higher reconstruction error rate and by thresholding the reconstruction error rate, SCEME can detect perturbed regions. Note that because SCEME works at region proposal level instead of the whole image, we cannot directly compare it with SCENE-BERT. To calculate the detection performance at the whole image level, we aggregate all reconstruction errors from each region proposal and use the highest one as the final score.

While SCENE-BERT performed slightly worse than SCEME over the datasets, we also observed cases where SCENE-BERT can detect attacks that SCEME cannot. Figure 6 shows two cases. In the first case, the left bird (peacock) is perturbed to be a boat; and in the second case, the left horse is perturbed to be a dining table. Both cases obviously violate the context consistency based on object co-occurrence hence were detected by SCENE-BERT. We suspect that the reason that SCEME did not detect these attacks is because it also considers the visual features of the object and the auto-encoders may focus more on the visual features instead of the context.

While analyzing the evaluation results, we also noticed that if the attack is context consistent (e.g., misclassifies a bus to a car), then SCENE-BERT cannot detect such attacks. We want to argue that context consistency is one way to check for an attack and need not supplant, but can complement, other methods that can check individual objects or removal/addition of objects. Moreover, such context consistent attacks are likely to be less disruptive. For example, mis-clsassifying a bus to a car is unlikely to cause an autonomous vehicle to collide with the bus, but changing a speed limit sign in the middle of a road to a stop sign can lead to disastrous outcomes.

In this subsection, we explain how the baseline Feature Squeeze is implemented. Algorithm 2 shows the algorithm. $\mathbf{Img_{O}}$ denotes the original image, $\mathbf{Img_{Q}}$ defined in line 1 denotes the quantized image after squeezing. $\mathbf{PR_{O}}$ defined in line 2 denotes the prediction result for the original image from object detector $g(\cdot)$, while $\mathbf{PR_{FS}}$ defined in line 3 denotes the prediction result of quantized image. $R_{O}$ and $R_{FS}$ defined in line 7, 9 denote one region from the prediction result for original image and quantized image respectively. Note that we take the highest distance among all regions as a represent to the distance of whole image. Furthermore, for each region, we take the lowest distance calculating from all its overlapped as the distance for the queried region. Under the consideration that we usually take the category with highest confidence score for the regions dumped from object detector, we manually set rule, i.e. the distance is $1$ for regions with different predicted categories.

In this subsection, we explain how the baseline SCEME model is implemented. Algorithm 3 illustrates how we adapted the original SCEME, which works at region proposal level, to make it work at whole image level. $\mathbf{PR}$ defined in line 2 denotes the prediction results. $CP$ defined in line 6 denotes the Context Profile extracted from intermediate layer of F-RCNN which will be passed to SCEME model to generate a reconstruction error. We take the highest reconstruction error as the reconstruction error of the whole image.

Note that because the categories in MS COCO dataset are biased and SCEME requires on auto-encoder for each category, it cannot be trained well on categories that rarely occur. For this reason, we did not measure SCEME’s performance on the MS COCO dataset.