RED – Robust Environmental Design

As autonomous driving systems become progressively more embedded in real-world systems, their safety becomes paramount. These systems and their sub-components, such as classification and segmentation modules, have been shown to be vulnerable to adversarial attacks Goodfellow et al. [2014], Madry et al. [2017], Kurakin et al. [2016] In this work, we focus on enhancing the safety of such systems by modifying the appearance of objects (specifically road signs) such that adversarial attacks applied to those objects are less effective (see Figure 1).

Salman et al. [2021] first propose modifying the appearance of physical objects by designing patterns that make them easier to recognize under naturally challenging conditions, e.g., foggy weather. These conditions occur naturally rather than being the result of an adversarial attack. Adversarially crafted perturbations pose a more significant challenge from a defender’s perspective for two key reasons: firstly, adversarial examples are explicitly designed to decrease model performance, and secondly, they are out of distribution with respect to training data (naturally challenging conditions are typically seen in training data, albeit scarcely for some domains).

In the context of autonomous driving, Eykholt et al. [2018], Yang et al. [2020] demonstrated the practical dangers of misclassification, such as small errors leading to severe accidents. Giving greater concern is the observation that these adversarial attacks can be physically realized Eykholt et al. [2018], Kurakin et al. [2016], Athalye et al. [2018]. Physically realizable attacks often take the form of adversarial patches, small regions of an image designed to deceive classifiers, detectors, segmenators. Brown et al. [2017], Eykholt et al. [2018], Liu et al. [2018], Karmon et al. [2018], Zhang et al. [2019] introduced physical adversarial patches for real-world objects.

Defenses can be categorized into Attack-Aware and Attack-Agnostic defenses. Attack-aware defenses, such as adversarial training, (Goodfellow et al. [2014], Madry et al. [2017], Shafahi et al. [2019]), rely on knowledge of specific attacks. In contrast, Attack-Agnostic defenses, such as randomized smoothing (Cohen et al. [2019], Lecuyer et al. [2019], Salman et al. [2019]) and image sanitization techniques (Xiang et al. [2020, 2022], Xu et al. [2023]), do not rely on specific attack information.

To counter adversarial attacks, we propose an environmental-centric approach, Robust Environmental Design (RED), in which we design the backgrounds of road signs such that the road signs are both robust and still easy to print (as shown in Figure 1). Our defense procedure and objective function learn object patterns so that once these road signs are deployed in the physical world, they achieve robustness against patch attacks without requiring adversarial training. The model $f$ is trained only on clean data, and at test time, even some naive image sanitizing defense ensures $f$ can accurately predict even on maliciously modified inputs $x^{\prime}$, eliminating the need for adversarial training after deployment. To demonstrate the efficacy of our method, we conduct experiments using two common benchmark datasets for road sign classification, LISA and GTSRB Eykholt et al. [2018], and test against several types of patch-based attack paradigms. Our approach achieves high levels of robustness compared to baseline models. Additionally, we conducted physical experiments by printing various common road signs (e.g., stop signs, speed limit signs, etc.) with patterns optimized via RED. We collected photos at different times of the day, under various lighting and weather conditions. We find that RED significantly improves robustness against attacks in both digital and physical settings.

Road Sign Classification Let $\mathcal{X}\subset\mathbb{R}^{W\times H}$ be a domain of possible road sign images of $W$ by $H$ pixels, and $Y$ be the set of possible road sign classes. Each road sign image $X\in\mathcal{X}$ has a corresponding true label $y\in Y$ (e.g., stop sign). To predict the class of a road sign, a classifier $f:\mathcal{X}\rightarrow Y$ is used, where $f(X)$ represents the predicted class of image $X$.

Adversarial Patch Attack We focus on patch-based attacks against image classification models. For a given image $X$ with true label $y$, the attacker’s goal is to find a maliciously modified version of $X$, say $X^{\prime}$ such that $f(X^{\prime})\neq y$. Patch-based attacks constitute the attacker modifying a region of at most $B$ pixels in $X$. The region can have various shapes but is constrained to be a contiguous region of the image, and is defined by a binary mask $M$ where $M[i][j]=1$ if the attacker is modifying pixel $(i,j)$ and $M[i][j]=0$. The attacker then applies a perturbation $\delta$ (with magnitude at most $\varepsilon$) to the pixels defined by $M$. The attacker finds their desired mask $M$ and perturbation $\delta$ via the following:

Where $|M|$ counts the number of 1’s in the mask $M$ and $\odot$ is elementwise multiplication.

Next, we outline our proposed method for robust environmental design (RED). RED works by crafting the background of a road sign such that any patch placed on that road sign is not effective at fooling the classifier (see Figure 1 for an example). RED has two key phases: a pattern selection phase in which we select the pattern for a given road sign class and a training phase in which we train our classifier on these newly selected patterns. The result of RED is a classifier that is robust to a wide array of patch-based attacks (without the need to simulate those attacks directly).

The key insight to our pattern selection is that road signs are manufactured objects, and their true label $y$ is known at manufacture time. Thus, we will seek to modify the road signs at manufacture time to contain a high level of class specific information, making them easier to detect and, more importantly, harder to attack. More formally, for each class $y$, let $\alpha_{y}$ represent the pattern on road signs of class $y$ (e.g., when $y$ is the class stop signs, the current design of $\alpha_{y}$ is a red background).

We employ image ablation defense at inference time, with the key advantage that no adversarial training is required to detect the adversarial patch. An ablation algorithm g masks image X, leaving only a subregion of unmasked pixels (see Figure 4). Several works (Xiang et al. [2020, 2021], Levine and Feizi [2020]) have explored this approach, differing mainly in ablation size and strategies for removing adversarial patches when specific attack information is available. Most of these defenses, however, are effective only for small patches, as they remove only a small portion of the image; when the patch covers more than 10%, as shown in Xiang et al. [2020, 2021], they tend to fail, limiting their effectiveness as a universal defense. In contrast, Levine and Feizi [2020] propose a technique that defends against patches of various sizes by applying a simple ablation technique that removes most of the image, preserving only a small portion for inference to avoid the patch. In this paper, we empirically demonstrate the robustness of redesigned road signs using this generic method Levine and Feizi [2020] to defend against both small and large adversarial patches. However, our road signs are not tied to any specific defense. In the extended version of this paper, we will present the effectiveness of these redesigned signs using various image sanitizing techniques like Xiang et al. [2020] and Xiang et al. [2022].

At inference time, we apply several different ablation algorithms $g_{1},\ldots g_{m}$, to image $X_{\alpha}$, producing masked images $g_{1}(X_{\alpha})\ldots g_{m}(X_{\alpha})$. The classifier then makes a prediction on each ablated image, and the final prediction is obtained via majority vote:

RED selects $\alpha_{y}$ such that $X_{\alpha_{y}}$ does contain enough class-specific information via Algorithm 1.

Training Given any image ablation algorithm $g$, a dataset with $N$ examples, and $N$ classes of road signs, our goal is to find a robust background $\alpha_{y}$ for each class $y$. We use gradient methods to find $\alpha_{y}$; we parameterize the background of each sign to be a colored grid (see Figure 1) where $\alpha_{y}$ gives the color of each element of the grid. We then minimize the classification loss with respect to color parameters $\alpha_{1},\ldots,\alpha_{N}$, i.e.,

Figure 4 of the Appendix illustrates the training process.

Our work on Robust Environmental Design (RED) enhances the robustness of visual recognition systems, particularly for self-driving cars, by improving the resilience of road signs against adversarial attacks. This contributes to the safety and reliability of autonomous navigation technologies as they integrate into society. The societal impact is significant, as our research reduces the risk of adversarial misclassifications, which could lead to traffic accidents or system failures. This directly improves public safety by strengthening the reliability of critical systems in urban and rural environments.

Our work addresses the challenge of ensuring AI systems are both performant and resistant to manipulation. By developing defenses against adversarial attacks, we contribute to more secure and fair AI technology deployment. With this in mind, we note that although RED significantly improves robustness, it requires the ability to edit objects (e.g., selecting the pattern applied to road signs at manufacture time). This may not be feasible for all objects (e.g., pedestrians, wild animals, plants, etc.).