Robust Physical-World Attacks on Face Recognition

Let $\bm{x}$ be an input facial image and $\bm{x}^{a}$ be an anchor facial image, $f:\mathbb{R}^{m}\rightarrow\mathbb{R}^{d}$ be the face recognition model and $f(\bm{x})\in\mathbb{R}^{d}$ denotes the learned feature embedding of $\bm{x}$. The state-of-the-art face recognition model is generally realized based on the similarity between $f(\bm{x})$ and $f(\bm{x}^{a})$. There are two types of adversarial attacks on face recognition: dodging attacks that aim to reduce the similarity between facial images from the same identity and impersonation attacks that aim to increase the similarity between facial images from different identities. For dodging attacks where $\bm{x}$ and $\bm{x}^{a}$ are captured from the same identity, the optimization of adversarial sticker $\bm{\delta}$ can be formulated as

where $\mathcal{L}_{sim}$ denotes the attack loss, e.g., the cosine loss $\mathcal{L}_{sim}={(1+cos(f(\bm{x}+\bm{\delta}),f(\bm{x}^{a})))}/{2}$. In contrast, impersonation attacks, where $\bm{x}$ and $\bm{x}^{a}$ are sampled from two different identities, can be optimized by minimizing $\mathcal{L}_{sim}={(1-cos(f(\bm{x}+\bm{\delta}),f(\bm{x}^{a})))}/{2}$.

The physical-world condition variations are challenging when attacking face recognition. Firstly, there would be physical-world variations w.r.t. the sticker: 1) spatial constraints that the sticker cannot cover all parts of faces, such as facial organs; 2) inevitable deformation and position disturbance when wearing the sticker and fitting it to the real face. 3) chromatic aberration of the sticker caused by the printers and cameras. The sticker is first fabricated by printers and then wore and photographed by cameras for attacking face recognition. Due to the limitation of printer resolution and different shooting conditions of photographing, there would be the chromatic aberration of the sticker between the digital space and the physical world.

Secondly, there would be physical-world variations w.r.t. the adversarial face: 1) photographing variations containing camera angles, head poses, and lighting conditions, etc.; 2) internal facial variations of attackers, such as different facial expressions and movements.

In this section, we propose a robust physical attack framework on face recognition, dubbed PadvFace, which considers and models the challenging physical-world conditions. Specifically, we adopt a rectangular sticker $\bm{\delta}$ pasted on the forehead of an attacker without covering facial organs. The overall framework of the proposed PadvFace is illustrated in Fig. 1.

The rectangular sticker $\bm{\delta}$ is firstly fed into a Digital-to-Physical (D2P) module $f_{D2P}$ to model the chromatic aberration induced by printers and cameras. Then, a sticker transformation module $\mathcal{T}^{A}$ is introduced to simulate variations w.r.t. the sticker $\bm{\delta}$ when pasting on a real-world face. In the meantime, an initial mask $m$ is also fed into $\mathcal{T}^{A}$ by sharing the transformation with that on $\bm{\delta}$, generating the blending mask $M$. After these, the sticker is blended with a randomly selected facial image $\bm{x}$ according to the blending mask $M$, resulting in an initial adversarial image $\hat{\bm{x}}$. This initial adversarial image further inputs a transformation module $\mathcal{T}^{B}$ to simulate the environmental variations such as different poses and lighting conditions, leading to the ultimate adversarial facial image $\bm{x}^{adv}$ for deceiving face recognition, i.e.,

where $\bm{v}\sim\mathcal{N}(\mu,\sigma)$ is a random Gaussian noise and $M=\mathcal{T}^{A}(m)$, $\bm{x}$ is a randomly selected facial image. More details of each module are introduced as follows.

Digital-to-Physical (D2P) Module. There are two types of chromatic aberration: 1) the fabrication error induced by printers, which mainly refers to the color deviation between the digital color and its printed version due to the limitation of printer resolution; 2) photographing error caused by cameras, such as sensor noises and different lighting conditions. To alleviate these issues, we develop a Digital-to-Physical (D2P) module to simulate the chromatic aberration of the sticker inspired by (Xu et al. 2020). Specifically, the proposed D2P module is realized by training a multi-layer perception (MLP) to learn a 1:1 mapping from a customized digital color palette to its physically printed and photographed version, as illustrated in Fig. 2 (a-b), where Fig. 2 (c) presents the learned color palette from the proposed D2P module. Note that only parts of these color palettes are shown for the brief illustration, and the full color palettes and details of D2P are provided in Appendix.

Sticker Transformation Module $\mathcal{T}^{A}$ contains: 1) sticker deformation when pasting on a real-world face, including the off-plane bending and 3D rotations. Following (Komkov and Petiushko 2020), we adopt a parabolic transformation operator to simulate the off-plane bending and a 3D transformation for rotations; 2) position disturbance as it is hard to paste the sticker precisely at the same position as designed, such as random rotations and translations.

Face Transformation Module. As analyzed above, the attacker also undergoes a set of physical-world variations induced by different poses, lighting conditions, and internal facial variations, etc. To model these variations, we sample facial images from both physical and synthetic transformations. Specifically, for internal facial variations, we capture real-world facial images with different facial expressions and movements, leading to a set of facial images $\mathcal{S}_{\bm{x}}$. To simulate the variations induced by different camera angles, poses and lighting conditions, we consider transformation $\mathcal{T}^{B}$ that includes random rotation, scaling, translation, contrast and brightness on the adversarial facial images.

The overall physical-world variations considered by the proposed PadvFace are summarized in Table 1. To address these challenging variations, EOT algorithm (Athalye et al. 2018) is commonly used to implement robust physical attacks. Specifically, EOT first samples $n$ transformations $\mathcal{K}=\left\{(\bm{x}_{i},\tau^{A}_{i},\tau^{B}_{i},\bm{v}_{i})\right\}_{i=1}^{n}$ from $(\mathcal{S}_{\bm{x}},\mathcal{T}^{A},\mathcal{T}^{B},\bm{v})$ and then optimizes the following objective as

where we adopt $k_{i}\triangleq(\bm{x}_{i},\tau^{A}_{i},\tau^{B}_{i},\bm{v}_{i})$ and $\mathcal{L}_{sim,k_{i}}\triangleq\mathcal{L}_{sim}(f(\bm{x}^{adv}_{i}|\bm{x}_{i},\bm{\delta},M_{i},\tau^{A}_{i},\tau^{B}_{i},\bm{v}_{i}),f(\bm{x}^{a}))$ for shorthand. $\bm{x}^{adv}$ is the corresponding adversarial image generated by Eq. (2). $\mathcal{L}_{TV}(\bm{\delta})=\sum_{i,j}\left((\bm{\delta}_{i,j}-\bm{\delta}_{i+1,j})^{2}+(\bm{\delta}_{i,j}-\bm{\delta}_{i,j+1})^{2}\right)^{\frac{1}{2}}$ is the total-variation loss introduced to enhance the smoothness of the sticker and $\alpha>0$ is a regularization parameter. The D2P module and the synthetic transformations in $\mathcal{T}^{A}$ and $\mathcal{T}^{B}$ are all differentiable. And model (3) can be solved by stochastic gradient descent algorithm.

We have the following observations for model (3). Firstly, for generating an adversarial sticker, the attack complexity varies with different physical-world conditions. In Fig. 3, we show the dodging attack performance of a fixed sticker under various facial variations, illumination variations, and 100 random sampled transformations from $(\mathcal{T}^{A},\mathcal{T}^{B})$. And higher adversarial cosine similarity indicates lower attack performance and higher attack complexity. The results show that the difficulty of physical attacks varies with different conditions. Secondly, due to the high nonlinearity of DNNs, model (3) generally leads to a non-convex optimization problem. Hence, directly fitting the adversarial sticker to all kinds of physical-world conditions $\mathcal{K}$ in model (3) could make the optimization difficult, resulting in inefficient solutions.

In light of these, we propose an efficient curriculum adversarial attack (CAA) algorithm to gradually optimize adversarial stickers from easy to complex physical-world conditions. Given an adversarial sticker $\bm{\delta}$, larger attack loss of $\mathcal{L}_{sim,k_{i}}$ indicates higher attack complexity under the condition $k_{i}$. Thus, $\mathcal{L}_{sim,k_{i}}$ can serve as an appropriate surrogate for the measurement of the complexity of $k_{i}$. Based on this, we assign a learnable weight parameter ${p}_{i}\in[0,1]$ for each transformation in $\mathcal{K}$ and formulate the objective of CAA as

where $g(p_{i})=\frac{1}{2}p_{i}^{2}-p_{i}$ is a regularizer and $\lambda>0$ is a curriculum parameter.

Let $\bm{p}=[p_{1},p_{2},\cdots,p_{n}]$, model (4) can be solved by alternatively optimizing $\bm{\delta}$ and $\bm{p}$ while keeping one of them fixed. Firstly, given $\bm{p}$, the optimization w.r.t. $\bm{\delta}$ reduced to

which can be solved by stochastic gradient descent method. Secondly, given $\bm{\delta}$, the optimal $p_{i}^{*}$ is determined by

leading to a closed-form solution as $p_{i}^{*}=1-\frac{\mathcal{L}_{sim,k_{i}}}{\lambda}$. Thus, easier transformation with lower attack loss $\mathcal{L}_{sim}$ will be assigned with a larger weight $p$ and dominate the updating of $\bm{\delta}$ in the following step. On the other side, the value of $\lambda$ is monotonically increased to involve more and more complex transformations during the optimization. As a result, the proposed CAA algorithm can generate the sticker $\bm{\delta}$ from easy to complex transformations gradually. The overall algorithm of CAA is reported in Algorithm 1.

CAA adheres to the principle of curriculum learning (Bengio et al. 2009), which learns from easy to complex tasks and has shown the benefits in obtaining better local minima and superior generalization for many non-convex problems (Huang et al. 2020b; Kumar, Packer, and Koller 2010; Fan et al. 2017; Cai et al. 2018). To the best of our knowledge, we are the first to explore the complexity of physical-world conditions in adversarial attacks and to aggregate them via curriculum learning towards the robust performance.

The constructed testing protocol is shown in Figure 4, which contains two phases: (1) attack launching stage for collecting facial images and generating adversarial stickers, and (2) attack evaluation stage for evaluating attacking performance under various physical conditions.

Attack Launching Stage. For each experimenter, we first take 4 videos under the normal light with 4 different facial variations: happy, sad, neutral, and mouth-open. We use iPhone-12 to take 1920$\times$1080 resolution videos and each video lasts about 3 $\sim$ 5 seconds. The camera is placed in front of the experimenter with a distance of 50cm and the experimenter is asked to sit steady without pose variations. Then, we randomly sample a single frame for each video as $\mathcal{S}_{\bm{x}}$, which would be taken as inputs of attack models for generating robust adversarial stickers.

Attack Evaluation Stage. The stickers generated in the launching stage are firstly printed by Canon Generic Plus PCL6 and then wore to deceive face recognition systems. We consider complicated environmental variations for the evaluations of this stage, including (1) different head poses. Existing methods usually acquire this by asking the experimenter to make certain pose changes, which is hard to control and cannot lead to a fair comparison between different methods due to the poor repeatability. To alleviate this issue, we customize a cruciform rail to make accurate movements for reducing the potential effects by uncontrolled experimenter movements. The experimenter is asked to sit in front of the cruciform rail with a distance of 50cm, and the cruciform rail carries the camera and moves in four directions (up, down, left, and right) sequentially to capture facial images with different poses. As a result, we can obtain the facial images with fine-controlled pose variations. (2) different lighting conditions. To imitate the illumination variations, we select a room without any extra window and use a KN-18C annular light as the light source, which is also placed in front of the experimenter with a distance of 50cm. We select three different light intensities, refer to dark/normal/light respectively, and examples of face images under different lighting conditions are provided in Appendix. (3) facial variations. As considered in the attack launching stage, we also involve four facial variations: happy/sad/neutral/mouth-open.

For each experimenter with a certain sticker, we take six videos: (a) three videos under the dark/normal/light lighting conditions with the neutral expression, and (b) three videos with the happy/sad/mouth-open variations under the normal illumination. The movement ranges of the cruciform rail are kept same for each video, i.e., left-right angles $(-{{5}^{\circ}},+{{5}^{\circ}})$ and the up-down angles $(-{{2}^{\circ}},+{{2}^{\circ}})$. Afterwards, about 35 frames are captured from each video as the adversarial images $\bm{x}^{adv}$. For a better evaluation, we also collect six videos for each experimenter without any stickers, i.e., benign faces ${{\bm{x}}^{b}}$.

We take the ‘neutral’ facial image captured from the attack launching round as an anchor ${\bm{x}^{a}}$, and calculate the benign cosine similarity ${co{s_{b}}}=cos({{\bm{x}}^{b}},{\bm{x}^{a}})$ and the adversarial cosine similarity ${co{s_{adv}}}=cos({{\bm{x}}^{adv}},{\bm{x}^{a}})$. Instead of the attack success rate, taking the cosine similarity as the metric would lead to more precise evaluation without the effects caused by the threshold setting.

The statistics of the physical evaluation cases are presented in Table 2, where we also tabulate those of Advhat (Komkov and Petiushko 2020) for comparison. It is worth noting that the evaluation of physical attacks is much more challenging than that of digital attacks, making the number of experimenters or attack cases relatively small. Yet our evaluations are already on the largest scale compared to existing works.

Experimental Setting. We take the state-of-the-art ArcFace model^†††https://github.com/deepinsight/insightface/wiki/Model-Zoo trained on the large-scale dataset MS1MV2 as our attacked face recognition model, which has 99.77% benign recognition accuracy on the LFW benchmark (Huang et al. 2008). The default sticker size is 400$\times$900 in pixels, obtained 13cm$\times$5.8cm in the physical world. All experiments are conducted with the TensorFlow platform and a NVIDIA Tesla P40 GPU. The detailed setting of the proposed CAA algorithm is provided in Appendix.

In the following, we denote the proposed method that considers all physical variations with the CAA optimizer as PadvFace-$F$ and introduce its two variants: PadvFace-$B$ that does not involve facial variations in $\mathcal{S}_{\bm{x}}$ and illumination transformations in $\mathcal{T}^{B}$, and PadvFace-$S$ realized by further substituting the standard EOT optimizer for the CAA optimizer in PadvFace-$B$.

In this section, we evaluate the proposed PadvFace with dodging and impersonation attacks in the physical world. And we take Advhat (Komkov and Petiushko 2020) as the main baseline, which is the most comparative sticker-based attack method for our evaluations on large-scale face recognition. Since Advhat did not capture the internal facial variations and illumination variations during the optimization, we align this setting and adopt the PadvFace-$B$ for a fair comparison. Nevertheless, our PadvFace-$B$ still keeps the D2P module and CAA optimization algorithm, which are the two main differences in contrast to Advhat. In consequence, we evaluate both methods with the ‘neutral’ expression under the ‘normal’ light and Fig. 5 provides some attack examples. The numerical results of dodging and impersonation attacks on 10 experimenters are reported in Table 3, where $cos_{b}$ and $cos_{adv}$ denote the benign and the adversarial cosine similarity, respectively. For dodging attacks, lower $cos_{adv}$ denotes better performance, while for impersonation attacks, higher $cos_{adv}$ denotes better performance.

The results of dodging attacks are shown in the left of Table 3. Compared to the benign similarity without any sticker, our proposed method achieves significantly lower adversarial cosine similarities, demonstrating its superior dodging attack performance. For example, for ID=01, after wearing the adversarial sticker, the cosine similarity drops from 0.91 to 0.27, posing a serious threat to the physical-world face recognition system. Furthermore, compared with Advhat, we also obtain significant performance improvements under most cases. On the average performance of all 10 experimenters, our method achieves 0.31 adversarial cosine similarity while that of Advhat is only 0.38, leading to 18.4% relative improvement.

As for impersonation attacks, we randomly select a single victim anchor from 10 experimenters for each attacker, obtaining 10 ‘attacker$\rightarrow$victim’ pairs. The evaluations are shown in the right of Table 3. As shown in the Table, our proposed method can significantly increase the cosine similarities of two different identities by wearing the adversarial stickers under all cases. For instance, for the attacking pair (10 $\rightarrow 03$), the adversarial cosine similarity increases from 0.03 to 0.41. Furthermore, we also obtain better attack performance than Advhat on most attacking pairs. On the average of all 10 cases, our method leads to an average cosine similarity of 0.30 while that of Advhat is 0.24, the relative performance improvement over Advhat is 25%. In addition, compared with dodging attacks, impersonation attacks that aim to deceive a specific identity are generally more difficult.

We also conduct the paired t-test further and observe that our PadvFace-B is significantly better than Advhat with p=0.005 for both dodging and impersonation attacks. In summary, the superior performance of dodging and impersonation attacks on PadvFace-$B$ over Advhat demonstrates the effectiveness of the developed D2P module and CAA optimizer.

With the proposed standardized testing protocol, we can make more quantitative analyses of environmental variations. In this section, we analyze in-depth the impact of internal facial variations and illumination variations. We compare the performance of PadvFace-B and PadvFace-F under both dodging and impersonation attacks.

The evaluation results under different lighting conditions are given in Table 4, where all experimenters are asked to keep the neutral expression during the attack evaluation stage. We have the following observations: (1) the attacking performance varies under different illumination conditions, and (2) PadvFace-F that incorporates the illumination transformations in $\mathcal{T}^{B}$ consistently outperforms PadvFace-B that learned with a fixed normal illumination under all cases. Specifically, for ID=09, PadvFace-F has 12.5% performance improvement than PadvFace-B (0.52 vs. 0.45) for dodging attacks, and 19.9% performance improvement (0.34 vs. 0.41) for impersonation attacks.

The evaluations under internal facial variations are presented in Table 5, where all adversarial images are collected under the normal illumination. It can be observed that when attackers make different facial variations, the attack performance would vary as well. This point can be further verified by the superior performance of PadvFace-F over PadvFace-B, where PadvFace-F involves facial variations during the optimization but PadvFace-B learns only based on neutral expression. Specifically, for ID=09, PadvFace-F achieves 9.4% dodging attack performance improvement (0.53 vs. 0.48) and 13.0% impersonation attack performance improvement (0.31 vs. 0.35) compared with PadvFace-B. These experimental results demonstrate that considering internal facial variations and illumination variations during the attacking process can largely boost the robustness of learned adversarial stickers.

D2P Module. We utilize PadvFace-S for evaluating the effectiveness of the proposed D2P module. We randomly select three experimenters and conduct dodging attacks in physical world. The evaluation results are presented in Table 6, where ‘w/ D2P’ refers to PadvFace-S and ‘w/o D2P’ refers to the variant of PadvFace-S without the D2P module. The benign and adversarial images are captured under the ‘neutral’ expression and ‘normal’ illumination condition. As can be observed, the D2P module can significantly benefit the performance of physical attacks for all three experimenters, demonstrating the effectiveness of modeling the chromatic aberration induced by printers and cameras.

Inconspicuousness. For physical attacks, the inconspicuousness aims to make adversarial perturbations unnoticed. However, based on our experiments, attacking physical-world face recognition is intrinsically hard, and the current attack performance is far from satisfactory even without the inconspicuousness constraint. Thus, in this paper, we primarily focus on efficiently modeling the complicated physical-world conditions in attacking face recognition, and leaving the inconspicuousness of the adversarial stickers to future work.