Adversarial Privacy-preserving Filter

With the increasing popularity of Online Social Networks(OSNs), privacy-preserving photo sharing has received considerable attention  (Xu et al., 2016; Such and Criado, 2016; Li et al., 2018). Existing attempts can be roughly categorized into preserving image metadata (Zhang et al., 2016), setting access control protocols (Klemperer et al., 2012) or sharing privacy policies (Such and Criado, 2016; Sun et al., 2018), and explicitly encrypting ROI region before uploading but decrypting after downloading (Xu et al., 2016; Sun et al., 2018). Different from these studies, the goal of this work is to hide the face identification information when sharing to OSN so that the potential image crawler and face cracker cannot use it for malicious usage.

In the last few years, it has been witnessed that the existing machine learning models, not just deep neural networks, are vulnerable to adversarial attack (Szegedy et al., 2014).  Szegedy et al. (2014) first introduced the problem of adversarial attack, and proposed a box-constrained L-BFGS method to find adversarial examples. To address the expensive computation,  Goodfellow et al. (2014b) proposed Fast Gradient Sign Method (FGSM) to generate adversarial examples by performing a single gradient step, which later becomes a widely-used baseline attack method.  Kurakin et al. (2016) extended this method to an iterative version (I-FGSM).  Dong et al. (2018) further improved adversarial examples by adopting momentum term (MI-FGSM).  Xie et al. (2019) proposed D$\rm I^{2}$-FGSM and M-D$\rm I^{2}$-FGSM by adopting input diversity strategy, which focused on improving the transferability and achieved state-of-the-art performance in black-box setting.

There exist many attempts to explore the adversarial attack problem in face recognition applications.  Sharif et al. (2016) and  Komkov et al. (2019) proposed face recognition attacks by modifying facial attributes like adding virtual eyeglasses to impersonate other subjects or prevent them from being recognized.  Dong et al. (2019) proposed a query-based method for generating adversarial faces in black-box setting, which requires at least $1,000$ queries to the face recognition system. Some other works, e.g., AdvFaces (Deb et al., 2019) focused on employing Generative Adversarial Network (GAN) (Goodfellow et al., 2014a) to craft new adversarial images. However, although the above face recognition attack solutions easily satisfy the privacy requirement mentioned in Section 1, they may violate either the utility or non-accessibility requirement: (1) Attack solutions like modifying facial attributes and employing GAN tend to introduce non-trivial perturbation and make the perturbed face images unsuitable for sharing. (2) The above solutions all request to access the original face images, which leaves privacy-leakage risk in the cloud. Different from the existing face recognition attack solutions, in this work, we propose an end-cloud collaborated adversarial attack solution to simultaneously satisfy the privacy, utility and non-accessibility requirements. Moreover, the solution is compatible to allow instantiation with most of the existing adversarial attack algorithms.

In addition to the above image-dependent adversarial attacks, Moosavi-Dezfooli et al. (Moosavi-Dezfooli et al., 2017) found a type of image-independent noise called universal adversarial perturbation (UAP), which can mislead the pre-trained model for different images. Following this, Poursaeed et al. (Poursaeed et al., 2018) trained a generative network for generating universal adversarial perturbations (GAP), and  Li et al. (2019) observed the property of regional homogeneity and generated regionally homogeneous perturbations (RHP). These studies demonstrate the possibility of attacking the classification of specific images with universal perturbation. Inspired by this, on the basis of image-specific perturbations, we further enhance the adversarial examples with universal adversarial perturbations in the cloud, with improved training convergence as well as attack performance verified in the later experiments.

Definition 1 (Adversarial Privacy-preserving Filter).

Given an original image $\mathbf{x}$, we assume that a face recognition model $f_{\theta}$ outputs an $l$-dimension vector as embedding feature $f_{\theta}(\mathbf{x})$. $\mathbf{x_{e}}$ represents the corresponding enrolled image, when the distance $d(f_{\theta}(x),f_{\theta}(x_{e}))$ between the two images is less than a certain threshold, they are judged to be the same subject, otherwise they are different subjects.

The main focus of this paper is to design the adversarial privacy-preserving filter (APF) that is an adversarial invisible noise $s$, making the adversarial example $\mathbf{x}+s$ look the same with $\mathbf{x}$ from the perspective of human observation but does not belong to the same subject from the perspective of face recognition model $f_{\theta}$. Under the condition of ensuring image quality, it preserves users’ portrait privacy without affecting their photo sharing experience.

The main notations of this work are summarized in Table 1.

The working flow of privacy-preserving filter is illustrated in Fig. 2, where three parts are involved as user end, privacy-preserving server cloud and photo sharing service. Given a portrait image, its image-specific gradient is first extracted in the user end, and then issued to the privacy-preserving server cloud for enhancement to derive adversarial noise. After perturbed by the enhanced adversarial noise, the resultant adversarial portrait image is expected to fool the potential face cracker on photo sharing service. It is noted that the original image is accessible only to user end during the whole process, preventing information leakage even on the privacy-preserving server.

The core solution is an end-cloud collaborated adversarial attack framework, consisting of three algorithm modules deployed respectively in the user end and server cloud: (1) Image-specific gradient generation is to obtain the image-specific gradient by employing a compressed probe model runnable in the end. (2) Adversarial gradient transfer module is to align the image gradient from probe model and the server model in the cloud, with the goal to recover the adversarial information fooling practical crackers. (3) Universal adversarial perturbation enhancement module is to append image-independent universal perturbation to further enhance the derived adversarial noise. The following will elaborate each module.

Following the non-accessibility requirement, users’ original face images should not be uploaded to the cloud server to avoid leakage. However, most of the existing adversarial attack algorithms are too complicated to be deployed in the end due to their high demands on computing resources. Therefore, our solution is to employ a compressed model in the user end to extract preliminary information from original image to mismatch the enrolled image, and then enhance the information in the cloud.

Specifically, we use the compressed model as probe model in the user end to extract image-specific adversarial gradient $g$. Noted that different adversarial attack algorithms are allowed, which lead to $g$ with different levels of intensity under the same maximum perturbation. In experiments we will discuss the influence of different adversarial gradient extraction algorithms. In this subsection, we employ FGSM as the example algorithm to introduce this process. Following standard adversarial attack operation, image-specific gradient $g$ for original image $\mathbf{x}$ is extracted by:

where $\mathbf{x_{e}}$ is the corresponding enrolled image, $\theta$ denotes the network parameters, $\epsilon$ ensures that the generated gradient is within the $\epsilon$-ball in the $L_{\infty}$ space, and sign($\cdot$) denotes the sign function. $L(\cdot)$ is the loss function that measures the distance between the original face image and the enrolled image:

where $f_{\theta}(\cdot)$ represents the embedding feature, $d(\cdot,\cdot)$ is distance function. We use Euclidian distance in our work.

The small-scale probe model extracts the image gradient in the user end for generating adversarial samples. However, since the structure and parameters of probe model in the end and server model in the cloud are different, the gradient cannot be used directly. To align the image gradient between probe model and server model, we propose the gradient transfer module $T$, which is defined as:

We consider the gradient transfer module as an image-to-image translation network. The U-Net architecture is widely used in image-to-image translation problem, which is an encoder-decoder network with skip connections between the encoder and the decoder (Ronneberger et al., 2015). We employ this architecture for transferring gradients from probe model to server model. Specifically, we train the gradient transfer module as the following optimization problem:

where $\tilde{g}$ is the gradient generated on server model following Eqn. (1) and $\theta_{T}$ represents the parameters of network $T$. This objective function enforces the network $T$ to learn the correlation between the probe model and server model.

Potential crackers may use multiple face recognition models, which is essentially a black-box adversarial attack problem. It is recognized that adversarial perturbation is transferable between models: if an adversarial image remains effective for multiple models, it is more likely to transfer to other models as well (Liu et al., 2017). Inspired by this, to improve the attack performance to unknown cracking models, we implement the gradient transfer module not only for one-to-one domain transfer but also for one-to-many.

Specifically, given $K$ white-box face recognition models with their corresponding gradient $\tilde{g}_{1}$, …, $\tilde{g}_{k}$, we re-formulate the loss function in Eqn. (4) by replacing $\tilde{g}$ with $\tilde{g}_{ensemble}$:

where $\alpha_{k}$ is the ensemble weight with $\sum_{k=1}^{K}\alpha_{k}=1$. For many face recognition models, the larger the value of $K$, the stronger the generalization capability of the derived adversarial images. However, an excessive $K$ value will lead to high computational complexity and trivial weight $\alpha_{k}$ to underemphasize single model. We select $K=2$ and evenly set $\alpha_{k}=1/2$ ²²2 Usually $\alpha_{k}=1/K$ except when prior is available to emphasize on some of the models.. In this study, we respectively select two of three state-of-the-art face recognition models as ensemble models for training and the remaining one as black-box model for testing. The performance of employing ensemble adversarial training to resist different face recognition models is reported in Section 4.3.1.

As claimed in previous studies (Moosavi-Dezfooli et al., 2017; Poursaeed et al., 2018; Li et al., 2019), universal adversarial perturbation contains image-independent information to mislead the classification of multiple images. This inspires us to integrate the image-specific information and image-independent information to enhance the performance of adversarial perturbation. Referring to the previous study (Moosavi-Dezfooli et al., 2017), universal adversarial perturbation is a vector $u$ that causes label change for most images sampled from the data distribution $\eta$:

where $\hat{C}$ is a classification function that outputs for each image $\mathbf{x}$ an estimated label $\hat{C}(\mathbf{x})$. The previous universal adversarial perturbation studies produce a universal perturbation with the goal to cause image misclassification (Moosavi-Dezfooli et al., 2017; Poursaeed et al., 2018; Li et al., 2019). Instead of directly misleading image classification, we expect such a universal adversarial perturbation $u$, that provides a fixed image-independent perturbation deviating the examined face image far away from other face images. Therefore, we derive the naive universal perturbation as follow:

where $n$ represents the number of all images contained in the dataset.

To integrate the image-specific perturbation and image-indepen-dent perturbation, we further design an enhancement module to adapt the domain of the universal perturbation $u$ with the domain of $\hat{g}$. The enhancement module $E$ consists of a scale transformation layer and a $1\times 1$ convolution layer, which is defined as follows:

where $\beta$ and $\gamma$ are trainable parameters and $\rm{conv(\cdot)}$ is the $1\times 1$ convolution layer. The overall optimization problem incorporating the two proposed modules is as follows:

where ${\rm{clip}}_{\epsilon}(\cdot)$ indicates clipping the input within the $\epsilon$-ball, $\theta_{T}$ denotes the parameters of network $T$, and $\theta_{E}$ denotes the parameters of network $E$.

Specifically, to employ ensemble adversarial attack, similar to Eqn. (5), given $K$ face recognition models with their corresponding embedding feature $f_{1},...,f_{k}$, we reformulate Eqn. (9) to derive the final objective function by replacing $f_{\theta}(\cdot)$ with $\tilde{f}(\cdot)$ defined as follows:

Network E and T are then trained by solving the above final objective function. The output $\hat{u}$ and $\hat{g}$ of E and T are added to derive $s$. The cloud will output $s$, and then return to user end to add it to the original image to derive the privacy preserved image, which can be safely shared to OSNs.