Breaking the Black-Box: Confidence-Guided Model Inversion Attack for Distribution Shift

Problem Formulation. We define the target classification model as $M$, with $x$ representing the input image to the target model and $c$ denoting the target class for the attack. To obtain a synthesized image $x^{\ast}$ that reflects the private features of the target class $c$, we optimize the following loss function:

Here, $\mathcal{L}$ can be the cross-entropy loss or other suitable loss functions. The purpose of this loss is to directly optimize the image $x$ in order to leak the private training data of the model. As directly optimizing the high-dimensional vector $x$ is not efficient. The following section will delve into generative MIAs.

Generative Model Inversion Attacks. The idea of training a generative model as an image prior to optimize the latent vector $z$ in the GAN for image synthesis was first introduced by [41]. This approach addresses the problem discussed in [10], where directly optimizing $x$ in the high-dimensional, nonlinear, and non-convex solution space when attacking deep neural networks can lead to the generation of meaningless results. Their method involves training a DCGAN [28] on publicly available data that does not overlap with the private training data, and then optimizing the latent input vector of the GAN to attack a target model trained on private data. By introducing the image prior GAN, the optimization problem in Equation 1 can be expressed as:

After optimizing Equation 2 to obtain $z^{\ast}$, then input $z^{\ast}$ into the GAN[28] to generate the synthesized image $x^{\ast}=G(z^{\ast})$. This approach helps mitigate the issue of generating meaningless images to a certain extent.

In this section, we will present our approach, Confidence-Guided Model Inversion (CG-MI), for attacking different data distribution models in a black-box scenario. An overview of CG-MI is illustrated in Figure 2.

Pre-Trained Publicly Available Image Prior. In the architecture of a GAN[28, 3, 21], the generator model learns to map latent vectors sampled from a simple distribution (e.g., Gaussian or uniform distribution) denoted as $z$, to the generated image $x$. However, StyleGAN2[21] consists of two main components: $G_{mapping}$ and $G_{synthesis}$. In StyleGAN2, the latent vector $z$ is first transformed into a style vector w using a non-linear mapping network $f$, implemented as an $8$-layer Multi-Layer Perceptron (MLP). The style vector $w$ is then transformed into a synthesized image $x$. Specifically, a mapping network $G_{mapping}:Z\to W$, where $z\in Z$ and $z\sim\mathcal{N}(0,1)$, a synthesis network $G_{synthesis}:W\to X$, generating the corresponding image $x$ based on the input style vector $w\in W$. Previous works on PPA[34] have demonstrated the tremendous potential of StyleGAN2 across diverse data distributions. In our paper, while ensuring the independence between the generator model and the target model, we leverage a pre-trained publicly available StyleGAN2 model as our image prior to perform attacks between different data distributions.

Synthesis Image Transferability in MIAs. Consider two well-trained face recognition models, denoted as $M_{1}$ and $M_{2}$, trained on the same dataset $P(X_{target})$. Let $x^{*}$ be a synthetic image generated through an attack on $M_{1}$, classification result as $max(M_{1}(x^{*}))=c$. In the ideal scenario of transferability, the generated image $x^{*}$ satisfies both $max(M_{1}(x^{*}))=max(M_{2}(x^{*}))=c$. This indicates that both $M_{1}$ and $M_{2}$ are able to recognize $x^{*}$ and classify it into the target class $c$. Conversely, if $x^{*}$ lacks transferability, it may result in $max(M_{2}(x^{*}))\neq c$, leading to attack failure.

Enhancing Synthesized Images Transferability with Meaningful Exploration. In previous works, such as Brep-MI [18] and RLB-MI [12], the chosen image prior model was DCGAN[28]. However, the overall quality of the synthetic images generated by DCGAN is not satisfactory, as illustrated in Figure 3, thereby undermining the transferability of the attack. In the white-box PPA[34] work, StyleGAN2 [21] was selected as a replacement for DCGAN. They achieved success by optimizing the $w[bs,14,512]$ vectors, where $bs$ represents the batch size, by leveraging the target model’s weights and utilizing a gradient descent algorithm. The objective function for their method is shown in Equation 3.

where $G_{synthesis}$ is the synthesis network of StyleGAN2, $c$ is the target class for the attack, $M$ denotes the target model, $w$ is the style vector of the generative model. However, in a black-box scenario, the use of gradient-free optimization algorithms for the direct optimization of the equation above encounters several model inversion issues, leading to the generation of images lacking meaningful features. The main issue arises because gradient-free optimization algorithms focus solely on minimizing the loss function without considering the preservation of the underlying structure of the GAN latent space. Without constraints, they can deviate from the natural image space of the GAN, leading to synthesized images that receive high confidence scores from the target model while the evaluation model assigns them low confidence scores. Moreover, the high dimensionality of $w$ hampers the efficiency of gradient-free optimization algorithms.

To address the issue of generating images lack meaningful features and to reduce the dimensionality of the optimization variables, we focus on the $z$ vectors before they enter the input mapping network. Regardless of how $z$ is changed during the optimization process, the mapping network consistently maps $z$ to a meaningful latent space, thereby avoiding the generation of meaningless images. In comparision to the high dimensionality of $w$, $z$ has dimensions of $[bs,512]$. Specifically, with a pre-trained StyleGAN2, we aim to solve the following optimization problem:

where $z\in\mathbb{R}^{k}$, $\mathcal{L}_{Inv}$ is confidence matching loss, $G_{mapping}$ is the mapping network of stylegan2. By incorporating a mapping network into the optimization process and utilizing a gradient-free optimization algorithm, we have successfully transitioned the problem of solving the black-box MIA from the unrestricted latent space to a space characterized by meaningful facial features.

Confidence Matching Loss. The Confidence Matching Loss encourages the solver to find images that can reflect the characteristics of the private training data in the image prior’s latent space by minimizing the loss between the target model output confidence $M(x)_{c}$ and the label $c$. We explore the following confidence matching loss: (1) Cross-Entropy Loss[4]; (2) Max-Margin Loss[40]; and (3) Poincaré Loss[34]. We performed several comparisons in Table 4 and we final chose poincaré loss. For specific details of the loss functions, please refer to Appendix Appendix A.

Gradient-free Optimizer. The optimization problem in Equation 4 is a non-linear and non-convex problem. Choosing a suitable optimization algorithm is crucial for achieving good performance. In this study, we consider the Covariance Matrix Adaptation Evolution Strategy (CMA-ES)[13], a gradient-free optimization algorithm that is particularly well-suited for high-dimensional problems. CMA-ES is a variant of evolutionary strategies [6, 14] and utilizes an adaptive covariance matrix to optimize the probability distribution. We initiate the optimization process by inputting an initial latent vector z. After obtaining the optimized $z^{*}$, we can generate an image $x^{*}$ that reflects the private training data features with a target model class label of $c$ using $G_{synthesis}(G_{mapping}(z^{*}))$. We maintain the parameter settings as specified in [14] for the parameters in the algorithm. Please refer to Algorithm 1 for details.

Transformation-based Selection. Following [34], we utilize transformation-based selection to choose the images from the optimized population that best reflect the private training data. Let $x=\{x_{1},x_{2}\}$ denote the set of optimized synthetic images. We define a transformation operation $T$, which includes scaling, modifying aspect ratio, and random horizontal flipping. By applying $T$ to $x_{1}$ and $x_{2}$, we obtain transformed images, $T(x_{1})$ and $T(x_{2})$, which are then input to the target model, yielding new confidence scores, $l_{1}$ and $l_{2}$. Typically, if $x_{1}$ captures the target class features more effectively than $x_{2}$, the confidence scores after transformations satisfy $l_{1}>l_{2}$.

Hence, the image with the highest logit after transformations, denoted by $x_{1}$, is selected as the final image. In Equation 5, the Monte Carlo estimation method are employed, where $N$ represents the number of applied transformations.

Datasets. In our experiments, we focus on five datasets: CelebA[25], FFHQ[19], FaceScrub[26], AFHQ Dogs[5], Metfaces[20] and Stanford Dogs[23]. For the purpose of conducting attack evaluations, we trained our target models on CelebA, FaceScrub, and Stanford Dogs datasets. As a prior for image synthesis, we utilized pre-trained stylegan2 models on FFHQ, Metfaces and AFHQ Dogs datasets, enabling us to perform attacks on different data distributions.

Models. Our models are divided into target models and evaluation models. To facilitate a fair comparison, we conducted attack experiments on several popular network architectures, including Resnet[15] and DenseNet[17], while selecting Inception-V3[35] as the evaluation model. For the facial recognition task, we trained Resnet18, Resnet152, Densenet169, and InceptionV3 models on the CelebA and FaceScrub datasets, respectively. Similarly, for the dog breed classification task, we trained Resnet18, Resnet152, Densenet169, and InceptionV3 models on the Stanford dogs dataset. To facilitate comparison with prior work, we attacked the Resnet18 model trained on the CelebA and FaceScrub datasets for comparative experiments. The details of the data partition used for training the target model and the parameters for model training, the attack process, and comparative experiments can be referenced in Section B.1.

Evaluation Metrics. To align with prior work, we followed PPA[34] to calculate various evaluation metrics. Firstly, we trained an independent Inception-v3 evaluation model on the training data of the target model. Then, we used the evaluation model to predict labels on the generated attack results and computed the TOP-1 and TOP-5 accuracy for the target class.

Next, we computed the shortest feature distance from each generated image to any training sample in the target class and denoted the average distance as $\delta_{eval}$. The distance was measured using the squared L2 distance between activation layers in the penultimate layer of the evaluation model. For facial images, we utilized a pre-trained FaceNet model [32] to measure the feature distance $\delta_{Face}$. Lower values indicate that the attack results are visually closer to the training data.

The third metric is the FID (Fréchet Inception Distance) score [16]. FID calculates the distance between the feature vectors of the generated attack results and the training data of the target. The feature vectors are extracted using an Inception-v3 model trained on ImageNet [7]. A lower FID score indicates a higher similarity between the two datasets.

Comparision with Previous MIAs Approachs. We compared CG-MI with various MIAs methods in different scenarios, including white-box, black-box, and MIAs in the label-only setting. For white-box MIAs, we used PPA[34] as the baseline method. In contrast to previous white-box attack methods (such as [41, 4, 40]), PPA focuses on scenarios with different data distributions, image priors, and independence from the target model, making it more meaningful for comparison. For the black-box attack scenario, we selected RLB-MI[12] and Brep-MI[18] as baseline methods, representing state-of-the-art MIA methods in black-box attacks and the label-only setting, respectively.

To ensure fair comparison, we trained the Resnet18 model on the CelebA and Facescrub datasets for conducting comparative experiments using different MIAs methods. In our CG-MI method, we first ran the CMA-ES optimization algorithm multiple times for each class, generating a batch of 200 synthetic images. This approach ensured the stability and reliability of the results and reduced the impact of randomness. Next, we adopted a transformation selection strategy to choose the most representative 50 images from the optimized batch of 200 synthetic images for evaluation. For the other MIAs methods, we combined the characteristics of each attack method and ran it multiple times to generate a total of 200 synthetic images. Additionally, these methods also incorporate a transformation-based selection strategy to choose 50 images. We then used the same evaluation models and metrics to assess all the methods.

It is worth mentioning that RLB-MI and Brep-MI both employ the same GAN[28] structure, which is designed specifically for generating low-resolution 64x64 pixel images. To ensure a fair comparison, we made adjustments to the aforementioned methods by using a deeper GAN architecture capable of generating higher-resolution images[34]. Firstly, we added two additional upsampling blocks (consisting of a transpose convolution layer and a batch normalization layer) to the generator. We also expanded the discriminator by adding two convolution blocks, with each block consisting of a convolution layer and an instance normalization layer. Subsequently, we trained the modified generator on the FFHQ256 dataset to generate 256x256 pixel images. We then utilized this enhanced GAN as the image prior for RLB-MI and Brep-MI. By adapting the GAN architecture and training on higher-resolution images, we ensured that the comparison between CG-MI and RLB-MI/Brep-MI was carried out on a level playing field.

Table 1 presents the evaluation results of CG-MI and baseline methods in attacking Resnet18 trained on Celeba and Facescrub datasets. The target model Resnet18 achieves test accuracies of 86.38% and 94.22% on Celeba and Facescrub, respectively, while the evaluation model InceptionV3 achieves test accuracies of 93.28% and 96.20% on the same datasets. Based on the data in Table 1, CG-MI outperforms other black-box methods in addressing distribution shift issues in black-box attack scenarios. It generates more transferability synthetic images, resulting in higher attack success rates, lower feature distances, and FID values.

The qualitative evaluation results shown in Figure 4 demonstrate that compared to previous black-box MIA methods, CG-MI is capable of generating more realistic images in different data distribution scenarios. It overcomes the limitations of distribution shift and produces synthetic images of comparable quality to state-of-the-art white-box methods.

Performance Evaluation on various Models and Datasets. We also evaluated the performance of CG-MI on deeper network architectures and datasets from different categories. Specifically, we trained Resnet152 and Densenet169 models on the CelebA, Facescrub, and Stanford Dogs datasets. Additionally, we trained Resnet18 models on the Facescrub and CelebA datasets, with the recognition accuracy on the respective test datasets indicated in parentheses.

The objective of this experiment was to explore the potential for our method to generalize to other model architectures or deeper network structures. It also aimed to evaluate the performance of CG-MI in scenarios involving significant data distribution shifts. From the data presented in Table 2, it is evident that as the target model becomes structurally deeper, CG-MI encounters increased difficulty when attacking CelebA and Facescrub. Consequently, this results in a slight reduction in attack effectiveness. In settings with significant data distribution shifts, such as using Metfaces to attack Celeba and Facescrub, CG-MI still maintains a certain level of attack effectiveness. In Figure 5, the visual experimental results illustrate that CG-MI continues to produce meaningful attack results, even when dealing with deeper network architectures, different classification tasks, and scenarios involving significant data distribution shifts. This further reinforces the robustness and versatility of our proposed method across various model architectures and datasets.

Experiments with Various Gradient-free Optimizer. We compared different gradient-free optimization algorithms, namely CMA-ES[13] and BO[8], under the setting of $P(X_{prior})=FFHQ$ and $P(X_{target})=Facescrub$, with a target model architecture of Resnet18. Bayesian Optimization (BO) is a global optimization algorithm that utilizes probabilistic models to efficiently search for the optimal solution of an expensive black-box function.

In the ablation study, we considered two probability distributions: $P(X_{target})$, which represents the distribution of images from the CelebA dataset, and $P(X_{prior})$, which represents the distribution of images from the FFHQ dataset. We first compared three different loss functions for the model inversion attack: Poincaré loss[34], max-margin loss[40], and cross-entropy loss[4]. Poincaré loss achieved the best performance in terms of generating effective synthesis images.

The experiments involved also replacing the StyleGAN2 architecture with the DCGAN architecture combine our proposed attack CG-MI. Ablation study results indicated that CG-MI is compatible with other GAN architectures. Our findings demonstrate that, even when replacing StyleGAN2 with the DCGAN, CG-MI still yields favorable evaluation results. We also conducted ablation experiments by employing the objective function proposed in [34] in combination with the gradient-free optimization algorithm CMA-ES. From the experimental results(No Mapping), we observed that in the black-box scenario, where gradient information is unavailable, directly using the objective function from PPA did not yield favorable attack results. This underscores the importance of leveraging the mapping network as an integral part of the optimization process. By using our newly proposed objective function, we significantly improved our attack capabilities in the black-box scenario.

We also investigated the influence of transformation-based selection techniques[34] on the attack outcomes (No Trans. Selection). Transformation-based selection involves applying certain transformations to the synthesis images after the optimization process. By employing selection transformations, we can enhance the stability of attack outcomes and, to some extent, increase the success rate of attacks. The results of our ablation experiments highlighted the significance of using our proposed objective function for generating synthesis images.