Sample IDs Protection in Vertical Federated Learning

With the increasing concerns on data security and user privacy in machine learning, federated learning (FL) [17] becomes a promising solution to these challenges. Based on how sensitive data are distributed among various parties, FL can be classified into different categories [31], notable among which are vertical FL and horizontal FL. Different from horizontal FL where the data are partitioned by examples, vertical FL assumes that the data are partitioned by different features (including labels). A typical example of vertical FL is when an online media platform $A$ displays advertisements of an ecommerce company $B$ to its users and charges $B$ for each conversion (e.g., user clicking the ad and buying the product). In this case, both parties have different features for the same set of users: party $A$ has features on users’ media viewing records and party $B$ has the users’ product browsing records and conversion labels. The conversion labels are not available to party $A$ because users’ buying processes happen entirely on party $B$’s website/mobile app.

If both parties want to train a deep learning model jointly to predict the conversion rate based on feature-partitioned data, they could sign legal agreements to allow them to share the data with each other. However, due to privacy and business concerns, recently companies are unwilling to share their data with other parties. Split learning [14, 29] enables the joint training of deep learning models without sharing sensitive data by splitting the execution of a deep learning model between the parties on a layer-wise basis. In vanilla split learning, the party without labels (called the passive party) sends the computation results of the intermediate layer (called the cut layer) rather than the raw data to the party with labels (called active party). The active party then completes the rest of the forward step computation, computes the gradients based on the labels, and sends the gradients with respect to the cut layer back to the passive party. The passive party then completes the back propagation with the gradients of the cut layer using chain rule.

At first glance, the process of split learning seems private because only the intermediates computation results of the cut layer, rather than raw features or labels, are communicated between the two parties. However, such “gradient sharing” scheme has been shown to be vulnerable in the setting of horizontal FL: in a recent work of Zhu et al. [33], it is shown that the central server could recover raw features and labels of a device using model parameters and gradients shared by that device. We wonder whether the raw data (specifically the labels from the active party) can also be leaked by sharing gradients of the cut layer in the split learning (vertical FL) setting.

In this work, we first identify a simple but accurate method that uses the norm of the communicated gradients from the active party to uncover the labels in imbalanced datasets. We then discuss several protection techniques to mitigate this issue. Among them, we have designed a principled approach that directly maximizes the worst-case error of label detection. This theoretically justified technique strategically perturbs the gradient randomly before communication. We experimentally demonstrate the effectiveness and competitiveness of the proposed protection method compared to other baselines.

In this paper, we first propose a more general approach than iDLG to extract the ground-truth labels from the shared gradients w.r.t hidden layers of any NN with cross-entropy loss over one-hot labels. By deviation, we demonstrated that the gradient norm of positive instance are larger than the negative ones’. With this rule, we can uncover the label of instances by ranking the corresponding gradient norms in a mini-batch and select the top ones as the positive instances.

The norm attack described above puts a server challenge to the current vertical FL and some split learning settings. We then propose to use gradient perturbation strategy to prevent label leakage. The amount of the perturbation is determined by minimizing the symmetrized KL divergence between perturbated gradients of positive and negative instances. To avoid add unlimited noise to the gradients and make the model converge, some constraints are added to the minimization problem. An approximate theoretical guarantee is achieved for this protection strategy. Extensive experiments demonstrated that we can achieve a reasonable trade-off between model performance and protection quality.

Uncovering the raw data. Even though raw data are not shared across different parties in federated learning, they are still not secure when gradients and model parameters are shared among different parties. In the horizontal FL setting, Zhu et al. [33] showed that an honest-but-curious server can uncover the raw features and labels of a participating device by knowing the model architecture, parameters, and the communicated gradient of the loss on the device’s data. Building upon the techniques in Zhu et al. [33], Zhao et al. [32] showed that the ground truth label of an example can be extracted by exploiting the relationship between the directions of the gradients of the weights connected to the logits of different classes. In this work, we study a different setting in FL — the two-party split learning setting for vertical FL [31], where no parties have access to the model architecture or model parameters on the other party’s side. We show one can recover the ground-truth labels (not the raw input features [30]) by using magnitude ($2$-norm) instead of the direction of the gradients.

Privacy protection methods. Because sharing intermediate computation results (model parameters, representations, gradients) might leak the raw data, FL communications will need to proceed in a privacy-preserving manner. There are generally three classes of approaches to communicate in a privacy-preserving manner: 1) cryptography-based methods such as Secure Multi-party Computation [9, 20, 6, 2] and homomorphic encryption[4, 25]; 2) system-based methods such as Trusted Execution Environments [26, 28]; 3) perturbation methods such as randomly perturbing the communicated message [1, 18], shuffling the messages [12, 8], reducing message’s data-precision, compressing and sparsifying the message [33], etc. Our work belongs to the third category where we add random perturbations to the gradients before communication to protect the labels. Many of the randomness-based privacy protection methods was proposed in the domain of horizontal FL where the aim is to protect the privacy of each example/ each device’s dataset [1, 13, 19, 18, 5, 27, 22, 24]. In this case, Differential privacy (DP) [10, 11] was used to quantitatively measure the proposed random mechanisms’ ability to protect the privacy of any individual records. To the best of our knowledge [16], in this paper, we introduce the first several randomness-based techniques to protect the active party’s label privacy with the goal of maximizing detection error during split learning in vertical FL.

In this paper, we identify a simple norm-based method to uncover the private labels of the active party using the communicated gradients in the two-party split learning problem for binary classification. We then propose a theoretically principled method to determine the best additive random perturbation to the gradients to reduce the probability of label leakage against a general class of label uncovering methods. We also conducted extensive experiments to demonstrate the effectiveness of our proposed protection methods.

We note that label leakage in the gradient-sharing scheme is a general phenomenon, as long as prior knowledge on the gradient distributions is known to the passive party. It remains open if the passive party can computational efficiently exploit this leakage when cryptographic scheme is applied in the learning protocol like in [3, 15]. Another open problem is whether there exist other methods to uncover the label information from the gradients of not necessarily imbalanced datasets and understanding how our method could protect against those attacks theoretically and empirically. Finally, our protection technique \marvell is designed to maximize the overall detection error. It would be interesting to see if similar ideas can be used to satisfy other protection requirements.

Acknowledgements We would like to express our thanks to the team members at ByteDance for providing the collaborative federated learning framework, Fedlearner⁴⁴4\urlhttps://github.com/bytedance/fedlearner. In addition, we would like to thank Liangchao Wu, Di Wu, and Xiaobing Liu for their discussions and supports. The code is availale at \urlhttps://github.com/bytedance/fedlearner/tree/master/example/privacy/label_protection.

By analyzing the KKT condition of this four variable problem we can find that the optimal solution must occur on the hyperplane $p\lambda_{1}^{(1)}+p(d-1)\lambda_{2}^{(1)}+(1-p)\lambda_{1}^{(0)}+(1-p)(d-1)\lambda_{2}^{(0)}=P$. Knowing from the proof above that in addition $\lambda_{2}^{(1)}=0$ if $u\geq v$ and $\lambda_{2}^{(0)}=0$ if $u<v$ further reduces this problem to a 3-variable problem. Keeping one of them fixed and optimizing the other two reduces the problem to a line search over a convex objective. Alternatingly fixing one of these three variables while optimizing the other two would eventually converge and give us the optimal solution.