Locally Differentially Private Distributed Deep Learning via Knowledge Distillation

Differential Privacy (DP) [19, 20] aims to protect the privacy of individuals while releasing aggregated information about the database, which prevents membership inference attacks [9] by adding randomness to the algorithm outcome. Two databases $D$ and $D^{\prime}$ are neighbors if they differ in only one entry. The formal definition is given as follows:

Local Differential Privacy (LDP) [13] is the local setting of DP, which does not require any trusted aggregator. In LDP, individuals (i.e., data owners) send their data to the data aggregator after privatizing data by perturbation. Hence, these techniques provide plausible deniability for individuals (i.e., data owners). Data aggregator collects all perturbed values and makes an estimation of statistics such as the frequency of each value in the population. The formal definition is given as follows:

Compared with DP, LDP provides more protection to the data owners. Other than sending the private data directly to a trusted aggregator, the data owners could perturb their private data with the mechanism that satisfies $\epsilon$-LDP, and then release the perturbed data. As such, LDP provides a stronger privacy protection, since the aggregator (i.e., data user) only receives the perturbed data and the true values of the private data never leave the hands of the data owners.

Knowledge Distillation (KD) [24, 14, 25] was originally designed for deep neural network (DNN) compression and knowledge transfer. KD usually considers a “teacher-student” paradigm, where the teacher model is a DNN (or an ensemble of a set of DNNs) that performs well on a given dataset, and the student model is another neural network that may or may not have the same architecture as the teacher model, but aims to mimic the performance of the teacher model(s) using another public dataset. Hinton, et al. [14] proposes an end-to-end knowledge distillation framework with a loss function, namely Distillation Loss, where the output of the teacher model is used as the soft target (i.e., soft label) for the student model, and the overall loss function is presented as below:

where $y$ is the true label of data $x$, $z_{s}$ is the output of the student model, $z_{t}$ is the output of the teacher model, $\sigma(z_{s};T=1))$ is the softened label of $z_{s}$ at temperature $T=1$, and $\sigma(\cdot;T=\tau))$ is the softened label at temperature $T=\tau$, and usually $\tau>1$.

In this work, we aim to develop a privacy-preserving distributed deep learning framework. As shown in Fig. 1, we consider the following problem: Given $L$ data owners, each data owner $l$ holds a set of private samples $(X^{l},Y^{l})$, where $X^{l}=\{x_{1}^{l},x_{2}^{l},\dots,x_{n^{l}}^{l}\}$, $Y^{l}=\{y_{1}^{l},y_{2}^{l},\dots,y_{n^{l}}^{l}\}$, $x_{i}^{l}\in\mathbb{R}^{d}$, and $y_{i}^{l}\in\{1,2,\dots,k\}$ is the label associated with sample $x_{i}^{l}$, $i=1,2,\dots,n^{l}$; the untrustworthy data user would like to learn a DNN model with the help of all the data owners, and a public dataset $X^{P}$ that comes from the same distribution (i.e., the same problem) as the data owners’ private datasets, but does not have the label information. In our problem setting, each data owner has two privacy requirements: (i) the value of the individual private data should not be shared to the data user, and (ii) any inference of the individual private data should be prevented from using the intermediate communication messages and the data user’s DNN model.

In our problem, we assume (i) the data user is untrustworthy, and (ii) the data owners are honest-but-curious, where each data owner follows the protocol honestly, but try to use the protocol transcripts to extract new information. We assume the value of the individual private data is what the adversaries would like to acquire during the whole protocol. Hence, the adversaries could be the data user, the participating data owners or an outside attacker that has the access to the intermediate communication messages or the data user’s DNN model. We also assume that the adversaries may have arbitrary background knowledge and might collude with each other. Our work aims to protect the privacy of each data owner’s individual private data while providing the reasonable utility to the data user’s DNN model. Since we assume that the data user is untrustworthy, it is of the data user’s own interest to correctly execute the algorithm or not. However, while using our proposed framework, if the untrustworthy data user behave dishonestly, it will not compromise data owner’s privacy, but will only hurt the utility of the data user’s DNN model. Furthermore, since the data owners are assumed to be honest-but-curious, the poisoning [26], backdoor [27, 28] or trojans [29] attacks (e.g., the data owner actively and maliciously modify their inputs to influence the performance of the data user’s DNN model) are beyond the scope of this work.

Our proposed privacy-preserving distributed algorithm framework, as shown in Fig. 2, consists of four stages that work synergistically between the data owners and the data user. Alg.1 shows the pseudo-code of our algorithm. Firstly, each data owner trains a teacher model using his/her own private dataset (i.e., lines 1-2), and the data user initialize the student models with random or pretrained (i.e., ImageNet [30]) parameters (i.e., line 3). The student model and all the teacher models do not have to use the same DNN architecture. Secondly, in each iteration, the data user efficiently selects a subset of the available public dataset (i.e., lines 5-6), that could better improve the performance of the current student model in the upcoming training, using our well-designed active query sampling approach. The active query sampling component aims to reduce the total number of queries to each teacher model, thus saving the privacy budgets. Thirdly, the data user uses the selected subset of the public data (no labels) to query each of the teacher model from its corresponding selected data owner to obtain the “knoweledge” (data’s soft label), and all the query results are sanitised by our local differential privacy techniques before being sent back to the data user (i.e., lines 7-15). Since the data owner might select a huge amount of query samples, it is not realistic to use all the selected samples to query all the data owners, which cost much on the privacy budge and the communication, but might not help a lot for the utility (per our experimental results). Therefore, we predefined a parameter $N_{Q}$ to control/specify the upper bound of the available data owners for each selected public sample to query (lines 10-11). Last but not least, the data user aggregates the received sanitised query results (i.e., the distilled knowledge) of each data, and leverage the knowledge distillation techniques (using the subset of the public data, and the distilled knowledge) to update/train the student model. The details of the most important three components in our framework (i.e, private query from teacher models, build student model via knowledge transfer and active query sampling) are described in the subsequent sections.

In our proposed algorithm, upon receiving the query data from the data user, each data owner evaluates it using his/her own teacher model and gets the data’s soft label. Each data owner perturbs the query data’s soft label (using LDP techniques) and then sends the perturbed value to the data user to transfer the distilled knowledge. The data user then aggregates the perturbed query results of each data to obtain the aggregated noisy soft label (i.e., averaged over the query results sent by all the selected data owners) of each data. As such, we could formulate this as a locally differentially private mean estimation problem, where we would like to protect the data owners’ private data from the inference attacks given the perturbed query results, and ensure the aggregated noisy soft label as close as the real value. As described in Section 2.1, while applying LDP, the adversaries could not distinguish the true value from a perturbed value with a high confidence (adjusted by the parameter $\epsilon$). To protect the privacy of the data owners’ private data, the randomization method which satisfies $\epsilon$-LDP is adopted. On the other hand, the performance of the aggregation of the perturbed data could be maintained with an error bound [23, 31], which provides us a way to control the utility of the distilled knowledge. Furthermore, since all the soft labels are multidimensional numerical values, different from the hard labels which are categorical values, we can not directly adopt the encoding-based LDP techniques [23].

To achieve our goal, we adopt the Piecewise Mechanism (PM) [13] that is designed to perturbed the multidimensional numerical values, and has an asymptotic optimal error bound for the mean estimation problem. Alg. 2 shows the PM for one-dimensional numerical data (i.e., PM-ONE). To simplify our explanation, in this section, the value to be perturbed (i.e., the soft labels of $k$ classes) is denoted as $z\in\mathbb{R}^{k}$, $z=[z_{1},z_{2},\dots,z_{k}]$. PM-ONE (Alg. 2) takes a one-dimentional numerical data $z_{i}\in[-1,1]$ as the input, and returns its perturbed value $z^{\prime}_{i}\in[-\Delta,\Delta]$, where $\Delta\leftarrow\frac{e^{\epsilon/2}+1}{e^{\epsilon/2}-1}$ is small and thus $z^{\prime}_{i}$ has relatively high probability (i.e., $\frac{e^{\epsilon/2}}{e^{\epsilon/2}+1}$) to be close to $z_{i}$. As shown in [13], while applying PM-ONE $|\frac{1}{n}\sum_{i=1}^{n}z_{i}^{\prime}-\frac{1}{n}\sum_{i=1}^{n}z_{i}|=O(\frac{\sqrt{log(1/\beta)}}{\epsilon\sqrt{n}})$ with at least $1-\beta$ probability for the task of mean estimation, which is an asymptotically optimal error bound.

Alg. 3 shows the PM for multidimensional numerical data (i.e., PM), where for each data of $k$ dimensions, it randomly selects $m$ (i.e., $m<k$) attributes to perturb. Alg. 3 is designed to reduce the amount of the noise in the task of mean estimation for multidimensional numerical data. While using Alg. 2 to perturb $k$ attributes, each attribute evenly shares a privacy budget of $\frac{\epsilon}{k}$, and the total amount of noise in the mean estimation is $O(\frac{k\sqrt{log(k)}}{\epsilon\sqrt{n}})$, which is super-liner to $k$. However, it has been shown [13] that while using Alg. 3, $E[max_{j\in[1,k]}|\frac{1}{n}\sum_{i=1}^{n}z_{i,j}^{\prime}-\frac{1}{n}\sum_{i=1}^{n}z_{i,j}|]=O(\frac{\sqrt{klog(k/\beta)}}{\epsilon\sqrt{n}})$ with at least $1-\beta$ probability for the task of mean estimation, which is still an asymptotically optimal error bound.

While the data user receiving all the query results from the data owners, our proposed framework uses the knowledge distillation technique to transfer the knowledge learned from the queried teacher models to the student model. Our usage of knowledge distillation is slightly different from the it convectional usage (as described in Section 2.2), where (i) we only focus on the knowledge transfer perspective of KD, but not the model compression, thus the student model and all the teacher models could use different and arbitrary DNN architectures; and (ii) in our case, the public dataset does not have the true label information, thus cannot directly use equation (3). Hence, in our framework, the student model is trained to minimize the gap between its own predicted soft label and the aggregated soft label from the teacher models, i.e., the knowledge distillation loss:

where $z_{s}$ is the soft label predicted by the student model, $z_{t}$ is the aggregated soft label from the teacher models, $T$ is the temperature parameter, and $\sigma(z;T)=softmax(z/T)$. The temperature parameter is usually set to 1. While $T>1$, the probabilities of the classes whose normal values are near zero would be increased. To better distill the knowledge to the student, two temperature values are adopted in our KD loss (i.e., $T=1$ and $T=\tau>1$).

As analyzed in Section 3.4.1, one direction to reduce the overall noise of the soft label estimation (thus, enhance the overall performance) is to decrease the size of the set of public data (i.e., $|X^{P}|$) utilized for private query. In this section, we present an active query sampling approach that could actively and adaptively choose samples from the public dataset batch-by-batch to query the teacher models. As shown in Fig. 3, we adopt a “least confidence” strategy [33], where in iteration, we attempt to select a set of query samples from the public dataset that the student model shows the “least confidence” at. To be specific, our active query sampling follows the procedure described below:

1. 1.

   Select an initial subset of $S$ unlabeled public data $X^{Q}$ uniformly at random from $X^{P}$. Update $X^{P}\leftarrow X^{P}-X^{Q}$.

2. 2.

   Use $X^{Q}$ to query the teacher models, and use the distilled knowledge to train the student initial student model $M_{S}$.

3. 3.

   For each available public data $x_{i}\in X_{P}$, evaluate it on the student model $M_{S}$. Let $P_{ij}$ denote the probability of $x_{i}$ belonging to class $j\in\{1,2,\dots,k\}$ predicted by $M_{S}$. Let $P_{i}=\{P_{i1},P_{i2},\dots,P_{ik}\}$, and suppose $\sum_{l=1}^{k}P_{il}=1$. Let $P_{i}^{*}$ be the largest value (posterior probability) in $P_{i}$. Then, repeat the procedure below for $S$ times to select $S$ query samples:

   $$\begin{split}x_{i}&\leftarrow X^{P}\\ P_{i}&\leftarrow M_{S}(x_{i})\\ X^{Q}&\leftarrow X^{Q}\cup\underset{x_{i}}{argmin}\frac{1}{m-1}\sum_{l=1}^{k}(P_{i}^{*}-P_{il})\\ X^{P}&\leftarrow X^{P}-X^{Q}\end{split}$$

   (5)

   Then, use $X^{Q}$ to query the teacher models, and use the distilled knowledge to train the student initial student model $M_{S}$.

4. 4.

   Repeat 3), until the student model meet the performance requirement or no more public data available (i.e., $X_{P}=\O$).

All the experiments were conducted on a PC with an Intel Core i9-7980XE processor, 128GB RAM, a Nvidia GeForce GTX 1080Ti graphic card, running 64-bit Ubuntu 18.04 LTS operating system. All the experiments are implemented using Python 3.7.

Three popular benchmark image datasets are utilized to conduct our experimental evaluation:

1. 1.

   CIFAR-10 [16]: CIFAR-10 (Canadian Institute For Advanced Research) is a widely used benchmark dataset to evaluate deep learning algorithms. This dataset is a subset of the 80 million tiny images dataset. It contains 60,000 32 x 32 color photographs of objects in 10 different classes, such as frogs, birds, cats, ships, etc. For each class, there are 6,000 images in total, where the testing set includes exactly 1,000 images that randomly selected from each class, and the training set contains the remaining 5,000 images in a random order.

2. 2.

   MNIST [17]: MNIST (Modified National Institute of Standards and Technology database) dataset is a collection of handwritten digits that is commonly used in the field of image processing and machine learning. This dataset is created by ”re-mixing” samples from the NIST dataset. It contains 70,000 28 x 28 grayscale images in 10 different classes, i.e., 10 digits, from 0 to 9. The handwritten digits have been size-normalized and centered in each images. The 70,000 samples have been separated to 60,000 training samples and 10,000 testing samples.

3. 3.

   Fashion-MNIST [18]: This dataset is a collection of Zalando’s article images, which is created as a drop-in (more challenging) replacement for MNIST to better represent modern computer vision tasks. It contains 70,000 28 x 28 gray-scale images in 10 different classes. Each class is a kind of cloth, such as T-shirt, dress, trouser, sneaker, etc. There are 60,000 training samples and 10,000 testing samples.

In our experiments, we assume the data owner’s teacher models are using ResNet50, and the data user’s student model is using ResNet18. For each experiment dataset, we assume each data owner has 4,000 private samples to train his/her teacher model (i.e., ResNet50). The data user would query 200 public samples in each iteration of the Active Query Sampling (AQS) process, and queries 5 iterations in total to train his/her student model (i.e., ResNet18). Each teacher and student model has been trained for 20 epochs with a batch size of 32.

As discussed in Section 3.4.1, there are three major parameters that we would like to tune and evaluate in our approach, such as the privacy budget ($\epsilon$), the number of queries of each public sample ($N_{Q}$) and the total number of participated data owners ($L$). We use various combinations of $\epsilon$, $N_{Q}$ and $L$ to evaluate our approach, where $\epsilon$ $\in$ $\{1$, $2$, $3$, $4$, $5$, $6$, $7$, $8$, $9$, $10\}$, $N_{Q}$ $\in$ $\{10$, $20$, $30$, $40$, $50$, $60$, $70$, $80$, $90$, $100\}$, and $L$ $\in$ $\{1,000$, $2,000$, $3,000$, $4,000$, $5,000$, $6,000$, $7,000$, $8,000$, $9,000$, $10,000\}$.

Then, we evaluate the performance of our approach while with and without the Active Query Sampling (AQS) process. Also, we evaluate the performance of our approach’s private query using three basic LDP mechanisms, including Piecewise mechanism [13], Duchi’s mechanism [22] and Laplace mechanism [34]. All the experiments have been repeated for 10 times and we take the average as the reported results.

In this section, we evaluate the effectiveness of our approach on three benchmark image datasets with different parameters and basic LDP mechanisms (as shown in Fig. 4, Fig. 5 and Fig. 6). We can observe that Piecewise mechanism always performs better than Duchi’s mechanism and Laplace mechanism in our framework. Moreover, the results with our AQS process consistently outperforms the ones without our AQS process, which demonstrates that our proposed AQS could dramatically save the privacy budget and prevent privacy budget exploding from happening in privacy-preserving distributed deep learning training.

In this section, we have compared our LDP-DL framework with 3 state-of-the-art approaches: DP-SGD [15], PATE [10] and DP-FL [12].

• •

  DP-SGD [15]: Differentially Private Stochastic Gradient Descent (DP-SGD) algorithm trains the deep neural network with differential privacy under a centralized setting. It utilizes the Gaussian mechanism on random subset of examples to produce average noisy gradient for model optimization. This approach does not have available code from public resource. Therefore, we directly refer the results presented in the original paper.

• •

  PATE [10]: Private Aggregation of Teacher Ensembles (PATE) proposed a distributed teacher-student framework. The privacy guarantee comes from the perturbation on teachers voting aggregation. The ensemble decision based on the noisy voting provides the label of student model’s training data. The student model is trained on semi-supervised learning with GANs. PATE [10] are evaluated on the code provided by the paper authors.

• •

  DP-FL [12]: Differentially Private Federated Learning (DL-FL) describes a federated optimization algorithm under private manner. Instead of directly averaging the distributed client models updates, an alter approach that use random sampling and Gaussian mechanism on sum of clients updates is introduced to approximate the averaging. The curator collects the noisy updates to optimize the center server model. Since the original paper aims at protecting the privacy at the client’s level but not at the sample’s level, DP-FL [12] are evaluated on the code published by the author with some minor changes to enable privacy preservation at the sample’s level.

Table I shows the results of different approaches under the same privacy budgets. The results of LDP-DL are evaluated under parameters $L=10,000$, $N_{Q}=30$. For the other approaches, we strictly follow the settings mentioned in the corresponding papers and keep the common parameters (such as the number of data owners (clients), privacy budgets) at the same level. Under the same level of privacy budgets, we can observe that LDP-DL consistently outperforms the other competitors. The improvement can be derived from the knowledge distillation and active query sampling. Knowledge distillation leverage richer information while transferring the knowledge from the teacher models to a student model. Meanwhile, active query sampling efficiently reduces the total number of queries from the data user (i.e., the student model) to the data owners (i.e., the teacher models). As such, the total cost of privacy budget is been reduced dramatically.