QI-DPFL: Quality-Aware and Incentive-Boosted Federated Learning with Differential Privacy
^††thanks: ^*Corresponding Author. ^††thanks: This work was supported by National Natural Science Foundation of China under Grant No. 62206320.

FL introduces an innovative decentralized machine learning paradigm in which a global model is collaboratively trained by tremendous geographically distributed clients with locally collected data. Each client engages in one or multiple epochs of mini-batch stochastic gradient descent (SGD), subsequently transmitting the updated local model to a central server for the local model aggregation and global model update. Then, the central server dispatches the updated global parameter to all clients to trigger a fresh cycle of local training until the global model converges or reaches a predefined maximum iteration.

Given $N$ clients participating in FL training and each client $\small i\in\{1,2,\ldots,N\}$ utilizes its localized dataset $\small\mathcal{D}_{i}$ with a data size of $\small|\mathcal{D}_{i}|$ to contribute to model training. In each global iteration, each client parallelly performs $L$ ($\small L\!\geq\!1$) epochs of SGD training to update its local parameter:

where $\small\eta_{i}\!\in\!(0,1)$ is the learning rate of client $i$, $\small l\!\in\!\{1,2,\ldots,\\ L\}$ and we define the local model $\small\boldsymbol{w_{i}^{L}(t)}\!=\!\boldsymbol{w_{i}(t)}$. The central server averages the local models from $N$ participating clients to update the global model $\boldsymbol{w(t)}$:

The goal of FL is to find the optimal global parameter $\boldsymbol{w}^{*}$ to minimize the global loss function $\small F(\boldsymbol{w})$:

For the sake of theoretical analysis in the later content, we employ the following commonly used assumptions [17, 18].

For tackling the attacks such as gradient inverse attack [5] that might disclose the original training data without accessing the datasets, $\rho$-zero-concentrated differential privacy ($\rho$-$z$CDP) was proposed in [7], which attains a tight composition bound and is more suitable for analyzing the end-to-end privacy loss of iteration algorithms [19].

Firstly, we define a metric $\small\mathcal{L}_{p}$ to measure the privacy loss. Specifically, for a randomized mechanism $\small\mathcal{M}\!:\!\mathcal{X}\!\rightarrow\!\mathcal{E}$ with domain $\small\mathcal{X}$ and range $\small\mathcal{E}$ given any two adjacent datasets $\small\mathcal{D},\mathcal{D}^{{}^{\prime}}\\ \!\subseteq\!\mathcal{X}$ with the same size but only differ by one sample, after observing an output $o\in\mathbb{R}$, the privacy loss $\small\mathcal{L}_{p}$ is given as:

Then, the formal definition of $\rho$-$z$CDP is given as follows:

Based on the Gaussian mechanism, given a query function $\small Q\!:\!\mathcal{X}\!\!\rightarrow\!\mathcal{E}$, the sensitivity of the query function $\small Q$ is defined as $\small\!\Delta{Q}\!=\!\max_{\mathcal{D},\mathcal{D}^{\prime}}\!\!\|Q(\mathcal{D})-Q(\mathcal{D}^{\prime})\|_{2}$ for any two adjacent datasets $\small\mathcal{D},\mathcal{D}^{\prime}\!\subseteq\!\mathcal{X}$. Specifically, in the $t$-th global training iteration, by adding the artificial Gaussian noise $\small\boldsymbol{n_{i}(t)}\!\sim\!\mathcal{N}(0,\sigma_{i}^{2}(t))$, the transmitted parameter of client $i$ becomes:

where $\frac{\Delta{Q}^{2}}{2\sigma_{i}^{2}(t)}$-$z$CDP is satisfied with $\rho_{i}^{t}\!=\!\frac{\Delta{Q}^{2}}{2\sigma_{i}^{2}(t)}$[6]. Based on the definition of the query function, we can easily derive the upper bound of the sensitivity $\small\Delta{Q}$ as given in Corollary 1.

In $t$-th global training iteration, based on Corollary 1 and note that $\rho_{i}^{t}=\frac{\Delta{Q}^{2}}{2\sigma_{i}^{2}(t)}$, the variance of the Gaussian random noise $\sigma_{i}^{2}(t)$ of client $i$ can be easily derived as follows:

From the perspective of the central server, to minimize the cost while reaching the accuracy threshold, it is crucial to select the clients with superior data quality by a metric to quantify clients’ potential contributions to the FL system. To disclose the pertinent information about the local data while ensuring privacy preservation, we focus on the critical attribute of local data (i.e., data distribution) [20].

In FL, the data distribution varies owing to the distinct preferences of heterogeneous clients, leading to a non-independent and identically distributed (Non-IID) setting. The characteristic of Non-IID data dominantly affects the model performance, such as training accuracy [21]. Inspired by [20], the accuracy attenuation is significantly affected by the weight divergence, which can be quantified by the Earth Mover’s Distance (EMD) metric. The larger EMD value indicates higher weight divergence, thus damaging the global model quality.

In the first layer, we assume that there are a total of $H$ clients who are willing to participate in the FL model training process. Considering a $P$ class classification task that defines over a compact space $\small\mathcal{Y}$ and a label space $\small\mathcal{Z}$. The data sample $\small\mathcal{D}_{h}\!=\!\{\boldsymbol{x_{h}},y_{h}\}$ of client $\small h\!\in\!\{1,2,\ldots,H\}$ with $\small\boldsymbol{x_{h}}\in\mathcal{Y}$ and $\small y_{h}\in\mathcal{Z}$ follows the distribution $\small\mathbb{P}_{h}$. Under the premise of the actual distribution $\small\mathbb{P}_{a}$ for the whole population, we denote the EMD of $\small\mathcal{D}_{h}$ by $\small\theta_{h}$, which can be calculated as follows:

where the actual distribution $\mathbb{P}_{a}$ is a reference distribution and can be the public information or estimated according to the historical data. If the EMD value $\small\theta_{h}$ of client $\small h\!\in\!\{1,2,\ldots,H\}$ is larger than the pre-set threshold (i.e., $\small\theta_{h}\!\!>\!\!\theta_{\text{th}}$), client $h$ encounters a failure in executing the FL task.

Supposing that $N$ clients are selected by the central server. The interactions between the central server and selected clients are modeled as a two-stage Stackelberg game. Specifically, at Stage I, the central server, which acts as the leader, decides the optimal payment $\small\mathcal{R}_{t}^{*}$ in $t$-th global iteration to minimize its cost $\small\mathcal{U}_{T}$. Then, at Stage II, based on the reward allocated by the central server, each selected client $i\in\{1,2,\ldots,N\}$, which acts as the follower, maximizes its utility function $\small\mathcal{U}_{i}^{t}$ by determining the optimal privacy budget $\rho_{i}^{t}$.

We adopt a backward induction approach to derive the optimal strategy of the central server and each client respectively. First of all, we analyze the optimal strategy of each selected client by determining the optimal privacy budget $\rho_{i}^{t*}$ and are presented in the following theorem.

Based on the optimal privacy budget $\rho_{i}^{t*}$, we derive the central server’s optimal reward $\mathcal{R}_{t}^{*}$ as summarized in Theorem 2.

As shown in Theorem 2, the optimal reward $\mathcal{R}_{t}^{*}$ increases with $t$, indicating that the central server necessitates data of superior quality to attain the desired accuracy level when approaching the end of the time horizon $T$.

Theorem 2 reveals that the optimal privacy budget of the selected clients in Theorem 1 and optimal reward of the central server in Theorem 2 are mutually optimal, which leads to the steady state of the FL system. The overall framework process is summarized in Algorithm 1.

In this section, we will analyze the range of the reward under the Stackelberg Nash Equilibrium to guarantee the pre-set model accuracy while simultaneously minimizing the compensation provided to clients. Firstly, we introduce the additivity property of the privacy budget $\rho_{i}^{t}$ of each selected client $i$ as summarized in Lemma 1 [19].

According to Lemma 1, in the $t$-th global iteration, it is equivalent that the global model satisfies $\rho_{t}$-$z$CDP, where the global privacy budget $\rho_{t}$ equals the summation of each client’s privacy budget $\small\rho_{i}^{t}$ (i.e., $\small\rho_{t}\!=\!\sum_{i=1}^{N}\!\rho_{i}^{t}$). Naturally, the larger privacy budget usually results in better model performance. To quantify the global model accuracy as a function of the privacy budget, we acquire the test accuracy of the training model with different privacy budgets based on different real-world datasets (i.e., MNIST, Cifar10, and EMNIST datasets). Fig. 2 illustrates the MNIST and EMNIST datasets as an example. We observe that, in $t$-th global iteration, the global model accuracy $\epsilon_{t}$ can be regarded as a simplified concave function concerning the global privacy budget $\rho_{t}$ (i.e., $\epsilon_{t}\!=\!I(\rho_{t})\!=\!I_{1}\!-\!I_{2}\!\times\!e^{-I_{3}\rho_{t}-I_{4}}$), where $\small I_{k}$, $\small k\!\in\!\{1,2,3,4\}$ are corresponding constants.

Our proposed approach is implemented on three real-world datasets (i.e., MNIST [23], Cifar10 [24] and EMNIST [25]) to demonstrate the generalization of our framework. We adopt Dirichlet distribution [26] to partition the datasets into Non-IID by setting the concentration parameter as 1.0. In addition, three classic machine-learning models with different structures are implemented for each dataset. Specifically, for the MNIST dataset, we leverage a linear model with a fully connected layer of 784 input and 10 output channels. As to the Cifar10 dataset, the local training model is consistent with [21]. For the EMNIST dataset, we adopt a CNN similar to the structure of LeNet-5 [23]. The correlated basic dataset information detailed parameter settings are summarized in Table I.

In the experiments, we denote the approach without client selection and DP mechanism as FedAvg, the approach only performs client selection as FedAvg-select, the approach only considers DP mechanism as FedAvg-DP, and our proposed method with both client selection and DP mechanism as QI-DPFL. Moreover, to verify the efficiency and effectiveness of our proposed FL framework, we compare it with two baselines, named Max and Random, which is similar to the comparison paradigm in [27]. Specifically, Max means the central server chooses the largest possible reward value in each global iteration to guarantee the best performance. Random refers to selecting a random reward for clients incentivizing. Other incentive mechanisms such as [19] and [9] are designed based on different objectives, namely to maximize the utility of both clients and the central server, and maximize the profit of the model marketplace by designing an auction scenario for DPFL, respectively, which can’t be compared directly with our QI-DPFL framework. Thus, We exclude these two closely related methods from our comparative analysis.

Based on the above experimental results, it becomes evident that the advantages of our proposed QI-DPFL are particularly pronounced on the Non-IID datasets, which is attributed to the heightened effectiveness of the client selection mechanism.