Quantum Federated Learning with Entanglement Controlled Circuits and Superposition Coding

Recent advances in quantum computing hardware and algorithms have recently lead to the emergence of quantum machine learning (ML) [2, 3, 4]. As opposed to classical computation at a linear scale in bits, quantum computing can perform calculations at an exponential scale in qubits [5]. The main enablers are the stochastic nature and the entanglement phenomenon of qubits, allowing one to make each qubit represent superimposed multiple states and to simultaneously control multiple qubits, respectively. Consequently, even in the current era of noisy intermediate scale quantum (NISQ) [6], i.e., with 50 to a few hundred qubits, quantum ML has achieved linear or sublinear complexity in various applications, as compared with the polynomial complexity of classical ML [7].

Quantum ML has recently established its standard framework. As analogous to the neural network (NN) of classical ML, the parameterized quantum circuit (PQC), also known as the quantum NN (QNN), has become a de facto standard quantum ML architecture [7, 8]. In a PQC, qubits flow through the gates associated with trainable classical parameters, during which the states of the qubits can be adjusted. For various applications ranging from image classification [9] to reinforcement learning [10], with a much smaller number of trainable parameters, PQC training has achieved the prediction accuracy on par with the neural network (NN) of classical ML.

Focusing on the parameter efficiency of PQCs, by integrating federated learning (FL) [11, 12, 13, 14] into standalone quantum ML, quantum FL (QFL) has recently attracted attention [15, 16]. Without communicating qubits via costly quantum communications, QFL enables distributed quantum ML at scale by communicating the PQC’s trainable parameters via classical communications, even over wireless channels [17]. This is not in the distant future, but is an upcoming application, especially considering the ever-increasing pace of innovation in quantum computers, e.g., IBM’s development roadmap planning to implement a 1K-qubit beyond-NISQ computer in 2023 [18] and a 100K-qubit computer in 2026 [19].

Motivated by this trend in QFL, the overarching goal of this article is to develop a communication-efficient QFL framework that can cope with heterogeneous and time-varying channel conditions and computing resources. To this end, we first revisit slimmable FL (SFL) in classical ML [1], wherein each device has a width-controllable local model, known as a slimmable NN (SNN) [20, 21], and communicates its superposition-coded local model with different width levels, enabling multi-level local information exchanges depending on channel conditions. Inspired from this, as visualized in Fig. 1, we propose an entangled slimmable quantum FL (eSQFL) framework with entangled slimmable QNNs (eSQNNs), which is a non-trivial extension from SFL with SNNs to their quantum versions as summarized later.

Unlike multi-width SNNs, the eSQNN is a multi-depth PQC wherein more depth levels incur higher von Neumann entanglement entropy on average. Unfortunately, the PQC trainability is often challenged by the problem of vanishing all gradients, known as the barren plateaus [22], which is exacerbated under higher entanglement entropy [23]. Meanwhile, too low entanglement may negate the benefit of quantum ML. To resolve this issue for an unknown target degree of entanglement, our proposed eSQNN entangles different qubits using the controlled universal (CU) quantum gates [24] such that the degree of entanglement is trainable.

Next, simultaneous local training of the multiple eSQNN depths may induce inter-depth interference, hindering convergence. In classical ML, SFL avoids its similar inter-width interference issue by adding the inplace knowledge distillation (IPKD) regularizer that penalizes the output difference from any smaller width to the largest width level [20]. Since the IPKD uses the Kullback-Leibler (KL) divergence, it becomes less accurate (or even diverging) for the larger differences. Alternatively, leveraging the Uhlmann’s fidelity function in quantum information theory [25], we propose a novel inplace fidelity distillation (IPFD) regularizer that is bounded within 0 and 1 while accurately measuring the quantum state difference even between the smallest and the largest levels.

Finally, for communication efficiency, the eSQNN parameters in multiple depths are superposition-coded and transmitted with a different transmit power allocation to each depth. Like SFL, the transmit power is optimized by deriving and minimizing the convergence bound of the eSQFL. Nevertheless, the convergence analysis is completely different since the gradient in PQC training is measured in a quantum computing way using the parameter shift rule [26].

Not only by analysis but also by extensive simulations, we corroborate that the proposed eSQFL with eSQNNs achieves convergence while each different width level can be trained to be a separate model with reasonable accuracy and fidelity, under various channel conditions as well as independent and identically distributed (IID) or non-IID data distributions. Note that unlike eSQFL, Vanilla quantum FL having fixed local PQC architectures cannot cope with different channel conditions [15, 27]. A recent work [28] also considers a slimmable architecture in the context of QFL. However, it does not theoretically guarantee convergence, and its specific architecture (i.e., angle/pole parameters) only allows two-level superposition coding, as opposed to generalized multi-level architectures using CU gates in eSQFL.

The major contributions of the work in this paper can be summarized as follows.

• •

  A multi-depth QNN architecture with CU gates, i.e., eSQNN, is proposed to enable superposition-coded transmissions while avoiding barren plateaus. We measure von Neumann entropy between inter-quantum states of different depths. Indeed, CU gates increase the trainability when designing multi-depth QNN.

• •

  A local eSQNN training algorithm with a fidelity-inspired regularizer, i.e., IPFD, is proposed in order to mitigate inter-depth interference. In eSQNN training, the proposed IPFD in this paper shows the crucial role.

• •

  With eSQNNs and IPFD, a novel quantum FL framework, i.e., eSQFL, is proposed, and its convergence bound is theoretically derived.

• •

  Based on the derived convergence bound, transmit power allocation in superposition coding is optimized. In addition, we corroborate that the derived convergence bound helps eSQFL achieve high accuracy.

The rest of this paper is organized as follows. Sec. II presents the related work to the proposed quantum federated learning. Sec. III introduces the eSQNN architecture and its local training with IPFD regularizer. Sec. IV describes superposition coding, successive decoding, and the proposed eSQFL framework. Sec. V provides the convergence analysis on eSQFL and its insight. Sec. VI presents the numerical experimental results to corroborate eSQFL empirically. Lastly, Sec. VII concludes this paper. Notice that the notations in this paper are in Tab. III.

The output of the PQC is the entangled quantum state that can be measured after applying a projection matrix $M\in\mathcal{M}\equiv\{M_{1},\cdots,M_{c},\cdots,M_{C}\}$ onto the reference $z$-axis. The measured output $\langle V\rangle_{\bm{\theta}}\in[-1,1]^{C}$ is called an observable, where $C$ denotes the output dimension. The operation of QNN corresponding to $c$-th observable is as follows,

where $(\cdot)^{\dagger}$ denotes the complex conjugate operator. Using the observable, a given loss function is calculated. Unlike classical NNs having visible activations in their hidden layers, the quantum states within QNNs are not measurable; otherwise, the quantum states collapse [29]. This does not allows quantum ML to compute the loss gradients via the chain rule, i.e., backpropagations. Alternatively, quantum ML evaluates the gradients using the zero-th order method called the parameter shift rule [26] (see Appendix -A).

Federated learning (FL) is a machine learning (ML) architecture made up of a server, local devices, and a global model [12]. The server transmits the global model to all the local devices, and each device produces local parameters by training the received global model. Then, these parameters are sent back to the central server, where all the data is aggregated to update the global model. Finally, the updated global model is transmitted to the local devices again for another iteration. Due to this mechanism, FL allows a large number of devices to learn a global model simultaneously without transmitting any data, ensuring data privacy as well. Considering the recent increase in the number and computational power of edge devices, FL is an extremely useful tool for reducing computational overhead and protecting data security which is both emerging challenges in the field of ML [33]. Within this architecture, various techniques with differing methods of aggregating data and training the global model exist, e.g., FedAvg [34], FedBN [35].

The convergence analysis of FL algorithms is especially challenging because of the data heterogeneity in FL, which forces researchers to rely on copious numbers of assumptions. Consequently, gaps in the understanding of FL analysis occur. Over the years, many major works have attempted to better understand FL by removing assumptions and exploring new techniques [36, 37, 34, 38, 39]. Even now, research on FL convergence in various aspects is still being carried out (i.e., non-convex, convergence bounds). For example, [40] has successfully proposed an analysis of local stochastic gradient descent (SGD) using only arbitrarily heterogeneous data while also using weaker assumptions than previous works. On the other hand, convergence analysis of most QFL algorithms has not been fully developed yet. This paper aims to further discuss the convergence analysis of QFL via the analysis of eSQFL with the characteristic of quantum computing. The convergence analysis on a dynamic QFL is elaborated in Sec. IV.

SFL is a framework that executes FL by using slimmable neural networks (SNNs) with SC and SD [1]. The architectural properties of SNN reduce memory costs of SFL [20] while with rigorous communication and computational efficiencies. SC is a process of compressing two different data signals into one signal. As the signals are encoded, different power levels are assigned to each data signal which is used to decide the priority of signals during SD. Additionally, the SNN is composed of the left-hand (LH) and the right-hand (RH) sides. The LH side is occupied by the high priority signal, while the lower priority signal goes to the RH side. After SC is finished, the encoded message will be uploaded to the server, which then undergoes SD. Assuming that the state of the communication channel is good, the LH of the SNN will be decoded first, followed by the RH signal. However, if the communication channel is not stable enough, only LH will be decoded, resulting in a small model. Finally, if the communication channel is completely unstable, no signal will be obtained. This flexible characteristic of SNN allows SFL to be extremely adaptable to dynamic communication environments, making it suitable for practical applications.

In this section, the concept of QFL is elaborated in depth. QFL is implementing FL via quantum computation by replacing all the NNs with QNNs. Chen et al., [15] is the first to propose a hybrid quantum-classical QFL architecture where the local devices are replaced with quantum devices, unlike FL models. After receiving global model parameters, the quantum computers carry out QML using QNNs. Then, the output of each device is aggregated to update the global model before repeating the process. In Chehimi et al., [27], a purely QFL framework is proposed. Similar to [15], this model is composed of a server and multiple quantum devices. However, instead of converting classic data into quantum states, the local devices generate quantum data by labeling qubits as excited or not excited according to the degree of rotation on the Bloch sphere. Both [15, 27] use FedAvg to aggregate data and execute training. As seen from the two examples above, a QFL and FL share an identical system structure, but QFL leverages QML instead of ML in order to exploit the advantages of quantum computing. For this work, Vanilla QFL is referring to a purely quantum version of [15]. In addition, quantum application of SFL is studied to improve the communication opportunities [28]. SQFL utilized trainable measurement parameters to configure two messages which contain both the trainable measurement parameters and PQC parameters, respectively. However, in this work, multiple layer architectures and local training algorithms are proposed, which are not present in [28].

The successful reception of a wireless signal is mainly affected by the signal-to-interference-plus-noise ratio (SINR) [45]. At a receiver, SINR can be expressed as,

where $P$, $P^{I}$, $d$, and $\sigma^{2}$ denote the transmission interference, reception interference, a transmitter-receiver distance, and noise powers, respectively. In addition, $\beta\geq 2$ is a path loss exponent and $\chi$ is small-scale fading parameter (i.e., Rayleigh fading). Following the Shannon’s capacity formula with a Gaussian codebook, the received throughput $R$ with the bandwidth $W$ is $R=W\log_{2}(1+\gamma)$ (bits/sec). When the transmitter encodes raw data with a code rate $u$, its receiver successfully decodes the encoded data if $R>u$. Finally, the decoding success probability can be given as follows,

where $u^{\prime}=2^{\frac{u}{W}}-1$. Consider transmitting $L$ messages from a transmitter to a receiver simultaneously. Before transmission, these messages are SC-encoded [46], and the whole transmission power budget $P$ is allotted to the $l$-th message, with $P_{l}=\nu_{l}P$ transmission power for $l\in[1,L]$. Note that $\nu_{l}$ is an allocation variable such that $\nu_{l}>u^{\prime}\sum^{L}_{l^{\prime}=l+1}\nu_{l^{\prime}}$, $\sum^{L}_{l=1}\nu_{l}=1$, and $\forall\nu_{l}\geq 0$.

The SC-encoded message is meant to be sequentially decoded at the receiver by first decoding the strongest signal, then canceling out the decoded signal, and finally decoding the next strongest signal, i.e., SD, also known as successive interference cancellation [47, 48]. The small-scale fading parameter $\chi$ under Rayleigh fading follows an exponential distribution, i.e., $\chi\sim\exp(1)$. Assuming $l^{\prime}>l$, the receiver may gradually decode the $l$-th message while experiencing the remaining messages as interference $P_{l}^{I}$, i.e.,

for $l\leq L-1$. However, $P^{I}_{L}=0$ as there is no interference for the last message. Assume that $R_{l}$ represents the throughput of the $l$-th message. Then, the distribution of $R_{l}$ is given as,

where $\bar{\gamma}=\frac{Pd^{-\beta}}{\sigma^{2}}$ denotes the averaged signal-to-noise ratio (SNR). By using this result, the $l$-th message’s decoding success probability $p_{l}$ can be expressed as follows,

This section describes the operations of eSQFL. Algorithm 2 shows the eSQFL algorithm. First of all, local devices are trained with Algorithm 1. The power allocation is conducted to configure SC-encoded model parameters, i.e., $\bm{\nu}=\{\nu_{l}\}^{L}_{l=1}$ for the gradient $\{g^{k}_{t}\odot\Xi_{l}\}^{L}_{l=1}$ of the subdivided model configuration. After that, the local devices transmit their SC-encoded model parameters to the server. The server decodes the devices’ SC-encoded model parameters with SD. If the server receives at least one local gradient for every model configuration, the server aggregates; otherwise, no aggregation occurs. In the aggregation of sub-divided model configuration, FedAvg is utilized [34]. The updates of eSQFL will be explained later.

In order to analyze the convergence rate of eSQFL, the following assumptions are considered. Firstly, the local-side decoding is always successful (Algorithm 2, lines 12–13) because the server-side transmission power is higher than the uplink power. Secondly, $K$ is assumed to be big enough such that $|X_{l}|\approx Kp_{l}$, for all $l$. During the $t$-th communication round, the server builds the global model which can be expressed as follows,

The objective function of the global model and the local objective functions are denoted as $F$ and $\{F^{k}\}$ respectively. The bar notation $\bar{\cdot}$ is used for the averaged value over $\{\zeta_{t}^{k}\}$, and the superscript ^∗ is used to indicate the optimum. For mathematical amenability, we consider the following assumptions on $F$ and $\{F^{k}\}$, as used in [49].

According to [40], the metric for the non-IIDness of $\mathbf{Z}$ is given as follows,

In classical ML, the convergence of FedAvg has been analyzed by assuming bounded local gradients in [49]. Without such an unrealistic assumption, the convergence bound of SFL has been derived in [1]. In quantum ML, local gradients can be shown to be inherently bounded thanks to the bounded fidelity and the parameter shift rule computing quantum gradients [26]. Hence, rather than adopting the methods in [1], we first derive the local gradient bound, and then derive the convergence bound of eSQFL by following the steps [49]. The detailed proofs are deferred to Appendix, and only the results are presented as elaborated next.

Note that Lemmas 2 and 3 are different, in the sense that Lemma 2 focuses on the actual gradient, whereas Lemma 3 is related to data distributions. The convergence analysis utilizes Lemmas 1–3 and eSQFL convergence can be proven by [49].

Theorem 1 exhibits several insights of eSQFL as follows.

1. 1.

   Failure under extremely poor channels: Consider an extremely poor channel condition, where the server cannot receive $[l,L]$-th model configurations, i.e., $p_{l^{\prime}}\simeq 0,\forall l^{\prime}\in[l,L]$. In this case, the RHS of (28) diverges.

2. 2.

   Importance of successful reception: The optimal gap of eSQFL becomes smaller by increasing the communication opportunities. Consider a perfect channel condition, where the RHS of (28) is minimized. By optimizing the SC transmission, the optimality gap is reduced which is referred to Proposition 1 and Corollary 1.

3. 3.

   Other important metrics: The optimality gap is affected by the local iterations per communication round $E$, balance factor $\lambda$, and the number of layers $L$.

To corroborate the main analysis and hypothesis of this paper, the experiments are designed as follows:

• •

  From Sec. V-A, the derived convergence bound is highly affected by the decoding success probability and non-IIDness. To corroborate these results numerically, we compare the top-1 accuracy of eSQFL in various channel conditions and degrees of non-IIDness with Vanilla QFL (referred to Fig. 1(a)).

• •

  We investigate the advantage of CU gates that compose eSQNN by designing an experiment which measures entanglement entropy and top-1 accuracy of eSQNN and standard QNNs under the same conditions. Then, the two metrics are compared to demonstrate the advantage of CU gates.

• •

  The increased effectiveness of local training with IPFD compared to IPKD is proven. IPFD trains the local models by regularizing the fidelity of two quantum states. In contrast, IPKD trains local models by ensuring that the small model follows the large model via its prediction. The benchmark scheme comparing the fidelity and top-1 accuracy of IPFD and IPKD is designed.

• •

  According to Proposition 1 and Corollary 1, the convergence bound is minimized by optimal transmission power allocation. To corroborate this, we compare the optimal power allocation scheme to its random power allocation counterpart.

• •

  Finally, we conduct experiments by controlling various variables and assess their various impact on the performance.

For the experiment, eSQFL and Vanilla QFL are evaluated. eSQFL is the proposed model which leverages eSQNN. This specific QNN consists of three sub-models named ‘L1’, ‘L2’, and ‘L3’. In contrast, Vanilla QFL uses a standard QNN which is made up of basic quantum gates [7], and does not consider SC and SD [15]. Despite the difference in structure, both eSQNN and standard QNN use equivalent number of parameters. Moreover, we conduct ablation studies on our eSQNN by comparing it with Vanilla SQNN, a depth-controllable yet entanglement-fixed QNN. Since the performance of QFL suffers under a system with a large number of qubits, many QFL works use a simple dataset [15]. In this paper, the MNIST dataset is transformed into a simpler form: the dimension of MNIST data is reduced to $4\times 4$ by inter-area interpolation, and only four classes are used (i.e., 0, 1, 2 and 3) [50]. The four classes are represented with red, blue, black, and green respectively. In addition, Dirichlet distribution is used to investigate non-IIDness of data [51]. Fig. 3 shows the data distribution with the different values of the Dirichlet concentration ratio $\alpha$. Data with high Dirichlet concentration ratio (i.e., $\alpha=10$) is IID while data with low Dirichlet concentration ratio (i.e., $\alpha=0.1$) is non-IID.

To compare IPFD and IPKD, we initialize the parameter of eSQNN identically. The simulation parameters used in these numerical experiments are summarized in Tab. I.

According to Theorem 1, the data distribution affects the convergence bound of eSQFL. With non-IID data, the convergence bound is widened. As shown in Fig. 5, we test various Dirichlet concentration, i.e., $\alpha=\{0.1,1,10\}$. The overall performance of all comparison models decreases as $\alpha$ decreases. However, eSQFL shows robustness under non-IID data distribution. Vanilla QFL shows low top-1 accuracy under $\alpha=1$ and $\alpha=0.1$. In contrast, eSQFL maintains the top-1 accuracy of $52\%$ and $41\%$ under $\alpha=1.0$ and $\alpha=0.1$ respectively. From the results in Fig. 4 and Fig. 5, eSQFL is robust under various channel conditions and non-IID data distribution.

Parameter shift rule [26], one of the most known quantum gradient calculators, is utilized to train the model. Subsequently, the eSQNN is trained accordingly using the zeroth stochastic gradient descent algorithm, e.g., quantum natural gradient. Consider that eSQNN consists of $I$ trainable parameters, i.e., $\bm{\theta}^{k}_{t,e}=[\theta^{k}_{t,e,1},\cdots,\theta^{k}_{t,e,i},\cdots,\theta^{k}_{t,e,I}]$. Then, the partial derivative of $k$-th device’s $c$-th observable over parameter $\theta^{k}_{t,e,i}$ is given as follows,

where $\mathbf{e}_{i}$ denotes the $i$-th standard basis, and $\varepsilon\in(0,\pi/2]$. We calculate the loss gradient using (33).

The true label is class $c$ and the predictions and its derivative is canceled out due to the definition of cross-entropy. Then, the cross-entropy loss is simplified as follows,

Hereafter, we denote $\hat{y}_{c}=y^{k,l,c}_{t,e}$, and $p(\hat{y}_{c})=p(y^{k,l,c}_{t,e}|\bm{x})$. Let’s denote the partial derivative of cross-entropy loss and fidelity loss as,

By the triangle inequality, the partial derivative of (12) is bounded as follows,

We have

The bound of $G_{1}$ obtained as follows,

The former step is due to $\sum\limits^{C}_{c^{\prime}\geq 1,c^{\prime}\neq c}\hat{y}_{c^{\prime}}\leq\sum\limits^{C}_{c=1}\hat{y}_{c}$, and the latter step is because the gradient using parameter shift rule is bounded to $1$ [26]. The term $G_{2}$ and its bound are given as,

The former step is due to $|\langle\bm{\psi}^{k,L}_{t,e,\bm{x}}|\cdot\bm{\psi}^{k,l}_{t,e,\bm{x}}\rangle|^{2}\leq 1$, and the latter step is due to parameter shift rule. Substituting the bound of $G_{1}$ and $G_{2}$ into LHS of (37), we have the bound,

Calculate LHS of (37) for all $i\in[1,I]$, the loss gradient is obtained, and its gradient is bounded as,

Applying these results to $g^{k}_{t}$, we complete the proof.

We expand the global gradient $f_{t}$ as follows,

The first step is due to Jensen’s inequality, i.e.,

and the next step is due to Cauchy-Schwarz inequality, i.e., $\|X\odot\Xi\|^{2}\leq\|X\|^{2}$ [1]. Combining Lemma 1 and latter term of (46), we finalize the proof.

According to (20) and Assumption 3, the distance between $f_{t}$ and $\bar{f}_{t}$ is as,

This step is due to Jensen’s inequality. With Assumption 3, we have $\mathbb{E}\|g^{k}_{t}-\bar{g}^{k}_{t}\|^{2}\leq\sigma_{k}^{2}$. Combining these results finalizes the proof.

Using (20), the distance between $\Theta_{t+1}$ to the optimal is as,