Distributed Machine Learning in D2D-Enabled Heterogeneous Networks: Architectures, Performance, and Open Challenges

To reconcile the demand for ML model training and privacy protection, a straight idea is to conduct the model training by distributed data owners to avoid sharing raw data, which facilitates distributed ML architectures. Specifically, federated learning (FL) [4] and split learning (SL) [5] represent two bright peals. Specifically, these two learning mechanisms implement distributed ML from different perspectives, while their representative learning architectures are depicted in Fig. 1.

Training process of FL: A typical FL scheme engages a set of smart devices termed as clients to participate in the iterative model training. At the beginning of each iteration, each client receives a global model from a parameter server, then conducts local training to update the model by performing stochastic gradient descent on its local training data. After the completion of local training, all clients involved in FL upload the trained model parameters to the FL server in parallel (step 1). The FL server next aggregates (e.g., FedAvg $Avg(\cdot)$ ) the overall received model parameters into a new global model, which will be broadcasted to clients (step 2) for the following training round. Specifically, each client only exchanges the model parameters with the FL server, preventing privacy disclosure to some extent.

Training process of SL: To achieve SL training, an ML neural network is firstly split into two subnetworks via splitting in the middle layer (i.e., a split layer) of the network. Generally, the subnetwork associated with input layer is deployed on the clients’ side, and the one related with output layer will be deployed at an SL server. In each iteration, a client starts training by performing forward propagation; and then, the activation data, i.e., the output of split layer of the client (with label data), will be transmitted to the SL server (step 1). Upon receiving the activation data, the SL server proceeds with forward propagation within its subnetwork to derive output results, and subsequently initiates the back propagation process, which involves the computation of the loss by comparing the output results with the provided label data[5]. Similarly, the output of cut layer can be transmitted back to the client for subsequent back propagation (step 2). Finally, subnetworks on both the client and server can be updated respectively. By offloading the model parameter of the current client to the next client (step 3) and repeating the above steps, the model can be trained sequentially over multiple clients.

Interestingly, FL and SL share some common advantages, such as communication/computation cost reduction, and privacy preservation. However, the implementations of FL and SL in heterogeneous wireless networks can also encounter non-negligible challenges:

• •

  Network heterogeneity: The SL server can only interact with the clients sequentially, leading to a low level of parallelization and may hinder the corresponding convergence speed. For FL, frequent model reporting from clients to the server over unstable wireless links usually result in model performance degradation due to transmission failures. Besides, since the implementation of conventional FL and SL generally relies on a star topology, where a central server coordinates the model update of all the clients. Thus, FL and SL may suffer from unsatisfying scalability and performance degradation in large-scale D2D-enabled heterogeneous networks due to single point failures[3]. For example, under a space-air-ground-ocean integrated 6G network architecture, satellites, unmanned aerial vehicles, smart vehicles, and underwater unmanned vehicles can be the clients, where frequent D2D communications among them can further complicate the network topology (e.g., hierarchical tree topology). Such disadvantages advocate combining both FL and SL architectures, to guarantee the training performance in heterogeneous wireless networks.

• •

  Client heterogeneity: Clients generally have heterogeneous computation/communication/energy capabilities. Each client in FL requires more computation/energy resources to support model training. Besides, since the stability of wireless communications are essential to guarantee successful model transmissions (e.g., the size of a large complete model can reach 1 GB[6]), FL prefers clients with sufficient computation/energy/communication resources. Differently, SL usually supports clients with constrained on-board computation/energy resources since each client only has to train a partial model, which, however, can incur heavy communication overhead. More importantly, imbalanced and non-independent and identically distributed (non-IID) data of clients can leave heavy impacts on training performance. To this end, exploring the combination of FL and SL architectures to make full use of clients’ heterogeneous capabilities and resources presents another major significance.

• •

  Optimization target heterogeneity: Typically, model test accuracy and convergence speed represent the key optimization targets of distributed ML[3]. Besides, when distributed ML is implemented in heterogeneous networks, common indicators such as datarate, throughput, delay, and energy consumption also represent major concerns of ML, which complicates problems such as client scheduling and resource allocation[7]. Thus, it is significant to improve distributed ML architectures according to the characteristics of heterogeneous networks while realizing multiple targets optimization.

Given the above discussed challenges and limitations of conventional FL and SL in heterogeneous networks, this article is motivated to investigate two comprehensive architectures upon considering the combination of FL and SL. Our main contributions are highlighted below:

• •

  We first introduce a hybrid split FL (HSFL) architecture by integrating the split architecture into FL. Then, we are interested in designing a hybrid federated SL (HFSL) architecture, which unifies federated architecture with SL. key topics such as the advantages and performance comparisons of the two architectures are discussed.

• •

  Several open challenges are comprehensively discussed to identify the future research directions of our proposed architectures for future implementations.

• •

  Preliminary simulations are conducted to verify the feasibility of our proposed architectures on three datasets, under highly non-IID data settings.

According to previous discussions, FL clients may undergo heavy communication costs since each of them has to transmit a complete model to the server, especially when facing with large size models over unstable wireless links. Besides, in typical FL, the model of a client is no longer useful in updating the global mode when confronting transmission failures, e.g., only partial of the corresponding model has been successfully transmitted to the server. Promoted by the principle of SL and to overcome the drawbacks of FL, we delve into splitting the model from a different perspective, namely, the number of model parameters. Accordingly, a comprehensive HSFL architecture is proposed as inspired by [6], in a multi-layer heterogeneous wireless networks (as depicted in Fig. 2(a)), consisting of D2D clients, cellular clients, edge servers, and a main server. Key modules of HSFL are detailed below.

Model splitting: The model is firstly split into $M$ ($\{S_{1},\dots,S_{M}\}$) segments with equal data size where each segment is identified by a unique identification number. $M$ represents a hyperparameter that can be different for various FL models. More importantly, a larger $M$ can bring a smaller model granularity, while a higher transmission efficiency could be reached. Considering different communication conditions, an appropriate value of $M$ can ensure a good trade-off between transmission capacity and communication efficiency. Although each client can set $M$ by itself theoretically, to facilitate model aggregation/storage, all clients will use the same value of $M$.

Model transmission and aggregation at clients: Each client first evaluates the wireless channel quality and transmission capacity, to determine the number of segments that can be transmitted successfully. Any specific segments for transmission can be randomly chosen or specified by the receiver, while different clients can transmit the same segments. Specifically, for D2D clients, suppose that each client can communicate with its neighboring clients within one hop, it will send/receive at least one segment to/from each neighbor. Two paradigms associated with model transmission and model aggregation are applied for D2D clients and cellular clients, respectively, as given in Fig. 2(a). On the right side, cellular clients can transmit model segments to the edge server in parallel, for example, client 1 sends segment 1 to the edge server while client 2 sends segments 2 and 3. On the left side of Fig. 2(a), D2D clients transmit model segments to their neighboring clients sequentially in a decentralized manner, while the last D2D client can send the aggregated model to the edge server. Specifically, edge servers or D2D clients proceed with segment-wise model aggregation, where the model segments are aggregated individually. For example, edge servers aggregate segment 1 of $W_{E,B}$ by averaging all the received segments 1.

Horizontal/Vertical model aggregation at edge servers: Both vertical aggregation and horizontal aggregation are considered in HSFL to improve communication efficiency. For vertical model aggregation, the model transmission and aggregation can be repeated for multiple rounds to obtain multiple model replicas, which thus reduces the communication cost between edge servers and main server[3]. Then, the model will be transmitted to the main server for a wide-range global aggregation. Horizontal aggregation among edge servers can be seen as a unique form of D2D communication at the edge server level. This approach effectively diminishes the overall model size transmitted to the main server, thereby reducing communication costs. Additionally, horizontal aggregation facilitates model parameter sharing, resulting in a significant acceleration of local model training and the mitigation of the impact of non-IID data distributions among clients.

Fig. 2(b) illustrates the proposed HFSL architecture, where the main principle is to parallelize SL training and average the model weights over multiple clients by applying FL. Similar to HSFL, both D2D clients and cellular clients are considered.

When HFSL is implemented over multiple D2D clients, clients can be clustered into several clusters, e.g., based on their communication/computation capabilities. Then, the ML model is split into multiple subnetworks and distributed to the clients within each cluster. As shown on the left side of Fig. 2(b), the model is split into 4, and 3 subnetworks for two clusters, respectively. Then, the forward propagation starts at the client (e.g., client 1) with the input layer while ending at the client (e.g., client 4) with the output layer. Next, the back propagation starts in reverse order. Thus, each D2D cluster can be regarded as a hyper FL client, training a complete model, i.e, ${W}_{1},{W}_{2}$. The models can be transmitted by the clients to the edge server/neighboring cluster for averaging aggregation. In the next training round, the training starts with different clients under a new model splitting setting. Apparently, the training process is sequential within each D2D client cluster, and parallel over different clusters.

For cellular clients, we are motivated by [8, 9, 11, 10], and integrate them into the proposed HFSL. As shown by the right side of Fig. 2(b), the ML model is split into two subnetworks $C$ and $H$, where each client trains the same subnetwork $C$ in parallel while the subnetwork $H$ is deployed at edge server $B$. When multiple clients forward the activation results to the server in parallel, the edge server $B$ first copies the model to obtain multiple model copies to conduct forward propagation and back propagation for different clients in parallel. The number of model copies should be equal to the number of clients, e.g., two copies for clients $A$ and $B$. Then, the gradients will be sent back to the clients for updating the corresponding subnetwork parameters, $C_{A}$ and $C_{B}$; meanwhile the server can update parameters $H_{1}$ and $H_{2}$. Then, the clients can send the updated model parameters to the edge server in parallel. Finally, the edge server aggregates the model copies of edge server $B$ with $H_{B}$, and $C_{B}$ of the two clients. Among which, $C_{B}$ will be sent back to the clients, helping with the next training round. Notably, the number of cellular clients associated with each edge server should be optimized so that to alleviate the model storage cost of the edge server. Similarly, edge server $B$ can further report the complete model parameter $W_{E,B}$ to the main server for wide-range global aggregation. Note that horizontal model aggregation can also be done between the edge servers in HFSL, which is omitted here.

Although both HSFL and HFSL represent concrete implementations combining FL and SL architectures, they exhibit both connections and distinctions. Firstly, the fundamental architecture of HSFL is FL, wherein all clients are responsible for training complete ML models and only need to transmit partial (or complete) model parameters. In contrast, HFSL’s core architecture is SL, where each client exclusively trains a specific segment of the model. The federated architecture in HFSL is designed for parallel training and multi-layer model aggregation. Consequently, while HSFL and HFSL fundamentally represent two different architectures, they are both applicable in D2D-enabled heterogeneous networks. Generally, HSFL and HFSL are interconnected and can be harmoniously combined to create a complex yet effective architectural solution.

To achieve better performance comparison regarding different learning architectures, we quantify the communication (comm.)/computation (comp.) cost of clients via a simple analytical analysis. Without losing generality, we mainly consider cellular clients, since it is challenging to compare the performance of different architectures under the same parameter settings for D2D clients. Besides, it is hard to find general settings for decentralized FL and SL. Assuming that there are $N$ clients, the total training data size is $D$ where each client has the same training data size $D/N$. The overall model data size is $|W|$. The model is split into two subnetworks for SL and HFSL, the size of the split layer is $b$, and the size of forward propagation or back propagation over the split layer can be calculated by $bD/N$[12]. The fraction of model size with clients is $\gamma$ and $(1-\gamma)$ with the server. For HSFL, the model is equally divided into $M$ segments, while each client transmits $m$ segments to the server at one training round. The computation cost, i.e., floating point of operations (FLOPs) of training a complete model is denoted by $F$; and the size of allocated computation load fraction at the client is $\lambda$. Thus, the total computation cost is $NF$ for FL and HSFL, and it is $N\lambda F$ for SL and HFSL. Correspondingly, HSFL reduces the total communication cost of FL from $2N|W|$ to $2Nm|W|/M$, while HFSL raises the communication cost of SL from $N(2bD/N+\gamma|W|)$ to $2N(bD/N+\gamma|W|)$, which, however, greatly reduces the training time through parallelization (as shown in the simulation).

According to the above-discussions, the advantages of HSFL and HFSL are summarized as follows:

• •

  Training cost reduction: When compared to FLSL, both HSFL and HFSL can significantly reduce communication costs and training time. However, HSFL is more effective for training large-sized models, especially when dealing with a large number of clients, in contrast to HFSL.

• •

  Communication efficiency improvement: Wireless spectrum resources can be efficiently reused through the in-band D2D communication mode for D2D and cellular clients, aided by an appropriate resource sharing scheme.

• •

  Data-efficient training: HSFL can mitigate potential model transmission failures and make effective use of the local models trained by clients. Furthermore, HFSL can enable the participation of more clients with limited computational resources in each global training round, thanks to its parallelism capability. Additionally, the integration of D2D communications and a multi-layer hybrid network architecture significantly enhances server-client coverage and boosts the efficient utilization of training data.

• •

  Good adaptation in time-varying network topology: Leveraging the hybrid network architecture, both HSFL and HFSL demonstrate strong adaptability to dynamic network conditions, including those resulting from client mobility, and they achieve excellent stability throughout the training process.

• •

  High privacy protection: HSFL and HFSL empower clients to train or transmit partial models, thereby diminishing the likelihood of privacy disclosure resulting from malicious attacks or eavesdropping.