Graph Neural Network-Based Scheduling for Multi-UAV-Enabled Communications in D2D Networks

Due to the fast and flexible deployment features, unmanned aerial vehicles (UAVs) have attracted extensive attention from both academia and industry [1]. In the past few years, UAVs have been widely used in various fields such as military operations, earth and environment monitoring, disaster management, good delivery, precision agriculture, intelligent surveillance etc [2]. In the field of communications, UAVs are envisioned as a vital element of future wireless network technologies that will expand network coverage, improve deployment flexibility and increase system throughput [3, 4]. Compared with terrestrial base stations (BSs) whose the locations are pre-determined and fixed, UAVs can adaptively control its position to react as needed to requests for on demand services in a flexible and low-cost way [5].

Despite promising prospects for UAVs in many fileds like those mentioned above, some key issues must be dealt with seriously to effectively use them to realize seamless connectivity and ultra reliable communication in the future [6]. Among them, UAV trajectory design and position deployment for sensing and communications are two important issues which have received great attention. The trajectory design mainly for the fixed-wing UAVs who are always on the move [7], while the position deployment mainly for the rotary-wing UAVs that remain relatively stationary [8]. For the relatively static UAVs, it may always set the position based on a pre-designed optimization algorithm. Under the assumption of quasi-static channel, such an off-line design method is effective to some extent. However, in fact, this method obviously has the defects of complex calculation, poor real-time and unrealistic. Once the environment changes, it is worth studying how to adjust the UAVs’ position in time to achieve reliable communication and coverage.

D2D communication is an effective way to increase the transmission range and throughput [9]. Since the channel between UAV and ground user is usually assumed to be line of sight (LoS) channel, the UAV-to-ground channel will cause serious interference to D2D communications and thus compromise the system performance, how to integrate the UAV-enabled information dissemination and ground D2D transmission has attracted much attention. The authors in [10] considered the power control of a single UAV in the D2D communication network, a centralized processing algorithm was proposed to maximize the system throughput. [11] focused on the scenario that the D2D users act as the relay for UAV communications, by jointly optimizing user association, UAV scheduling, transmission power and UAV trajectory, the minimum security rate between UAVs was maximized.

Moreover, the authors in [12] focused on D2D-enhanced UAV nonorthogonal multiple access (NOMA) network architecture, in which the ground users can reuse the time-frequency resources assigned to NOMA links. It proposed a novel and efficient graph-based file dispatching method with a hypergraph-based grouping algorithm. D2D transmission was applied to extend wireless coverage of UAV in [13]. It focused on the optimal set of UAV-served users rather than the transmission performance of D2D users. An UAV-assisted D2D network was studied in [14], in which a new trade off between the throughput and channel switching cost was proposed. Instead of transforming the optimization problem into a convex form, it introduced game theory and constructed a distributed framework, which greatly reduced the computational complexity and makes it more suitable for scenarios with high-speed changed channels. Random walk model was also considered, in which D2D users move continuously and UAV flies around a central point [15]. Deep reinforcement learning was applied to solve the real-time resource allocation problem.

We note that the solutions of the above mentioned studies are not limited to convex optimization approaches. On the one hand, we know that some mathematical problems modeling of many complex network scenes are difficult to be transform into the equivalent convex form, and most of the widely used convex approximation skills can undermine optimality [16]. On the other hand, the popularity of smart devices enable users a certain degree of mobility, which makes the acquisition of real-time channel state information even worse. As a result, some efficient and low-complexity methods such as graph theory, game theory and deep reinforcement learning have been explored and exploited [17].

In many fields, such as communication signal processing, image processing and natural language processing, machine learning has become a common method to solve large-scale complex non-convex problems, as long as the training samples are available [18]. Machine learning can be mainly divided into supervised learning, unsupervised learning and semi-supervised learning. The difference among them is reflected in the proportion of labeled samples [19].

Recently, graph neural networks (GNNs) have been introduced to solve large-scale wireless resource management problems [20]. Some important properties were proposed in [20], including permutation equivalence property and the equivalence with distributed optimization algorithms, which can be used to analyze the performance and generalization of GNN-based methods. The authors in [21] proposed an intelligent reflecting surface (IRS)-based wireless network. To obtain the channel state information (CSI) of each communication link, a deep neural network was applied to parameterize the mapping from the received pilots to the optimized system configuration. Part of the channel information that can be obtained was included in the pilot information, and the interaction between users was captured through GNN. [22] considered a novel graph embedding based method for link scheduling in D2D networks, in which the graph embedding process was based on the distances of both communication and interference links without requiring the CSI. Two ways of learning were reflected in the simulation experiment of [22], i.e., supervised learning and unsupervised learning. For supervised learning, it was utilized to approximate optimal link scheduling strategy, while the unsupervised learning was employed to maximize the sum throughput of the proposed system.

From the above analysis, one may note that unsupervised learning is somewhat more popular than supervised learning. This is because that the formulated mathematical objective can be treated as the main component of the loss function, which is suitable for unsupervised learning. On the other hand, supervised learning requires a large number of labeled samples, who are not readily available since many mathematical problems of modeling are always highly non-convex and difficult to solve. Traditional convex optimization methods may obtain the optimal or near optimal solutions at the expense of high computing overhead. As a consequence, unsupervised learning becomes a better alternative.

In this paper, we focus on the deployment of relative static UAVs in D2D communication scenarios. Specifically, we consider the coexistence of the UAV-to-ground transmissions and D2D communications. Taking into account the interference of two patterns, we map the normal communication links to vertices of a graph, and map the co-channel interference links to edges of a graph. Based on the established heterogeneous graph, and taking the CSI and the coordinates of ground users as inputs, as well as taking the location of UAVs and the transmission power of all transmitters as outputs, we obtain the mapping from inputs to outputs through training the parameters of GNN. In addition, as wireless channels change with the movement of ground users, the UAVs’ positions need to be constantly adjusted accordingly. According to the real-time feedback of the CSI, the positions can be adjusted through the forwarding of trained GNN rather than the high-volume recalculation, which can help reducing the flight energy consumption and the computing delay effectively.

As shown in Fig. 1, we consider a heterogeneous wireless communication system with the coexistence of multi-UAV-enabled downlink transmissions and D2D communication network, which includes $N$ UAVs, $K$ downlink users (DUs) and $M$ D2D pairs. Each D2D pair consists of one D2D transmitter (DT) and one D2D receiver (DR). Due to the limited energy storage and computing ability, multiple UAVs in the air cooperatively send the same message (such as the live high-definition video obtained from some of them) to all DUs on the ground. Meanwhile, each DT sends information to its corresponding DR. All the nodes in our system are single-antenna. It is assumed that the UAV-to-DU links share the same spectrum with D2D communications. Therefore, the co-channel interference among them must be handled carefully to achieve globally optimal system utility.

Without loss of generality, we use the three dimensional Cartesian coordinate system [23], with which the location of DR $m$ can be denoted as $[x_{M_{m}},y_{M_{m}},0]$ and that of DU $k$ is $[x_{K_{k}},y_{K_{k}},0]$, with $m\in\left\{{1,2,...,M}\right\}$ and $k\in\left\{{1,2,...,K}\right\}$. For simplicity, we assume that all UAVs are flying at the same altitude, and the location of UAV $n$ is expressed by $[x_{N_{n}},y_{N_{n}},H]$, where $H$ is the minimum required height to circumvent ground obstacles and $n\in\left\{{1,2,...,N}\right\}$.

Here, we assume that the UAV-to-DU channel is the LoS channel, and locations of all DUs are given. Then, the channel gain from UAV $n$ to DU $k$ can be expressed as [5, 24]

where $\beta_{0}$ is the channel power gain at the reference distance $d=1$ meter.

Similarly, the channel gain from UAV $n$ to DR $m$ can be expressed as

It is worth noting that expression (1) and expression (2) have the similar structure. However, expression (1) refers to the transmission channel, while expression (2) denotes the interference channel. All air-to-ground channels are highly related to the position of UAV which is one of the main design concerns of this paper. By taking into the co-channel interference and the broadcast characteristics of the UAV-to-DU channels, the signal-to-interference and noise ratio (SINR) of DU $k$ can be given by

where $P_{N_{n}}$ and $P_{M_{m}}$ are the transmission power of UAV $n$ and DT $m$, respectively. The Rayleigh fading channel $g_{m,k}^{u}={\left({\frac{\lambda}{{4\pi{d_{1}}}}}\right)^{2}}{\left({\frac{{{d_{1}}}}{{d_{m,k}^{u}}}}\right)^{\gamma}}$ is the interference channel from DT $m$ to DU $k$ with $\lambda$ as the wavelength, ${d_{1}}$ as the reference distance outdoors, $\gamma$ as the path loss parameter relied on the transmission scene and $d_{m,k}^{u}$ as the distance between DT $m$ and DU $k$. ${\sigma_{k}}$ is the Gaussion noise. Similarly, the SINR of DR $m$ is expressed as

for $m\in\left\{{1,2,...,M}\right\}$, where ${g_{m}}={\left({\frac{\lambda}{{4\pi{d_{1}}}}}\right)^{2}}{\left({\frac{{{d_{1}}}}{{{d_{m}}}}}\right)^{\gamma}}$ represents the channel between DT $m$ and DR $m$ with ${d_{m}}$ as the distance between them, the Rayleigh fading channel $g_{i,m}^{d}={\left({\frac{\lambda}{{4\pi{d_{1}}}}}\right)^{2}}{\left({\frac{{{d_{1}}}}{{d_{i,m}^{d}}}}\right)^{\gamma}}$ is the interference channel from DT $i,\;\left({i\neq m}\right)$ to DR $m$. ${\sigma_{m}}$ is the Gaussion noise. It is worth noting that the interference received at DR $m$ contains two parts, one part is that from other D2D pairs and other part is from UAVs.

Our goal is to maximize the sum throughput of DUs while guaranteeing the basic QoS requirements of all the D2D pairs. In summary, the problem can be formulated as [5, 16]

where $R_{\min}$ is the required minimum transmission rate of each D2D pair.

The first two constraints in (5) refer to the power consumption of UAVs and DTs, which cannot exceed their tolerance limit. The third constraint indicates that the interference from the UAVs and that of other D2D transmitters must ensure that the transmission rate of each D2D link cannot be lower than the threshold ${R_{\min}}$. It can be found that problem (5) is non-convex and difficult to solve directly, since the variables are highly coupled in the objectives and constraints. In the next section, instead of resorting to traditional optimization skills, such as the alternating optimization, block coordinate descent, etc, we will provide the details of GNN-based approach to solve this problem optimally and efficiently.

In this section, we first convert the proposed multi-UAV D2D communication system into a graph model, and then provide the detailed training process.

In this section, we provide numerical results to validate the effectiveness of the GNN-based scheme. Here, we introduce three schemes for comparison, including 1) alternate optimization (AO) scheme for the original problem, 2) random deployment of UAVs’ position, and 3) fixed power allocation. Some of the main parameters are listed in Table 1.

The graph neural network adopted includes one input layer, three hidden layers and one output layer. The number of parameters of the hidden layer is $\left\{{32,64,32}\right\}$. In the training process, we take each user’s coordinates and CSI as input, and the each UAV’s power allocation and coordinates as output.

We use program simulation to generate 500 samples as the sampling set, and generate another 200 samples as the test set. For the purpose of updating the network weights more efficient, we use the Adam optimizer [25] instead of the classic stochastic gradient descent method. From Fig. 6, it can be seen that the result of the UAV-to-DU total transmission rate tends to be stable after about 100 iterations, the convergence of the proposed network is guaranteed.

Fig. 7 shows the the UAV-to-DU total transmission rate versus the number of D2D pairs. It reveals that our proposed algorithm has a greater improvement than other benchmark schemes in terms of downlink transmission rates. Specifically, as the number of D2D users increases, the total transmission rate of the system decreases. Since D2D users are randomly distributed in the area, the increase in the number of D2D users will not only increase the interference with DUs and DRs, but also make the deployment of UAVs more difficult. In this case, the UAV tends to fly far away from the communication area or reduce the transmission power to ensure the transmission rate of D2D users. Compared to $M=10$, when $M\geq 30$, the transmission rate will decrease by more than two thirds. Hence, in this situation, increasing the number of UAV may be a feasible way to improve the performance, as what we will discussed in the next simulation result.

In Fig. 8, we take the total transmission rate versus the number of UAVs. It can be found that as the number of UAVs increases, the transmission rates increase significantly. This means that increase the number of UAV will not significant increase the interference with the DUs, and the interference with DRs from the newly added UAVs can be effectively suppressed by location deployment and power control. All of this reveals the obvious advantages of multi-UAV-enabled communications.

In this paper, the power control and position optimization problem was investigated for multi-UAV-enabled communications in D2D network. An GNN-based method was proposed to tackle the highly non-convex fractional programming problem optimally. In particular, we formulated a graph-based model for the proposed network via mapping the transmission links and interference links into vertexes and edges, respectively. Then, we trained the GNN with unsupervised learning. Finally, the trained GNN can be used to perform scheduling based on the input channel state information and the coordinates of ground users. Simulation results demonstrated the outstanding performance of UAV in facilitating the information broadcasting to ground downlink users. Moreover, the proposed GNN-based approach shows lower complexity and better system throughput than that of the benchmark scheme.

This work was supported in part by the National Natural Science Foundation of China (61901231), in part by the National Natural Science Foundation of China (61971238), in part by the Natural Science Foundation of Jiangsu Province of China (BK20180757), in part by the open project of the Key Laboratory of Dynamic Cognitive System of Electromagnetic Spectrum Space, Ministry of Industry and Information Technology (KF20202102), in part by the China Postdoctoral Science Foundation under Grant (2020M671480), in part by the Jiangsu Planned Projects for Postdoctoral Research Funds (2020Z295).