DNN Task Assignment in UAV Networks: A Generative AI Enhanced Multi-Agent Reinforcement Learning Approach

The primary challenge is that conventional scene data collection and monitoring tasks rely on staged task processing. For example, in [10, 19], the methods involve an initial phase of task planning, followed by data collection, and conclude with online or offline processing of related tasks, such as target detection and positioning. This sequential approach is evidently inadequate to meet the real-time requirements of sudden events or emergency scenarios. In addition, the limited computational power, battery life, and other onboard resources of UAVs further constrain their ability to perform real-time tasks effectively.

Secondly, when DNN tasks need to be offloaded, many studies explore the transfer of intermediate data to the ground nodes [13, 15, 18]. However, non-line-of-sight communication and the high relative mobility between UAVs and ground nodes—especially dynamic nodes—introduce significant uncertainty into the offloading process. Furthermore, existing methods for offloading computationally intensive tasks, such as heuristic-based, decomposition-based [10] and game-based approaches [20], typically require numerous iterations to converge to a satisfactory and stable assignment decision. This iterative decision-making process is often impractical for latency-sensitive tasks and energy-constrained systems.

Third, non-uniformities in the DNN-based task processing pipeline are apparent. To elucidate this, we conducted a pilot study that analyzes execution latency by layer, intermediate output data size, and computational demands for Yolov5 [21], AlexNet [22], and VGG16 [23]. The results, shown in Fig. 2, demonstrate that most of the computational cost is concentrated in the first half of the DNN, with considerable variability in latency, output data size, and computation across different layers of these DNNs. This underscores the need for uniform task partitioning. Arbitrary division can lead to significant imbalances in task assignment between UAVs, which may adversely affect the operational efficiency and survival time of multi-UAV, thus hindering its overall effectiveness in multi-UAV operations [12]. To delineate the differences between the proposed DNN task assignment strategy and existing studies in this field, a comparison of various works is presented in Table I. Performance metrics that meet the specified criteria are indicated with a $\checkmark$.

In this paper, we develop a collaborative approach of multi-UAVs for the assignment of DNN tasks. This approach considers the computing resources, energy consumption, and number of layers they can execute each UAV. Moreover, our method introduces a finer-grained task partitioning compared to previous research. Our main contributions are as follows.

• •

  We propose a mother-and-child UAV system that employs a high-altitude platform (HAP), such as an airship, equipped with multiple UAV launchers. From this HAP, several UAVs are launched and networked to support collaborative tasks. Then we design DNN tasks directly on UAVs, performing inference independently of the ground base station. This approach enhances the scalability of the system. Furthermore, we propose a novel greedy algorithm for UAV path planning that utilizes a fitness function to minimize flight distance, taking into account the varying sizes of tasks.

• •

  We formulate the optimization problem of joint energy consumption, task completion rate, and load balancing in a multi-UAV network as a Mixed Integer Nonlinear Program. Subsequently, we introduce a multi-agent reinforcement learning (MARL) algorithm based on a generative diffusion model (GDM) to address the problem. Our work tackles constraints related to the number of UAVs, the AoI of the DNN task, the cache capacity, and the limitations of energy consumption. Notably, our research is the first to integrate a GDM into the MARL framework for multi-UAV networks.

• •

  The proposed algorithm leverages an innovative application of a GDM to determine optimal DNN task assignment decisions. We use the reverse denoising process of the GDM instead of the actor network of the multi-agent deep deterministic policy gradient (MADDPG) to generate specific DNN task assignment actions based on the agent’s observations in a dynamic environment. The GDM functions by sequentially reducing noise through a series of denoising steps, thereby extracting optimal decisions from an initial state of Gaussian noise. By decomposing complex computations into simpler subtasks, we achieve substantial reductions in latency and improve the overall efficiency.

The organization of this paper is as follows: Section II reviews the related work, the system model is formulated in Section III, Section IV describes the proposed problem, Section V formulates the path planning approach, Section VI describes the GDM-MADDPG approach, and Section VII presents the simulation results, followed by the conclusion in Section VIII.

The partition of the DNN task represents a significant research challenge that involves dividing a DNN into several segments and distributing these segments between designated locations. The existing work provides valuable information to support the efficient DNN inference task. The authors in [24] present a joint design for task partitioning and offloading in DNN-enabled edge computing, involving a server and multiple devices. In [25], the authors propose a multi-task learning-based asynchronous advantage actor-critic approach, and explore distributed DNN inference through fine-grained model partitioning in edge computing. These studies overlook the mobility of UAV networks. The authors in [26] explore accelerating DNN-based task processing in vehicular edge computing through task partitioning and offloading. In [17], the authors propose an efficient allocation strategy using RL and investigate fast DNN inference in UAV swarms through multi-UAV collaboration. However, studies have not addressed the adaptation challenges associated with the partitioning of DNN tasks. In addition, the oversight of these studies may result in decreased network stability in certain resource-constrained networks due to suboptimal task assignment within the DNN.

MARL enables each agent to optimize its policy through observations of both the environment and the policies of other agents [27, 28]. Many existing MARL-based approaches significantly improve task offload efficiency within dynamic networks [29, 30]. The authors in [31] propose a dedicated actor neural network for coordination and a scalable training algorithm for the offloading of UAV-aided MEC tasks. In [32], the authors present a DRL-based joint safe offloading and resource allocation scheme for vehicular edge computing networks. These studies have paid limited attention to this aspect. The authors in [33] present a mobility-aware service offloading and migration scheme with Lyapunov optimization and an MADDPG algorithm. The authors in [34] focus on optimizing task offloading and UAV trajectory under energy and queue latency constraints. However, most of these studies focus on optimizing network performance metrics, including task completion latency and energy consumption. Our research emphasizes DNN inference services, where the AoI for DNN tasks is a critical performance metric.

GAI has the potential to effectively overcome the limitations of conventional AI and can be applied to optimize UAV networks, particularly to improve transmission rates, communication capacities, and energy efficiency [35, 36]. In [37], the authors present a radio propagation model with a conditional generative adversarial network. The authors in [38] introduce a two-stage generative neural network to predict link states and generate path losses, latencys, and arrival and departure angles in millimeter wave communication for UAVs. The authors in [39] explore the long-term optimization of joint task offloading and resource allocation in a multi-access edge computing network for UAVs. In [40], the authors present a hierarchical aerial computing system where UAVs collect tasks from ground devices and offload them to HAPs. Unlike the above schemes, our approach incorporates a path planning, GDM-based MARL decision-making framework specifically tailored for multi-UAV networks. This methodology aims to minimize the cost associated with DNN task completion while ensuring the system’s stability.

The network architecture is primarily composed of an HAP layer and a UAV swarm layer, as illustrated in Fig. 3. The HAP layer consists of a tethered platform with extensive computing and communication capabilities, such as an airship. This platform exchanges information with UAVs via wireless links to facilitate model training [41] and path planning (see Section V). The airship will transmit the final result of the path planning to the leader UAV. The UAV swarm comprises a leader UAV and multiple follow UAVs. All UAVs are isomorphic, with the total number of UAVs being ${n}=\left\{{1,2,\cdot\cdot\cdot,N}\right\}$, and each is equipped with trained DNN models. The leader UAV receives the results of the path planning transmitted by the airship and operates according to the predetermined route. It is also responsible for collecting and processing images and point cloud data of the target area, including tasks such as target detection and mapping. Thus, it functions as both a task producer and executor, as well as a decision maker for task assignment. The follow UAVs primarily execute the tasks assigned by the leader UAV, with the assignment of DNN tasks between follow UAVs being streamlined. The leader UAV evaluates the task size and the remaining resource status of the follow UAVs to determine whether to assign tasks. Therefore, to prevent task overload on the UAVs, extend the operational lifespan of the UAV swarm, and improve the timeliness of task processing and network stability, efficient task assignment decisions for the UAV swarm are particularly crucial.

The area where the task is performed in this paper consists of $q$ center coordinates of the target area, denoted as $q=1,2,...,\mathbb{W}$, and the sequence of the UAV inspection target coordinates is represented as $\mathfrak{q}(q)=[\mathfrak{q}(1),\ldots,\mathfrak{q}(\mathbb{W})]$. The data size in the target area is $\ell_{q}=\{1,...,\mathbb{L}_{\mathbb{W}}\}$. Assume that the number of DNN tasks in each synchronization cycle is $i=\{1,2,...,I\}$, the DNN type is $\kappa=\{1,...,\mathbb{K}\}$. The DNN model typically comprises $l$ layers, denoted as $l=\{1,2,...,L\}$. Each layer has a cache capacity $m_{i,l}$ and a computing requirement ${c}_{i,l}$, represented by a tuple $(m_{i,l},c_{i,l})$. $l_{\kappa}=\{1,2,...,L_{\kappa}\}$ represents a set of layers associated with a specific type of DNN task, including convolutional and pooling layers for feature extraction, as well as fully connected layers for classification [8]. The DNN task is decomposed into multiple subtasks for serial processing, and each follow UAV serves as an execution stage in this serial processing. This approach allows each subtask to offload and transfer intermediate results among different follow UAVs without requiring the processing of the entire task, thereby enhancing the computational efficiency and real-time performance of computationally intensive and latency-sensitive tasks [11].

In response to such practical needs, determining the optimal split point of the task is crucial in DNN task assignment. Assume that the split point of the ${i}$-th DNN task model is $P_{i,n^{\prime}}$, and $n^{\prime}\in\{1,2,...,N-1\}$ represents the number of subtasks generated after the DNN task model is divided into blocks, its size depending on the initial task data size. Based on the size of the task data, we can categorize the task assignment modes into swarm assignment, partial assignment, and binary assignment. Swarm assignment indicates that the DNN task is processed serially through pipeline assignment across all follow UAVs. Partial assignment means that the DNN task is processed serially through pipeline assignment among some follow UAVs. Binary assignment entails that the DNN task is completely offloaded to a single follow UAV for processing. The DNN task assignment mode ${M}$ is expressed as follows

where $f(\cdot)$ represents the function that determines the task assignment mode (see Section VI). $\tau_{i}^{max}$ denotes the maximum latency allowed for the total completion time of each task, and $O$ represents the Observation space of the follow UAV.

DNN task assignment is performed within the UAV swarm, and during the execution of the task, the UAV swarm maintains a fixed formation flight. At time slot $t$, the coordinates of UAV are designated as $\mathcal{P}^{n}=[x^{n}_{t},y^{n}_{t},z^{n}_{t}]$, and the distance between the two UAVs is defined as follows

The UAV-to-UAV data link utilizes line-of-sight communication, and its channel model is described by the Close-In Free Space Reference Model (CI) [29, 42]. The path loss is expressed as follows

where $PL_{{FS},ref}(\mathfrak{f})$ represents the free space path loss per unit length. $\mathfrak{f}$ means the frequency of a radio wave is equal to the speed of light divided by the wavelength. $n_{CI}$ is the path loss index, and $\xi_{\sigma,CI}$ is the shadow fading parameter, which is typically assumed to follow a zero-mean Gaussian distribution.

The signal-to-noise ratio between the two UAVs is formulated as follows

where $P_{n}$ is the information transmission power of the UAV. $P_{I}$ is the interference power. $P_{N}$ is the noise power, denoted as $P_{N}=kBT$, where ${k}$ is the Boltzmann constant. ${B_{n}}$ is the channel bandwidth. ${T}$ is the absolute temperature.

Therefore, the data transmission rate between the UAVs can be expressed as follows

Let $\boldsymbol{\varphi}_{q}$ denote the binary decision variable for the coordinate of the inspection target $\boldsymbol{g}_{\mathfrak{q}(q)}$; specifically, when the target coordinate $\boldsymbol{g}_{\mathfrak{q}(q)}$ is inspected by the UAV, $\boldsymbol{\varphi}_{q}$ is equal to 1. Otherwise, $\boldsymbol{\varphi}_{q}$ is equal to 0. After the UAV swarm collects data from the target coordinate $\boldsymbol{g}_{\mathfrak{q}(q)}$, it must process the data before reaching the next target coordinate $\boldsymbol{g}_{\mathfrak{q}(q+1)}$. The Euclidean distance between the two target coordinates is denoted as $D^{\mathfrak{q}(q),\mathfrak{q}(q+1)}$. The total moving distance of the UAV to complete the inspection task is the sum of the distances of each segment along its flight path, as illustrated below

Assuming that the UAV flies at a constant speed $\nu$, the time taken by the UAV swarm to travel from one coordinate to the next one is expressed as follows

The total flight time for the process of the UAV executing the task is represented as follows

AoI defined in this paper aims to measure the duration from data collection to processing completion, incorporating waiting latency, transmission latency, and computation latency to reflect the real-time performance of DNN task assignment. We assume that each UAV has a maximum memory capacity $m_{n}^{threshold}$, available energy $e_{n}^{threshold}$, and computing capacity $f_{n}$. The AoI is optimized by minimizing the UAV’s moving distance and refining the DNN task assignment strategy. The waiting latency for the data collected by the leader UAV to be processed by the DNN model is denoted as

where $T_{i}$ represents the moment when the collected raw data is obtained on the leader UAV or when the subtask ${n}$ is generated on a follow UAV. $T_{i^{\prime}}$ signifies the moment when the collected raw data is processed in the leader UAV or when the subtask ${i}$ is about to be sent from the follow UAV ${n}$ to the follow ${n+1}$.

The latency for the intermediate result of the completed DNN task transmitted by follow UAV ${n}$ is expressed as

where $W_{i,l}$ is the data size output by the previous follow UAV, specifically the data size before the split point $p_{i,n^{\prime}+1}$.

The computation latency of the task is represented as

where $\frac{c_{i,l}}{f_{n}}$ denotes the computing time of the ${i}$-th subtask. $c_{i,l}$ indicates the computing requirement of the ${i}$-th subtask, and $f_{n}$ represents the computing capacity of the $n$-th UAV.

Therefore, the latency $\mathcal{T}$ for the complete DNN task and the execution latency of the entire DNN task ${AoI}$ are as follows

The energy consumption of UAVs in performing DNN tasks primarily consists of computing, transmission, and flight energy consumption.

1) The computing energy consumption of the UAV is denoted as

where $k_{0}$ represents the energy efficiency parameter and $c_{i,l}$ denotes the computing requirement or complexity of the ${l}$-th layer in the ${i}$-th task.

2) The transmission energy consumption is approximated as the product of the transmission power $P_{n}$ and the transmission time $t_{i,n}^{trans}$ of the intermediate result of the DNN task as follows

3) As one of the energy consumptions that must be considered, the flight energy consumption of the UAV is adopted in [30]. The flight energy consumption is approximated as the product $e_{n}^{fly}$ of the propulsion power $P^{fly}$ of the UAV and the total flight time during the task execution as follows

where $P_{1}$ represents the blade shape power in the hovering state, $P_{2}$ is the induced power, $\nu_{tip}$ is the speed of the rotor blade tip, and $\nu_{0}$ is the average induced rotor speed in the hovering state. Additionally, $\zeta$, $\rho$, $\varsigma$, and $s$ refer to the fuselage drag ratio, air density, solidity of the rotor, and disk area, respectively. Under hovering conditions, the power consumption of the UAV is the sum of $P_{1}$ and $P_{2}$.

Consequently, the energy consumption of a follow UAV is $e_{n}=e_{n}^{comp}+e_{n}^{trans}+e_{n}^{fly}$. And then the total energy consumption of all DNN tasks executed by the UAV swarm is denoted as

This paper employs a greedy algorithm to minimize the total flight distance of the UAV while jointly considering the task size of the target area to address the UAV path planning problem. Before executing the DNN task assignment, the UAV must plan an optimal flight path to reduce task completion time and energy consumption. Given that the task sizes of each target area vary, we denote the task size of the coordinate of the $q$ -th target as $\mathbb{L}_{q}$, and let ${\upsilon}_{q}$ represent the average rate at which the UAV swarm processes the tasks. Consequently, the time required to process the task size at the $q$-th target coordinate is denoted as

To improve task processing efficiency and AoI, the UAV swarm is required to complete the task after inspecting the target coordinate and before reaching the next target coordinate, thus establishing the condition $t_{q}\leq t^{next}$. Furthermore, we define $\Delta t=t^{next}-t_{q}$ as the difference between flight time and task processing time between two target coordinates of the task.

Accordingly, the fitness function integrates the UAV flight distance and the quantity of tasks processed by the DNN model, represented as

where $\vartheta$ and $\rho$ are weight coefficients utilized to balance the trade-off between the UAV flight distance and the amount of data processed by the DNN model.

The greedy algorithm is a straightforward, lightweight heuristic solution method that is easy to implement [3]. However, this approach can lead to suboptimal outcomes, as selecting local optimal solutions does not guarantee a global optimal solution. To address this limitation, this paper incorporates randomness into the design of the greedy algorithm to mitigate the risk of converging on a local optimal solution.

Specifically, the candidate target coordinates $k$ are randomly selected from the set of uninspected target coordinates, denoted $\widetilde{\mathbb{W}}$. If $|\widetilde{\mathbb{W}}|\leq\mathrm{k}$ indicates that all uninspected target coordinates are chosen as candidate target coordinates $k$, then we define this set as $\tilde{\mathrm{g}}=\widetilde{\mathrm{W}}$. Otherwise, $\widetilde{\mathbb{W}}$ uninspected target coordinates $k$ are randomly selected from $\tilde{\mathrm{g}}\subset\widetilde{\mathbb{W}}$. For each candidate target coordinate, the fitness function value ${F}$ is calculated and the target coordinate with the smallest value $F$ is selected as the next target coordinate. A lower value of $F$ corresponds to a lower total cost. The chosen target coordinate is then added to the flight path sequence and removed from the set of candidate target coordinates. This process is repeated until all target coordinates have been inspected once, at which point the UAV returns to the initial target coordinate.

The path planning algorithm based on the greedy algorithm is illustrated in Algorithm 1. In this context, the path planning problem is denoted as $\mathbb{P}_{2}$

where $\mathrm{C}_{8}$ signifies the requirement that the UAV swarm completes the task at the previous target coordinate before proceeding to the next target coordinate.

DNN Task assignment decision generation differs from traditional applications of GDM, primarily because decision optimization often lacks a large, dedicated dataset for offline training. In this paper, the DNN task assignment decision process involves the GDM predicting noise distribution through iterative refinement, followed by training its reverse denoising process, specifically denoising Gaussian noise to yield optimal action decisions. This approach transforms the challenge of limited datasets into an opportunity for dynamic online learning and agent self-optimization. GDM-MADDPG takes the observation of the agent as the condition of the denoising network of the diffusion model, and replaces the actor network of MADDPG with the reverse denoising process to better capture the correlation between the data, as shown in Fig. 4. In the initialization phase, each agent will establish its own Actor network and Critic network, as well as the corresponding target network, and create an experience replay buffer to store interaction data. The Actor network determines the action based on the local observation output of the agent, while the Critic network evaluates the expected reward of the state-action pairs of all agents. During training, the agent performs actions and collects environmental feedback, which is stored in the experience replay memory. In order to stabilize the training, GDM-MADDPG introduces a target network whose parameters are updated at a certain rate to achieve a smooth learning process. Next, samples are randomly drawn from the memory to update the Critic network so that it can more accurately predict the rewards of Observation-action pairs. Subsequently, the output of the Critic network is used to update the Actor network with the goal of maximizing the expected reward.

The GDM inverse denoising process serves as a reverse denoising technique that reconstructs the original data from noisy observations. In this context, the denoising network, once fully trained, is capable of generating optimal decisions for DNN task assignment in any dynamic environment. Both the selection of assignment modes and the design of utility functions typically adapt to the dynamic variations present in the environment. This capability to respond to changing conditions and generate appropriate actions is invaluable in decision optimization. Consequently, the GDM inverse denoising process can leverage information such as the UAV observation space ${O}$, task data size $\ell_{q}$, and task type $\kappa$ as conditions within the denoising network, thus facilitating adaptive decision generation.

For the agent ${n}$, the objective of the inverse process is to infer the probability distribution $\mathbf{x}_{0}^{n}$ of each task that is selected from the Gaussian noise $\mathbf{x}_{T}^{n}{\sim}N(0,I)$. If the inverse distribution $p(\mathbf{x}_{t-1}|\mathbf{x}_{t})$can be learned, it becomes possible to sample $\mathbf{x}_{t}$ from the standard normal distribution ${N}(0,{I})$ and subsequently obtain samples from $p(\mathbf{x}_{0})$ through the inverse process. However, statistical estimation $p(\mathbf{x}_{t-1}|\mathbf{x}_{t})$ requires calculations that involve the complexity of the data distribution, which is often challenging to manage in practice. Therefore, our goal is to estimate $p(\mathbf{x}_{t-1}|\mathbf{x}_{t})$ using the parameterized model $p_{\theta}$ as outlined below

In this manner, we can derive the trajectory from $\mathbf{X}_{T}$ to $\mathbf{x}_{0}$ as follows

By conditioning the model in the time interval ${t}$, it can learn to predict the parameters of the Gaussian distribution, specifically the mean $\mu_{\theta}(\mathbf{x}_{t},t)$ and the covariance matrix $\boldsymbol{\Sigma}_{\theta}(\mathbf{x}_{t},t)$ at each time step.

The training of GDM involves optimizing the negative log-likelihood of the training data. According to the literature [43], by incorporating conditional information $g$ in the denoising process, it can be modeled as a noise prediction model with the covariance matrix fixed to $p_{\theta}(\mathbf{x}_{0:T})$

The construction of the mean is as follows

We first sample $\mathbf{x}_{T}$ from ${N}(0,{I})$ and then sample $x_{t-1}|x_{t}=\frac{x_{t}}{\sqrt{\alpha_{t}}}-\frac{\beta_{t}}{\sqrt{\alpha_{t}(1-\overline{\alpha}_{t})}}\epsilon_{\theta}(x_{t},g,t)+\sqrt{\beta_{t}}\epsilon$ through the inverse denoising process parameterized by $\theta$ where $\epsilon\sim{N}(0,{I})$ and $t=1,\ldots,T$. The loss function for the model training of the denoising network is given by

GDM effectively addresses challenges within the DNN task pipeline assignment environment, generating optimal assignment decisions that maximize the utility function. This method showcases the ability of GDM to manage complex data distributions and produce high-quality decisions, particularly in dynamic and evolving environments [8]. By integrating DRL, more efficient and adaptable action strategies can be developed. Consequently, the utility maximization problem is formulated as a multi-agent Markov decision process (MDP), represented by a tuple $(n,O,\mathcal{A},{r},\gamma)$, where n denotes the number of agents, ${O}$ is the observation space of the agent, $\gamma$ represents the discount factor, and ${r}$ signifies the reward obtained by the agent. The agent is represented by the UAV. It learns an optimal task assignment strategy based on GDM-MADDPG according to task size, computing resources, and task latency to achieve optimal load balancing and AoI. The DNN task assignment decision generation based on GDM-MADDPG is detailed in Fig. 4 and Algorithm 2.

Observation Space: In each time slot t, the UAV collects observations from its environment. The observation space is denoted as $O_{t}=\{o_{t}^{1},o_{t}^{2},\ldots,o_{t}^{N}\}$, with the observation of a single agent represented by

where $\ell_{t}$ indicates the amount of data associated with the DNN task at time slot ${t}$, $\kappa_{t}$ represents the type of DNN model, $l_{\kappa,t}^{completed}=\left\{l_{1}^{completed},l_{2}^{completed},\ldots,l_{l}^{completed}\right\}$ indicates the degree of completion of the DNN task (i.e., the number of completed layers), $\mathbf{\tau}_{i}^{max}=\{\tau_{1}^{max},\tau_{2}^{max},\ldots,\tau_{I}^{max}\}$ denotes the maximum latency that each task can tolerate at time slot ${t}$, $\mathbf{e}_{t}$ represents the energy consumption constraint of the UAV (the remaining energy at time slot ${t}$, $\mathbf{m}_{t}$ indicates the memory constraint of the UAV (the remaining memory at time slot ${t}$), $\boldsymbol{\mathcal{P}}_{t}=\{\mathcal{P}_{t}^{1},\mathcal{P}_{t}^{2},\ldots,\mathcal{P}_{t}^{n}\}$ represents the positions of all UAVs at time slot ${t}$, and $\mathfrak{q}_{t}(q)$ signifies the positions of all targets.

Action Space: The action space is defined as

$\mathcal{A}^{n}=\{a_{1}^{n},a_{2}^{n},\ldots a_{t}^{n}\}$. In each time slot t, the agent’s action is denoted by $a_{t}^{i}$, where $i=\{1,2,...,I\}$ represents the task chosen by the agent for execution and $a_{t}^{n}=\{a_{t}^{1},a_{t}^{2},...,a_{t}^{i}\}$ means the action taken.

Reward Function: The agent’s actions are constrained by AoI and are closely related to load balancing and task completion rates. To maximize the system utility, the agent observes $o_{t}^{n}$ and executes action $a_{t}^{i}$ to obtain the reward $r_{t}^{n}$ as follows

where $r_{t}^{n,{indiv}}$ and $r_{t}^{n,{group}}$ represent individual and global rewards, respectively. $\sigma$ and ${\vartheta}$ are weight coefficients, and $e^{rendez}$ denotes the energy consumption required for the UAV to return.

1) Training Environment

The hardware configuration consists of an Intel i7-13700K CPU and an NVIDIA RTX 4090 with 24 GB of RAM. The software environment includes pytorch GPU version 1.11.0 and Python version 3.8. The parameters for DNN are specified as follows: for the Actor network, the number of neurons in the first and second hidden layers are set to 256, respectively, while the number of neurons in the third hidden layer is determined by the dimensionality of possible actions. For the Critic network, the neuron counts for the three hidden layers are set to 256, respectively. During training, the learning rates for the Actor and Critic networks are set to $\gamma=10^{-3}$. The mini-batch size during training is 512. The discount factor is set to $\gamma_{m}=0.9$, and the initial value and the decay rate for exploration are defined as $\epsilon_{0}=0.9$ and $\beta=10^{-4}$, respectively. In our simulation, we examine the performance of collaborative DNN task assignment in different data sizes. To simplify calculations, we assume a constant transmission model, where the data transmission rate remains uniform during a complete collaborative task assignment. The experimental parameters and their corresponding values are detailed in Table II [44].

2) Simulation Scenario

We perform simulations in the following scenario: an airship positioned at an altitude of 10 km, accompanied by nine UAVs at an altitude of 3 km, covering an area of 12×12 km². The airship is located at the center of this area, whereas the leader UAV and follow UAVs are uniformly distributed throughout it. The airship obtains task directives from the ground control center, formulates the optimal flight sequence for the designated task targets, and transmits the planned route information to the lead aircraft. The lead aircraft, along with several wingmen UAV, then executes a coordinated flight along the predetermined route. It is important to note that all UAVs fall within the coverage area of the airship and are interconnected and capable of communicating with one another. Furthermore, considering overall accuracy in detecting target amounts, the Yolov5 model outperforms the seven attention-optimized Yolov5 variants, as well as other state-of-the-art object detection models [4]. In this study, we employ Yolov5 as the task model and perform experiments to evaluate the effectiveness of the proposed approach.

3) Benchmark Algorithms

To demonstrate the advantages of the proposed method, we examine the following benchmark algorithms.

Greedy algorithm involves making the best choice at each step based on the current state, with the aim of achieving a global optimal solution through a series of local optimal solutions.

MADDPG use a centralized training and distributed execution framework. It extends single agent RL algorithms from the actor-critic framework to accommodate multi-agent scenarios [34].

MADDPG with path planning integrates the proposed path planning approach with MADDPG, facilitating the generation of task assignment decisions that align with the requirements for task execution.

1) Path Planning

This experiment simulated a UAV inspection area consisting of ten target coordinates, each defined by a distinct task size. Fig. 5(a) shows the proposed UAV path planning method. By comprehensively considering both task size and flight path, this method generates the global optimal path and target coordinate sequence, prioritizing target coordinates with larger task sizes while ensuring coverage of all target coordinates. In contrast, Fig. 5(b) presents a greedy algorithm which selects the best option at each step based solely on the current state, and is prone to local optima. As a result, it can lead to a significant increase in flight path length, reducing the survival time of the UAV swarm and hindering its ability to conduct long-term operations under energy constraints.

As illustrated in Fig. 5(c), the cost associated with the optimal path finding of the UAV increases as the number of target coordinates increases. In particular, the path planning algorithm proposed in this study consistently exhibits lower costs compared to the greedy algorithm in all scenarios evaluated. Specifically, for target coordinate counts of 10, 20, 30, 40, and 50, the total cost produced by the proposed algorithm is reduced by approximately 19.3%, 21.1%, 23.0%, 25.6%, and 27.1%, respectively, compared to the greedy algorithm. This enhancement can be attributed to the expanding solution space that accompanies an increase in the number of target coordinates. As the candidate set of target coordinates grows, the proposed algorithm has greater opportunities to explore solutions that yield lower total costs, ultimately facilitating the identification of the optimal path with minimal total cost. Thus, it is evident that the algorithm developed in this study is particularly adept at addressing path planning challenges that involve a larger number of target coordinates.

2)Training Procedures

The convergence of GDM-MADDPG in various tasks of the DNN model is illustrated in Fig. 6(a). Despite the differences in structural complexity between DNN models, the proposed method successfully accommodates these tasks and achieves convergence. The AlexNet model, characterized by fewer layers and lower computational requirements, demonstrates favorable convergence performance. In contrast, the VGG16 model, with its greater number of layers and higher computational demands, experiences increased processing delays, resulting in lower rewards and reduced convergence efficacy for our method. Although the convergence performance of our method on the YOLOv5 model is slightly inferior to that of AlexNet, YOLOv5, as a widely adopted real-time object detection algorithm, has a model complexity that lies between the two aforementioned models. Consequently, subsequent experiments employ YOLOv5 as the experimental task model to validate the superior performance of this method in terms of AOI and task completion rate.

We perform ten independent training runs for each algorithm. As illustrated in Fig. 6(b), although the proposed GDM-MADDPG converges at a slower rate than the two algorithms, GDM-MADDPG demonstrates superior performance compared to the other solutions. The total reward attained by GDM-MADDPG increases throughout the training process, eventually converging to approximately 5000 after around 600 training episodes. This enhancement is attributed to the generative capabilities of GDM, which markedly improve action sample efficiency by progressively reducing noise through multiple denoising steps. Finally, the low performance of the two algorithms further highlights the effectiveness of the proposed GDM-MADDPG method.

By conducting a series of systematic experiments, we aim to identify the optimal values for two key parameters that influence the performance of the MADDPG algorithm: 1) learning rate and 2) batch size. The experimental results, illustrated in Fig. 7(a), clarify the convergence behavior of the reward function under varying learning rates. A learning rate of 0.01 is excessively large, preventing convergence even after 1000 training iterations. Similarly, learning rates of 0.0001 do not facilitate convergence. In particular, the 0.001 learning rate promotes faster convergence. The suitable learning rate of 0.001 is used to assess the effects of different batch sizes on training performance. As illustrated in Fig. 7(b), a batch size of 512 is recognized as the most advantageous option.

3) Performance Comparison

This study compares the AoI of different methods in varying task sizes, as illustrated in Fig. 8(a). The AoI performance of the MADDPG-based method is significantly improved when combined with the path-planning approach. Furthermore, the GDM-MADDPG method proposed in this paper, which is based on MADDPG and incorporates the advantages of GDM in handling complex data distributions and generating high-quality task assignment decisions, improves the timeliness and generalizability of task assignment decision generation. This results in the generation of the most suitable task assignment decisions for the current state, thereby reducing the transmission time during serial task execution. According to Eq. (25), this method also addresses the optimization of path planning for target coordinates and tasks, which reduces waiting times associated with tasks and improves AoI.

The task completion rates of different methods were compared in various task sizes, as shown in Fig. 8(b). The method proposed in this paper outperforms other methods in optimizing system utility, indicating that optimized task path planning and task offloading decision generation achieve a joint optimization of multiple objectives. In terms of path planning, the method not only considers the optimal path but also accounts for the task-size requests at the target coordinates, enhancing the selectivity of local optima. Regarding DNN task assignment, factors such as task partitioning decisions and load balancing are incorporated. Additionally, the introduced GDM reverse denoising process improves the learning and generation capabilities of agent decisions, leading to well-generalized action decisions. Consequently, as the task size gradually increases, the MADDPG-based decision method lacks robust dynamic adaptability under multiple constraints. Its learning capability is constrained by the fixed learning mechanism of the actor-critic network, resulting in the absence of a strategy with generalization capabilities.

The utility of different methods was compared under various task sizes, as shown in Fig. 8(c), as task sizes increase, the need for additional task assignment work is incurred under latency constraints, directly increasing the latency of task completion. This results in a downward trend in task completion rates for all methods. The proposed method considers task size constraints during the path planning stage, and task assignment decisions can be rapidly learned and generated through the GDM method, further reducing task processing delays and allowing a greater number of tasks to be completed within the specified time. In addition, these findings indicate that the proposed method is suited for large-scale tasks.