Self-Improving Interference Management Based on Deep Learning With Uncertainty Quantification

We consider a downlink network comprising $N$ pairs of single-antenna transceivers, each pair consisting of a base station (BS) and a corresponding scheduled user. The set of BSs is defined as $\mathcal{N}=\{1,2,...,N\}$. Let $h_{nn}$ denote the channel gain between BS $n$ and its scheduled user, and $h_{nm}$ the interference channel gain from BS $m$ to the scheduled user of BS $n$. The channel gain matrix is defined as $\mathbf{H}\coloneqq[h_{ij}]_{i,j\in\mathcal{N}}$. We assume that the channels main unchanged during each timeslot. We denote the transmission power of BS $n$ as $p_{n}$ and define the vector of transmission powers as $\mathbf{p}\coloneqq[p_{n}]_{n\in\mathcal{N}}$. Then, the signal to interference-plus-noise ratio (SINR) for the user of BS $n$ is given by

where $\sigma_{n}^{2}$ is the noise power at that user. The sum-rate of the network is given by

To maximize the weighted sum-rate, a power allocation problem for managing downlink interference is formulated as

where $\mathcal{P}\coloneqq[0,P^{\textrm{max}}]^{N}$ with $P^{\textrm{max}}$ being the maximum transmission power. The optimal solution for a given $\mathbf{H}$ is denoted as $\mathbf{p}^{*}(\mathbf{H})$.

In the literature, a number of algorithms have been developed to address the interference management problem as formulated in (3) with a given channel gain matrix $\mathbf{H}$ [1, 2, 3]. Most of these algorithms employ an iterative approach, and their effectiveness has been demonstrated through simulation results and theoretical analyses. However, implementing these algorithms in practical systems proves, in many cases, challenging due to their computational complexity.

To overcome this issue, recently, deep learning-based approaches to interference management have gained significant attention [4, 5]. These approaches treat an interference management algorithm as a function $f(\mathbf{H})$ that takes $\mathbf{H}$ as input and yields $\mathbf{p}^{*}(\mathbf{H})$ as output. Then, a deep neural network (DNN) model, $g_{\bm{\uptheta}}(\mathbf{H})$, is trained to approximate $f(\mathbf{H})$, where $\bm{\uptheta}$ represents the weights of the DNN model. After training, the approximated solution $\hat{\mathbf{p}}^{*}(\mathbf{H})$ can be rapidly computed using the DNN model, requiring far fewer calculations than traditional algorithms. Fig. 1 illustrates a typical deep learning-based approach to interference management. This approach involves generating a dataset from the target algorithm for given channel gain matrices, i.e., pairs of $(\mathbf{H},\mathbf{p}^{*}(\mathbf{H}))$. Then, the DNN model is trained using this dataset, as shown in Fig. 1a, and subsequently applied to interference management, as shown in Fig. 1b. This implies that the efficacy of the DNN model highly depends on the dataset used for training. However, generating a comprehensive dataset that encompasses all possible channel conditions and corresponding power allocations is impractical. Consequently, it is inherently challenging to ensure the DNN model’s effectiveness in all conceivable interference scenarios, especially as some scenarios might exceed the generalization capabilities of deep learning and fall outside the scope of the training dataset.

The quantification of predictive uncertainty in deep learning has been widely studied due to its role in assessing the trustworthiness of predictions. Predictive uncertainty mainly stems from two sources: aleatoric and epistemic uncertainties [6]. Aleatoric uncertainty is associated with inherent variabilities in the data distribution, rather than a characteristic of the DNN models. As such, it is considered irreducible, meaning that it cannot be reduced through the collection of more data or improvements in the models. In contrast, epistemic uncertainty stems from the limitations in DNN models, often due to insufficient training or inadequate experiences. This type of uncertainty is typically prominent in regions of the input feature space that are not well-represented by the training dataset and is, therefore, reducible. For more detailed theoretical background on uncertainty quantification, readers are referred to [6].

Deep ensemble is one of the representative uncertainty quantification methods in deep learning [7]. In uncertainty quantification, the output for a given input feature is treated as a random variable encompassing uncertainty. Therefore, each DNN model in deep ensemble aims to approximate both the mean and variance of the output value of a given input feature, in contrast to typical DNN models that focus on approximating a single output value. To effectively quantify uncertainties, deep ensemble involves training multiple models, each with distinct random initialization. The prediction from these models are subsequently integrated to derive the final estimate.

We now describe an uncertainty quantification approach in deep learning-based interference management using deep ensemble.¹¹1In this paper, we use deep ensemble for quantifying uncertainty in deep learning due to its simplicity of implementation and adaptability to various types of NNs. However, other uncertainty quantification methods, such as Bayesian NN [8] and conformal regression [9], can also be applied within our proposed framework. To quantify uncertainty in interference management, we consider $M$ DNN models that jointly approximate the following distributional parameters of transmission power $p^{*}_{n}$ for a given channel gain matrix $\mathbf{H}$: the mean function $\mu_{n}(\mathbf{H})$ and the variance function $\sigma^{2}_{n}(\mathbf{H})$, where $\mu_{n}$ and $\sigma^{2}_{n}$ describe the solution $p^{*}_{n}(\mathbf{H})$ and the uncertainty of the estimate, respectively. We denote the estimation of $\mu_{n}$ and $\sigma^{2}_{n}$ by the $m$th DNN model as $\hat{\mu}_{n,\bm{\uptheta}_{m}}$ and $\hat{\sigma}^{2}_{n,\bm{\uptheta}_{m}}$, respectively. Note that we cannot simply use a mean squared error (MSE) loss function, a standard loss function for regression problems, as a loss function for training the models since the variance $\sigma^{2}_{n,\bm{\uptheta}_{m}}$ cannot be trained via the MSE. Hence, to train the models for approximating both mean and variance, we employ a negative log-likelihood function as the loss function:

where $c$ is a constant added to the loss function for practical reasons, such as numerical stability, without altering the fundamental behavior of the optimization process. Then, by using the loss function, the DNN models with random initialization are trained by using given dataset.

We now estimate the uncertainty of the predicted solution $\hat{}\mathbf{p}^{*}$ by averaging the predictions of $M$ DNN models, which implies that the individual distributions are ensembled as a uniformly-weighted mixture model. For transmission power $p_{n}^{*}$, the averaged mean is given by $\hat{\mu}_{n}(\mathbf{H})=\frac{1}{M}\sum_{m=1}^{M}\hat{\mu}_{n,\bm{\uptheta}_{m}}(\mathbf{H})$, and the averaged variance is given by $\hat{\sigma}^{2}_{n}(\mathbf{H})=\frac{1}{M}\sum_{m=1}^{M}\left(\hat{\sigma}^{2}_{n,\bm{\uptheta}_{m}}(\mathbf{H})+\hat{\mu}^{2}_{n,\bm{\uptheta}_{m}}(\mathbf{H})\right)-\hat{\mu}_{n}$. We can further decompose the variance as

where the first term, the averaged estimated variance over all models, signifies aleatoric variance, and the second term, which goes to zero as the mean predictions of all models become identical, signifies epistemic variance. Consequently, as shown in Fig. 2a, the transmission power $p^{*}_{n}(\mathbf{H})$ is predicted as $\hat{\mu}_{n}(\mathbf{H})$, and its corresponding uncertainty is estimated as $\hat{\sigma}_{n}^{2}(\mathbf{H})$ by employing the DNN models.

The quantified uncertainty of predictions can be used to improve deep learning [10, 11]. Commonly, typical uncertainty-aware deep learning methods in machine learning literature focus on maximizing prediction accuracy by leveraging uncertainty. However, in the realm of wireless communications, the goal of deep learning-based interference management is not to precisely estimate the power control solution, but to improve the system performance, specifically by maximizing the sum-rate as in (3). Hence, when designing self-improving deep learning, it is insufficient to simply consider the quantified uncertainty of the predicted solution as in (5) for effectively pursuing the goal of interference management.

To address this issue, we propose a novel framework for self-improving interference management that elaborately considers actual system performance in light of the quantified uncertainty in deep learning. In the framework, first during the training stage, the interference management and corresponding uncertainty quantification models are trained as described in Section III-B. In the subsequent self-improving stage, for each instance of $\mathbf{H}$, the framework obtains the predicted solution $\hat{\mathbf{p}}(\mathbf{H})$ and its quantified uncertainty. Then, based on the quantified uncertainty, the credibility of the prediction is qualified. If the predicted solution is deemed credible, it is used as is. If not, the predicted solution is refined using existing optimization-based methods,²²2Note that the predicted solution itself is a plausible solution. Hence, enhancing this predicted solution is substantially more cost-efficient compared to directly seeking the optimal solution from scratch. This efficiency is evident in our empirical experiments. In one sample case, using the predicted solution $\hat{\mathbf{p}}^{*}$ as an initial point significantly reduces computational time. The WMMSE algorithm requires only $0.096$ ms of CPU time when starting with $\hat{\mathbf{p}}^{*}$, as opposed to $8.628$ ms when no initial point is provided. and this improved solution is then used to further enhance the DNN models, as illustrated in Fig. 2b.

To qualify the credibility, we need to design a qualifying criterion that can assess how much the predicted solution is close to the optimal one in terms of the system performance. For design, we first construct a confidence interval for each predicted transmission power, using the standard deviation of epistemic uncertainty $\hat{\sigma}_{n,{\rm epi}}$, as

where $\alpha$ is the control parameter that determines the confidence level. With the confidence interval, we can define an uncertainty-aware feasible set of solutions as $\tilde{\mathcal{P}}=\prod_{n=1}^{N}\tilde{\mathcal{P}}_{n}$. Since the optimal solution is likely to belong to the uncertainty-aware feasible set, comparing the sum-rate with the predicted solution (i.e., $\hat{R}=R(\mathbf{H},\hat{\mathbf{p}})$) and the maximum sum-rate achievable within the set $\tilde{\mathcal{P}}$ (i.e., $\tilde{R}=\max_{\mathbf{p}\in\tilde{\mathcal{P}}}R(\mathbf{H},\mathbf{p})$) is a straightforward way to design the criterion. If $\hat{R}$ is sufficiently close to $\tilde{R}$, the predicted solution $\hat{\mathbf{p}}^{*}$ is considered credible from the perspective of system performance, and if not, it requires enhancement. However, identifying the maximum achievable sum-rate within the feasible set is equivalent to solving the power allocation problem for interference management. Consequently, the comparison of $\hat{R}$ and $\tilde{R}$ is not practical.

Instead, we design a qualifying criterion to evaluate whether the uncertainty-aware feasible set is sufficiently small to minimally affect the sum-rate. This approach is plausible for achieving a sum-rate close to the optimal one, as the feasible set likely contains the optimal solution; the sum-rate achieved by the predicted solution will be close to the optimal one if the feasible set is sufficiently small. To this end, we define the maximal and minimal interference solutions as $\hat{\mathbf{p}}^{U}=[\hat{p}_{n}^{U}]_{n\in\mathcal{N}}$ and $\hat{\mathbf{p}}^{L}=[\hat{p}_{n}^{L}]_{n\in\mathcal{N}}$, respectively, and calculate their corresponding sum-rates as $\hat{R}^{U}=R(\mathbf{H},\hat{\mathbf{p}}^{U})$ and $\hat{R}^{L}=R(\mathbf{H},\hat{\mathbf{p}}^{L})$, respectively. Then, we design a qualifying criterion involving $\hat{R}$, $\hat{R}^{U}$, and $\hat{R}^{L}$ as

where $\textrm{maxdist}(a,b,c)=\max\{\lvert a-b\rvert,\lvert a-c\rvert,\lvert b-c\rvert\}$, and $\epsilon$ is a criterion value that controls the criterion’s sensitivity. This criterion evaluates whether the uncertainty-aware feasible set is small enough so that the sum-rates achieved by the extreme solutions within the feasible set and the predicted solution (i.e., $\hat{R}^{U}$, $\hat{R}^{L}$, and $\hat{R}$) appear similar. If this criterion is satisfied, the predicted solution $\hat{\mathbf{p}}^{*}$ is considered credible since $\tilde{R}$ is expected to be close to $\hat{R}$. On the other hand, if not due to a large uncertainty-aware feasible set, the prediction solution requires enhancement.

Here, we describe the self-improving interference management framework, illustrated in Fig. 2 and detailed in Algorithm 1. First, in the training stage (Fig. 2a), a dataset for interference management is generated using the target algorithm and channel instances as $(\mathbf{H},\mathbf{p}^{*}(\mathbf{H}))$’s (line 1). We denote this dataset by $\mathcal{D}$. For prediction and uncertainty quantification, we construct $M$ DNN models, each outputting the means and variances of the transmission power $p^{*}_{n}$’s as $g_{\bm{\uptheta}_{m}}(\mathbf{H})=[\hat{\mu}_{n,\bm{\uptheta}_{m}}(\mathbf{H}),\hat{\sigma}^{2}_{n,\bm{\uptheta}_{m}}(\mathbf{H})]_{n\in\mathcal{N}}$ (line 2).³³3It is worth noting that any kind of DNN model capable of producing mean and variance as outputs can be used for self-improving interference management. The weights of each DNN model $m$, $\bm{\uptheta}_{m}$, are randomly initialized (line 3) and trained to minimize the negative log-likelihood loss function in (4) with respect to $\bm{\uptheta}_{m}$ (lines 4–6). During this training stage, various learning techniques such as cross-validation or early stopping can be employed.

In the self-improving stage (Fig. 2b), for each arrival of interference management request with instance $\mathbf{H}$, the predicted solution $\hat{\mathbf{p}}^{*}(\mathbf{H})$ and its epistemic standard deviation $\hat{\sigma}_{n,{\rm epi}}(\mathbf{H})$ are computed using deep ensemble with $M$ DNN models as in Section III-B (lines 8–9). Using these calculations, the uncertainty-aware feasible set $\tilde{\mathcal{P}}(\mathbf{H})$ is constructed with $\tilde{\mathcal{P}}_{n}$’s in (6) (line 10). Then, the qualifying criterion in (7) is examined with $\tilde{\mathcal{P}}$; if the criterion is met, the predicted solution $\hat{\mathbf{p}}^{*}(\mathbf{H})$ is directly used for transmission (line 12). If not, it is enhanced by using the target algorithm starting from the initial point $\hat{\mathbf{p}}^{*}(\mathbf{H})$, and the enhanced solution is then used for transmission (line 14). The enhanced solution (i.e., $\hat{\mathbf{p}}^{*}$) is collected into a self-improving dataset $\mathcal{D}_{\rm SI}$ (line 15). The DNN models, $\bm{\uptheta}_{m}$’s, are improved using $\mathcal{D}_{\rm SI}$ if the number of collected enhanced solutions is large enough to improve the models, i.e., $\lvert\mathcal{D}_{\rm SI}\rvert\geq N_{\rm SI}$, where $\lvert\mathcal{X}\rvert$ denotes the cardinality of set $\mathcal{X}$, and $N_{\rm SI}$ denotes the required number of samples for model improvement. The self-improving dataset is cleared once it is used (lines 17–19). The model improvement can be achieved either by simply retraining the DNN models with the combined dataset $\mathcal{D}\cup\mathcal{D}_{\rm SI}$ or by adopting continual learning techniques [12].