Meta-Gating Framework for Fast and Continuous Resource Optimization in Dynamic Wireless Environments

As mentioned in Section I, we attempt to utilize the learning ability of NNs to achieve the selective plasticity. Therefore, a dual-network structure is proposed, where the outer network extracts the characteristics of each CSI distribution. It aims to ensure the performance of the inner network under the previous CSI distribution when the inner network adapts to samples from a new CSI distribution. Specifically, as shown in Fig. 1, the proposed meta-gating network consists of an inner network, an outer network, and a non-linear layer, where both inner and outer networks are implemented as general NNs. Except that the number of outputs of the inner and outer networks need to be the same, there are no other constraints, e.g., the kind of NNs, the number of neurons in the hidden layer and the number of hidden layers. The inner and outer networks are connected through the gating operation, which refers to as element-wise multiplication of the output vectors of the inner and outer networks. After the multiplication, the results are input to the non-linear layer to obtain the final outputs.

In this part, to achieve aforementioned three important goals, we design a training procedure for the proposed meta-gating framework, which is based on the model-agnostic meta-learning (MAML) algorithm[23] and the unsupervised learning.

Different from the general DL where the wireless networks with different channel states can be directly used as different training samples, the training sample in the proposed training method refers to as a task. Specifically, one task consists of a support set and a query set as shown in Fig. 2, both containing the samples in general DL. The samples in each support set and query set are randomly selected from the different channel distributions.

The proposed training procedure consists of an inner loop and an outer loop, where the inner loop is employed to update the inner network parameters $\bm{\theta}$ on the support set and the outer loop is for updating the outer network parameters $\bm{\phi}$ on the query set. Specifically, the parameters of the inner network are optimized by Adam optimizer[29] for $J$ iterations on the support set with the loss function $\mathcal{L}(\bm{\theta},\bm{\phi})$. During each of these $J$ forward propagation, the outputs of the inner network are gated, i.e. element-wisely multiplied, by the outputs of the outer network, which enables selective activation of the inner network by modifying its ultimate outputs during the forward propagation. Moreover, the gating to the inner network influences the update of the Adam optimizer and would result in selective plasticity during back propagation. After $J$ inner loop iterations, the parameters of the inner networks on task $i$ are denoted by ${\bm{\theta}}_{J}^{i}$, which will be used in the subsequent outer loop. As for the outer loop, the parameters of the outer network $\bm{\phi}$ are updated with the query sets and a meta loss $\mathcal{L}_{meta}$, which is calculated based on ${\bm{\theta}}_{J}^{i}$ and $\bm{\phi}$. The detailed training procedure of the proposed meta-gating framework is summarized in Algorithm 1.

Following the aforementioned training procedure, the framework can well achieve aforementioned three goals and the reasons are as follows. First, the proposed framework can achieve the fast adaptation with small amount of samples because of the suitable initialization obtained by the MAML method. Thus, the proposed training method can well achieve the goal of ‘seamlessness’ and ‘quickness’. Secondly, the selective plasticity is achieved by the gating operation. Specifically, the importance of model parameters in response to different CSI distributions is different. The outer network is trained by the outer loop of Algorithm 1 with tasks from multiple CSI distributions. Therefore, the outer network can learn to evaluate the importance of inner network’s parameters under different CSI distributions, and then decide which subset of the inner network should be activated. By the gating operation, the meta-learned outer network can convey the decision to the inner network and thus indirectly influence the back propagation of the inner network. In this way, the inner network can perform well under both the previous and current CSI distribution, so as to overcome the CF problem.¹¹1Similarly, the graceful forgetting ability can be achieved by adjusting the learning abilities of inner network and outer networks, e.g, increasing the number of layers of the inner network within a certain range or adding a mask to the gating operation between the inner and outer network. The testing procedure of the proposed framework is summarized in Algorithm 2.

In this part, we first model the wireless environment as a graph, and then formulate Problem $\mathcal{P}_{2}$ as a graph optimization problem.

In general, the wireless environment can be modeled as a weighted directed graph with both node and edge features. Formally, a graph can be represented as a four tuple $\mathcal{G}=(\mathcal{V},\mathcal{E},f,\bm{\alpha})$, where $\mathcal{V}$ is the set of nodes and $\mathcal{E}$ is the set of edges. For each node in $\mathcal{V}$, function $f$ maps it to its corresponding feature vector. For each edge in $\mathcal{E}$, it has a corresponding weight $\alpha(i,j)\in\bm{\alpha}$.

Then the SINR in (2) can be rewritten with the notations $\mathbf{Z}$, $\bm{\alpha}$, and $\mathbf{V}$ as follows

Finally, Problem $\mathcal{P}_{2}$ in each period can be reformulated as

As depicted in Fig. 4(b), the input data of meta-gating GNN is the graph model in Fig. 4(a) and the final outputs are the optimal beamformer matrix $\mathbf{V}$ in each period. Both inner and outer networks are implemented as wireless communication graph convolution network (WCGCN) [6], which belongs to the message passing graph neural network (MPGNN). Before introducing the WCGCN model, we first describe the mechanism of MPGNN. Specifically, the update process (key operation of GNNs) of the $n$-th layer at node $k$ in an MPGNN is describe as

where $\bm{x}_{k}^{n}$ represents the hidden state of the $n$-th layer at node $k$, $\bm{x}_{k}^{0}$ is the input node feature vector of node $k$, $\mathcal{N}(k)$ denotes the neighbors of node $k$, and $\bm{e}_{jk}$ is the input edge feature vector of edge $(j,k)$. Moreover, $\beta(\cdot)$ is the function that aggregates information from the neighbors of node $k$ and $\gamma(\cdot)$ is the function that combines the aggregated information with its own information, which can be seen in Fig. 4(b). Furthermore, $\beta(\cdot)$ can be further simplified by applying NNs as follows

where $\psi$ is implemented by some simple functions, such as ${\rm{MAX}}$ and ${\rm{SUM}}$, and $f$ is the existing NN structure.

In the WCGCN model, ${\rm{MAX}}$ is utilized as function $\psi$, and two different multi-layer perceptrons (MLPs) are applied to function $\gamma(\cdot)$ and $f(\cdot)$, respectively. Thus, its update process of node $k$ can be expressed as $\bm{x}_{k}^{n}={\rm{MLP}}_{2}(\bm{x}_{k}^{n-1},{\rm{MAX}}_{j\in\mathcal{N}(k)}\left\{{\rm{MLP}}_{1}([\bm{x}_{j}^{n-1},\bm{e}_{jk}])\right\})$. Then, the forward propagation of node $k$ in the proposed meta-gating GNN can be expressed as

where $\bm{x}_{k}^{0}=\hat{\bm{x}}_{k}^{0}=\bm{Z}_{(k,:)}$ is the input node feature, $\bm{\alpha}_{(j,k)}$ is the weight matrix that represents the input edge feature, $N$ and $M$ denote the number of layers in the inner and outer networks, respectively, $\sigma(\cdot)$ is a differentiable normalization function in the non-linear layer, and $\odot$ denotes the element-wise multiplication operation.

Due to the lack of an optimal solution to Problem $\mathcal{P}_{2}$, unsupervised training that directly maximizes the sum rate is applied for solving the considered problem. The loss function to be minimized can be written as

For the meta-gating GNN, the inner and outer networks are implemented by WCGCNs. The complexity of WCGCN is $\mathcal{O}(L(|E|+|V|))$, where $L$ is the number of layers of WCGCN, $|E|$ denotes the size of edge set, and $|V|$ denotes the size of node set. Therefore, the complexity of the proposed framework is $\mathcal{O}({\rm{MAX}}\{N,M\}(|\mathcal{E}|+|\mathcal{V}|))$.

CNN has been widely used in DL, e.g., it can extract spatial features from an image for classification. In a wireless environment, CNN is generally utilized to exploit the spatial features in channel state. It is because that the nearby receiver plays more significant role in determining the beamformer of the $k$-th transmitter. Besides, CNN has fewer number of trainable parameters compared to DNN and can greatly reduce the training overhead. Based on this, we model $\mathbf{H}=\{\mathbf{h}_{j,k},\forall j,k\}$ in Problem $\mathcal{P}_{2}$ as a picture-like pixel structure, as shown in Fig. 5(a).

As depicted in Fig. 5(b), the input data of meta-gating CNN is the picture-like pixel structure and the final outputs are the the optimal beamformer matrix $\mathbf{V}$ in each period. Both inner and outer networks are implemented as general CNNs[31]. The forward propagation in the proposed framework can be expressed as

where $\mathbf{x}^{0}=\hat{\mathbf{x}}^{0}=\mathbf{H}$, $\mathbf{x}^{n}$ and $\hat{\mathbf{x}}^{m}$ denote the $n$-th and $m$-th hidden state of the inner network and the outer network, respectively. $\mathbf{u}^{N}$ and $\mathbf{u}^{M}$ represent the outputs of the inner and outer networks, respectively. ${\rm{ReLU}}$ represents the rectified linear unit layer to prevent the negative values, ${\rm{FC}}$ represents the fully-connected layer, and ${\rm{MP}}$ denotes the max-pooling operation. $\sigma(\cdot)$ is a differentiable normalization function, and $\mathbf{y}$ denotes the final outputs. $N$ and $M$ represent the layer number of the inner and outer networks, respectively, and ${\rm{Conv}}$ represents the convolution layer that performs two-dimensional spatial convolution of the input data. The size of the convolution layer is denoted as $s(\hat{s})$ and its depth is set to $c_{\rm{in}},c_{\rm{out}}(\hat{c}_{\rm{in}},\hat{c}_{\rm{out}})$.

Similarly, we employ the unsupervised training for the considered problem and the loss function to be minimized can be written as

For the meta-gating CNN, the inner and outer networks are implemented by CNNs. The complexity of CNN is $\mathcal{O}(\sum\limits_{l=1}^{L}Q_{l}^{2}S_{l}^{2}C_{l-1}C_{l})$, where $L$ is the number of layers of CNN, $Q_{l}$ denotes the output size of $l$-th layer, $S_{l}$ represents the size of convolution kernel, and $C_{l}$ is the number of channels in the $l$-th layer. Therefore, the complexity of the proposed framework is $\mathcal{O}({\rm{MAX}}\{\sum\limits_{n=1}^{N}Q_{n}^{2}s_{n}^{2}c_{n-1}c_{n},\sum\limits_{m=1}^{M}{\hat{Q}_{m}}^{2}{\hat{s}_{m}}^{2}\hat{c}_{m-1}\hat{c}_{m}\})$, where $Q$ and $\hat{Q}$ denote the output size of inner and outer networks, respectively.

(i) Seamlessness. Fig. 8 compares the proposed meta-gating framework with the above benchmark algorithms on the sum-rate performance, where the channel state samples in ‘Mismatch’ are collected from ${\rm{channel}}_{3}$. Obviously, applying a model trained under one distribution to test the samples on another distribution does lead to a quite large sum-rate performance loss. Moreover, we can observe that the sum-rate performance of our meta-gating framework is better than that of TL under the premise of the same number of fine-tune samples, update rounds, and optimizer. It is mainly because that the proposed framework has better initializations compared to the TL and thus achieves better sum-rate performance using only a small number of update rounds. Besides, compared with TL, the MAML algorithm can obtain a suitable model initialization for all three channels. Thus, the average sum-rate performance is better than that of TL.

Similar with the TL, the sum-rate performance of ‘WoGate’ on ${\rm{channel}}_{2}$ and ${\rm{channel}}_{3}$ are affected by the online update according to the previous channel state samples, which can be seen from the gap between ‘proposed’ and ‘woGate’. In the proposed framework, the inner network updates according to the current channel sample data, where only part of the model parameters that are selected by the meta-learned outer network will be updated. Therefore, the proposed framework can still achieve good sum-rate performance on the current channel state sample and is not largely affected by the previous channel state samples.

Table III presents the variance of the sum-rate performance among different channel distributions under different methods. It can be observed that the proposed framework has the smallest variance, indicating that the sum-rate performance on different channel distributions is basically similar. Therefore, the proposed framework can well achieve the goals of ‘seamlessness’.

Furthermore, we compare the sum-rate performance of the proposed meta-gating framework under different values of maximum transmit power $P_{\rm{max}}$, and the results are depicted in Fig. 8. From the figure, the proposed framework achieves good sum-rate performance under different $P_{\rm{max}}$, i.e., the sum-rate performance is basically the same as the WMMSE algorithm. Moreover, Table IV presents the variances of the normalized sum rate among different channel distributions under different values of $P_{\rm{max}}$. From the table, these variances are all quite small which further highlights the advantage of the proposed meta-gating framework in terms of ‘seamlessness’.

(ii) Quickness. Fig. 10 depicts the impact of different values of $J_{q}$ on the sum-rate performance. From the figure, the proposed framework only needs a small value of $J_{q}$ to achieve good sum-rate performance under each channel distribution ($J_{q}=10$ in this simulation scenario). It is mainly because that the proposed meta-gating framework has good model initializations via the proposed training procedure. Besides, we can see that the sum-rate performance under different channel distributions first increases and then decreases with the increase of $J_{q}$. The degradation of the sum-rate performance is mainly caused by the severe overfitting on the small amount of adaptation samples $N_{a}$. In fact, the fine-tune process with small amount of samples exactly achieves the goal of ‘quickness’, where the value of $J_{q}$ is small to avoid serious overfitting phenomenon. The simulation results are consistent with the theoretical analysis in Section IV-B.

(iii) Continuity. Fig. 10 depicts the ‘continuity’ capability of different methods. Specifically, it shows the sum-rate performance of the proposed framework on ${\rm{channel}}_{j}$ (the vertical axis) after updating according to the ${\rm{channel}}_{i}$ (the horizontal axis). In order to clearly illustrate the capability for continuous adaptation of different methods, we set the value of the normalized sum rate in ${\rm{channel}}_{j}$ as $1$ when $j\textgreater i$. From the figure, the sum-rate performance of TL on the previous channel suffers from a significant degradation when it adapts to the following new channel distribution. It is mainly because that the model in TL is only fine-tuned on the latest new samples. After learning the knowledge of new samples, the knowledge from the previous model may be altered or even overwritten, which thus results in significant performance deterioration on the previous samples. Similar results can be seen from ‘WoGate’. On the other hand, the proposed meta-gating framework utilizes the outer network to evaluate the importance of inner network’s parameters under different CSI distributions and then decide which subset of the inner network should be activated through the gating operation. Therefore, it can ensure the capability for continuous adaptation.

According to the analysis in Section IV, we compute the similarity between the considered three channel distributions and the results are given in Table V. From the table, the distance between ${\rm{channel}}_{1}$ and ${\rm{channel}}_{2}$ is the largest, indicating that the distribution between these two channels are quite different. Therefore, it will cause the largest performance loss in ${\rm{channel}}_{1}$ when the model is updated according to the samples in ${\rm{channel}}_{2}$ in TL. Moreover, the CDS metric shown in Table V can explain the capability for continuous adaptation of the EWC method in Fig. 10. Specifically, channel samples under each distribution are sequentially input during the EWC training stage. In order not to forget the knowledge learned on ${\rm{channel}}_{1}$, the update of model’s parameters on ${\rm{channel}}_{2}$ will be affected by the consolation operation, where the distance between ${\rm{channel}}_{1}$ and ${\rm{channel}}_{2}$ is the largest. Therefore, it finally leads to poor sum-rate performance on ${\rm{channel}}_{2}$.

Furthermore, Fig. 12 compares the capability for continuous adaptation of the proposed meta-gating framework under different values of $P_{\rm{max}}$. From the figure, the proposed framework achieves the good capability for continuous adaptation under each value of $P_{\rm{max}}$, i.e., the sum-rate performance on ${\rm{channel}}_{i}$ does not largely degrade when the model is updated according to the samples of the ${\rm{channel}}_{j}$. It further indicates the advantage of the proposed framework in terms of ‘continuity’.

It is quite important for TL to select a suitable pre-train model since the model initializations make great influence on the adaptation. Fig. 12 presents the impact of different pre-train models on the sum-rate performance, where TL1, TL2, and TL3 represent the models trained with the samples of ${\rm{channel}}_{1}$, ${\rm{channel}}_{2}$, and ${\rm{channel}}_{3}$ as pre-train models, respectively. Since there does exist differences between each channel distribution, the sum-rate performance with different pre-train models will be quite different. In contrast, the proposed meta-gating framework achieves a better sum-rate performance because of its adaptivity on different channel distributions.

The performance of the EWC method largely depends on the coefficient of the penalty term, $w_{p}$. To verify it, we test the sum-rate performance on each channel distribution and the capability for continuous adaptation with $w_{p}=10^{2},10^{4},10^{6}$, which can be seen in Fig. 13. It is observed that a large value of $w_{p}$ can indeed improve the sum-rate performance on the previous channel, i.e., enhance the capability for continuous adaptation of the model, but with the cost of significant sum-rate performance loss on the current channel. Therefore, it is important for the EWC method to select an appropriate value of $w_{p}$, which is the disadvantage of the EWC method. Different from the EWC method that needs to be manually implemented, the proposed meta-gating framework can continuously achieve the good sum-rate performance because of the proposed training procedure and the gating operation.

In this part, we test the sum-rate performance and the capability for continuous adaptation with different numbers of users ($K=10$, $20$ and $30$) in the same area with radius $R=1000$ m, where the number of update round $J_{q}$ is chosen as $10$. As depicted in Fig. 15 and Table VI, the proposed meta-gating framework achieves good sum-rate performances with different numbers of users and the variances of the normalized sum rate among different channel distributions under different numbers of users are all quite small, which further highlights the advantage of the proposed meta-gating framework in terms of ‘seamlessness’. Besides, as shown in Fig. 15, the proposed framework achieves the good capability for continuous adaptation with different numbers of users, i.e., the sum-rate performance on ${\rm{channel}}_{j}$ does not largely degrade when the model is updated according to the samples of the ${\rm{channel}}_{i}$. These results demonstrate the good scalability of the proposed framework with more practical simulation scenario settings.

(i) Seamlessness. Fig. 16 compares the proposed meta-gating framework with other benchmark algorithms on the sum-rate performance. The proposed meta-gating framework has the best sum-rate performance on each channel distribution compared to the benchmark algorithms due to its better model initializations. Table IX presents the variance of the sum-rate performance among each channel distribution with different methods. It can be observed that the proposed framework has quite small variance, which indicates that the proposed framework can achieve similar sum-rate performances on different channel distributions. Therefore, it can well achieve the goals of ‘seamlessness’. These results are similar as those observed in Fig. 8 and Table III.

(ii) Quickness. Fig. 18 depicts the impact of different values of $J_{q}$ on the sum-rate performance, where the similar conclusion can be concluded as those in Fig. 10.

(ii) Continuity. Fig. 18 depicts the capability for continuous adaptation of different methods and we set the value of the normalized sum rate in ${\rm{channel}}_{j}$ as $0.5$ when $j\textgreater i$. Similarly, we compute the similarity among these three channels, which can be seen in Table X. From the table, the distance between ${\rm{channel}}_{1}$ and ${\rm{channel}}_{3}$ is the largest. Therefore, it would cause a large performance loss in ${\rm{channel}}_{1}$ when the model is updated according to the samples in ${\rm{channel}}_{3}$ in TL, as shown in Fig. 18.

To conclude, simulation results demonstrate that the proposed framework is model-agnostic, i.e., it can well achieve three goals compared with the state-of-the-art algorithms on the proposed problem for both CNN and GNN models.

To verify Theorem 2 with simulation experiments, we test the sum-rate performances on the Nakagami-$m$ channels. The reason why we apply the Nakagami-$m$ channel is that it is a more general fading channel model and the Nakagami fading can be transformed into a variety of fading models by changing the value of $m$ (e.g., it can be degenerated into Rayleigh fading when $m=1$). Specifically, we randomly generate four kinds of channels where $|h_{j,k}|\sim Nakagami(m,\Omega)$ and $m$ follows a uniform distribution between $[0.5,2]$ in the training stage. Similarly, we generate four kinds of channels in the testing stage, where the latter two kinds of channels are unseen channels following the Nakagami-$m$ distribution with different values of $m$ from the training stage.

Table XI presents the sum-rate performances of joint training method and proposed framework in both seen and unseen channels with different values of $P_{\rm{max}}$. It is observed that the proposed framework achieves a similar sum-rate performance with the joint training method on seen channels but significantly outperforms the joint training method on unseen channels, where the sum-rate performance gaps are further depicted in Fig. 19. It is mainly because that the joint training method only focuses on the distribution on the seen channels, but the proposed framework can better take further optimizations into account because of the dual-loop optimization for better model initializations.