Attention Mechanism Based Intelligent Channel Feedback for mmWave Massive MIMO Systems

Considering a typical mmWave FDD MIMO communication system, we assume that the BS is equipped with $N_{t}$ antennas in the form of uniform linear array (ULA)¹¹1We adopt the ULA model here for simpler illustration, nevertheless, the proposed approach does not restrict to the specifical array shape. and the UE is equipped with $N_{r}$ antennas $(N_{t}\gg N_{r})$. Meanwhile, the orthogonal frequency division multiplexing (OFDM) technique is applied to the link-level channel model. Then, the received signal at the UE can be expressed as,

where $\mathbf{H}\in\mathbb{C}^{N_{RB}\times N_{r}\times N_{t}}$ is the downlink CSI between the BS and the UE, $n$ denotes the noise vector. For an OFDM system, it is necessary to consider multiple subcarriers and OFDM symbols. In this paper, resource blocks (RBs) are used as the channel matrix resolution. Thus, $N_{RB}$ represents the number of RBs used in link-level channel model. Considering a single RB and a pair of transmit and receive antennas, a common multi-path fading channel model [32] is used and can be expressed as,

where $N$ and $M$ denotes the number of scattering clusters and ray pathes, respectively, $P_{m,n}$ represents the power of $m$-th ray in the $n$-th scattering cluster, $c_{n,m}$ is the coefficient calculated by field patterns and initial random phases, $\theta_{n,m}$ is the corresponding azimuth angle-of-departure (AoD) of ray path, $v_{n,m}$ stands for the speed and can be understood as Doppler shift parameter. Then, the the steering vector $\bm{\alpha}(\theta_{n,m})\in\mathbb{C}^{N_{t}\times 1}$ can be formulated as,

where $d$ and $\lambda$ are the antenna element spacing and carrier wavelength, respectively.

After channel model, we further discuss the probability distribution of LOS channel. Considering an urban macro (UMa) scenario defined by 3GPP TR38.901 [33], we assume that the plane straight-line distance from the UE to the BS is $d_{2D}$ and the LOS probability is $\mathrm{Pr}_{LOS}$. If $d_{2D}\leq 18~{}{\rm m}$, then $\mathrm{Pr}_{LOS}=1$, else the $\mathrm{Pr}_{LOS}$ can be calculated via

where the $C(h_{UT})$ can be found in (5), and the $h_{UT}$ denotes the antenna height for the UE.

To sum up, NLOS channel is a more common scenario with the popularization of mmWave systems.

This subsection shows the advantages of SVD transformation and its application in wireless communication. In order to reduce the conflict between multi-rays and increase the channel capacity in massive MIMO system, the transmitter needs to use the beamforming technology to precode the data flow according to the quality of channel. A conventional precoding matrix is based on SVD transformation of CSI matrix.

Considering channel model mentioned above with CSI matrix as $\mathbf{H}\in\mathbb{C}^{N_{RB}\times N_{r}\times N_{t}}$. For simplicity of description, we discuss only one RB here²²2This assumption is only for the brief illustration of SVD, and this solution is also applicable to OFDM systems., i.e. $RB=1$ and $\mathbf{H}\in\mathbb{C}^{N_{r}\times N_{t}}$. First, the CSI matrix $\mathbf{H}$ should carry on SVD transformation as

where $\mathbf{U}\in\mathbb{C}^{N_{r}\times N_{r}}$ and $\mathbf{V}\in\mathbb{C}^{N_{t}\times N_{t}}$ are the left-singular and the right-singular matrices³³3Both $\mathbf{U}$ and $\mathbf{V}$ will be called as eigenmatrix in the follows., respectively. What is more, $\mathbf{U}\mathbf{U}^{*}=\mathbf{I}_{N_{r}},\mathbf{V}\mathbf{V}^{*}=\mathbf{I}_{N_{t}}$ ⁴⁴4$\mathbf{X}^{*}$ denotes conjugate transpose matrix of $\mathbf{X}$.. Note that $\mathbf{\Sigma}=(\Lambda,0)$ and $\Lambda$ can be expressed as follows:

which represents the singular value matrix. And we define the eigenvalues of $\mathbf{H}\mathbf{H}^{*}$ as $\mathbf{s}=\left[\lambda_{1},\lambda_{2},\cdots,\lambda_{N_{r}}\right]$. Next, the application of SVD transformation is introduced in detail. The unitary matrices $\mathbf{V}$ and $\mathbf{U}$ are used as precoding matrices for transmitter and receiver, respectively. When BS needs to sent the parallel data flow $\bm{x}=[x_{1},x_{2},\cdots,x_{N_{t}}]^{T}$ to multiuser, right-singular matrix $\mathbf{V}$ will be used for precoding: $\bm{x}_{t}=\mathbf{V}\cdot\bm{x}$. Thirdly, we consider a typical signal transmission model as

where $\bm{y}$ is the received data flow and $\bm{n}$ denotes the noise vector. The channel matrix $\mathbf{H}$ can be expressed by (6), and we can obtain

Finally, the receiver will use $\mathbf{U}^{*}$ for receiver combining, which can be expressed as

The noise component in (10) will be filtered out by the receiver. Therefore, the receiver can recover the data flow $\bm{x}$ by $\mathbf{\Sigma}$.

In summary, as a transmitter, the BS should pay more attention to eigenmatrix $\mathbf{V}$, which can support the following precoding module. And eigenvector $\mathbf{\Sigma}$ should be applied to receiver combining the data flow when the BS is a receiver.

As analyzed above, the CSI matrix $\mathbf{H}$ of a massive MIMO system is too complex to compress and reconstruct at the BS accurately. Meanwhile, considering the specific application of downlink CSI at the BS, we find that the BS prefers to obtain a perfect eigenmatrix $\mathbf{V}$ (right-singular matrix) and eigenvector $\mathbf{s}$. What’s more, the eigenmatrix $\mathbf{V}$ is unitary matrix which is symmetric and easily compressible, and the eigenvector $\mathbf{s}$ is a simple real-value vector. Therefore, this paper proposes to jointly compress the unitary matrix $\mathbf{V}$ and the corresponding eigenvector $\mathbf{s}$, and feed back the codeword to the BS.

Although downlink channel estimation is challenging, this topic is beyond the scope of this paper.We assume that perfect CSI has been acquired and focus on the feedback scheme. Cosidering a classical mmWave massive MIMO FDD system described above, we focus on the eigenmatrix $\mathbf{V}\in\mathbb{C}^{N_{RB}\times N_{t}\times N_{t}}$ and the eigenvector $\mathbf{S}\in\mathbb{R}^{N_{RB}\times N_{r}}$. The UE needs to deploy an encoder to jointly encode $\mathbf{V}$ and $\mathbf{S}$, which can be formulated as,

where $\varepsilon$ represents the codewords encoded by the UE, $\Theta_{en}$ is the weight parameter of the encoder, and $f_{en}(\cdot)$ stands for the framework of encoder. The role of the encoder is to extract high-dimensional features from $\mathbf{V}$ and $\mathbf{S}$ respectively, and match a suitable mapping function to convert them into codewords. When received the codewords $\varepsilon$, the BS needs to switch the corresponding decoder to interpret and obtain the required $\mathbf{V}$ and $\mathbf{S}$. The decoder can be expressed as,

where $\widehat{\mathbf{V}},\widehat{\mathbf{S}}$ are the reconstructed eigenmatrix and eigenvector at the BS, $f_{de}(\cdot)$ denotes the framework of decoder, and $\Theta_{de}$ is the corresponding weight value.

This paper utilizes neural network (NN) based encoder and decoder to complete the compression and feedback of $\mathbf{V}$ and $\mathbf{S}$. When training NN based auto-encoder, the loss function used by the optimizer is mean squared error (MSE) and can be expressed as,

where $\mathbb{E}(\cdot)$ stands for mathematical expectation, $\|\cdot\|^{2}_{2}$ denotes the Euclidean norm, and $\Gamma(\cdot)$ is a joint loss estimation function, weighted average function in general. The main problem explored is to solve the optimal weights of the NN-based encoder and decoder, which can be formulated by,

where $\Theta_{en}^{*}$ and $\Theta_{de}$ are the optimal weights of encoder and decoder, respectively.

This part is the overview of DL-based EMEV feedback architecture. We aim to explore efficient feedback schemes for beamforming. As is shown in Fig. 1, the whole process starts when the UE estimates the real-time downlink CSI through pilot. And then the CSI matrix $\mathbf{H}$ is divided into eigenmatrix $\mathbf{U},\mathbf{V}$ and eigenvector $\mathbf{S}$ by SVD transformation. From the figure we can find that $\mathbf{H}$ is complex and irregular, but $\mathbf{U}$ and $\mathbf{V}$ are unitary matrices and $\mathbf{S}$ exhibits a scatter distribution. Furthermore, the power distributions of $\mathbf{U}$ and $\mathbf{V}$ are symmetric. Thirdly, the UE inputs $\mathbf{U}$ and $\mathbf{S}$ to the NN-based channel identification to obtain the exact channel type. The detailed NN-based channel identification is shown in our conference paper [34], called EMEV-IdNet. Considering the clustered delay line (CDL) channel model compliant with 5G new radio (NR) standards[33], we explores five common channel types, composed of three none line of sight (NLOS) channels and two line of sight (LOS) channels. Since the eigenmatrix distributions of five channel types are quite different [35], we cascade the channel identification before EMEV encoder to improve the system performance. After channel identification, the UE will joint-encode $\mathbf{V}$ and $\mathbf{S}$ into codewords and feedback to the BS. As is analyzed above, $\mathbf{V}$ and $\mathbf{S}$ will be able to meet the requirements of beamforming and communication for BS. Finally, the BS receives and decodes the codewords $\varepsilon$ and reconstructs eigenmatrix $\widehat{\mathbf{V}}$ and eigenvector $\widehat{\mathbf{S}}$. The algorithm flow of proposed EMEV feedback architecture is described in Algorithm 1.

This subsection shows the overall framework of proposed DL-based EMEV feedback neural network, called EMEVNet. Fig. 2 is the illustration of its framework.

As is described in Fig. 2, the EMEVNet is an auto-encoder which is combined with an encoder at the UE and a decoder at the BS. And the encoder can be further divided into feature extraction and transcoding modules. We design a feature extraction module with dual-channel input layer. Different convolution layers are used for different inputs, i.e. three-dimensional convolution layer (Conv3D) is for $\mathbf{V}\in\mathbb{C}^{N_{RB}\times N_{t}\times N_{t}}$ and two-dimensional convolution layer (Conv2D) for $\mathbf{S}\in\mathbb{R}^{N_{RB}\times N_{r}}$. The high-dimensional feature maps after convolution layers will be compressed to one dimensional feature $\xi_{V}$ and $\xi_{S}$ by fully-connected layers. Thus, the feature extraction module can be described as,

where $\mathcal{L}_{fc}(\cdot),\mathcal{L}_{conv}(\cdot)$ represent fully-connected layer and convolution layer respectively, and $\Omega_{fc},\Omega_{conv}$ are their corresponding weight values. Then, $\xi_{V}$ and $\xi_{S}$ pass through the transcoding module and output codewords $\varepsilon$ at specified system compression ratio. The attention mechanism⁵⁵5The attention mechanism will be described separately when analyzing the transcoding module. based attention residual block and fully-connected layer are combined into a transcoding module, which is formulated as,

where $\mathcal{L}_{att}^{(5)}$ indicates 5 loops of attention residual block, $\beta_{CR}$ is the input system compression ratio, and $\Omega_{att}$ is the corresponding weight value. Therefore, the length of $\varepsilon$ is determined by system $\beta_{CR}$, which can be defined as,

where $L[\cdot]$ denotes the length of the variable, $\Re(\cdot)$ and $\Im(\cdot)$ are the real and imaginary parts of complex numbers, $L_{\varepsilon}$ is the length of codewords $\varepsilon$, and $id$ represents the control symbol of channel identification result. Finally, the BS can reconstruct eigenmatrix $\widehat{\mathbf{V}}$ and eigenvector $\widehat{\mathbf{S}}$ from received $\varepsilon$ by utilizing decoder. The decoder is composed of fully-connected layers, convolutional residual blocks and convolution layers, which can be written as,

where $\mathcal{L}_{res}^{(3)}$ stands for 3 loops of convolutional residual block and $\Omega_{res}$ is its weight values. In summary, the concrete steps of EMEVNet algorithm are given in the Algorithm 2.

In this subsection, feature extraction, transcoding and decoding modules are discussed one by one. We analyze the proposed NN from two aspects: time complexity and space complexity [36]. Time complexity refers to the number of operations of the model, which can be measured by FLOPs, i.e., the number of floating-point operations. The space complexity includes two parts: parameter quantity and output characteristic map. Meanwhile, the hyper-parameters and activation functions set for each layer are given in detail.

This subsection will show some simulation results and discuss the feasibility of our proposed architecture. In order to verify the feasibility of proposed EMEVNet, this part utilize $50,000$ samples of CDL-A channel, i.e. $\mathbb{D}_{sp}^{A}=\{\mathbf{H}_{A}^{(50k)}\}$, as simulation datasets. We set the training, validation and testing datasets obeying the ratio of $70:15:15$. Before training EMEVNet, the datasets $\mathbb{D}_{sp}^{A}$ need to carry out SVD transformation according to Algorithm 1. And the generated $\mathbf{V},\mathbf{S}$ are the input of EMEVNet. After NN training stage according to Algorithm 2, we can obtain the trained $\mathbb{N}_{sp}^{A}$, which is the specific EMEVNet for CDL-A channel. In this part, we have trained and tested seven different compression ratios set as Section IV-A4.

The experimental results can be found in Fig. 6. Considering that $\mathbf{V}\in\mathbb{C}^{N_{RB}\times N_{t}\times N_{t}}$ is a 3D complex-valued matrix, so we split one RB for more intuitive exhibition. The program randomly selects the third resource block, i.e. $\mathbf{V}(3)\in\mathbb{C}^{N_{t}\times N_{t}}$. Then, the Euclidean norm of $\mathbf{V}(3)$ is applied for convenient plotting, which can be interpreted as the power distribution. Fig. 6a shows the initial $\mathbf{V}$ sample at UE and Fig. 6b to Fig. 6h show the reconstructed $\widehat{\mathbf{V}}$ at BS with different compression ratio from $L_{\varepsilon}=416$ to $L_{\varepsilon}=6$. As for eigenvector $\mathbf{S}\in\mathbb{R}^{N_{RB}\times N_{r}}$, it is a 2D real-valued matrix. We directly plot the figure without any processing. Fig. 6i shows the initial $\mathbf{S}$ at UE and Fig. 6j to Fig. 6p show the reconstructed $\widehat{\mathbf{S}}$ at BS with the same settings as mentioned above. After many experiments, the shape of $\mathbf{V}$ of different RBs is almost the same. They all present diagonal power distribution, and the edges are serrated. Moreover, the power distribution of $\mathbf{V}$ is a symmetrical image.

As shown in Fig. 6, with the reduction of feedback codewords $L_{\varepsilon}$, the reconstructed $\widehat{\mathbf{V}}$ by the BS will lose more information. This loss is mainly reflected in the sawtooth energy at the edge of eigenmatrix, while the loss of power principal component (diagonal distribution) is limited. When the length of codewords comes to $L_{\varepsilon}=26$, i.e. $\beta_{h}=256$ and $\beta_{emev}=4096$, the reconstructed $\widehat{\mathbf{V}}$ losses almost all edge information. It can be seen from the figure that when $L_{\varepsilon}\leq 26$, the edge power of $\widehat{\mathbf{V}}$ cannot be reconstructed. However, we can ensure that the distribution of $\widehat{\mathbf{V}}$ remains unchanged and the diagonal power is hardly affected in any case. As for $\widehat{\mathbf{S}}$ at the BS, it can be well reconstructed under almost any $\L_{\varepsilon}$. The detailed numerical results of $\mathbb{N}_{sp}^{A}$, i.e. $NMSE,\rho$, will be shown and discussed in Section IV-E. From the exhibition and analysis of this subsection, we vividly proved the feasibility of EMEVNet by visualizing some experimental results.

In this subsection, we hope to prove the superior performance of the proposed architecture. As is mentioned in the previous section, we assign a channel identification module before encoding $\mathbf{V}$ and $\mathbf{S}$. Therefore, this part compares the performance of specific networks $\mathbb{N}_{sp}^{*}$ trained with large datasets $\mathbb{D}_{sp}^{*}$ and general network $\mathbb{N}_{mix}$ trained with mixed datasets $\mathbb{D}_{mix}$. In order to completely verify the superiority of the proposed architecture, we have checked five CDL channels defined by 3GPP. The testing objects are $\left\{\mathbb{N}_{sp}^{A},\mathbb{N}_{sp}^{B},\mathbb{N}_{sp}^{C},\mathbb{N}_{sp}^{D},\mathbb{N}_{sp}^{E}\right\}$, and the baseline is set as $\left\{\mathbb{N}_{mix}\right\}$. For each experiment, we give four evaluation indexes defined in Section IV-A5, including $NMSE(\mathbf{V},\widehat{\mathbf{V}}),NMSE(\mathbf{S},\widehat{\mathbf{S}}),\rho(\mathbf{V},\widehat{\mathbf{V}})$ and $\rho(\mathbf{S},\widehat{\mathbf{S}})$.

The experimental results can be seen in Fig. 7. Each channel type has been tested and verified by 7 different compression ratios mentioned in Section IV-A4. The horizontally adjacent figures respectively show the reconstruction performance curves of $\widehat{\mathbf{V}}$ and $\widehat{\mathbf{S}}$ under the same channel, e.g. Fig. 7a and Fig. 7b show the performance under CDL-A channel environment. In addition, the vertically adjacent figures show the performance of the same feedback information with different channel types, e.g. Fig 7a and Fig. 7c show the performance with different channels. Each figure has two y-axes of different scales, corresponding to $NMSE(dB)$ and $\rho$ respectively. In addition, there are four performance curves in each figure, among of which the solid blue lines show $NMSE$ performance, the dotted red lines represent $\rho$ performance, the circle marked lines represent specific network $\mathbb{N}_{sp}^{*}$, and the triangle marked lines represent general network $\mathbb{N}_{mix}$.

As can be seen from Fig. 7, the two blue solid lines of each figure show an upward trend, while the two red dotted lines show a downward trend. This result is the same as the analysis in Section IV-A5. The performance of reconstruction is getting worse with the decrease of $L_{\varepsilon}$, i.e., $NMSE(dB)$ tends to be larger and $\rho$ deviates from 1. What’s more, all the blue lines with circle mark are lower than the triangle mark, and all the red lines with circle mark are higher than the triangle mark. These results mean that the performance of specific network $\mathbb{N}_{sp}^{*}$ is better than general network $\mathbb{N}_{mix}$. It can also be found that two lines with the same color are almost parallel, which shows that the performance improvement of $\mathbb{N}_{sp}^{*}$ is relatively stable compared with $\mathbb{N}_{mix}$, and is less affected by $L_{\varepsilon}$. In summary, we can conclude that the channel identification module is necessary, and the performance of feedback and reconstruction can be improved by selecting a specific network $\mathbb{N}_{sp}^{*}$.