Enhancing Deep Learning Performance of Massive MIMO CSI Feedback

The massive multiple-input multiple-output (MIMO) is one of the key technologies for next generation communication systems, e.g., 5G and above. Unlike traditional cell-based communication paradigms, the massive MIMO makes better use of spatial diversity and serves users in a cell-free way. A massive MIMO system typically is equipped with a large number of antennas at the base station (BS), which aims to make full use of spatial diversity by conducting beamforming to concentrate signal energy to a specific user equipment (UE).

BS requires the downlink channel state information (CSI) to conduct beamforming. Especially for modern cellular systems that work on frequency division duplexing (FDD) mode, the UE may have to explicitly feed back the downlink CSI to the BS due to the lack of channel reciprocity. As the overhead of CSI feedback grows quadratically with the number of transmitting antennas, CSI compression is needed before the feedback to reduce the overhead. Therefore, how to feedback CSI efficiently and accurately becomes one most important problem for massive MIMO [1].

Traditional compressive sensing (CS) based CSI feedback methods rely on specific assumption and require iterative reconstruction of CSI, which makes it hard to adapt to fast-changing channels in real life. Ever since the first DL-based method CSINet [2] was proposed, a series of subsequent works [3, 4, 5, 6] appear because of the superior performance brought by deep learning. Generally, DL-based methods utilize the auto-encoder framework [7], where the encoder learns to compress the original CSI at the UE side and the decoder learns to reconstruct the original CSI at the BS side. The auto-encoder is trained in unsupervised manner without the need for labeled data and only requires a single run upon deployment for continuous CSI reconstruction, which overcomes the computation inefficiency of traditional CS-based approaches.

Despite the success of DL-based CSI feedback methods, current approaches have not fully exploited the advantages of deep learning due to the lack of analysis of the CSI feedback problem and the characteristics of CSI data. Current DL-based approaches generally migrated some of the well-known neural network building blocks, such as residual blocks [2], transformer [6], LSTM [8] and etc., to the CSI feedback task. Those modules are proposed mainly for the recognition problem in computer vision or natural language processing domain, which differs from the CSI compression task. As Fig. 1 illustrates, the recognition task aims to capture the most distinctive features of the original data while the compression task should aim at preserving the information approximately but as complete as possible.

Especially for the CSI data that is usually sparse, the encoder tends to discard sparse parts when doing compression, then zero-pads those parts during reconstruction. Although the absolute value of the original CSI matrix and the reconstructed CSI matrix are similar by doing the zero-pad operation, the reconstruction error depends on the relative position of those zero-padded parts. In addition, the CSI matrix carries physical information with the delays and angles of the propagation paths, so the relative position of sparse parts matters. As a result, the physical information carried by CSI becomes inaccurate when the encoder discards more parts without the knowledge of the relative positions of them in the CSI matrix. Therefore, improving performance of CSI feedback task requires a mechanism that helps retain complete but approximative information rather than discarding relatively unimportant information and only preserving the most distinguishable parts. In particular, the physical information of path delays and angles that are encoded as position information in the CSI matrix should be considered.

To this end, we propose a jigsaw puzzles aided training strategy (JPTS), which involves an auxiliary jigsaw puzzle-solving task during training so the neural network is forced to fuse the position information across different local regions (puzzle pieces) of the CSI matrix while encoding the most representative information so that even the model has better knowledge about where to zero-pads when reconstruction and preserve the holistic CSI information as complete (but approximated) as possible. We evaluate the effectiveness of JPTS by adopting it on top of three open sourced SOTA CSI feedback methods and experiment with both the indoor and outdoor CSI data. The experimental results demonstrate that the proposed JPTS effectively improves the performance of SOTA CSI feedback approaches. The strategy helps to improve the overall average accuracy of CSINet, CRNet and CLNet by 25.66%, 6.98% and 3.58% for indoor scenario and 16.67%, 2.18% and 2.17% for outdoor scenario, respectively. There is an overall accuracy lift of 14.80%, 6.32%, 10.14%, 10.09% and 6.35% at corresponding compression ratios of 1/4, 1/8, 1/16, 1/32 and 1/64, respectively. The highest improvement is 39.43% which is obtained from outdoor JPTS-CSINet with 1/4 compression ratio.

The main contributions are summarized as follows:

• •

  To the best of our knowledge, JPTS is the first training strategy proposed for massive MIMO CSI feedback.

• •

  The proposed JPTS helps improve the performance of DL-based CSI feedback approaches.

• •

  JPTS is effective to different deep neural network architectures and thus can be generally applied across different SOTA approaches for DL-based CSI feedback.

For simplicity, a single cell massive MIMO system operating in FDD mode is considered, where the BS is equipped with $N_{t}$ antennas and the UE side has $N_{r}$ antennas. $N_{t}$ $\gg N_{r}$ ($N_{r}$ equals to 1 for simplicity). The orthogonal frequency division multiplexing (OFDM) is adopted with $N_{c}$ subcarriers. The received signal $\textbf{{y}}\in\mathbb{C}^{N_{c}\times 1}$ can be expressed as follows:

where $\textbf{{x}}\in\mathbb{C}^{N_{c}\times 1}$ indicates the transmitted symbols and $\textbf{{z}}\in\mathbb{C}^{N_{c}\times 1}$ is the complex additive Gaussian noise. A is a diagonal matrix that can be expressed as $\operatorname{diag}\left(\textbf{{h}}_{i}^{H}\textbf{{p}}_{i},\cdots,\textbf{{h}}_{N_{c}}^{H}\textbf{{p}}_{N_{c}}\right)$, $i\in\left\{1,\cdots,N_{c}\right\}$, where $\textbf{{h}}_{i}\in\mathbb{C}^{N_{t}\times 1}$ is the downlink channel coefficients and $\textbf{{p}}_{i}\in\mathbb{C}^{N_{t}\times 1}$ represent beamforming precoding vector for subcarrier $i$.

Due to the asymmetry of uplink and downlink, in order to derive the beamforming precoding vector $\textbf{{p}}_{i}$, the BS needs the knowledge of corresponding downlink channel coefficient $\textbf{{h}}_{i}$ that is fed back by the UE. The downlink channel matrix is $\textit{{H}}=\left[\textbf{{h}}_{1}\cdots\textbf{{h}}_{N_{c}}\right]^{H}$ which contains $N_{c}N_{t}$ elements. As the CSI matirx H is complex-valued, the total number of parameters that need to be fed back is $2N_{c}N_{t}$, which is proportional to the number of antennas. Since the characteristic of massive MIMO is assembled with an extremely large antenna array, the overhead of direct CSI feedback is unacceptable, and thus how to better compress CSI becomes the bottleneck problem to enable massive MIMO.

The common practice is first to obtain angular-delay domain CSI representation $\textit{{H}}^{\prime}$, because the channel matrix H is often sparse in the angular-delay domain. $\textit{{H}}^{\prime}$ can be obtained by performing the 2D discrete Fourier transform (DFT) on H such that

where $\textit{{F}}_{c}\in\mathbb{C}^{N_{c}\times N_{c}}$ and $\textit{{F}}_{t}\in\mathbb{C}^{N_{t}\times N_{t}}$ are the DFT transform matrices. Each element in $\textit{{H}}^{\prime}$ represents a certain path delay with a certain angle of arrival (AoA). Since the time delay of multi-path arrivals is within a finite time, only the first few rows contain useful information, while the rest rows are made up of near-zero values, and can be omitted without much information loss. We may obtain the informative $\textit{{H}}^{\prime}$ as $\textit{{H}}_{a}\in\mathbb{C}^{N_{t}\times N_{t}}$. However, even after that, $\textit{{H}}_{a}$ is still too big to feedback because $N_{t}$ is large.

The DL-based solution therefore achieves affordable overhead by assembling an encoder on the UE side for $\textit{{H}}_{a}$ compression and a decoder on the BS side for $\textit{{H}}_{a}$ reconstruction. Mathematically,

where $f_{\mathcal{E}}$ and $f_{\mathcal{D}}$ denote the encoding process and the decoding process, respectively, and $\mathit{\Theta}_{\mathcal{E}}$ and $\mathit{\Theta}_{\mathcal{D}}$ represent a set of learned parameters of the encoder and the decoder, respectively. The output of the encoder is the compressed CSI that is denoted as v. The compression ratio $\eta$ is defined as the ratio of v and the encoder’s input such that:

The goal of DL-based solutions is to find the parameter sets of encoder and decoder with minimize the difference between the original $\textit{{H}}_{a}$ and the reconstructed $\hat{\textit{{H}}}_{a}$:

The compression task essentially is to maximize the mutual information (MI) between the original input and the compressed output [9], which differs from the recognition task that focus to learn the most distinctive part of the input. The MI between the input $\textit{{H}}_{a}$ and output v of the encoder is defined as:

which measures the inherent dependence of the joint distribution of $\textit{{H}}_{a}$ and v relative to the marginal distribution of $\textit{{H}}_{a}$ and v under the assumption of independence. $I(\textit{{H}}_{a};\textit{{v}})=0$ means $\textit{{H}}_{a}$ and v are independent. Base on the definition, larger MI indicates less uncertainty between $\textit{{H}}_{a}$ and v, higher amount of information obtained of v through observing the $\textit{{H}}_{a}$.

The current DL-based CSI compression methods use auto-encoder structure with reconstruction loss, which in fact maximizes the MI in a global way [10]. Later research demonstrates that maximizing the MI in local regions (e.g. patches rather than the complete image) can greatly improve the representation’s quality [11], which also explains why various attention mechanisms [3, 5] and multi-resolution [4] blocks are useful in CSI feedback task. Nevertheless, the attention mechanisms only focu on the local patche/cluster with signal path information while those sparse parts are simply ignored. Although the value of sparse parts is negligible, the position information of sparse parts matters because they actually help determine the position of signal paths that carry the physical information of path delays and angles.

Therefore, we introduce the idea of jigsaw puzzles, which were first introduced by John Spilsbury as a pretext for learning geography. Solving jigsaw puzzles is another way to maximize MI in a local way with the natural benefit of learning position information. In order to solve the puzzles correctly, the network should have the ability to recognize the relative positions of different parts of the original data, whose characteristic is suitable for CSI feedback task as the positions of CSI sub-matrices also carry physical information. Currently, using deep learning to solve jigsaw puzzles by predicting the location of puzzle fragments is frequently used as a self-supervised learning pretext for representative learning [12]. We design a jigsaw puzzle-solving task as an auxiliary task in the training phase to enforce the network to encode the relative position even for those sparse parts, namely, Jigsaw Puzzles aided Training Strategy for the CSI feedback task.

Concretely, the original CSI matrix $\textit{{H}}_{a}\in\mathbb{C}^{N_{t}\times N_{t}}$ is divided to $n$ equal-size tiles $T$ followed by a random shuffle to get the permutation index. The learning goal is to be able to get the original version from the shuffled version. The $n$ determine the difficulty of the solving problem, if the number of tiles is 9, there are 9!=362,880 possible permutations. In particular, as Fig. 2 illustrates, we use $n=4$ as an example. First, the input $\textit{{H}}_{a}$²²2This is the CSI data from the dataset visualized with matlab’s ’lines’ colormap. is divided into four equal-size tiles $\textit{{T}}_{i}\in\mathbb{R}^{16\times 16}$, $i=1,...,4$, followed by a random shuffle, e.g. $\textbf{{s}}=[4,1,2,3]$, to rearrange the position of each tile to get the permutation version of $\textit{{H}}_{a}$ termed as $\overline{\textit{{{H}}}_{a}}$. Finally, $\textit{{H}}_{a}$ and $\overline{\textit{{{H}}}_{a}}$ are paired together to form the input of the network for training. The auxiliary jigsaw puzzle-solving task lets the encoder recognize the original position of each tile by giving a list of integer index l that is the same as the actual permutation index s. Concretely, we add an extra fully connected layer, $f_{S}$, at the end of the encoder to structure the output to be a matrix with a fixed dimension $\textbf{{J}}\in\mathbb{R}^{4\times 4}$. The matrix further goes through the Softmax function $\sigma$ to re-scale each column of the matrix so they lie in the range of [0,1] and sum to 1. In the end, each column of the matrix is a vector with the probability value of the position for each tile.

The goal of the training now turns to finding the parameter sets of encoder and decoder that not only minimize the difference between the original $\textit{{H}}_{a}$ and the reconstructed $\hat{\textit{{H}}}_{a}$ but also minimize the error of the permutation prediction such that:

in which, $\textbf{{J}}=f_{S}\left(f_{\mathcal{E}}\left(\overline{\textit{{{H}}}_{a}},\mathit{\Theta}_{\mathcal{E}}\right),\mathit{\Theta}_{\mathcal{S}}\right)$ and the $argmax$ denotes getting the index of the maximum element in the vector. The $\alpha$ in equation 7 is a weighting parameter that controls the difficulty of the jigsaw puzzle-solving task. If $\alpha\to 0$, the training degrades to learning the 2D absolute position of the tiles without learning their semantic content. If $\alpha\to 1$, the training degrades to the original task where the encoder just keeps more discriminative parts and simply discards those sparse parts.

As such, in order to meet the goal of reconstructing $\textit{{H}}_{a}$ and solving the puzzle correctly at the same time, which requires the encoder to preserve the local region information for recognizing each tile and its relative position within the CSI matrix, instead of just preserving the most discriminative parts of the entire CSI matrix. Especially for CSI data, most part of the $\textit{{H}}_{a}$ is sparse, e.g. yellow annotated part in Fig. 2, without the JPTS, the encoder may simply discard the sparse part. The decoder just zero-pads the discarded sparse region during reconstruction as the absolute value almost the same. By adding the auxiliary jigsaw puzzle-solving task, the sparse part may also be discarded, but the relative position information is preserved by the encoder so that the decoder has better knowledge of where to zero-pad those sparse parts and as a result benefit to the entire CSI matrix. The JPTS maximizes the MI in local regions, which is complementary to the auto-encoder framework that maximizes the MI in a global way. As a result, the whole MI is maximized and the power of those deep learning building blocks is unleashed.

This section details the experiment setting and reports the results of utilizing Jigsaw Puzzles aided Training Strategy with existing open-source state-of-the-art (SOTA) DL-based CSI feedback approaches.

This paper proposes a novel jigsaw puzzles aided training strategy (JPTS) for DL-based massive MIMO CSI feedback task. The JPTS aims to promote better CSI compression by maximizing the mutual information in a local way with the fact that the relative position information matters in CSI matrix. The experiment results of adopting JPTS to different SOTA approaches demonstrate the efficacy of JPTS as well as support the argument of this paper, which is massive MIMO CSI feedback is a compression task, other than adopting powerful deep learning building blocks to better learn the most distinguishable parts of the CSI matrix, to preserve the information of entire CSI matrix as complete (but approximated) as possible also matters.

As JPTS is a training strategy that can be generally applied to any DL-based massive MIMO CSI feedback approaches, we open source the code to boost the following research as well as for reproducible.

This work is supported by Singapore MOE AcRF Tier 2 MOE-T2EP20220-0004 and MOE AcRF Tier 1 RT13/20.