FusionNet: Enhanced Beam Prediction for mmWave Communications Using Sub-6GHz Channel and A Few Pilots

Let $\mathbf{h}_{s}[k]\in\mathbb{C}^{N_{s}\times 1}$ denote the uplink channel vector of the $k$-th subcarrier at the sub-6GHz band. The received uplink signal at the BS can be expressed as

for $k=1,2,\cdots,K_{s}$, where $s_{s}[k]$ is the pilot of the $k$th subcarrier and $\mathbf{n}_{s}[k]\in\mathcal{N}(0,\sigma^{2})$ is the corresponding noise. Either least-square (LS) or linear minimum mean-squared error (LMMSE) channel estimation, in the frequency domain or the time domain can be used, the detailed process is omitted here.

Since there is only one RF chain, the regular uplink or downlink channel estimation at mmWave band needs to be repeated $N_{m}$ times via switching the RF chain onto different antennas, or vary the weights of the phase shifters when each single RF chain connects to $N_{m}$ antennas simultaneously. Such process costs a large amount of time resource, especially when $N_{m}$ is large.

Here, we estimate the channels only on $\tilde{N}_{m}\ll N_{m}$ antennas to assist the beam prediction from the sub-6GHz to the mmWave band. The mmWave antennas that participate in the mmWave uplink channel estimation is called as active antennas. Note that the estimated channels corresponding to this $\tilde{N}_{m}$ antennas may not even be sufficient to recover the whole $N_{m}$ channel elements via compressive sensing or the deep learning techniques. To make the overall illustration clear, we will first present how to estimate the uplink $\tilde{N}_{m}$ channels by changing the weights of the phase shifters.²²2The other way that switches the RF chain to the $\tilde{N}_{m}$ antennas sequently is not stable in practical applications and will not be adopted here.

The switchers of $(N_{m}-\tilde{N}_{m})$ antennas are in off status during the training stage and the mmWave RF chain connects to the $\tilde{N}_{m}$ antennas via the $\tilde{N}_{m}$ phase shifters, simultaneously. Denote the channel on the $k$th subcarrier of these $\tilde{N}_{m}$ antennas as $\tilde{\mathbf{h}}_{m}[k]$. In order to estimate these $\tilde{N}_{m}$ channels, the user should send the training OFDM block $\tilde{N}_{m}$ times since there is only one RF chain. The pilot signal on the $k$th subcarrer in the $i$th training block is denoted as $s_{m,i}[k],i=1,\ldots,\tilde{N}_{m}$. For simplicity, we assume that the pilot on all $K_{m}$ subcarriers and all $\tilde{N}_{m}$ training blocks are same i.e., $s_{m,i}[k]=1,\forall(i,k)$.³³3It is also possible to consider the combo type training where only part of the subcarriers are used for pilots while the rest are used for unknown data.

Moreover, let us denote $\tilde{f}_{i,j}$ as the value of the phase shifter for the $i$th training block and the $j$th antenna, $i,j=1,\ldots,\tilde{N}_{m}$, which is universal for all subcarriers. Denote $\tilde{\mathbf{f}}_{i}=[\tilde{f}_{i,1},\tilde{f}_{i,2},\tilde{f}_{i,\tilde{N}_{m}}]^{T}$ with $|\tilde{f}_{ij}|=1$. As in Fig 1.b, the received signal on the $k$th subcarrier of the $i$th training block is

Stacking $y_{i}[k]$ from $\tilde{N}_{m}$ training block into one vector, we have

where $\tilde{\mathbf{F}}=[\tilde{\mathbf{f}}_{1},\tilde{\mathbf{f}}_{2},\ldots,\tilde{\mathbf{f}}_{\tilde{N}_{m}}]^{T}$ and $\mathbf{n}[k]=[n_{1}[k],n_{2}[k],\ldots,n_{\tilde{N}_{m}}[k]]^{T}.$ Normally, $\tilde{\mathbf{F}}$ is selected as the $\tilde{N}_{m}\times\tilde{N}_{m}$ normalized DFT matrix during the training process.

During the downlink data transmission, the BS uses all $N_{m}$ antennas to form one single narrow beam, $\mathbf{f}\in{\cal C}$. Since we assume only one RF chain, the downlink beamforming can be represented as $\mathbf{f}=[f_{1},f_{2},\ldots,f_{N_{m}}]^{T}$ with $|f_{i}|=1$. The received signal on the $k$-th subcarrier in one OFDM block can be written as

where $n_{m}[k]$ and $s_{d}[k]$ represent the corresponding noise and signal with powers $\sigma_{n}^{2}$ and $P_{s}$, respectively.

The achievable rate of the downlink transmission can be expressed as

Since the phase shifters are generally constrained with limited bits, the size of ${\cal C}$ is finite. In the previous works [35, 3, 36], exhaustive search is performed to find optimal $\mathbf{f}^{\star}$ to maximize the achievable rate, $R(\mathbf{h_{m}},\mathbf{f})$, according to the channel conditions.

A typical example is shown in Fig. 2, where the power delay profile (PDP) of the channels from both 28GHz and 3.5GHz bands with 100MHz bandwidth are measured in the corridor environment[37]. From the figure, though the PDP of 28GHz channel is more sparse compared to that of 3.5GHz, the contour of both frequency bands are similar, which inspires to use the congrugency of the channel vectors in [13, 15, 16]. Specifically, it is possible to predict optimal mmWave downlink beam $\mathbf{f}^{\star}$ directly from the sub-6GHz uplink channel in [32, 33]. For instance, it has been proven in [32] that there exists a deterministic mapping from the sub-6GHz channel to the mmWave beams. In [33], the optimal mmWave beam is predicted by estimating the PDP of the sub-6GHz channel, where the PDP is considered as a fingerprint for the UE position and thus contains essential angular information for beam selection in a cell-specific manner with given environments. The training dataset of the two frequency bands is generated from the Wireless System Engineering (WiSE). Then, a DNN similar to [32] is trained to predict the best beam within the candidate set ${\cal C}$.

Although predicting the mmWave beam from sub-6GHz channel CSI is theoretically and experimentally demonstrated effective. From [32, 33], there is still a big gap to improve its performance, especially when the SNR of the sub-6GHz channel is low. In fact, even without channel noise, the prediction accuracy is still below 85$\%$ and does not meet the practical communications requirement, which motivates us to improve the accuracy by combining features of the sub-6GHz channel and pilots and the mmwave channel.

The best beam for mmWave downlink almost corresponds to the strongest path in the angular domain. Hence, the beam prediction problem is readily solved if the angular information of the mmWave channel is available. As discussed before, predicting directly from the sub-6GHz channel is not accurate enough due to the following factors:

• 1)

  With limited number of sub-6GHz antennas the BS cannot offer high resolution in the angular domain;

• 2)

  As pointed out in [38], the support for the mmWave channel in the angular domain is only a subset of that for the sub-6GHz channel with the same spatial grid quantization, and thus the angle of the strongest path for sub-6GHz channel may not be the mmWave counterpart;

• 3)

  The DL approximation of the channel mapping from sub-6GHz to mmWave is not accurate enough due to limited number of training samples and limited network size.

Though the beam direction predicted from the sub-6GHz channel usually deviates from the true one as we can imagine, it still preserves certain channel spatial information and could serve as a valid starting point to find the best beam. Hence, the limited mmWave pilots would be very helpful to “calibrate” such deviation and significantly enhance the prediction accuracy, as illustrated in Fig. 3. Note that since the mmWave pilots are mainly used for calibration but not estimation, the number of required pilots is much smaller than that to estimate the complete mmWave channel, i.e., $\tilde{N}_{m}\ll N_{m}$.

This key rationale is simple but is rather practical and useful for beam prediction, channel estimation, or data detection, etc. In all, one should always count on a few pilots to achieve the ultimate precision after using the DL to get a coarse starting point. In the following, we will present a detail design of the neural network architecture to effectively merge the mmWave pilots and sub-6GHz channel information, to accurately predict the beam in the mmWave band.

Following [32], we here adopt multi-layer perceptron (MLP) to represent the mapping function from the sub-6GHz channel to the optimal beam $\mathbf{f^{\star}}\in{\cal C}$.

One straightforward way is to directly concatenate the sub-6GHz channel and the mmWave channel as the input of a DNN, as illustrated in Fig. 4(a), which is known as the shallow model [39]. However, as the correlation between the sub-6GHz and the mmWave channel is highly non-linear, it is hard for a neural network to learn their individual features effectively from a concatenated vector. The shallow model still has some performance loss, especially when the sub-6GHz channel is with low SNR.

Inspired by [40, 39], we design a dual-input network, called FusionNet, as shown in Fig. 4(b). The FusionNet first extracts the features from the sub-6GHz and mmWave channels separately, and fuses them in a concatenation layer to generate a probability vector $\mathbf{p}$ whose $i$th entry, $p_{i}$, represents the probability for the $i$th beam (in the given codebook $\mathbf{{\cal C}}$) being the optimal one. We denote the inputs corresponding to the sub-6GHz and the mmWave channels as $\mathbf{x}_{s}$ and $\mathbf{x}_{m}$, respectively, which are the vectorization of channels on all subcarriers estimated in Section II.

As in Fig. 4(b), the FusionNet is comprised of three sub-networks, i.e., mmw-network, sub6-network and classify-network. The mmw-network, with $L_{m}$ fully connected layers, extracts the features, such as angular, delay, and path gain information from the mmWave channel input. The sub6-network, with $L_{s}$ layers, extracts information from the sub-6GHz input. The classify-network takes the concatenated feature as the input, and is followed by a Softmax layer. Each fully connected layer in these sub-networks is followed by a BatchNorm layer, a Relu layer, and a dropout layer.

Since the frequency discrepancy between the sub-6GHz and the mmWave channel is very large, intuitively, using the sub-6GHz channel to predict the best beam in mmWave band will definitely need more layers compared to using the mmWave channel itself. Hence, we set $L_{s}>L_{m}$ in the proposed design. Denote the numbers of neurons in the $l$th layer in the sub6-network, mmw-network, and classify-network as $n_{l}^{m},n_{l}^{s},n_{l}^{c}$, respectively. The mmw-network and sub6-network extract the various path information of the corresponding channels, while the classify-network yields probability vector, $\mathbf{p}$, with length $n_{l}^{c}=\mathbf{|{\cal C}|}$. The other parameters of the FusionNet will be further discussed in Section V.

In the training stage, a supervised learning approach is adopted, where the training label, denoted as $\mathbf{t}$, is a one-hot vector representing the best beam for the mmWave downlink transmission. The detailed calculation of $\mathbf{t}$ will be presented later in the next section. We adopt the cross-entropy loss as the loss function:

which is minimized by the ADAM optimizer.

In the evaluation stage, new sub-6GHz and mmWave channels will be generated. After pre-processing the data and feeding to the trained FusionNet, the optimal beam can be predicted.

For the FusionNet, the total number of floating point operations (FLOPs) can be computed as:

Compared with [32] that only needs to process the training data from the sub-6GHz channel, the first two sub-networks of the FusionNet take training data from the two frequency bands, resulting in a little bit higher complexity. For example, when $L_{m}=4,L_{s}=6,L_{c}=3$ and $n_{l}^{m},n_{l}^{s}=2048,n_{l}^{c}=64$, the total complexity is mainly determined by the first two sub-networks and is approximately twice of the complexity in [32]. Nevertheless, the neural network is mainly trained offline and is deployed online at the BS, where the computing resources are always assumed abundant.

The dataset for the FusionNet comprises of the mmWave channel, the sub-6GHz channel estimated at different user positions, and the corresponding best beam index. We first generate the sub-6GHz and mmWave channels using the “O1 scenario” in DeepMIMO dataset [41]. The parameters used in the data generation process are summarized in Table. I.

The DeepMIMO dataset contains parameters of the $R$ strongest rays for each user and is represented by ($\psi_{r},\theta_{r},\alpha_{r},\tau_{r}$), where $\psi_{r}$ denotes the azimuth angle at the BS of the $r$-th path, $\theta_{r}$ denotes the elevation angle, $\alpha_{r}$ is the complex gain, and $\tau_{r}$ is the delay. The mmWave channel is constructed using a geometric channel model whose channel vector at the $k$-th subcarrier is

where $\mathbf{a}$ is the steering vector and $T_{s}$ is the reciprocal of the OFDM subcarrier interval. The sub-6GHz channel is also generated using (8) while the DeepMIMO dataset will automatically adjust the parameters according to the environment and frequency band. Therefore, the geometric channel model is capable to capture the physical characteristics of the signal propagation process including the dependence on the environment geometry, materials, frequency band, etc., which are of vital importance for DL based beam prediction.

We will place pilots on all subcarriers of the OFDM blocks at both frequency bands. With the received uplink signals, the BS will estimate channels $\mathbf{h}_{m}[k]$ and $\mathbf{h}_{s}[k]$ at the two frequency bands as illustrated in Section II. Similar data normalization [32] is then carried out for input signals in both frequency bands.

Let $\mathbf{h}_{m}^{u}[k]$ be the mmWave channel vector at the $k$th subcarrier for the $u$th user from DeepMIMO and $(\mathbf{h}_{m}^{u}[k])_{n}$ be the mmWave channel at the $k$th subcarrier on the $n$th antenna. The channel vectors are normalized by a global normalization value $\Omega$, which is the largest absolute value in the whole dataset, i.e.,

After normalization, the magnitudes of all channel elements will be between 0 and 1. We then spilt the normalized complex channel into real part and image part, which will then be stacked together. The final mmWave training dataset can be obtained with size $U_{m}\times(2\times K_{m}\times N_{m})$ where $U_{m}$ is the number of total user positions at the mmWave band.

Next, we will present how to get label $\mathbf{t}$ at each user position. The achievable rate for the mmWave downlink channel of the $u$th user with beam $\mathbf{f}_{c}$ can be computed as

for $c=1,2,\cdots,\mathbf{|{\cal C}|,}$ where SNR denotes the signal-to-noise ratio at the transmitter (user’s side). Then the index of the best beam can be obtained from the offline searching by

An one-hot vector $\mathbf{t}^{u}$ can be obtained for each user to represent $(c^{u})^{\star}$, whose $(c^{u})^{\star}$th element is 1 while other elements are 0, which serves as the label in the training and validating stage.

In Section II, the estimated sub-6GHz and mmWave channels, $\mathbf{h}_{s}[k]$ and $\mathbf{h}_{m}[k]$, are in the spatial-frequency domain. By stacking these channel vectors together, channel matrices for the sub-6GHz band and the mmWave band can be obtained as

Inspired by [38, 42, 43, 44], the FusionNet can improve the prediction performance by leveraging the sparsity of the channel matrices. Since the sparsity at the two frequency bands are similar, we will only discuss the mmWave channel matrix as an example. A 2-D Discrete Fourier Transform (DFT) is performed on $\mathbf{H}_{m}^{sf}$ to find the new channel matrix in the angle-delay domain,

where $\mathbf{F}_{a}$ and $\mathbf{F}_{d}$ are $N_{m}\times N_{m}$ and $K_{m}\times K_{m}$ normalized DFT matrices.

As illustrated in [24], limited scattering of channels as well as the large number of antennas at the BS (i.e the large $N_{m}$) ensure the sparsity of $\mathbf{H}_{m}^{ad}$ in the angular-delay domain, i.e., only $N_{a}\ll N_{m}$ rows and $N_{d}\ll K_{m}$ columns of $\mathbf{H}_{m}^{ad}$ have significant values. Nevertheless, since there are a limited number of sub-6GHz antennas in the considered scenario and only $\tilde{N}_{m}\ll N_{m}$ mmWave antennas are used for channel estimation, the condition that a large number of antennas are employed at the BS is not satisfied. Thus, we will only adopt one dimension DFT to obtain the sparse representation, $\mathbf{H}_{m}^{sd}$, in the delay domain by

The sparse representation for the sub-6GHz channel can also be leveraged following (12). Both the sparse channels, $\mathbf{H}_{m}^{sd}$ and $\mathbf{H}_{s}^{sd}$, will be used as the input data to train the FusionNet.

The proposed FusionNet is mainly comprised of fully connected layers with a large number of parameters. If trained with insufficient number of data, the neural network will most likely be over-fitted on the training set and would fail to yield a good generalisation on the test set. Many techniques, e.g., the dropout and batch normalization function, have been used to solve the over-fitting problem. However, as the number of parameters increases, the flexibility of the network becomes extremely high and these techniques fall out too.

Data augmentation has been used in DL to generate additional training data and has achieved great success in the realm of speech recognition[45], image classification[46, 47, 48] and deep reinforcement learning [49]. Despite various applications in other areas, the data augmentation approach has not yet been used to generate extra data for the wireless communications, to the best of the authors’ knowledge. We next introduce a novel data augmentation approach to generate new artificial samples as the input of the FusionNet.

A plausible data augmentation transformation should preserve label information. That is, one can perform any kind of transformation on the mmWave and the sub-6GHz channels as long as the transformed channels yield the same label as before. From the achieved rate objective (10), a simple observation is that if the mmWave channel vector, $\mathbf{h}_{m}^{u}[k]$, is multiplied by any $e^{-j2\pi\phi}$, then both the achievable rate and the corresponding best beam index for each user remain the same. This observation inspires us to insert a random phase $\phi$ into $\mathbf{h}_{m}$ to augment the mmWave data. We generate $\phi$ from the uniform distribution, i.e. $\phi\sim\mathcal{U}(0,1)$. Note that, different subcarriers for a specific user should share the same $\phi$ while different users may have different $\phi$’s.

Data augmentation is also needed for the sub-6GHz channel. At the first look, it seems hard to directly tell what kind of transformation performed on the sub-6GHz channel will not affect the original label because the sub-6GHz channel does not directly determine the optimal beam (training label). Actually, the sub-6GHz channel provides underlying information of the mmWave channel, such as angular feature and frequency feature, about the propagation environment. Therefore, any kind of transformation that preserves the information in sub-6GHz channel is valid. Hence, multiplying a random value to the sub-6GHz channel vector is also plausible since this linear transformation will not cause any loss of channel information. For simplicity, we augment the sub-6GHz data by

where $\chi$ is a random phase with $\chi\sim\mathcal{U}(0,1)$.

Using the above data augmentation approaches, a large number of new training samples can be generated. However, as the number of training samples increases, the computational complexity increases accordingly and the computer memory may also become insufficient. Most importantly, their underlying information remains the same even though the number of synthesized samples grows. As a result, the neural network’s performance would not be further improved when the number of the augment samples reaches a certain value.

Fig. 5 compares prediction accuracy of the FusionNet with other neural network architectures, where the top-3 accuracy of FusionNet is also displayed. The baseline curve is the performance of the DNN in [32] that merely takes the sub-6GHz channel as input while the “shallow model” curve stands for predicting directly from the concatenated sub-6GHz and mmWave channel as shown in Fig. 4(a). Both the “shallow model” network and the FusionNet are trained with $\tilde{N}_{m}=8$ active mmWave antennas and with pilot SNR $=20$dB. From Fig. 5, the prediction accuracy of “shallow model” network and the FusionNet both outperform the baseline curve. Since the shallow model does not fully exploit the individual features of the two channels, the “shallow model” curve is always below the FusionNet curve. Moreover, the top-3 accuracy of FusionNet reaches 100$\%$ even when the SNR of the sub-6GHz channel is merely 0 dB, hence we will omit the top-3 accuracy in the rest of the simulation.

Fig. 6 displays prediction accuracy and the corresponding achievable rate versus SNR of the sub-6GHz channel estimation. The number of active antennas $\tilde{N}_{m}$ is 2, 4, 8, 16, respectively, and the mmWave pilot SNR is 20 dB. From Fig. 6(a), prediction accuracy of the FusionNet with any number of $\tilde{N}_{m}$ is always much better than the baseline method, especially at the low SNR region. An approximately 5 dB SNR gain in the sub-6GHz band is observed in terms of beam prediction accuracy even if we turn on only 2 mmWave antennas. Moreover, the beam prediction accuracy of the FusionNet significantly improves as the number of active mmWave antennas increases while the improvement slows down beyond 8 active mmWave antennas. The achievable rate in Fig. 6(b) follows the similar trend. Another observation is that there is almost no rate loss in mmWave downlink data transmission when the SNR of the sub-6GHz signal is merely 5 dB and when $\tilde{N}_{m}=4$ mmWave antennas are used for beam calibration. All these observations clearly demonstrate the effectiveness of the proposed FusionNet.

Fig. 7 depicts prediction accuracy and the achievable rate versus the sub-6GHz channel SNR under different mmWave pilot SNR with 8 active mmWave antennas. From the figure, the FusionNet significantly outperforms the the baseline for most cases except when the mmWave pilot SNR is extremely low and the sub-6GHz training SNR is very high, which is not unexpected since in this case the calibration effect is not accurate enough, and then may drag down the performance predicted from the sub-6GHz channel. Nevertheless, the calibration effect with a low SNR is still positive when the sub-6GHz SNR is low. Therefore, the mmWave pilots would greatly help enhance the performance of a pure DNN in most practical scenarios. Moreover, further increasing the pilot SNR beyond 5 dB does not present more positive effect, which is essential for mmWave transmission when there is a severe path loss. Similar observations can also be found in the achievable rate in Fig. 7(b).

In previous examples, the FusionNet is examined with fully-loaded pilot subcarriers in both frequency bands. However, the practical protocol may assign limited number of pilot subcarriers in each OFDM block and it is of interest to check whether the FusionNet still works in this case.

Table. II demonstrates the FusionNet’s performance when a fraction of OFDM subcarriers of the mmWave band are used as pilots, with $\tilde{N}_{m}=4$ active antennas. From the table, prediction accuracy drops very little compared to the fully loaded pilots even if we use only one pilot because that the beam calibration process mainly uses the angular information of the mmWave channel, thus the additional OFDM subcarriers cannot provide further prediction improvement. Therefore, in practice, one may reduce the mmWave pilot number to enhance the data throughput.

Table. III shows the FusionNet’s performance using a fraction of OFDM subcarriers of the sub-6GHz channel. All OFDM subcarriers on $\tilde{N}_{m}=4$ mmWave active antennas are used as pilots. Different from Table. II, prediction accuracy drops when the SNR of the sub-6GHz channel is low, however, it ceases as the sub-6GHz SNR increases. This is because the sub6-network of FusionNet might easily fit to the noise at the low SNR region, using more OFDM subcarriers for training will help reduce the effect of noise.

In this part, two proposed data pre-processing approaches are evaluated. The augmentation rate, $R_{aug}$, is defined as the size of the augmented dataset divided by the size of the original dataset.

Fig. 8 shows the prediction performances utilizing channel sparsity, data augmentation, as well as the combination of these two approaches. For this simulation, ${\tilde{N}}_{m}=2$, the mmWave pilot SNR is 20 dB and the augmentation rate is 8. From the figure, the prediction performance is improved by exploiting the channel sparsity and adopting data augmentation approach especially at the low sub-6GHz SNR, it can be further improved combining the two pre-processing approaches.

Fig. 9 displays prediction accuracy versus the augmentation rate when the SNR of the sub-6GHz and the mmWave channel are $-10$ dB and 20 dB, respectively, where the channel sparsity is also exploited. The prediction accuracy improves rapidly at first and then slows down with 2 or 4 active antennas, showing the synthesized data has fully exploited the underline information after certain rate. However, when $\tilde{N}_{m}$ increases to 16, the improvement brought by data augmentation is limited since the prediction accuracy with the original dataset is good enough.

With the significant improvement brought by a few mmWave pilots, one natural question arises: is the enhancement because $\tilde{\mathbf{h}}_{m}[k]$ itself is good enough to predict the optimal downlink beam? To answer this question, the prediction performance using mmWave channel only along with the baseline and the FusionNet, under pilot SNR 20 dB is shown in Fig. 10 where the FusionNet adopts 8 active mmWave antennas.

From the figure, the prediction accuracy based on the active mmWave antennas is not satisfactory, which is still below 85 $\%$ even if half of the total mmWave antennas (i.e., $\tilde{N}_{m}=32$) are active. In brief, Fig. 10 clearly demonstrates the intriguingly novel aspect of the FusionNet, which can merge the two “mediocre” ways and results in an extremely precise prediction.