Beam-Delay Domain Channel Estimation for mmWave XL-MIMO Systems
Abstract
This paper investigates the uplink channel estimation of the millimeter-wave (mmWave) extremely large-scale multiple-input-multiple-output (XL-MIMO) communication system in the beam-delay domain, taking into account the near-field and beam-squint effects due to the transmission bandwidth and array aperture growth. Specifically, we model the sparsity in the delay domain to explore inter-subcarrier correlations and propose the beam-delay domain sparse representation of spatial-frequency domain channels. The independent and non-identically distributed Bernoulli-Gaussian models with unknown prior hyperparameters are employed to capture the sparsity in the beam-delay domain, posing a challenge for channel estimation. Under the constrained Bethe free energy minimization framework, we design different structures on the beliefs to develop hybrid message passing (HMP) algorithms, thus achieving efficient joint estimation of beam-delay domain channel and prior hyperparameters. To further improve the model accuracy, the multidimensional grid point perturbation (MDGPP)-based representation is presented, which assigns individual perturbation parameters to each multidimensional discrete grid. By treating the MDGPP parameters as unknown hyperparameters, we propose the two-stage HMP algorithm for MDGPP-based channel estimation, where the output of the initial estimation stage is pruned for the refinement stage for the computational complexity reduction. Numerical simulations demonstrate the significant superiority of the proposed algorithms over benchmarks with both near-field and beam-squint effects.
Index Terms:
Channel estimation, millimeter-wave, extremely large antenna array, near-field effect, beam-squint effect.I Introduction
Driven by the demands of higher data rates, both millimeter-wave (mmWave) and extremely large-scale multiple-input multiple-output (XL-MIMO) technologies are expected to play pivotal roles for the future cellular communication systems due to the vast spectrum and high spectral efficiency [1, 2, 3, 4, 5]. Accurate channel estimation is crucial for enabling the performance gains of mmWave and XL-MIMO, which is significantly challenged by the near-field and beam-squint effects [6, 7, 8, 9, 10].
Due to the extremely large aperture array (ELAA) employed in the mmWave XL-MIMO systems, the Rayleigh distance, which is proportional to the square of the array aperture, becomes comparable to the cell coverage radius [6]. This fact allows mobile terminal (MTs) and scatterers to appear in the near-field region, thereby leading to the special wave propagation [7]. On the other hand, large bandwidths are typically employed in the mmWave systems, such as the maximum transmission bandwidth of up to GHz in the fifth-generation (5G) New Radio (NR) specifications [11], which results in a rather low sampling interval. Therefore, when mmWave is integrated with ELAA, the propagation delay difference over the antenna array will be higher than the sampling interval, resulting in the beam-squint effect [8, 9, 10]. With the above analysis, it is shown that the near-field and beam-squint effects in mmWave XL-MIMO systems render its channel model significantly different from that of the conventional massive MIMO (mMIMO) system, which calls for new approaches for accurate channel estimation.
I-A Previous Works
At millimeter-wave frequencies, the channels are typically contributed by very few paths due to the sparse scattering nature [12, 13, 14], which facilitates compressed sensing techniques to enhance the channel estimation performance. For conventional mMIMO systems, the orthogonal matching pursuit (OMP)-based channel estimation algorithm is proposed in [15], which exploits beam domain sparsity based on the angle sampling and reconstructs the channel by extracting the dominant paths. For orthogonal frequency division multiplexing (OFDM) systems, [16] and [17] present the simultaneously weighted OMP (SW-OMP) algorithm and subcarrier selection-simultaneous iterative gridless weighted-orthogonal least squares (SS-SIGW-OLS) to exploit the joint sparsity of beam domain channels under different subcarriers. Furthermore, channel estimation based on beam-delay domain sparsity is proposed in [18] to explore the inter-antenna and inter-subcarrier correlations.
Since the near-field effect in XL-MIMO systems destroys the plane wave assumption, the beam domain sparsity based on the angle sampling no longer holds, necessitating the novel beam domain representation. Following the spherical wave in the near-field channels, the beam domain sparsity based on the angle-distance sampling, denoted as polar domain, is proposed in [19, 20], followed by the OMP algorithm for channel estimation. To alleviate the estimation error introduced by discrete sampling, [19] presents the polar domain simultaneously iterative gridless weighted (P-SIGW) algorithm, which updates the angle-distance sampling grids based on the gradient descent. As the scatterers and MTs may be distributed in both far-field and near-field regions, the hybrid-field model is employed in [21, 22, 23, 24] for such scenarios, which exploit both angle and angle-distance sampling.
With the transmission bandwidth and array aperture growth, the beam-squint effect becomes more significant, which destroys the joint sparsity of beam domain across subcarriers. To address this issue, [25] and [26] propose the pattern of beam domain sparsity variations with subcarriers in far-field channels, followed by the generalized simultaneous OMP (GSOMP) the beam-split pattern detection (BSPD) algorithms. Furthermore, by exploiting the beam-delay domain sparsity with beam-squint effects, the super resolution-based channel estimation algorithms are proposed in [27, 28, 29]. Most recently, the channel estimation with both near-field and beam-squint effects is investigated in [30, 31]. Specifically, [30] explores the bilinear patterns of beam domain sparsity variations with subcarriers in the common sensing matrix, facilitating the bilinear patterns detection (BPD)-based channel estimation algorithm. Besides, the near-field beam-split-aware OMP (NBA-OMP) algorithms is proposed in the [31] based on the GSOMP algorithm, which exploits the frequency selective sensing matrices to explore beam domain sparsity based on angle-distance domain sampling and the joint sparsity across subcarriers. The exploration of joint sparsity variations due to beam squint effects in [31, 30] provides one solution for the channel estimation in mmWave XL-MIMO systems, paving the way for subsequent work.
I-B Motivations and Contributions
In mmWave XL-MIMO systems, the near-field and beam-squint effects induce substantial changes in channel models, necessitating more accurate modeling. Besides, there are significant correlations between subcarriers in the OFDM systems, which are under-explored in such scenarios. Finally, the sparse model based on discretized sampling encounters an inherent mismatch with continuous parameters, thereby warranting further investigation into effective mitigation strategies for this issue.
Motivated by the previous works, we investigate the channel estimation for mmWave XL-MIMO systems with near-field and beam-squint effects. The main contributions of this paper are summarized as follows:
-
•
By incorporating the near-field and beam-squint effects, we model the channel in the beam-delay domain to explore inter-antenna and inter-subcarrier correlations. In doing so, we end up with the beam-delay domain sparse representation of the spatial-frequency domain channel. To capture the beam-delay domain sparsity, the independent and non-identically distributed Bernoulli-Gaussian models with unknown prior hyperparameters are employed, facilitating the formulation of the channel estimation problem.
-
•
To cope with the unknown hyperparameters in the prior models, we introduce the constrained Bethe free energy (BFE) minimization framework, which enables the joint estimation of the beam-delay domain channel and the hyperparameters in the prior model. The hybrid message passing (HMP) algorithm is established by designing different belief structures to estimate beam-delay domain channels and their prior hyperparameters, thereby, spatial-frequency domain channels.
-
•
To further improve the model accuracy, we introduce the multidimensional grid point perturbation (MDGPP)-based representation, which assigns individual perturbation parameters to each multidimensional discrete grid and thus mitigates mismatches between continuous parameters and discrete grids. Under the constrained BFE minimization framework, the MDGPP parameters are treated as unknown hyperparameters, facilitating the HMP algorithm for MDGPP-based channel estimation. To reduce the computational complexity, we propose the two-stage HMP algorithm, where the output of the initial estimation stage is pruned for the refinement stage.
Organization: The rest of this paper is organized as below. Section II describes the system model and formulates the channel estimation problem. In Section III, the HMP algorithm is developed based on constrained BFE minimization framework. The two-stage HMP algorithm for the MDGPP-based channel estimation is proposed in Section IV, whose computational complexity is also analyzed. Numerical simulations in Section V shows the superiority of the proposed algorithms over the benchmarks. Finally, Section VI concludes the paper.
Notation: Throughout this paper, imaginary unit is represented by . Lowercase, bold lowercase, and bold uppercase letters denote scalars, column vectors, and matrices, respectively. The transpose, conjugate, and conjugate-transpose operations are represented by the superscripts , , and , respectively. The Kronecker and Hardmard products are represented by the operator and , respectively. and are the -th element of a vector and the -th element of a matrix, respectively. and represent the and norms, respectively. and denote the maximum and minimum operators, respectively. and denote the expectation and variance operators, respectively. , , and denote the floor, ceil, and modulo operations, respectively. The element-wise inverse is represented by the superscripts . The symbols denote the complex number fields. denotes the random variable obeying circularly symmetric complex Gaussian distribution with mean and variance .
II System Model and Problem Formulation
In this paper, we consider the XL-MIMO-OFDM system in mmWave frequencies, which consists of a BS equipped with a uniform linear array (ULA) of () antennas and single-antenna MTs. In the time dulplex division (TDD) systems, the channel is estimated from the uplink pilot symbols. It is assumed that the pilot subcarriers in each training symbol are equally spaced without overlapping for each MT. Let and denote the number of subcarriers and subcarrier spacing, respectively. Then the transmission bandwidth and pilot subcarrier spacing can be defined as and , respectively.
II-A Received Signal Model and Channel Model
In the mmWave XL-MIMO systems, the BS usually adopts hybrid precoding architecture, which makes only a far smaller number of baseband received signals at each training symbol observed. With the hybrid precoding architecture, we assume that the BS is equipped with RF chains (), where the RF chains and the antennas are connected by phase shifters network[32]. Besides, quasi-static channel assumptions are considered in this paper, where BS, MTs and scatters remain nearly unchanged during channel estimation. Thus, the received signal under the -th pilot subcarrier at the -th training symbol can be expressed as111Since non-overlapping pilot subcarriers are allocated for different MTs, we can focus on only the single MT and omit the MT index to simplify the expression.
(1) |
where denotes the pilot under the -th pilot subcarrier at the -th training symbol, which can be set to without loss of generality, and denotes the additive white Gaussian nosie (AWGN) under the -th pilot subcarrier.
In the hybrid precoding architecture, the number of measurements required for reliable channel estimation cannot be satisfied with only one pilot symbol, and thus pilot symbols are required. Stacking all the received signal under the -th pilot subcarrier in a column vector, we can obtain
(2) |
where denotes the hybrid precoder and denotes the number of measurements. Then, all received signals under pilot subcarriers are stacked in a column vector, so that the received signal can be expressed as
(3) |
where denotes the hybrid precoder of all pilot subcarriers, and denotes the spatial-frequency domain channel.
We assume that the channel between the target MT and the BS has paths. Then the channel between the MT and the -th antenna of BS under the -th pilot subcarrier can be expressed as
(4) |
where denotes the complex gain of -th path, denotes the distance between the -th antenna of BS and the last-hop scatter (LHS)222For LOS path and NLOS path, the LHS is defined as the MT and the last scatterer before arriving at the antenna array, respectively. of -th path, denotes the propagation delay between the MT and LHS of -th path, denotes the operating frequency of the -th pilot subcarrier, and denotes the speed of light.

As shown in Fig. 1, the distance between the -th antenna of BS and LHS of the -th path for MT satisfies [7]:
(5) |
where denotes the physical angle between the -th antenna of ULA and LHS of the -th path, denotes the index of reference antenna, denotes the antenna index difference between the -th antenna and the reference antenna, and follows from the fact that the geometric constraint is constant, . With the Taylor series expansion of at , can be approximated as
(6) |
where denotes the angle between the -th antenna of BS and LHS of the -th path, denotes the half-wavelength antenna spacing, and is obtained by ignoring the higher order terms of .
For convenience, and are regarded as the location parameters of the -th path for MT without causing confusion. Substituting (II-A) into (4) and stacking the spatial domain channels into vector form, we can obtain
(7) |
where denotes the propagation delay between the MT and reference antenna at BS, denotes phase differences under the -th pilot subcarrier, which can be defined by
(8) |
denotes power differences, which can be defined by
(9) |
where denotes the normalized factor, such that
(10) |
and the common phase and gain between all antennas independent of is merged into the path gain . For convenience, we define as beam domain steering vector.
In (7), the differences between the mmWave XL-MIMO and mMIMO channel model are two-fold:
- 1.
-
2.
Beam-squint effect: In mmWave XL-MIMO systems, the propagation delay differences on the antenna array are not negligible due to the large bandwidth of mmWave systems, which leads to the frequency-dependent beam domain steering vector [8].
To illustrate the deviation from the plane wave assumption due to near-field effects and facilitate subsequent beam domain representations, we can introduce the slope parameter to characterize the rate of instantaneous angle change with , which is shown by
(11) |
Then, the angle-distance sampling can be converted to the angle-slope sampling by replace the with , resulting in the beam domain steering vector .
By stacking the spatial channel of all pilot subcarriers, the spatial-frequency channel is expressed as
(12) |
where denotes the delay domain steering vector defined as , and denotes aggregate beam domain steering vector given by
(13) |
Remark 1.
When specific effects and structure are ignored, the conventional mMIMO channel model can be considered as special cases of the proposed channel model.
- 1.
- 2.
- 3.
- 4.
II-B Beam-Delay Domain Sparse Representation
From (12), the spatial-frequency domain channel is superposed by several paths, resulting in the inter-antenna and inter-subcarrier correlations. To exploit such correlations, we develop the beam-delay domain sparse representation. Toward this end, we assume that the angles, slopes and delays for each MT satisfy , , and , respectively. The maximum slope parameter , where denotes the minimum service distance, and denotes the maximum allowable delay, which is determined by the cyclic prefix length and the array aperture under the beam-squint effect. Then, the angle, slope, and delay domain are sampled uniformly at intervals , 333The sampling interval is determined by the discrete Fresnel function with the coherence threshold [19]., and to obtain the sampling sets, which can be defined by (14a), (14b), and (14c), respectively.
(14a) | |||
(14b) | |||
(14c) |
where denotes the sampling point index set, , , and denote the number of sampling points in angle, slope, and delay domains, respectively. Since the sampling intervals decrease as and increase, the spatial-frequency domain channel is well approximated as a linear combination of basis functions determined by angle-slope-delay tuples for sufficiently large and .
Therefore, with the pre-defined sampling set of the angles, slopes, and delays, (12) can be approximated as
(15) |
where denotes the angle-slope-delay domain basis function defined as
(16) |
and denotes the complex gain of the path with respect to angle-slope-delay tuple , which is given by
(17) |
Reformulating (15) into matrix form and assuming that the spatial-frequency domain channels can be represented perfectly, we can obtain
(18) |
where and denote the beam-delay domain transformation matrix and the beam-delay domain channel defined as
(19) |
and
(20) |
respectively. Without causing confusion, maps to the simple index in a one-to-one manner, which satisfies
(21) |
and .
II-C Channel Estimation Problem Formulation
With the proposed sparse representation, the channel estimation problem is to estimate the beam-delay domain channel and ultimately the spatial-frequency domain channel based on the received signal. The MMSE channel estimator can be expressed as the a posteriori mean [33], given by
(22) |
where and denote the auxiliary vector and measurement matrix, respectively, denotes the posterior probability density function (PDF) of and .
From the received signal model and the beam-delay domain channel representation, can be expressed as
(23) |
where , , and denote received noise model, baseband transfer model, and beam-delay domain channel prior model, respectively.
Since AWGN channels are employed, the conditional PDF can be detailed as
(24) |
where and denote the -th element of and , respectively. Then the conditional PDF can be factorized as
(25) |
where denotes the -th row of .
With the assumption of wide-sense stationary uncorrelated scattering Rayleigh fading channel, the elements of are statistically independent for sufficiently large and [34]. Moreover, the sparse nature of mmWave channels is considered, implying that only a few number of elements in have significant power, with most elements being zero. Thus, we can model as an independent and non-identically distributed Bernoulli-Gaussian distribution, and the prior PDF controlled by hyperparameters can be decomposed into
(26) |
where , denotes the sparsity level, and denotes the power of when is nonzero.
Building on the probabilistic modeling, the MMSE estimation of spatial-frequency domain channel can be given by
(27) |
due to the commutability of the MMSE estimator over linear transformations [33].
III Constrained BFE Minimization-based Channel Estimation
The MMSE estimator in (22) requires the multi-dimensional integrals of , which introduces an unacceptable complexity in large-scale systems. In this section, we approximate with structured trial beliefs based on the Bethe method, and then develop the HMP algorithm to achieve efficient joint estimation of beam-delay domain channel and prior hyperparameters.
III-A Constrained BFE Minimization Framework
Following the VFE minimization framework [35, 36, 37], we approximate by the trial belief , which can be obtained by minimizing the Kullback-Leibler (KL) divergence of and . The KL divergence minimization problem can be expressed by
(28) |
where denotes the constrained set of , denotes the KL divergence, denotes the VFE defined by
(29) |
where denotes the joint PDF, and denotes the Helmholtz free energy independent of . Thus, the KL divergence minimization is equivalent to the VFE minimization.
The VFE minimization is still intractable in large-scale systems, which motivates the constrained set design of . To balance the exactness and tractability, the Bethe method is adopted to design the constrained set and convert the VFE minimization into the BFE minimization. Following Bayes rule and the probabilistic model, the joint PDF can be factorized as
(30) |
where , and denote the factor nodes. Based on the Bethe method [38, 39, 40], we introduce auxiliary beliefs , and for factor nodes , and , respectively. Similarly, we introduce auxiliary beliefs , , and for variable nodes , , and , respectively. Following this, the trial belief and the BFE can be expressed as
(31) |
and
(32) |
respectively, where denotes the entropy.
Inspired by the variational message-passing algorithm [36], we can factorize complicated beliefs into the product of simple beliefs by exploiting the slow-varying characteristic of hyperparameters, which can be expressed as
(33) |
where , and denote the auxiliary belief with respect to , and , respectively. Furthermore, the hyperparameter beliefs can be constrained by the Dirac delta function, which is given as
(34a) | |||
(34b) |
where and denote the ground-truth hyperparameters.
To guarantee the global dependence of factor nodes, the auxiliary beliefs of factor and variable nodes in Bethe method should satisfy marginal consistency constraints (MCCs), which is intractable for continuous random variables. To address this issue, the MCCs can be relaxed to mean and variance consistency constraints (MVCCs), which can be given by
(35a) | |||
(35b) | |||
(35c) | |||
(35d) |
where the beliefs without index subscripts denote the vector version of the scalar beliefs. The variance consistency constraints with respect to and are further relaxed as (36) by averaging, which can be given by
(36) |
Finally, the KL divergence minimization problem can be converted into the constrained BFE minimization problem, which is expressed by
(37) |
III-B HMP Algorithm for Channel Estimation
The optimization problem in (37) can be solved by the Lagrange multiplier method [41], which sets the derivative of the Lagrangian function with respect to the auxiliary beliefs to zero. Since the auxiliary beliefs of the hyperparameters can be completely factorized in (33), their MCCs can be satisfied constantly and thus can be ignored in the Lagrangian function. Substituting constraints (33) and (34) into , the Lagrangian function of (37) can be expressed as
(38) |
where denotes the BFE with factorized hyperparameter constraints, which can be obtained by replacing the KL-divergence with in (III-A), and , and defined in (39) denote the subfunctions of mean consistency constraints and variance consistency constraints, respectively, , , , and denote the Lagrangian multiplier for mean consistency constraints, , , and denote the Lagrangian multiplier for variance consistency constraints.
(39a) | ||||
(39b) |
Due to space limitations, the stationary-point equations are provided in Appendix A. Building on the derived equations, the resulting HMP algorithm for channel estimation is summarized in Algorithm 1, where the adaptive damping based on the BFE evaluation [42] is introduced to improve the convergence.
Remark 2.
In the constrained BFE minimization problem, we leverage the properties of different beliefs to design hybrid constraints, thus incorporating the existing message passing algorithms into Algorithm 1. It is noteworthy that the Bernoulli-Gaussian distribution is only one of the prior models, and other specific forms of can be obtained by different prior models, such as Bernoulli-Gaussian-Mixture [43] and Laplacian [44] distributions.
IV MDGPP-Based Channel Estimation
Due to the finite number of antennas and pilot subcarriers in the practical system, the sampling sets fail to match the continuous parameters in the channel perfectly [19, 28], resulting in the modeling inaccuracy [45, 46]. To address this issue, we introduce the MDGPP-based representation and propose the two-stage HMP algorithm based on the constrained BFE minimization framework.
IV-A MDGPP-Based Channel Representation
To eliminate the gap between the sampling sets and the continuous parameters, the spatial-frequency domain channel can be decorrelated by the MDGPP-based channel basis functions, which is expressed as
(40) |
where , , and denote the perturbation parameters in angle, slope, and delay of the -th sampling tuple, respectively, denotes the complex gain of the -th sampling tuple, and denote the perturbed basis function of the -th sampling tuple defined as
(41) |
where the index tuple and the simple index satisfies the mapping rule (21). The proposed MDGPP-based representation assigns individual perturbation parameters to each beam-delay domain sampling grid, while the conventional perturbation assigns common perturbation parameters based on grid line constraints [47]. To visualize advantages of the MDGPP-based representation, the schematic of the two-dimensional example is shown in Fig. 2, where the perturbations are shown on the parameter. It can be observed that, with conventional perturbations, the perturbation of “Grid 1” for matching “Path 1” causes the mismatch between “Grid 2” and “Path 2” due to the common perturbation parameter , which can be avoided by assigning different perturbation parameters and in the MDGPP.
Remark 3.
The conventional perturbation can be considered as a special case of the MDGPP. Specifically, when the MDGPP parameters exhibits the specific structure as
(42a) | |||
(42b) | |||
(42c) |
the MDGPP-based representation degenerates to the form in [47], where , and denote vectors of MDGPP parameters in the angle, slope and delay domains, respectively, and , and denote vectors of conventional perturbation parameters in the angle, slope and delay domains, respectively.


Benefiting from MDGPP-based representations, each path can be associated with the sampling tuple in which is closest to the path parameters, and the deviations are described by the MDGPP parameters. By stacking the perturbed basis function into matrix form, the spatial-frequency domain channel can be given by the bilinear form as
(43) |
where denotes the perturbed transformation maxtrix with perturbation parameter vector , , and . To balance the exactness and complexity, the perturbed transformation maxtrix is approximated by the Taylor series [48], which can be expressed as
(44) |
where , , and denote the first-order partial derivatives of with respect to angle, slope and delay.
IV-B Two-Stage HMP Algorithm for MDGPP-based Channel Estimation
Following the proposed representation, the channel estimation problem aims to estimate both beam-delay domain channel and the MDGPP paramters, leading to the structured bilinear model. To tackle this problem, the joint PDF with the MDGPP paramters is factorized as
(45) |
where the MDGPP parameters are assumed to be mutually independent of each other and beam-delay domain channels. With the independent assumptions, the prior PDF of MDGPP parameters can be fully factorized to , , and , which can be denoted by , , and , respectively. The conditional PDF can be factorized as
(46) |
Building on factorizations of the joint PDF, the factor graph of MDGPP-based channel estimation is shown in Fig. 3. We introduce auxiliary beliefs , , , and for factor nodes , , , and , respectively. Similarly, the auxiliary beliefs , , and are introduced for variable nodes , , and , respectively. To guarantee the tractability of the structured bilinear inference problem, the complicated beliefs can be factorized as
(47) |
where , , and are further constrained by the Dirac delta function with ground-truth perturbation parameters , , and .

With the formulation above, the perturbation parameters can be acquired efficiently by the constrained BFE minimization framework. Compared with Algorithm 1, the HMP algorithm for MDGPP-based channel estimation differ in the update of perturbation parameters, which is derived in Appendix B. The update rule of angle domain perturbation parameters is
(48) |
where and can be defined by
(49) |
and
(50) |
where , and denotes the posterior covariance matrix whose diagonal elements are in Algorithm 1. To solve the above optimization problem, the closed-form solution without bounding constraints is obtained and then checked whether the bounding constraints are satisfied. When the closed-form solution does not satisfy the bounding constraints, the perturbation parameters are adjusted sequentially by
(51) |
where . Similarly, the slope and delay domain perturbation parameters and can be obtained in the same manner.
However, the closed-form solution of the MDGPP parameters requires matrix inversion, which has a prohibitively high computational complexity in large-scale systems. Motivated by the mmWave channel sparsity, the two-stage HMP algorithm for MDGPP-based channel estimation can be proposed in Algorithm 2, which involves the initial estimation and refinement stages. In the first stage, the initial estimation of the beam-delay domain channel is obtained by Algorithm 1 without model perturbations. Building upon the initial estimation, the elements with energies below a certain threshold are pruned to significantly reduce the number of perturbation parameters to be estimated next. In the second stage, the pruned beam-delay domain channel and its corresponding perturbation parameters are iteratively acquired.
IV-C Computational Complexity
The per-iteration computational complexity of Algorithm 1 is , which are dominated by matrix-vector multiplications. For Algorithm 2, the per-iteration computational complexities in the initial estimation stage and the refinement stage are and , where the latter is dominated by the perturbation parameter acquisitions. Since the size of beam-delay domain channel after pruning can be assumed to keep the same order of magnitude as path number (), we have , such that the per-iteration computational complexity in the refinement stage can be approximated as . Therefore, the total computational complexity of Algorithm 2 is about , where and denote the number of iterations for the initial estimation stage and refinement stage, respectively. It can also be found that controls the tradeoff between the computational complexity and performance. Specifically, a larger leads to higher complexity, while smaller results in performance degradations since the true beam-delay domain channel components will be pruned.
It can be observed that the exploration of inter-subcarrier correlations increases the computational complexity but delivers significant performance gains, as verified in subsequent simulations. Since the proposed algorithms are inherently compatible with the prior models, the further reduction of the computational complexity is feasible by exploiting the prior information, which can be acquired by the sophisticated statistical channel state information estimation [49] and channel knowledge map [50]. This reduction unfolds two aspects:
-
•
The reduction in unknown variables: With the prior information, the support acquisition of beam-delay domain channels is avoided, which enables the reduction of the unknown variables to the order of the number of paths.
-
•
The reduction in observations: The received signals are highly correlated with support-aided beam-delay domain channels. To eliminate such redundancy, the row-orthogonalization [51] can be introduced to reduce the effective observations to the order of unknown variables based on an equivalent BFE representation.
Furthermore, the subsequent simulations also indicate that the proposed algorithms requires fewer pilot symbols than benchmarks, enabling a further reduction in computational complexity while maintaining performance.
V Numeical Simulations
Unless otherwise specified, the simulation configurations follow the system model in Section II with the paramters summarized in Table I. In the subsequent simulations, the SNR is defined as the received SNR defined as .
Paramter | Value |
---|---|
Carrier Frequency | GHz |
Pilot Subcarrier Spacing | MHz |
Number of BS Antennas | |
Number of BS RF Chains | |
Number of Pilot Subcarriers | |
Number of Pilot Symbols | |
Minimum Service Distance | m |
Number of Paths |
To demonstrate the superiority of the proposed algorithms, we select the following excellent channel estimation algorithms as benchmarks:
-
•
P-SIGW [19]: This channel estimation algorithm is designed for near-field channels without beam squint effect, which models the beam domain sparsity with sampling in the angle and distance doamin.
-
•
GSOMP [25]: This channel estimation algorithm is designed for far-field channels with beam squint effect, which employs frequency-dependent sensing matrices.
- •
For the comparison with the proposed algorithms and benchmarks, the normalized mean square error (NMSE) of the spatial-frequency channels is adopted as the performance metric, which is defined by:
(52) |
where denotes the numebr of simulations, and denote the ground-truth and estimated spatial-frequency channels in the -th simulation, respectively.

The convergence performance comparison of Algorithm 1 and Algorithm 2 at the SNR of dB is provided in Fig. 4. It can be seen that the MDGPP-based representation introduced in Algorithm 2 significantly reduces the NMSE from dB to dB. Since both Algorithm 1 and Algorithm 2 have rapid decline rate of NMSE in the beginning iterations, we can terminate the algorithm iterations earlier to reduce the computational complexity based on the performance requirements of practical systems. To show the effectiveness of the proposed beam-delay domain sparse representation, the beam-delay domain channel power of the on-grid case estimated from Algorithm 1 without specific effects are provided in Fig. 5. The proposed beam-delay domain sparse representation can decorrelate the spatial-frequency domain channel effectively, while ignoring the specific effects induces significant energy leakage.


The NMSE of the proposed algorithms and the benchmarks with the variation of SNR is shown in Fig. 6. It can be seen that the proposed algorithms significantly outperforms benchmarks at the full SNR region, with the performance gain increasing as SNR grows. Due to the mismatch between the assumptions and this scenario, the GSOMP and P-SIGW fail to achieve an NMSE below dB at the SNR of dB. By exploiting the joint sparsity variations across subcarriers and modeling the spherical wave propagation, the NMSE performances of BPD and NBA-OMP are almost identical, both outperforming GSOMP and P-SIGW. However, there is still a significant gap between the BPD, NBA-OMP and proposed algorithms, which stems from the further exploration of inter-subcarrier correlations. From the performance comparison of Algorithm 1 and Algorithm 2, it is clear that MDGPP-based representations can significantly improve model accuracy and thus Algorithm 2 achieves the lowest NMSE.

The trend of the channel estimation performance with the number of pilot symbols is crucial in the system design, hence, we provide the NMSE of the different algorithms as the number of pilot symbols varies in Fig. 7. Although the received SNRs are kept identical for different numbers of pilot symbols, the NMSE of the GSOMP, P-SIGW, BPD, NBA-OMP, and the proposed algorithms still decreases with the increase in the number of pilot symbols. It indicates that the number of measurements is also an essential factor in the channel estimation problem besides the received SNRs. The proposed algorithms significantly outperform all benchmarks for the full range of pilot symbol number, implying that the pilot overhead can be effectively reduced in the proposed algorithms with the same NMSE requirement. Specifically, Algorithm 1 can reduce the pilot overhead by about compared to the NBA-OMP for dB NMSE, while Algorithm 2 can further reduce the pilot overhead by about compared to Algorithm 1 for dB NMSE.

To illustrate the impact of near-field and beam-squint effects on different algorithms, the NMSEs of the proposed algorithms and the benchmarks as maximum slope parameter of MTs and the transmission bandwidths varies are shown in Fig. 8 and Fig. 9, respectively. As the near-field effect growing more significant, the deviation of the plane wave assumption adopted by the GSOMP rises, resulting in increasing NMSE. For the beam-squint effect, the P-SIGW that relies on the frequency-independent beam domain assumption exhibits the increasing NMSE as the transmission bandwidth grows. Besides, although the BPD and NBA-OMP model the frequency-dependent beam domain and spherical wave propagation to achieve better NMSE than other benchmarks, the proposed algorithms reach the lowest NMSE near dB under both near-field and beam-squint effects.

VI Conclusion
In this paper, we investigated the uplink channel estimation of mmWave XL-MIMO communication systems in the beam-delay domain, taking into account the near-field and beam-squint effects. Specifically, we modeled the sparsity in the beam-delay domain, and captured it by the independent and non-identically distributed Bernoulli-Gaussian models with unknown prior hyperparameters. To improve the model accuracy, we introduced the MDGPP-based representation, and treated the MDGPP parameters as unknown hyperparameters. Under the constrained BFE minimization framework, we established the two-stage HMP algorithm for MDGPP-based channel estimation, which enables efficient joint estimation of the beam-delay domain channel and its prior hyperparameters, and reduces the computational complexity by pruning the output of the initial estimation stage. With extensive numerical simulations, the proposed algorithms exhibits superiority over benchmarks in mmWave XL-MIMO communication systems. For future work, channel estimation of mmWave XL-MIMO systems with spatial non-stationary can be investigated, which further destroys the sparsity in the beam domain.
Appendix A HMP Algorithm for Constrained BFE Minimization
Based on the Lagrangian multiplier method, we set the derivative of the Lagrangian function with respect to the auxiliary beliefs to zero. The stationary-point of the auxiliary belief can be given in (53), where . By substituting the stationary equations in (53) into the MVCCs in (35) and (36), we obtain a series of iteration equations for the Lagrange multipliers. The iteration equations are organized following the message flow backwards and forwards from the observations to the beam-delay domain sparse channel, which yields Algorithm 1 except for the hyperparameter learning.
(53a) | ||||
(53b) | ||||
(53c) | ||||
(53d) | ||||
(53e) |
For the hyperparameters , the constrained BFE minimization with respect to can be expressed as
(54) |
where denotes the fixed-point solution of in the -th iteration. The derivative of the objective function with respect to can be obtained by
(55) |
Splitting the domain of integration in (54) into closed ball and its complement , the first order condition can be given with the limit of :
(56) |
Following this, the learning rule of is
(57) |
whose bounding constraint constantly holds.
In the similar manner, the learning rule of is given by
(58) |
where denotes the posterior likelihood of the -th angle-slope-delay tuple, which can be defined by
(59) |
Appendix B Perturbation Parameter Learning in the MDGPP-based Channel Estimation
Due to the similarity of perturbation parameter updating in the angle, slope and delay domains, we will give the derivation of the perturbation parameter updating rules in the angle domain as an example.
The optimization objective function of constrained BFE minimization with respect to can be expressed as
(60) |
To avoid the singularity of Dirac delta function, the condition PDF can be approximated by the limitation form:
(61) |
where denotes the -th row of , and . With this approximation, the optimization objective function can be simplified as
(62) |
For the first term of (62), it can be simplified as
(63) |
For the second term of (62), it can be simplified as
(64) |
where the last equality is based on the identity equation as
(65) |
for the proper dimensions of matrices , and vectors , .
Based on the simplification above, the constrained BFE minimization with respect to can be equivalently expressed in (48), which can be solved efficiently by closed-form solution without bounding constraints and sequential adjustment in (51). The perturbation parameter updating rules in the slope and delay domains can be derived in the similar manner, which are omiited due to the space limitations.
References
- [1] W. Roh, J.-Y. Seol et al., “Millimeter-wave beamforming as an enabling technology for 5G cellular communications: theoretical feasibility and prototype results,” IEEE Commun. Mag., vol. 52, no. 2, pp. 106–113, Feb. 2014.
- [2] M. Xiao, S. Mumtaz et al., “Millimeter wave communications for future mobile networks,” IEEE J. Sel. Areas Commun., vol. 35, no. 9, pp. 1909–1935, Sep. 2017.
- [3] Y. Han, S. Jin et al., “Channel estimation for extremely large-scale massive MIMO systems,” IEEE Wireless Commun. Lett., vol. 9, no. 5, pp. 633–637, May 2020.
- [4] G. Bacci, L. Sanguinetti, and E. Björnson, “Spherical wavefronts improve MU-MIMO spectral efficiency when using electrically large arrays,” IEEE Wireless Commun. Lett., vol. 12, no. 7, pp. 1219–1223, Jul. 2023.
- [5] Z. Wang, J. Zhang et al., “Extremely large-scale MIMO: Fundamentals, challenges, solutions, and future directions,” IEEE Wireless Commun., pp. 1–9, 2023.
- [6] K. T. Selvan and R. Janaswamy, “Fraunhofer and fresnel distances: Unified derivation for aperture antennas,” IEEE Antennas Propag. Mag., vol. 59, no. 4, pp. 12–15, Aug. 2017.
- [7] Z. Zhou, X. Gao et al., “Spherical wave channel and analysis for large linear array in LoS conditions,” in Proc. IEEE Globecom Workshops (GC Workshop), 2015, pp. 1–6.
- [8] B. Wang, F. Gao et al., “Spatial-and frequency-wideband effects in millimeter-wave massive MIMO systems,” IEEE Trans. Signal Process., vol. 66, no. 13, pp. 3393–3406, Jul. 2018.
- [9] M. M. Mojahedian, M. Attarifar, and A. Lozano, “Spatial-wideband effect in line-of-sight MIMO communication,” in Proc. IEEE Int. Conf. Commun. (ICC), 2022, pp. 3972–3977.
- [10] H. Luo, F. Gao et al., “Beam squint assisted user localization in near-field integrated sensing and communications systems,” IEEE Trans. Wireless Commun., 2023.
- [11] 3GPP, “NR; user equipment (UE) radio transmission and reception; part 2: Range 2 standalone,” 3rd Generation Partnership Project (3GPP), Technical Specification (TS) 38.101-2, 2023.
- [12] M. R. Akdeniz, Y. Liu et al., “Millimeter wave channel modeling and cellular capacity evaluation,” IEEE J. Sel. Areas Commun., vol. 32, no. 6, pp. 1164–1179, Jun. 2014.
- [13] I. A. Hemadeh, K. Satyanarayana et al., “Millimeter-wave communications: Physical channel models, design considerations, antenna constructions, and link-budget,” IEEE Commun. Surveys Tuts., vol. 20, no. 2, pp. 870–913, Secondquarter 2018.
- [14] C.-X. Wang, J. Bian et al., “A survey of 5G channel measurements and models,” IEEE Commun. Surveys Tuts., vol. 20, no. 4, pp. 3142–3168, Aug. 2018.
- [15] J. Lee, G.-T. Gil, and Y. H. Lee, “Channel estimation via orthogonal matching pursuit for hybrid MIMO systems in millimeter wave communications,” IEEE Trans. Commun., vol. 64, no. 6, pp. 2370–2386, Jun. 2016.
- [16] J. P. González-Coma, J. Rodríguez-Fernández et al., “Channel estimation and hybrid precoding for frequency selective multiuser mmWave MIMO systems,” IEEE J. Sel. Topics Signal Process., vol. 12, no. 2, pp. 353–367, May 2018.
- [17] N. González-Prelcic, H. Xie et al., “Wideband channel tracking and hybrid precoding for mmWave MIMO systems,” IEEE Trans. Wireless Commun., vol. 20, no. 4, pp. 2161–2174, Apr. 2021.
- [18] K. Venugopal, A. Alkhateeb et al., “Channel estimation for hybrid architecture-based wideband millimeter wave systems,” IEEE J. Sel. Areas Commun., vol. 35, no. 9, pp. 1996–2009, Sep. 2017.
- [19] M. Cui and L. Dai, “Channel estimation for extremely large-scale MIMO: Far-field or near-field?” IEEE Trans. Commun., vol. 70, no. 4, pp. 2663–2677, Jan. 2022.
- [20] Y. Lu and L. Dai, “Near-field channel estimation in mixed LoS/NLoS environments for extremely large-scale MIMO systems,” IEEE Trans. Commun., vol. 71, no. 6, pp. 3694–3707, Jun. 2023.
- [21] Z. Hu, C. Chen et al., “Hybrid-field channel estimation for extremely large-scale massive MIMO system,” IEEE Commun. Lett., vol. 27, no. 1, pp. 303–307, Jan. 2023.
- [22] X. Wei and L. Dai, “Channel estimation for extremely large-scale massive MIMO: Far-field, near-field, or hybrid-field?” IEEE Commun. Lett., vol. 26, no. 1, pp. 177–181, Jan. 2022.
- [23] X. Peng, L. Zhao et al., “Channel estimation for extremely large-scale massive MIMO systems in hybrid-field channel,” in Proc. IEEE/CIC Int. Conf. Commun. China (ICCC), 2023, pp. 1–6.
- [24] Y. Li and M. Jiang, “ADMM-based hybrid-field uplink channel estimation for extremely large-scale MIMO systems,” in Proc. IEEE/CIC Int. Conf. Commun. China (ICCC), 2023, pp. 1–5.
- [25] K. Dovelos, M. Matthaiou et al., “Channel estimation and hybrid combining for wideband Terahertz massive MIMO systems,” IEEE J. Sel. Areas Commun., vol. 39, no. 6, pp. 1604–1620, Jun. 2021.
- [26] J. Tan and L. Dai, “Wideband channel estimation for THz massive MIMO,” China Commun., vol. 18, no. 5, pp. 66–80, May 2021.
- [27] I.-S. Kim and J. Choi, “Spatial wideband channel estimation for mmWave massive MIMO systems with hybrid architectures and low-resolution ADCs,” IEEE Trans. Wireless Commun., vol. 20, no. 6, pp. 4016–4029, Jun. 2021.
- [28] B. Wang, M. Jian et al., “Beam squint and channel estimation for wideband mmWave massive MIMO-OFDM systems,” IEEE Trans. Signal Process., vol. 67, no. 23, pp. 5893–5908, Dec. 2019.
- [29] M. Jian, F. Gao et al., “Angle-domain aided UL/DL channel estimation for wideband mmWave massive MIMO systems with beam squint,” IEEE Trans. Wireless Commun., vol. 18, no. 7, pp. 3515–3527, Jul. 2019.
- [30] M. Cui and L. Dai, “Near-field wideband channel estimation for extremely large-scale MIMO,” Sci. China Inf. Sci., vol. 66, no. 7, p. 172303, Jun. 2023.
- [31] A. M. Elbir, K. V. Mishra, and S. Chatzinotas, “NBA-OMP: Near-field beam-split-aware orthogonal matching pursuit for wideband THz channel estimation,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2023, pp. 1–5.
- [32] R. Méndez-Rial, C. Rusu et al., “Hybrid MIMO architectures for millimeter wave communications: Phase shifters or switches?” IEEE Access, vol. 4, pp. 247–267, Jan. 2016.
- [33] S. M. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory. Prentice-Hall, Inc., 1993.
- [34] J. Yang, A.-A. Lu et al., “Channel estimation for massive MIMO: An information geometry approach,” IEEE Trans. Signal Process., vol. 70, pp. 4820–4834, 2022.
- [35] M. I. Jordan, Z. Ghahramani et al., “An introduction to variational methods for graphical models,” Mach. learn., vol. 37, pp. 183–233, Nov. 1999.
- [36] J. Winn, C. M. Bishop, and T. Jaakkola, “Variational message passing,” J. Mach. Learn. Res., vol. 6, no. 4, Apr. 2005.
- [37] C. M. Bishop and N. M. Nasrabadi, Pattern Recognition and Machine Learning. Springer, 2006, vol. 4, no. 4.
- [38] J. Yedidia, W. Freeman, and Y. Weiss, “Constructing free-energy approximations and generalized belief propagation algorithms,” IEEE Trans. Inf. Theory, vol. 51, no. 7, pp. 2282–2312, Jul. 2005.
- [39] D. Zhang, X. Song et al., “Unifying message passing algorithms under the framework of constrained Bethe free energy minimization,” IEEE Trans. Wireless Commun., vol. 20, no. 7, pp. 4144–4158, Jul. 2021.
- [40] X. Liu, W. Wang et al., “Sparse channel estimation via hierarchical hybrid message passing for massive MIMO-OFDM systems,” IEEE Trans. Wireless Commun., vol. 20, no. 11, pp. 7118–7134, Nov. 2021.
- [41] S. P. Boyd and L. Vandenberghe, Convex Optimization. Cambridge University Press, 2004.
- [42] J. Vila, P. Schniter et al., “Adaptive damping and mean removal for the generalized approximate message passing algorithm,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2015, pp. 2021–2025.
- [43] J. P. Vila and P. Schniter, “Expectation-maximization Gaussian-mixture approximate message passing,” IEEE Trans. Signal Process., vol. 61, no. 19, pp. 4658–4672, Oct. 2013.
- [44] F. Bellili, F. Sohrabi, and W. Yu, “Generalized approximate message passing for massive MIMO mmWave channel estimation with Laplacian prior,” IEEE Trans. Commun., vol. 67, no. 5, pp. 3205–3219, May 2019.
- [45] Y. Chi, L. L. Scharf et al., “Sensitivity to basis mismatch in compressed sensing,” IEEE Trans. Signal Process., vol. 59, no. 5, pp. 2182–2195, May 2011.
- [46] G. Tang, B. N. Bhaskar et al., “Compressed sensing off the grid,” IEEE Trans. Inf. Theory, vol. 59, no. 11, pp. 7465–7490, Nov. 2013.
- [47] G. Liu, A. Liu et al., “Angular-domain selective channel tracking and Doppler compensation for high-mobility mmWave massive MIMO,” IEEE Trans. Wireless Commun., vol. 20, no. 5, pp. 2902–2916, May 2021.
- [48] T. M. Apostol, Mathematical Analysis. Addison-Wesley, 1974.
- [49] A.-A. Lu, Y. Chen, and X. Gao, “2D beam domain statistical CSI estimation for massive MIMO uplink,” IEEE Trans. Wireless Commun., 2023.
- [50] Y. Zeng and X. Xu, “Toward environment-aware 6G communications via channel knowledge map,” IEEE Wireless Commun., vol. 28, no. 3, pp. 84–91, Jun. 2021.
- [51] H. Fan, W. Wang et al., “Generalized approximate message passing detection with row-orthogonal linear preprocessing for uplink massive MIMO systems,” in Proc. IEEE Veh. Technol. Conf. (VTC-Fall), 2017, pp. 1–6.