22institutetext: M. V. Lomonosov Moscow State University, Russia
22email: [email protected] 33institutetext: National Research University Higher School of Economics, Russia
33email: [email protected]
⋆ Equal contribution
On Probability Shaping for 5G MIMO Wireless Channel with Realistic LDPC Codes
Abstract
Probability Shaping (PS) is a method to improve a Modulation and Coding Scheme (MCS) in order to increase reliability of data transmission. It is already implemented in some modern radio broadcasting and optic systems, but not yet in wireless communication systems. Here we adapt PS for the 5G wireless protocol, namely, for relatively small transport block size, strict complexity requirements and actual low-density parity-check codes (LDPC). We support our proposal by a numerical experiment results in Sionna simulator, showing 0.6 dB gain of PS based MCS versus commonly used MCS.
Keywords:
QAM MCS OFDM 5G PS FEC BICM LDPC1 Introduction
In the 5G New Radio downlink procedure, the user equipment proposes the serving base station for use in the next signal transmission of the optimal modulation and coding scheme (MCS) [1] based on quadrature amplitude modulation (QAM). In order to achieve the capacity of the additive white Gaussian noise (AWGN) channel, the transmit signal must be Gaussian distributed. The use of uniformly distributed QAM symbols with optimal coded modulation (CM) leads to a shaping loss of up to 1.53 dB for high order constellations [4]. Bit-interleaved coded modulation (BICM) with parallel bit-wise demapping as currently employed in LTE leads to an additional loss.
Non-uniform constellations (NUC) and geometric shaping (GS) have been recently adopted for the next-generation terrestrial broadcast standard [12]. The QAM constellations are optimized for each target signal-to-noise ratio (SNR) by maximizing the BICM capacity for uniformly distributed bits. Note that, in contrast to standard Gray-labeled QAM (Fig. 1), the Non-Uniform constellations do not allow for a simple independent demapping of the real and imaginary part. Therefore, one-dimensional NUCs for each real dimension were also studied in [12], which provide a reduced shaping gain. The performance of BICM can be improved by using non-uniform constellations (NUC), but there remains a gap to the capacity with Gaussian transmit signal.
In the traditional approach of data transmission, each point in a particular constellation has an equal chance of being transmitted. While this technique gives the highest bit rate for a given constellation size, it ignores the energy cost of the individual constellation points. So, as an alternative to GS, it is also possible to adjust the probabilities of the constellation points such that they follow an approximate discrete Gaussian distribution, using the probability shaping (PS) method [11]. Probabilistically shaped coded modulation (PSCM) enables the BICM system to close the gap to the capacity with Gaussian transmit signal. PS is a CM strategy that combines constellation shaping and channel coding.
In the literature, Gallager’s error exponent approach has been used to study the achievable information rates of PS [5, Ch. 5]. In particular, it was shown that the PS method has achievable capacities for additive white Gaussian noise channels [2]. In [6], the authors revisit the capacity achieving property of PS. The concept of selecting constellation points using a nonuniform Maxwell-Boltzmann PS is investigated in the study [11]. Nonuniform PS signaling scheme reduces the entropy of the transmitter output and, as a result, the average bit rate. However, if low-energy points are picked more frequently than high-energy points, energy savings may (more than) compensate for the bit rate reduction. Authors of [16] proposed a new PS distribution that outperforms Maxwell-Boltzmann is studied for the nonlinear fiber channel. In [9] the authors successfully tested the suitability of PS constellations in a German nationwide fiber ring of Deutsche Telekom’s R&D field test network. In [3] the PS method is implemented in 64-QAM coherent optical transmission system. In [10], a proposed extension to the 5G New Radio polar coding chain is the introduction of a shaping encoder in front of the polar encoder, which will improve the performance with higher order modulation using this PS scheme.
The main objectives of the study:
-
•
In this paper, we investigate the PS Enumerative Sphere Shaping (ESS) [7] method known in the literature with respect to a realistic MIMO OFDM wireless channel with LDPC at a given coderate.
-
•
We provide numerical experiments on the modern Sionna [8] simulation platform and find local optimal parameters for the ESS method, minimizing BLER and providing a gain of up to 0.6 dB over the QAM-16 baseline.
-
•
This study could be interesting from a scientific point of view, since there are almost no published papers on PS that consider such realistic and contemporary scenarios, while considering only theoretical distributions [15].
The basic principle of PS method is presented in Fig. 2. We change the probability of constellation points, which allows us to scale their coordinate with preserving of the mathematical expectation of constellation power.
Complex plane
QAM-4
Complex plane
QAM-16
Complex plane
QAM-64
2 System Model
A block diagram of the proposed PSCM transmitter and receiver is shown in Fig. 6. The main difference to conventional BICM is the distribution matcher that maps the uniformly distributed data bits to bit streams with a desired distribution, which determine the amplitudes of the transmitted QAM symbols. The forward error correction (FEC) encoder generates additional parity bits, which are uniformly distributed and determine the signs of the transmitted QAM symbols. This results in an approximately Gaussian distributed transmit signal using the same constellation mapping as in Long-Term Evolution (LTE).
At the receiver side, the QAM demapper calculates the bit-wise log-likelihood ratios (LLRs) based on the observed receive signal, taking the non-uniform transmit symbol distribution into account. These LLRs are fed to the FEC decoder as in conventional BICM, and the decoder output is finally mapped back to data bits by the distribution deshaper. Note that both the distribution matcher and deshaper correspond to simple one-to-one mappings, which can be efficiently implemented.
3 Optimal Distribution for Probability Shaping
Let be the vector of probabilities of each constellation point and is its random variable — transmitted points on the constellation, where is a number of the constellation points.
Let be a random variable — received points:
(1) |
where is a Gaussian random variable — noise of a channel.
The energy of the constellation is equal to the expectation of , i.e.
Our goal is to minimize the energy of the constellation to reduce errors in symbols. In this paper, we study the case when the random variable is distributed on the QAM constellation.
Example. Initial distribution of is uniform. For instance, the energy of QAM-16 is equal to 10 since
Now if we change the distribution of in such a way that
-
•
four points with coordinates have probability 0.125
-
•
eight points with coordinates have probability 0.0375
-
•
four points of coordinates have probability 0.05.
In this case, the energy will be equal to 7.6 since
and if we shift the points by multiplying them by the square root of ratio of the energies of the constellations , then the energy again become equal to 10. It follows that the points of constellation are further apart, and the variance is the same. So, the probability of error are less.
3.1 Problem statement
The physical meaning of the problem (2) is to minimize the constellation energy at a fixed constellation entropy. The entropy means the amount of information transmitted by the constellation, and the energy means the power the transmitter has to expend in transmitting the data.
(2) |
There is no analytical expression for the problem (2), and so the constellation points are assumed to have a Maxwell-Boltzmann distribution (3) since it is close to the optimal distribution [11] and maximizes the entropy of the constellation with a constraint on its energy:
(3) |

Parameter — is a scaling of constellation points:
(4) |
where is the uniform distribution, is the optimal distribution, and is the complex coordinates of the points.
4 Coded Modulation Design for QAM-16
According to the labelling procedure, we can notice that the first two bits in the binary representation of the constellation points are responsible for symmetry about the coordinate axes, and the last two bits are responsible for absolute value. (Fig. 3). In what follows, we will refer to the first two bits as sign bits, and the last two bits as amplitude bits. Thus, amplitude bit zero corresponds to points with coordinates , and amplitude bit corresponds to points with coordinates .

4.1 Constellation Energy Minimisation
For the practical finite block-length codes, it is required to implement the Enumerative Sphere Shaping (ESS) method [7].
The energy minimization process is fairly straightforward. We take the constellation points with the smallest absolute value with a higher probability, and the points with the largest absolute value with a lower probability. Thus, we are more interested in constellation points that have more zeros than ones at the amplitude bit positions in the binary representation, and then it is sufficient to maximize the probability of zero at the amplitude bit positions. We also assume that the sign bits are uniformly distributed, i.e. the probability of zero and one of the first two bits in the binary representation of each constellation point is equal to .
As noted above, amplitude bit one corresponds to more distant points from the origin, and amplitude bit zero corresponds to closer points. Thus, we can assume that the energy of a sequence of amplitude bits consisting of ones and zeros, is equal to
It can be seen that the nearest points to the origin have the lowest energy.
For a given number of input amplitude bits and block length , the most efficient way to change probabilities is to map all possible realisations to the sequences of amplitude bits with minimal energy. After that, we can calculate the probability of one and probability of zero in a set of blocks of length .
Now if we know the distribution of the amplitude bits, then we can find the probability of the constellation points. Each constellation point contains two sign bits and two amplitude bits, so the probability of a point is equal to
(5) |
where is the number of zero amplitude bits in the bit representation of the constellation point.
After we have changed the distribution of constellation points, we can calculate the scaling parameter as the ratio of the initial energy to the received energy:
(6) |
where is a new random variable with distribution 5. Finally, we shift points of the constellation by multiplying them by the parameter , thereby reducing the probability of error.
4.2 Amplitude Shaper and Sign Delay

In this subsection we describe the model provided in Fig. 4. Initially, the input is informational bits with a uniform distribution. These bits are divided into two groups, one of which will be the amplitude bits, and the other group will be part of the sign bits. Amplitude bits are transmitted through the shaper block, which works according to the algorithm described above. The shaper output is a block of a different length, in which the amplitude bits are already distributed according to the algorithm. We will denote the number of bits in the first group by , the number of bits in the second group by and the number of bits at the shaper output by .
After that, bits and amplitudes bits are concatenated and encoded using the LDPC procedure. The LDPC procedure, in turn, generates additional check bits, which are also considered to be uniformly distributed. We will denote the number of bits at the encoder output as . Note that the bits are sign bits, which have a uniform distribution, while the amplitude bits are distributed according to the algorithm. The number of sign and error correction bits is equal to the number of shaper output bits, i.e. .
For this procedure, coderate is fixed, while shaper input size and shaper output vary. The values of and can be calculated using the code rate formulas.
Extra bits are now shared between Shaper with rate and Encoder with rate , afterall .
The problem is to find the optimal proportion between and .
4.3 Example of generated probabilities using ESS
In Tabs. 1, 2 examples of generated probabilities for QAM-16 and QAM-64 (Fig. 1) using the ESS method [7] are given. Note that for QAM-64 and above the probabilities of zeros and ones depend on each other, so joint probabilities need to be determined.
40 | 80 | 120 | 160 | 200 | 240 | 256 | |
---|---|---|---|---|---|---|---|
0.97 | 0.94 | 0.90 | 0.84 | 0.76 | 0.64 | 0.5 | |
0.03 | 0.06 | 0.10 | 0.16 | 0.24 | 0.36 | 0.5 |
300 | 400 | 500 | 600 | 700 | 800 | 900 | 1000 | 1024 | |
---|---|---|---|---|---|---|---|---|---|
0.87 | 0.80 | 0.73 | 0.66 | 0.59 | 0.52 | 0.44 | 0.32 | 0.25 | |
0.13 | 0.18 | 0.23 | 0.27 | 0.30 | 0.32 | 0.31 | 0.28 | 0.25 | |
0.00 | 0.01 | 0.03 | 0.06 | 0.09 | 0.13 | 0.18 | 0.23 | 0.25 | |
0.00 | 0.00 | 0.00 | 0.01 | 0.01 | 0.03 | 0.07 | 0.16 | 0.25 |
4.4 Probability Shaping Mapping
For mapping purposes, we form a special PS matrix (Fig. 5) with uniform sign bits and non-uniform amplitude bits, following the data flow scheme (Fig. 4). We generate sign bits with equal probability of zeros and ones , and amplitude bits with unequal probability of zeros and ones: such that . The probability of amplitude bits can be determined by the proper values of and using the ESS method [7].
Finally, after the PS matrix is constructed, the mapping to the QAM is performed. With mapping procedure, bits are converted to the constellation points (or symbols) using the mapping table, which gives a specific coordinate on the complex plane for each unique sequence of bits. Notice that, given bit probabilities, there is a one-to-one correspondence to symbol probabilities.
After mapping procedure is done the symbols go through the MIMO channel, demodulation and decoding, probability deshaping and BLER calculation, which are described in Sec. 2 and Fig. 6. The demodulation and deshaping methods are the same procedures described earlier and are performed in reverse order. The decoding procedure is a complex process, which uses loopy belief propagation [14] to iteratively recover the correct bits (LLRs).

4.5 Arrangement of Finite Code Block Shapes
To consistent all the shapes , and , and form the PS matrix (Fig. 5) we solve the system of integer equations (7) finding LCM. Hereafter, the values of and define the multiplicative constants balancing these equations. The values of and define the number of amplitude and sign bits in the code block. It is implicitly assumed that everywhere in Figs. (4) (5) the values are , , , :
(7) |
where the value of defines the constellation system, i.e. — QAM-16, — QAM-64, — QAM-256 and so on.
5 Numerical Experiments
In the experiments, Coded BLER is the average error of transmitted block of bits before the LDPC encoder and after the decoding in Fig. 6, which takes into account the realistic coding-encoding procedure.

5.1 Energy per Bit and Noise Ratio
The energy per bit to noise ratio is a normalized SNR measure, also known as SNR per bit. The measure can be used to express the relationship between signal power and noise power.
The energy per bit measure is the energy we use to transmit one bit of information with the total power and the LDPC coderate :
The noise measure is the noise variance per real and imaginary parts:
Thus, can be expressed in terms of SNR:
In decibel, is
(8) |
We use the value of in the Monte Carlo experiments. For a given value of with the power and coderate the variable noise power disturbs the symbols transmitted over the channel (1).
5.2 Realistic Simulations using Sionna
This study considers OFDM MIMO with a base station and a user equipped with multiple cross-polarised antennas. We provide simulations in Sionna [8] on the OFDM channel using 5G LDPC codes. The architecture of the system consists of LDPC, Bit Interleaver, Resource Grid Mapper, LS Channel Estimator, Nearest Neighbor Demapper, LMMSE Equalizer [13], OFDM Modulator and presented in Fig. 6. Optimization variables are constellation type, a bit order, coderate, BLER, SNR, code block sizes and 5G model (LOS D, NLOS A).
The system uses soft estimates of LLRs for the decoder. Channel model is chosen to be OFDM 5G 2.6 GHz with delay spread of 40ns. The block size is 1536 with number of Monte-Carlo trials in the simulations and so in total bits were processed for each point of . For all simulation, 20 iterations of LDPC have been used. The code source is random binary tensors. The system use 3GPP wireless both Line of Sight (LOS) and Non Line of Sight (NLOS) channel models D and A. The model is simulated in real time domain considering inter-symbol (IS) and inter-carrier (IC) interferences. In Tab. 3 we provide simulation parameters for Sionna.
In Figs. 7 and 8, we provide an experiment for both QAM-16 LOS Model D and NLOS Model A probability shaped (PS) constellations. We present experiments Coded BLER (see Fig. 6) with Gaussian transmit signal with QAM16 Baseline and amplitude PS with different shaping parameters, where coderate is , block size is and parameters of PS are and .
There is a local optimum for PS QAM-16 in both LOS and NLOS models. It is noteworthy that the optimal parameter is the same for the different LOS and NLOS models, which tells us that the chosen parameters are stable. Note that with wrong parameter settings, e.g. a strong shaping factor , the quality of the PS method is worse than that of baseline QAM.
In Fig. 9 present experiments with AWGN channel and LDPC, which show a higher gain than for the OFDM channel model.
In Tab. 4, we provide gains in dB of the proposed PS method at 10% BLER. The PS method achieves 0.56 dB gain in Model A NLOS Uplink compared to the baseline. From the massive experiments, we conclude that the proposed PS method constellations superior the baseline QAM for real 5G Wireless System using FEC LDPC.
The model codes and related experiments can be found in the repository111https://github.com/eugenbobrov/On-Probabilistic-QAM-Shaping-for-5G-MIMO-Wireless-Channel-with-Realistic-LDPC-Codes.
Carrier frequency | 2.6e9 |
---|---|
Delay spread | 40e-9 |
Cyclic prefix length | 6 |
Num guard carriers | [5, 6] |
FFT size | 44 |
Num user terminal antennas | 2 |
Num base station antennas | 2 |
Num OFDM symbols | 14 |
Num LDPC iterations | 20 |



Model D LOS | Model A NLOS | |
---|---|---|
Uplink | 0.52 dB | 0.56 dB |
Downlink | 0.5 dB | 0.5 dB |
6 Conclusions and Suggested Future Work
In this paper, for a MIMO OFDM wireless channel with realistic LDPC code at a given code rate, we study the PS scheme of Enumerative Sphere Shaping (ESS) known from the literature. We find local optimal parameters for the ESS method that minimise the BLER and provide a gain of up to 0.6 dB over the QAM-16 baseline through numerical experiments on the state-of-the-art Sionna simulation platform, modeling physical communication system level. Since there are almost no published works on PS that consider such realistic and contemporary scenarios, while only considering theoretical distributions, this study could be of scientific interest. In the future, a detailed study of BLER performance of a combination of PS and GS methods is possible, which could be very promising in communication applications.
Acknowledgements
The authors are grateful to Sergey Loktev, Dmitry Minenkov, Dmitry Shmelkin, Sviatoslav Panchenko, Alexandr Khodunin and Ivan Sobolev.
References
- [1] Bobrov, E., Kropotov, D., Lu, H., Zaev, D.: Massive MIMO adaptive modulation and coding using online deep learning algorithm. IEEE Communications Letters 26(4), 818–822 (2021)
- [2] Böcherer, G.: Achievable rates for probabilistic shaping. arXiv preprint arXiv:1707.01134 (2017)
- [3] Buchali, F., Steiner, F., Böcherer, G., Schmalen, L., Schulte, P., Idler, W.: Rate adaptation and reach increase by probabilistically shaped 64-qam: An experimental demonstration. Journal of Lightwave Technology 34(7), 1599–1609 (2016)
- [4] Forney, G., Gallager, R., Lang, G., Longstaff, F., Qureshi, S.: Efficient modulation for band-limited channels. IEEE Journal on Selected Areas in Communications 2(5), 632–647 (1984)
- [5] Gallager, R.G.: Information theory and reliable communication, vol. 588. Springer (1968)
- [6] Gültekin, Y.C., Alvarado, A., Willems, F.M.: Achievable information rates for probabilistic amplitude shaping: An alternative approach via random sign-coding arguments. Entropy 22(7), 762 (2020)
- [7] Gültekin, Y.C., Fehenberger, T., Alvarado, A., Willems, F.M.: Probabilistic shaping for finite blocklengths: Distribution matching and sphere shaping. Entropy 22(5), 581 (2020)
- [8] Hoydis, J., Cammerer, S., Aoudia, F.A., Vem, A., Binder, N., Marcus, G., Keller, A.: Sionna: An open-source library for next-generation physical layer research. arXiv preprint arXiv:2203.11854 (2022)
- [9] Idler, W., Buchali, F., Schmalen, L., Lach, E., Braun, R.P., Böcherer, G., Schulte, P., Steiner, F.: Field trial of a 1 tb/s super-channel network using probabilistically shaped constellations. Journal of Lightwave Technology 35(8), 1399–1406 (2017)
- [10] İşcan, O., Böhnke, R., Xu, W.: Probabilistic shaping using 5G new radio polar codes. IEEE Access 7, 22579–22587 (2019)
- [11] Kschischang, F.R., Pasupathy, S.: Optimal nonuniform signaling for Gaussian channels. IEEE Transactions on Information Theory 39(3), 913–929 (1993)
- [12] Loghin, N.S., Zöllner, J., Mouhouche, B., Ansorregui, D., Kim, J., Park, S.I.: Non-uniform constellations for ATSC 3.0. IEEE Transactions on Broadcasting 62(1), 197–203 (2016)
- [13] Mineev, D., Bobrov, E., Kuznetsov, V.: On the interference cancellation by reduced channel zero forcing class of precodings in massive mimo systems. Telecommunication Systems pp. 1–12 (2023)
- [14] Murphy, K., Weiss, Y., Jordan, M.I.: Loopy belief propagation for approximate inference: An empirical study. arXiv preprint arXiv:1301.6725 (2013)
- [15] Neskorniuk, V., Carnio, A., Marsella, D., Turitsyn, S.K., Prilepsky, J.E., Aref, V.: Model-based deep learning of joint probabilistic and geometric shaping for optical communication. In: CLEO: Science and Innovations. pp. SW4E–5. Optica Publishing Group (2022)
- [16] Sillekens, E., Semrau, D., Liga, G., Shevchenko, N.A., Li, Z., Alvarado, A., Bayvel, P., Killey, R.I., Lavery, D.: A simple nonlinearity-tailored probabilistic shaping distribution for square QAM. In: 2018 Optical Fiber Communications Conference and Exposition (OFC). pp. 1–3. IEEE (2018)
Abbreviations
5G | Fifth Generation |
AWGN | Additive White Gaussian Noise |
BICM | Bit-interleaved Coded Modulation |
BLER | Block Error Rate |
CM | Coded Modulation |
ESS | Enumerative Sphere Shaping |
FEC | Forward Error Correction |
GS | Geometric Shaping |
IC | Inter-Carrier |
IS | Inter-Symbol |
LCM | Least Common Multiple |
LDPC | Low-Density Parity-Check Code |
LLR | Log-Likelihood Ratio |
LMMSE | Linear Minimum Mean Squared Error |
LOS | Line-of-Sight |
LS | Least Squares |
LTE | Long-Term Evolution |
MCS | Modulation and Coding Scheme |
MIMO | Multiple-input multiple-output |
NLOS | Non-Line-of-Sight |
NUC | Non-uniform Constellations |
OFDM | Orthogonal Frequency-Division Multiplexing |
PS | Probability Shaping |
PSCM | Probability Shaped Coded Modulation |
QAM | Quadrature Amplitude Modulation |
SNR | Signal-to-Noise Ratio |