Rate-Distortion Optimized Skip Coding of Region Adaptive Hierarchical Transform Coefficients for MPEG G-PCC

When conducting RAHT from top to bottom, the number of transform blocks gradually increases with the depth of layers, leading to an increasing number of AC coefficients that need to be encoded. Taking the point cloud “dancer_vox11_00000001” as an example, Table I summarizes the total number of AC coefficients that need to be encoded for each layer. Herein, layer 0 represents the root node. From the table, it is evident that, for all bitrate configurations (r01, r02, r03, r04, r05, r06, from low bitrate to high bitrate), the number of coefficients in deep layers occupy the majority of the total coefficients. Additionally, as the layers deepen, a large transform block is progressively divided into several small blocks by the octree structure, and the attributes in a large block are allocated to the small blocks, resulting in a gradual decrease of attributes within the small blocks. This, in turn, leads to a decrease of AC coefficients after RAHT of the small blocks. After the prediction and quantization, in the last few layers near the leaf nodes, the proportion of zero residuals exceeds 95%. Fig. 4 illustrates the variations of zero values in each layer of the dense point cloud “dancer_vox11_00000001” and the sparse point cloud “Arco_Valentino_Dense_vox12”. We can see that both dense and sparse point clouds exhibit the aforementioned trend. Particularly, for bitrates, i.e., r01 and r02, the proportion of zero values in the last two layers of Luma components and the last four layers of Chroma components exceeds 99%. This suggests that the proportion of zero values within a layer at low bitrates is larger than that at high bitrates, and the proportion of zero values in the last few layers of Chroma components is also higher than that in Luma component. Therefore, encoding a significant amount of residuals of so many zero values in the last few layers leads to unnecessary waste of bitrate.

To reduce the bit rate waste of the last few layers in RAHT, we propose an adaptive skip coding method for the transformed coefficients by using RDO. As the proportion of zero residuals in the last several layers is large, we first estimate the attribute bitrates of skipping all AC residuals of the last one, the last two, the last three, and the last four layers, and calculate the corresponding distortion, thus calculate the RD cost of the four cases, respectively. Then, we estimate the attribute bitrate for encoding all the AC residuals and also calculate the corresponding distortion to obtain the corresponding RD cost. Subsequently, the RD costs of the five cases are compared, and the case which gives the minimum RD cost is selected and indicated in the bit stream for the decoder. Besides, the skip coding method is independently applied to Luma, Cb, and Cr color components.

From Parseval’s theorem [37], the distortion in the transform domain is equal to that in the attribute domain [38]. Therefore, to save computational complexity, distortion is calculated directly in the transform domain to avoid the additional computational complexity induced by inverse RAHT transform. For the reconstructed point cloud obtained by encoding all residuals, the distortion can be calculated as

where $AC_{org}^{i}$ represents the i-th AC coefficient, $AC_{recon}^{i}$ represents the reconstructed i-th AC coefficient, $AC_{pre}^{i}$ is the i-th predicted AC coefficient, $AC_{res_{q}}^{i}$ is the i-th quantized AC residual, $Q$ denotes the quantization step size, and $M_{t}$ is the total number of AC coefficients. When all residuals of the last $k$, $k\in\left\{1,2,3,4\right\}$, layers are skipped, all residuals of the AC coefficients in the last k layers are ignored during encoding, they are reconstructed as zero in the decoder. Therefore, these reconstructed AC coefficients are replaced by their predicted values, and their distortion can be calculated by ${\textstyle\sum_{i=1}^{M_{k}}}(AC_{org}^{i}-AC_{pre}^{i})^{2}$. Therefore, the distortion in this case, denoted as $D_{k}$ can be represented as

where $M_{k}$ represents the number of AC coefficients in the last k layers of RAHT.

Due to the fact that most of the residuals after RAHT are zero, zero-run length coding [39] is employed in G-PCC to encode non-zero residuals and run lengths of consecutive zeros. In practical implementation, three flags, i.e., utilizing_sign, iszero, and isone are used for non-zero residuals to distinguish negative values, 1, or 2. The remaining values are encoded using an exponential golomb encoder [40]. For run length of consecutive zeros, truncated unary coding combined with exponential golomb encoder is used. The bitrate can be estimated by the cumulative number of bits required for encoding. For truncated unary coding, the bitrate can be estimated by the length of the codeword. For exponential golomb coding, the number of bits can be estimated by the probability of the remaining values, which is updated in real time. The procedure of encoding and bitrate estimation of quantized coefficient’ residuals are illustrated in Fig. 5. In the proposed method, we use $R_{org}$ to denote the attribute bits when all residuals are encoded and $R_{k}$ to represent the attribute bits (including the flags) when all AC residuals of the last $k$, $k\in\left\{1,2,3,4\right\}$ layers are skipped. Therefore, the RD cost can be calculated by

where $D$ represents the distortion ($D_{k}$ resp. $D_{org}$), $R$ represents the bits ($R_{k}$ resp. $R_{org}$), and $\lambda$ is the Lagrange multiplier which follows the basic style of H.266/VVC, i.e.,

where $c$ is a content dependent constant and $QP$ is the quantization parameter.

In our study, we did extensive statistical experiments to obtain $c$. First, we implemented the proposed method onto the latest software platform GeS-TM v4.0 [41] and encoded some typical test point clouds “queen”, “8ivfbv2_redandblack_vox10” in the dynamic point cloud category Cat2-A, “8ivfbv2_longdress_vox10” in the dynamic point cloud category Cat2-B, and “basketball_player_vox11” in the dynamic point cloud category Cat2-C provided by MPEG to evaluate the coding performance with respect to $c$. During the experiment, $c$ ranges from 0.05 to 0.5 with an incremental step size of 0.01. To find the optimal $c$, we use the End-to-End BD‑Attribute Rate (BDBR) [42] to quantitatively represents the attribute bitrate savings at the same distortion when comparing the proposed method with GeS-TM v4.0. A negative BDBR indicates a positive gain compared to G-PCC, whereas a positive BDBR indicates a negative gain. We tested all the bitrates (from the lowest bitrate to the highest bitrate) by following the common test condition (CTC) [43] of Ges-TM, and presented the average performance. By taking all the color components into account, we also define $BDBR_{Total}$ to represent the sum of BDBRs for Luma and Chroma components,

where $BDBR_{Luma}$, $BDBR_{Cb}$ and $BDBR_{Cr}$ represent the BDBR of Luma, Cb, and Cr components, respectively, compared to GeS-TM v4.0. The parameter $a$ equals to 7 because the importance of Luma is roughly 7 times greater than that of Chroma [44]. The statistical results are given in Table II, and the variation of average $BDBR_{Total}$ with respect to $c$ is shown in Fig. 6. Finally, we can conclude from Table II and Fig. 6 that the optimal $c$ can be set as 0.26.

By using the optimal $c$, we can compare the $RDcost_{k}$ of skipping the residuals of the last $k$, $k\in\left\{1,2,3,4\right\}$, layers with the $RDcost_{org}$ of normally encoding all the coefficients by using (6) and (7), and pre-set a flag to indicate the best case for the decoder. If $RDcost_{org}$ is smaller than all $RDcost_{k}$, we set the flag to be $k=0$. In this case, the encoder will encode all RAHT residuals as their original values. If $RDcost_{k}$ is smaller than $RDcost_{org}$, we find the smallest $RDcost_{k}$, $k\in\left\{1,2,3,4\right\}$, and set the flag as $k$ to skip the residuals of the last $k$ layers of RAHT.

The decoder first extracts the flag from the attribute bitstream. If the flag is $k=0$, the residuals are decoded normally. If the flag is not zero, the decoder will deem all the residuals of the last $k$ layers of RAHT as zero. Subsequently, following the original G-PCC decoding process, the decoded residuals are inversely quantized to obtain $AC_{res}$. If intra-frame or inter-frame prediction was performed at the encoder side, the obtained $AC_{res}$ is added to the predicted coefficients $AC_{pre}$ to obtain the reconstructed coefficients $AC_{recon}$. Finally, inverse RAHT is applied to obtain the reconstructed attributes. The encoding and decoding framework for the proposed method can be illustrated in Fig. 7. Moreover, as there are three color components, we perform the proposed method for the three color components independently and use three flags, namely flag_luma, flag_cb, and flag_cr, to indicate how to decode the residuals, respectively.

In the field of 3D point cloud compression standardization, Bits Per Output Point (BPOP) is usually adopted to measure the average bitrate required for each point. For the same reconstruction quality, a lower BPOP indicates higher encoding efficiency. The reconstruction quality of a point cloud is usually measured by Peak Signal-to-Noise Ratio (PSNR), where a higher PSNR indicates higher reconstruction quality. In the CTC, each point cloud is first encoded at 6 different quantization parameters, resulting in 6 bitrates (i.e., r01, r02, r03, r04, r05, and r06) and PSNRs. Then, Bjøntegaard rate (BD-rate, in percentage) can be calculated to quantitatively evaluate the RD performance [42]. A negative BD-rate denotes a decrease in BPOP at the same reconstruction quality, indicating better encoding efficiency.

Table III shows the performance of dynamic point clouds of the proposed method compared to GeS-TMv4.0 under the C1 and C2 conditions, where End-to-End BD-AttrRate (%) denotes the change in attribute BPOP at the same reconstruction quality. Table IV gives the results when attribute inter-frame prediction is disabled and only intra-frame prediction is used under the C1 and C2 conditions.

From Table III, we can see that the RD performance gain of C2 condition is larger than that of C1 condition. This is because the number of points under the lossless geometry compression of C1 condition leads to larger total coding bits, and therefore, the ratio of saved bits by the proposed method is not so significant. Besides, in C1 condition, the values of AC residuals in the last few RAHT layers are also large, leading to more reconstructed distortion and PSNR loss if they are skipped. Specifically, at C1 condition, the average BD-rates for Luma, Cb, and Cr components achieves -0.46%, -1.49%, and -0.68%, respectively, while at C2 condition, the corresponding average BD-rates are -3.50%, -5.56%, and -4.18%, respectively. It is noteworthy that the average BD-rates of the three color components achieve -14.35%, -20.68%, and -18.15%, respectively, for “queen”. This is because “queen” is a computer-generated imagery (CGI) sequence, which has stronger inter-frame correlation compared to regular sequences, making the color attributes much easier to be predicted within the RAHT. Therefore, when inter-frame prediction is enabled, a larger proportion of the quantized AC coefficient’ residuals are zero values. Accordingly, using the proposed method achieves for significant bitrate savings. From Table IV, we can also get the conclusion that the RD performance gain of C2 condition is larger than that of C1 condition.

By comparing Table III with Table IV, we can observe that the proposed method achieves better performance when inter-frame prediction is enabled. This is because, when we skipped the residuals in the last few layers, $AC_{pre}$ is directly used to represent $AC_{recon}$ on the decoder side. The accuracy of inter-prediction is typically better than intra-prediction, resulting in higher PSNR and RD performance.

To analyze the reconstruction quality of point clouds at different bitrates, we conducted an analysis on the dynamic point cloud “queen”, “8ivfbv2_soldier_vox10”, “8ivfbv2_redandblack_vox10” in Cat2-A category, “8ivfbv2_longdress_vox10” in Cat2-B category, and “dancer_player_vox11” in Cat2-C category under the “Octree-RAHT-inter-C2” condition. Fig. 8 depicts the RD curves of the proposed method and G-PCC, in which the weighted average (7:1:1) of PSNR for Luma, Cb, and Cr components is used. We can see that the proposed method exhibits better performance at lower bitrates. This is because lower bitrates entail a larger quantization step, leading to a substantial number of zero values in the last few layers of AC residuals in RAHT. Therefore, significant bit rate saving and minimal PSNR loss can be achieved by the proposed method. At high bit rates, as the significant distortion leads to a high RD cost, the proposed method is hard to be selected by RDO.

Due to the difficulty of visualizing dynamic point cloud sequences, we randomly selected certain snapshots from the dynamic point cloud sequences. By comparing the subjective quality of these snapshots, we aim to approximately evaluate the performance of our method. Fig. 9 shows some snapshots of the point clouds reconstructed by the proposed method and GesTMv4.0. Specifically, we selected the point cloud “queen_frame_1.ply”, “redandblack_frame_8.ply”, “longdress_vox10_frame_6.ply” and “dancer_vox11_frame_2.ply” for illustration, as shown in Fig. 9 (a), (b), (c), and (d). We can see that the proposed method can achieve similar subject quality by consuming much smaller bit rates. For instance, under the C2, r02 condition, the point cloud “queen_frame_1.ply” was reconstructed by GPCC (GeS-TMv4.0) with a color BPOP of 27.6$\times 10^{3}$, whereas our method only required 4.7$\times 10^{3}$ color BPOP. We saved 82% of the bitrate while achieving almost the same subjective visual quality. Besides, the proposed method can also eliminates block artifacts effectively and thus producing smoother visual appearance compared with GeS-TMv4.0. This is because the residuals in the last few layers are skipped and the AC prediction is directly used as the reconstruction, resulting in more smooth reconstruction quality.

For multi-frame dynamic point clouds, the average encoding and decoding complexity $C$ is formulated according to:

where $T_{end/dec}^{pro}$ and $T_{end/dec}^{anc}$ represent the average encoding/decoding time of the proposed method and anchor (i.e., G-PCC), respectively. Table V shows the compares time complexity of the proposed method with GeS-TMv4.0 under the C1 and C2 conditions, respectively. Compared to the GeS-TMv4.0, there is a certain increase in encoding time for the proposed method, however, the decoding time remains more or less the same. We can also see that the encoding time increase of the proposed method under C1 condition is greater than that under C2 condition. This is because, during RDO, distortion and bitrate are calculated block by block. As the number of blocks is quite large in lossless geometry conditions (C1 condition), leading to higher time complexity. Additionally, the increase of encoding time under inter-frame coding configuration is less than that under intra-frame coding configuration, as inter-frame prediction results in smaller AC coefficients’ residuals (i.e., more zero residuals), making faster distortion and bitrate estimation. Furthermore, when inter-frame prediction is enabled, the proposed method is used more frequently, reducing more time complexity of attribute reconstruction at the encoder side. We also observe a slight decrease in decoding complexity for certain point clouds. This is because the proposed method does not need to decode the residuals in the last $k$ layers.