TSC-PCAC: Voxel Transformer and Sparse Convolution Based Point Cloud Attribute Compression for 3D Broadcasting

Traditional point cloud compression methods are mainly divided into two categories: octree based and projection based. Octree can efficiently and losslessly represent point clouds, where adjacent nodes exhibit high spatial correlation for point cloud compression. G-PCC [1] employs an octree-based compression method. The attribute compression of G-PCC integrates three key compression methods, which are Region-Adaptive Hierarchical Transform (RAHT) [15], predictive transform, and lifting transform [16]. RAHT is a variant of the Haar wavelet transform, using lower octree levels to predict values at the next level. Predictive transform generates different levels of detail based on distance and performs encoding using the order of detailed levels. Lifting transform is performed on top of predictive transform with an additional updating operation [17, 18]. Pavez et al. [19] utilized a Regionally Adaptive Graph Fourier Transform (RAGFT) to obtain low-pass and high-pass coefficients, and then used the decoded low-pass coefficients to predict the high-pass coefficients. Gao et al. [20] exploited the statistical correlation between color signals to construct the Laplace matrix, which overcame the interdependence between graph transform and entropy coding and improved the coding efficiency. V-PCC is a projection-based compression method. It divides the point cloud sequences into 3D patches and projects them into 2D geometry and attribute videos, which are then smoothed and compressed by the conventional 2D codecs. Zhang et al. [21] proposed a perceptually weighted Rate-Distortion (RD) optimization scheme for V-PCC, where perceptual distortion of point clouds [22] was considered in objectives of attribute and geometry video encoding. These are traditional coding schemes exploiting spatial-temporal correlations, symbol and visual redundancies.

Deep learning and neural networks have achieved great success in solving vision tasks [23, 24, 25]. Learning-based image compression [26, 3, 27, 28, 29, 30] has been hotspot in recent years, which has surpassed the traditional image compression methods, such as JPEG2000 and VVC Intra coding. The framework of learned image compression mainly transforms an image into a compact representation, which then undergoes quantization and entropy encoding for transmission. At the decoder side, the output image is synthesized from the decoded compact representation from entropy decoded bit stream. The entire end-to-end compression framework is optimized using RD loss as the objective function [26]. Ballé et al. [26] applied a hyper-prior module into the Variational AutoEncoder (VAE) compression framework. Minnen et al. [30] proposed a context module exploiting autoregressive and hierarchical priors to improve compression efficiency, which was further extended to model channel-wise context [29]. Tang et al. [27] utilized graph attention mechanism and asymmetric convolution to capture long-distance dependencies. Zou et al. [28] proposed to combine local perceptual attention mechanism with global-related feature learning. Then, a window-based image compression was proposed to exploit spatial redundancies. Liu et al. [3] proposed a mixed transformer-Convolutional Neural Network (CNN) module, which was incorporated to optimize channel context module for learned image compression. However, unlike the regular distributed 2D image, a point cloud is dispersed in 3D space in an unstructured and sparse manner. These 2D image compression methods cannot be directly applied to encode 3D point clouds.

To compress point clouds effectively, a number of learning-based compression methods have been proposed, which are categorized into learned geometry and attribute compression.

Generally, end-to-end learned Point Cloud Geometry Compression (PCGC) can be categorized into point-based methods [4, 5] and voxel-based methods [6, 7, 8]. The point-based methods process point clouds in 3D perspective. Huang et al. [4] developed a hierarchical autoencoder that incorporated multi-scale losses to achieve detailed reconstruction. Gao et al. [5] utilized local graphs for extracting neighboring features and employed attention-based methods for down-sampling points. These point-based methods struggle to efficiently aggregate spatial information. Voxel-based methods require an initial voxelization of point clouds before input to the compression network. Song et al. [31] designed a grouping context structure and a hierarchical attention module for the entropy model of octrees, which supported parallel decoding. Wang et al. [8] extended the Inception-Resnet block [32] to 3D domain, and then incorporated it into the VAE based compression network to address the gradient vanishing problem from going deeper. However, the employed dense convolution required intensive computations and large memory, which were unaffordable in processing large-scale point clouds. Wang et al. [6] introduced sparse convolution to accelerate computations and reduce memory cost. Additionally, a hierarchical reconstruction method was proposed to enhance compression performance. Liu et al. [7] introduced local attention mechanisms in PCGC, which achieved promising coding efficiency and demonstrated the potential of attention mechanisms. Qi et al. [33] proposed a variable rate PCGC framework to realize multiple rates by tuning the features. Akhtar et al. [34] proposed to predict a latent potential representation of the current frame from those of previous frames by using a generalized sparse convolution. Zhou et al. [35] proposed using transformers to capture temporal context information and reduce temporal redundancy. They also used a multi-scale approach to capture the temporal correlations of multi-frame point clouds over both short and long ranges. These methods are proposed to encode the point cloud geometry. However, the point cloud attribute is attached to the geometry, which is significantly different from the geometry.

To compress the point cloud attribute effectively, a number of Point Cloud Attribute Compression (PCAC) methods were developed, which can be categorized as lossless [9, 10] and lossy compression [11, 12, 13, 14, 36]. In lossless compression, Nguyen et al. [10] constructed the context information between the color channels to reduce redundancy among RGB components. Wang et al. [9] generated multi-scale attribute tensor and encoded the current scale attribute conditioned on lower scale attribute. In addition, it further reduced spatial and channel redundancy by grouping spatial and color channels. However, lossless compression can hardly achieve high compression ratio, which motivates lossy compression. In lossy compression, Isik et al. [36] modeled point cloud attribute as a volumetric function and then compressed the parameters of the volumetric function to achieve attribute compression. However, fitting the function leads to high coding time complexity. Fang et al. [13] employed the RAHT for initial encoding and designed a learning-based approach to estimate entropy. Pinheiro et al. [14] proposed a Normalized Flow Point Cloud Attribute Compression (NF-PCAC) method. It incorporated strictly reversible modules to improve the reversibility of data reconstruction, which extended the upper bound of coding performance. Similarly, Lin et al. [37] utilized the normalized flow paradigm by proposing the Augmented Normalizing Flow (ANF) model, in which additional adjustment variables were introduced. Zhang et al. [38] used two-stage coding, the point cloud was roughly coded using G-PCC in the first stage and it was augmented with a network in the second stage.

There are two more advanced PCAC methods based on the VAE framework, which are the Deep PCAC (Deep-PCAC) [11] and the Sparse convolution-based PCAC (Sparse-PCAC) [12]. In Deep-PCAC, a PointNet++ [39] based feature extraction was proposed to provide a larger receptive field for aggregating neighboring features. However, since the Deep-PCAC processes irregular points directly, it cannot utilize convolution operators on regular voxels to extract local features[32]. The latter Sparse-PCAC processed voxelized point clouds, which used convolution to aggregate neighboring features and sparse convolution to reduce the computation and memory requirements [40]. However, Sparse-PCAC used stacked convolutional layers with fixed kernel weights, degrading the model’s ability of focusing highly relevant voxels.

Point cloud is sparsely distributed in 3D space, which suits for sparse processing in neighboring feature extraction. In addition, attention mechanisms that can capture correlations between voxels are also applicable to the point cloud compression. Therefore, the properties of sparse convolution and local attention are analyzed for the attribute coding.

Given a point cloud $\mathbf{x}_{pc}$, a learned point cloud compression framework to encode and decode $\mathbf{x}_{pc}$ can be expressed as

where $E_{pc}(\cdot)$ and $D_{pc}(\cdot)$ represent the point cloud encoder and decoder, respectively. $AE(\cdot)$ and $AD(\cdot)$ respectively represent arithmetic encoder and decoder. $\mathbf{x}_{pc}$ and $\tilde{\mathbf{x}}_{pc}$ represent the source and the reconstructed point clouds, respectively. $\hat{\mathbf{y}}_{pc}$ and $B(\hat{\mathbf{y}}_{pc})$ represent the compact latent representations after quantization and the bitstream containing the $\hat{\mathbf{y}}_{pc}$ information. $Q(\cdot)$ is a quantization operation.

A point cloud $\mathbf{x}_{pc}$ consists of geometry $\mathbf{x}_{G}$ and attribute $\mathbf{x}_{A}$, i.e., $\mathbf{x}_{pc}=[\mathbf{x}_{A},\mathbf{x}_{G}]$. Since attribute attaches to geometry and is compressed after geometry [2, 1]. Eq.(2) is rewritten as

where $\tilde{\mathbf{x}}_{G}$ and $\tilde{\mathbf{x}}_{A}$ represent the reconstructed point cloud geometry and attribute. $\hat{\mathbf{y}}_{G}$ and $\hat{\mathbf{y}}_{A}$ represent the quantified latent representations of geometry and attribute. $E_{G}(\cdot)$ and $D_{G}(\cdot)$ represent point cloud geometry encoder and decoder. $E_{A}(\cdot)$ and $D_{A}(\cdot)$ represent the point cloud attribute encoder and decoder. $B(\hat{\mathbf{y}}_{G})$ and $B(\hat{\mathbf{y}}_{A})$ represent the bitstream containing the $\hat{\mathbf{y}}_{G}$ and $\hat{\mathbf{y}}_{A}$ information. Attribute $\mathbf{x}_{A}$ is on the basis of geometry ${\mathbf{x}}_{G}$, thus geometry is compressed before the attribute, and then the attribute is compressed with the reconstructed geometry information $\tilde{\mathbf{x}}_{G}$ as an input.

In this work, suppose that point cloud geometry has already been compressed, we focus on optimizing the PCAC. Consequently, our goal is to optimize $E_{A}(\cdot)$ and $D_{A}(\cdot)$, as well as the entropy model required in arithmetic codec while encoding and decoding $\hat{\mathbf{y}}_{A}$. We propose an efficient TSC-PCAC framework based on the VAE, as shown in Fig. 3, in which TSCM and a TSCM based context module are proposed to improve the point cloud coding. The TSC-PCAC combines the inductive bias capabilities and memory-saving advantages of sparse convolution, and also leverages the ability of transformers in capturing correlations between voxels. The TSC-PCAC consists of a pair of learning-based encoder/decoder and a pair of entropy model, which is

where ${\mathbf{y}}_{A}$ and $\hat{\mathbf{y}}_{A}$ are the original and quantized latent representations of attribute, respectively. $E_{{TSCM},A}(\cdot)$ and $D_{{TSCM},A}(\cdot)$ are TSCM-optimized attribute encoder and decoder with parameters $\boldsymbol{\phi}$ and $\boldsymbol{\theta}$, respectively. $Entropy_{A}(\cdot)$ represents the entropy model for $\hat{\mathbf{y}}_{A}$. The mean $\boldsymbol{\mu}$ and scale $\boldsymbol{\sigma}$ from the entropy model are used for the arithmetic codec $AE(\cdot)$ and $AD(\cdot)$. Specifically, the entropy model consists of a hyper-prior encoder-decoder and context module. In estimating the probability distribution of $\hat{\mathbf{y}}_{A}$, the initial mean and scale are firstly achieved by a hyper-prior encoder-decoder, and then refined by a context module. The TSCM based channel context model and entropy coding can be represented as

where $B(\hat{\mathbf{z}}_{A})$ represent the bitstream containing the quantized hyperprior representations $\hat{\mathbf{z}}_{A}$ information. $HE_{{TSCM},A}(\cdot)$ and $HD_{{TSCM},A}(\cdot)$ represent the hyperprior encoder and decoder with parameters $\boldsymbol{\phi}_{h}$ and $\boldsymbol{\theta}_{h}$, respectively. $\boldsymbol{\mu^{\prime}}$ and $\boldsymbol{\sigma^{\prime}}$ represent initial mean and scale of $\hat{\mathbf{y}}$. $C_{{TSCM},A}(\cdot)$ denotes the TSCM based channel context module with parameters $\boldsymbol{\theta}_{c}$, which establishes relationships between channels to provide contextual information for higher compression efficiency.

To enhance the network’s capability of allocating more importance to voxels with higher relevance and obtaining a compact representation, TSCM blocks are placed in the encoder $E_{{TSCM},A}(\cdot)$ and decoder $D_{{TSCM},A}(\cdot)$. Specifically, due to memory constraints, only two TSCM blocks are used in the encoder and decoder. Similarly, one TSCM block is employed in the hyper-prior encoder and decoder. Since the attribute is attached to geometry, the geometry information is utilized in the entire process of attribute coding, shown as gray arrows in Fig. 3. For simplicity, geometry variables are not explicitly indicated in subsequent diagrams.

Transformer models the correlations dependencies between voxels through the self-attention mechanism, which has the potential to enable the network to focus on voxels of higher relevance. However, it may reduce the generalization ability. Convolution, on the other hand, has a strong inductive bias and can achieve excellent generalization ability. Combining convolution and transformer can gain their advantages. However, when using the transformer to process large-scale color point clouds that contain millions of points, global attention is able to capture long-range dependencies at the cost of the unaffordable memory, while local attention reduces the memory cost at the cost of reducing the receptive field. To take the advantages of these methods, we propose a two-stage TSCM, which consists of local attention based transformer [41], residual block, and voxel-based global block to increase the receptive field.

Fig. 4 shows the structure of the proposed TSCM. In the first stage, residual block and local attention based transformer are utilized to extract local features. To take advantage of these features, we use a dual-branch structure to extract features separately and fused by concatenation. Specifically, an input $\mathbf{F}_{{in}}\in\mathbb{R}^{N\times 128}$ performs sparse convolution with a kernel size of 1 and splits along the channel dimension, resulting in $\mathbf{F}_{{att}}\in\mathbb{R}^{N\times 64}$ and $\mathbf{F}_{{cnn}1}\in\mathbb{R}^{N\times 64}$. They are used as inputs for the local attention-based transformer and residual block, respectively. The output is then concatenated to restore the feature channel to the original dimension, i.e., 128. This fusion combines both two types of features to form local features $\mathbf{F}_{{out}1}\in\mathbb{R}^{N\times 128}$, which captures interdependencies among voxels and has effective inductive bias. These features are the output of the first stage and the input of the second stage.

In the second stage, to alleviate the computation/memory cost and achieve the global features, the residual block and the voxel-based global block are employed. The global block in [42] uses linear layers and global pooling to extract and sharpen global features. To further capture spatial neighboring redundancy, voxel-based global block is proposed by replacing the linear layer in global block with convolutional layers. Specifically, $\mathbf{F}_{{glo}}\in\mathbb{R}^{N\times 64}$ and $\mathbf{F}_{{cnn}2}\in\mathbb{R}^{N\times 64}$ are input to the voxel-based global block and residual block, respectively. The voxel-based global block is shown in Fig. 4. It utilizes global pooling to extract global spatial features $\mathbf{F}_{{OS}}\in\mathbb{R}^{1\times 64}$ and global channel features $\mathbf{F}_{{OC}}\in\mathbb{R}^{N\times 1}$. $\mathbf{F}_{{OC}}$ and $\mathbf{F}_{{OS}}$ undergo matrix multiplication to obtain global spatial-channel features $\mathbf{F}_{{CS}}\in\mathbb{R}^{N\times 64}$. Subsequently, the global spatial-channel features are subtracted from the input features of voxel-based global block $\mathbf{F}_{{glo}}$ to obtain sharpened global spatial-channel features. The outputs of the voxel-based global block and residual block are concatenated to restore the feature channel to the original dimension. This fusion combines both two types of features to form global features with a large receptive field and inductive bias capabilities.

In TSCM, the integration of these two stages effectively captures both global and local inter-point relevance for reducing data redundancy. We use both the residual block and the transformer with local self-attention to extract local features and correlations. Meanwhile, the global features are obtained through channel and spatial pooling within voxel-based global blocks. Finally, the features of TSCM are output as $\mathbf{F}_{\text{out}}\in\mathbb{R}^{N\times 128}$.

In the Sparse-PCAC, the context module is a per-voxel context network, which processes voxels sequentially. While dealing with large-scale point clouds, this sequential processing leads to extremely high encoding and decoding delays. We propose a TSCM based channel context module to improve the network prediction probability distribution by constructing contextual information over quantized latent representations. Features are processed sequentially along the channel dimension instead of voxels, which reduces the sequential number.

The architecture of the proposed TSCM based channel context module is illustrated in Fig. 5, where the hyper-prior module is detailed in Fig. 3. $\hat{\mathbf{y}}_{i}$ represents the latent representation of the $i$-th quantized slice, while $\overline{y}_{i}$ represents the latent representation of the $i$-th slice with the addition of quantization error estimation, where $i\in{\{1,2,3,\ldots,C\}}$. $C$ represents the number of equal divisions in the channel. Besides the mean and scale, we also employ a module to predict quantization error. Specifically, the channel context module processes each slice sequentially by dividing the quantized latent representation $\hat{\mathbf{y}}$ proportionally into slices along the channel. Predicting the mean of each slice depends on the previously decoded slices and the initial mean value. Similar process is applied to predict the scale. Since the obtained probability distribution can be used to decode the current slice, the quantization error can be predicted from the decoded current slice. It is worth noting that the network architectures for predicting the mean, scale, and quantization error are the same. So, we use the term ‘TSCM based Transform’ for these three cases.

The detailed structure of the TSCM based transform is depicted in Fig. 5, where inputs are concatenated and processed through sparse convolutions and TSCM block. Due to the computational complexity of TSCM, we reduce the number of feature channels to 32. The mathematical expressions of obtaining the scale $\boldsymbol{\sigma}_{i}$, mean $\boldsymbol{\mu}_{i}$, and quantization error ${\mathbf{y}}_{e,i}$ are

where $\boldsymbol{\mu}_{i}$ and $\boldsymbol{\sigma}_{i}$ represent the refined mean and scale of $\hat{\mathbf{y}}_{i}$. ${\mathbf{y}}_{e,i}$ is the estimated quantization error. $\overline{\mathbf{y}}_{i}$ denotes the refined latent representation of group $i$, i.e., $\overline{\mathbf{y}}_{i}={\mathbf{y}}_{e,i}+\hat{\mathbf{y}}_{i}$. $\overline{\mathbf{y}}_{<i}$ indicates a set of $\overline{\mathbf{y}}_{j}$ where $j$ is smaller than $i$, $\overline{\mathbf{y}}_{<i}=\left\{\overline{\mathbf{y}}_{1},\overline{\mathbf{y}}_{2},\ldots,\overline{\mathbf{y}}_{i-1}\right\}$. $f_{\text{TSCM\_T }}(\cdot)$ is the ‘TSCM based Transform’ learning parameters of slices, which are $\boldsymbol{\theta}_{\sigma,i}$, $\boldsymbol{\theta}_{\mu,i}$, and $\boldsymbol{\theta}_{y,i}$, respectively. The mean $\boldsymbol{\mu}_{i}$ and scale $\boldsymbol{\sigma}_{i}$ of $i$-th group are utilized for entropy encoding and decoding of the current group’s $\hat{\mathbf{y}}_{i}$. The quantization error $\mathbf{y}_{e,i}$ predicted for each group is then employed to refine $\hat{\mathbf{y}}_{i}$. Based on Eq. 6 and Fig.5, the mean $\boldsymbol{\mu}_{i}$, scale $\boldsymbol{\sigma}_{i}$, and quantization error ${\mathbf{y}}_{e,i}$ can be predicted by the TSCM based transform, which improve the channel context module in entropy coding.

The overall objective of point cloud compression is to minimize the distortion of the reconstructed point cloud at a given bitrate constraint. As a point cloud consists of geometry and attribute, the objective is to minimize

where $d_{A}(\cdot)$ and $d_{G}(\cdot)$ are distortion measurements. $R_{A}(\hat{\mathbf{y}}_{A})$ and $R_{G}(\hat{\mathbf{y}}_{G})$ represent the bitrate required for transmitting the quantized attribute and geometry latent representations, $\hat{\mathbf{y}}_{A}$ and $\hat{\mathbf{y}}_{G}$. Due to the hyper-codec, the total bit rate includes the rate of hyper-prior feature, i.e., $R_{A}(\hat{\mathbf{z}}_{A})+R_{G}(\hat{\mathbf{z}}_{G})$. The trade-off between distortion and bitrate is controlled by $\lambda_{A}$ and $\lambda_{G}$. Furthermore, in this paper, we focus on the PCAC and assume the geometry information is shared and fixed at the encoder and decoder sides, Therefore, the objective for attribute coding is simplified to minimize

In this paper, $d_{A}(\cdot)$ uses Mean Squared Error (MSE) loss in the YUV color space.

Fig. 6 illustrates the coding performance comparison of Deep-PCAC, Sparse-PCAC, G-PCC, NF-PCAC and TSC-PCAC, where the vertical-axis is PSNR and the horizontal-axis is bit rate in terms of $bpp$. It can be observed that the Sparse-PCAC significantly outperforms the Deep-PCAC, but it is worse than the G-PCC in the majority of point clouds. NF-PCAC has higher compression efficiency than Sparse-PCAC. As for the proposed TSC-PCAC, it outperforms the NF-PCAC by a large margin in all point clouds. Our TSC-PCAC achieves the best performance on most test point clouds, such as Soldier and Dancer. However, it ranks second behind G-PCC v23 on a few point clouds such as Thaidancer and Redandblack.

Table I presents the quantitative BD-BR and BD-PSNR comparisons. ‘Average’ is the average value over 9 point clouds, while the ‘Average*’ is the average result over all including point clouds marked with ‘*’. It can be observed that, excluding the point clouds marked with ‘*’, the proposed TSC-PCAC achieves bit-rate savings ranging from 30.12% to 50.14%, and 36.62% on average compared to the Sparse-PCAC. In terms of BD-PSNR, our method achieves BD-PSNR gain ranging from 1.15 dB to 2.35 dB, and 1.66 dB on average, which is significant. While including the seven point clouds marked with ‘*’, the proposed TSC-PCAC achieves bit rate savings ranging from 30.87% to 54.00%, and 38.53% on average as compared with the Sparse-PCAC. Meanwhile, the proposed TSC-PCAC achieves an average of 1.72 dB BD-PSNR gain. Compared to NF-PCAC, TSC-PCAC achieves bitrate savings ranging from 12.85% to 29.61%, and 22.27% on average. In terms of BD-PSNR, it achieves from 0.47 dB to 1.25 dB and 0.85 dB on average. While including point clouds marked with ‘*’, the proposed TSC-PCAC achieves an average of 21.30% bitrate saving and 0.80 dB BD-PSNR gain, which is significantly superior to the NF-PCAC.

As compared to the G-PCC v23, the proposed TSC-PCAC saves bitrate up to 53.07% across different point clouds. However, as shown in Fig. 6 and Table I, the proposed TSC-PCAC increases the BD-BR by 28.01% for the Thaidancer. This is mainly because the geometry and color attribute of Thaidancer are relatively complex, and it has low PSNR value even at higher bit rates. These characteristics do not align with the training set, which primarily consists of point clouds with relatively simple color attributes. Therefore, TSC-PCAC exhibits better compression performance for simpler-colored point clouds like basketball player and dancer. Increasing more colorful point clouds to training set may improve coding the complex sequences. The TSC-PCAC is inferior to G-PCC v23 with bitrate increasing of 1.26% on average. As we include the seven point clouds marked with ‘*’, the proposed TSC-PCAC achieves an average of 11.19% bit rate saving and 0.34 dB BD-PSNR gain, which proves the effectiveness of the proposed TSC-PCAC. The TSC-PCAC achieves coding gains for most point clouds and is capable of learning from the first few frames to enhance coding the subsequent frames.

In addition, visual quality comparisons were performed to validate the effectiveness of TSC-PCAC. Fig. 7 illustrates the visual quality comparison of the compressed point clouds from five different encoding methods across two test sequences. Figs. 7 to 7 display the enlarged images reconstructed by each encoding method for Redandblack. To ensure fair comparisons, examples with similar bpp were selected. The face of Redandblack reconstructed from G-PCC v23, showed in Fig. 7, exhibits noticeable block artifacts when compared to the original. The bitrate and PSNR for the reconstruction are 0.102 $bbp$ and 33.10 dB. In contrast, bitrate and PSNR of the point clouds reconstructed by the Sparse-PCAC shown in Fig. 7 and NF-PCAC shown in Fig. 7 are 0.143 $bbp$ and 31.75 dB, and 0.116 $bbp$ and 32.21 dB, respectively. The reconstructed point clouds appear smoother with a lower PSNR. The reconstructed point cloud of our TSC-PCAC is presented in Fig. 7 with bitrate and PSNR of 0.132 $bpp$ and 33.24 dB. Compared with those from the Sparse-PCAC and NF-PCAC, it exhibits a lower bit rate and higher PSNR. Additionally, in terms of visual quality, it can be observed that TSC-PCAC is smoother and offers better visual quality. Similar results can be found for Soldier from Figs. 7 to Fig. 7. The visual results further validate the superiority of TSC-PCAC for Redandblack and Soldier sequences comparing with Sparse-PCAC, NF-PCAC and G-PCC v23 methods.

We also evaluated the computational complexity of TSC-PCAC on all the testing point clouds. We processed each point cloud sequentially using a single GPU and the operating system was Ubuntu 22.04. For G-PCC, the encoding and decoding time for each point cloud corresponds to the average time required to encode and decode using different QP settings, while for the learned methods, it corresponds to the average time required to encode and decode using networks with different $\lambda_{A}$ values.

From Table II, it can be observed that Deep-PCAC and Sparse-PCAC have relatively long total encoding/decoding time, which are 41.16s/35.28s and 82.13s/459.63s, respectively. The furthest point sampling in the network results in longer encoding and decoding time for Deep-PCAC. For Sparse-PCAC, the sequential processing of the voxel context module results in longer encoding and decoding time. This issue becomes particularly severe when processing denser point clouds, such as Basketball Player and Dancer, where the decoding time can exceed a thousand seconds. For NF-PCAC, the average encoding/decoding time is 2.49s/3.08s. For G-PCC v23, the encoding/decoding time ranges from 3.34s/2.99s to 14.82s/13.37s, with an average of 7.40s/6.65s. The encoding/decoding time of TSC-PCAC ranges from 1.30s/3.22s to 3.14s/10.66s, with an average of 1.90s/5.61s. TSC-PCAC reduces encoding/decoding time by 97.68%/98.78%, 23.74%/-82.22% and 74.33%/15.66% on average as compared with the Sparse-PCAC, NF-PCAC and G-PCC v23. Note that the encoding and decoding of G-PCC were performed on CPU, while learning-based methods were performed on the GPU+CPU platform.

We also analyze the computational complexity for the modules of learned compression methods. Table III illustrates the network parameters, memory consumption in testing, the encoding and decoding time. ‘Y Enc.’ and ‘Y Dec.’ represent the average computing time of encoder $E_{A}()$ and decoder $D_{A}()$. Firstly, the parameter counts for Sparse-PCAC, TSC-PCAC, and NF-PCAC are 15.88M, 23.62M, and 51.16M respectively. NF-PCAC has the largest number of network parameters while Sparse-PCAC has the smallest. Secondly, the encoder and decoder runtime for Sparse-PCAC is the shortest, while NF-PCAC takes the longest. However, the encoding/decoding time of Sparse-PCAC and TSC-PCAC are longer compared to NF-PCAC because of the context modules. Sparse-PCAC employed the voxel context module performing autoregression in spatial dimensions, while our TSC-PCAC utilizes the channel context module performing autoregression in channel dimensions. Thus, TSC-PCAC greatly reduces the number of autoregression steps. This results in longest entropy encoding/decoding time for Sparse-PCAC, moderate for TSC-PCAC, and shortest for NF-PCAC, which are 82.02s/459.29s, 1.70s/5.16s, and 1.39s/1.13s, respectively. Finally, the average total encoding time of the TSC-PCAC is 1.90s, which is the shortest among all schemes. The average total decoding time of the TSC-PCAC is 5.61s, which is in the second place and a little longer than that of the NF-PCAC, i.e., 3.08s.

Ablation studies were performed to validate the effectiveness of the TSCM, the TSCM based channel context module, and the voxel-based global module. ‘TSCM w/o transformer’ represents the transformer in TSCM is replaced with residual block, ‘TSCM w/o global block’ represents the global block in TSCM is replaced with residual block, and ‘TSCM w/o both’ represents the global block and transformer in TSCM are replaced with residual block. ‘Point-based Global block’ indicates the TSCM that uses the global block from [42].

Table IV show the BD-BR and BD-PSNR gains achieved by the TSC-PCAC using different coding modules compared to the baseline Sparse-PCAC. Firstly, TSCM without Transformer achieves an average of 27.47% bitrate saving and an average of 1.19 dB PSNR gain. If without both transformer and global block, the TSC-PCAC achieves an average bitrate saving of 28.38% and 1.23 dB PSNR gain. As for the TSCM w/o Global block consisting of only residual blocks and the transformer, it achieves an average of 36.09% bitrate savings. Secondly, to evaluate the effectiveness of using the TSCM, we replaced TSCM in the channel context module with two convolutional layers, denoted as ‘context module w/o TSCM’. The channel context module using conventional convolutions has larger parameters than that using TSCM. It is also found in Table IV that if removing TSCM from the channel context, only 32.16% bitrate on average can be saved. Thirdly, to validate the effectiveness of using voxel-based global block in the TSC-PCAC, we replaced the voxel-based global block with point-based global block [42] in TSCM, and it is able to achieve an average of 36.89% bit rate reduction. Finally, the overall TSC-PCAC achieves an average of 38.53% bitrate saving, which is the best. Fig. 8 shows the coding performance comparison of the ablation studies in TSC-PCAC, where the TSC-PCAC is the best for Basketball Player, Redandblack and average$*$ of all sequences. These results prove the effectiveness of each module in TSC-PCAC, including transformer and global block in TSCM, voxel-based global block and TSCM based channel context module.