PointGL: A Simple Global-Local Framework for
Efficient Point Cloud Analysis

Point-based methods, notably exemplified by PointNet [22] and PointNet++ [8], stand as pioneering instances in this domain, furnishing a foundational framework for subsequent endeavors. PointNet++ undertakes the hierarchical abstraction of intrinsic geometric features across multiple stages, facilitated by a collection of $\rm{N}$ points, each characterized by Cartesian coordinates denoted as $\bm{\mathcal{P}}=\left\{\bm{p}_{i}\in\mathbb{R}^{\rm{1}\times 3}|i=1,\cdots,\rm{N}\right\}$. Within the $s$-th stage, PointNet++ employs the farthest point sampling (FPS) algorithm to select ${\rm N}_{s}$ points. For each of these sampled points, $\rm{K}$ neighbors are queried, thereby constituting ${\rm N}_{s}$ local point sets. Subsequently, PointNet++ proceeds to acquire insights into the local patterns within each local point set through the expression:

Here, $\bm{f}_{i,j}$ signifies the feature of the $j$-th neighboring point pertaining to the $i$-th sampled point and $\bm{g}_{i}$ denotes the aggregated local feature for the $i$-th sampled point. The function $\bm{h}(\cdot)$ is implemented by a multi-layer perceptron (MLP), thus conducting spatial encoding for a given point. Meanwhile, $\bm{\mathcal{A}}$ represents a symmetric function, like max pooling, employed to consolidate the encoded point features. Through the aggregation of multiple stages, featuring diminishing ${\rm N}_{s}$ and augmented spatial coverage within each local point set, PointNet++ progressively engenders abstracted local geometric features coupled with an expanding receptive field.

Subsequent point-based methodologies [10, 35, 25, 13, 36] have primarily directed their efforts towards enhancing the effectiveness of $\bm{h}(\cdot)$ to more adeptly capture interrelations between points, achieved by integrating convolution, graph, or attention mechanisms. Notably, the majority of these methodologies adhere to the overarching framework established by PointNet++: during each stage, input points are conglomerated into a sequence of local point sets, wherein feature embedding is executed for the points within each point set. Subsequently, local features are distilled through an aggregation function. However, a comprehensive scrutiny of this framework has illuminated two inherent limitations that could potentially impede computational efficiency.

Primarily, it is evident that during the $s$-th stage, there are ${\rm N}_{s-1}$ input points. Following the process of grouping, ${\rm N}_{s}$ local point sets emerge, with each assemblage accommodating $\rm{K}$ neighboring points. Typically, ${\rm N}_{s}$ constitutes merely half of ${\rm N}_{s-1}$, and $\rm{K}$ is selected from the range of $[16,64]$, thus implying that the majority of input points are represented in multiple point sets. Given that $\bm{h}(\cdot)$ operates on every point within each point set, on average, every input point necessitates a cumulative count of ${\rm N}_{s}{\rm K}/{\rm N}_{s-1}$ feature embeddings. While point embeddings within distinct point sets might encompass varying spatial details, such as relative coordinates concerning a point set center, we posit that subjecting an individual point to multiple feature embeddings might not be pivotal for performance enhancement. However, this approach unavoidably introduces a substantial degree of computational superfluity. It’s paramount to recognize that this impact becomes significantly magnified upon the integration of intricate spatial encoders in lieu of conventional MLPs.

Secondly, a noteworthy observation pertains to the fact that PointNet++ embarks on the abstraction of geometric features within a local point set via point-wise MLP, succeeded by a symmetric function. Regrettably, this approach is prone to diluting intricate geometric subtleties, primarily stemming from its limited capacity to model interpoint relationships. This shortcoming has prompted subsequent studies to address the concern through the introduction of intricate spatial encoders or aggregation functions. Yet, the unintended consequence of these endeavors has been the imposition of exorbitant computational demands and a significant expansion in memory usage, thus compromising overall efficiency. Hence, the endeavor to enhance efficiency while upholding accuracy necessitates a more streamlined approach to the modeling and aggregation of interpoint relationships. These two limitations collectively impel us to embark on an exploration of a novel point-based paradigm.

The comprehensive architecture of our PointGL is elucidated in Fig. 2, detailing its process of learning hierarchical features from input point clouds through the stacking of a total of $\rm S$ learning stages. In the context of the $s$-th stage, the input is constituted by ${\rm N}_{s-1}$ points denoted as $\bm{\mathcal{P}}_{s-1}=\left\{(\bm{p}_{i},\bm{f}_{i})\ |\ i=1,\cdots,{\rm N}_{s-1}\right\}$. Here, each point $i$ is characterized by its $xyz$ Cartesian coordinates represented as $\bm{p}_{i}\in\mathbb{R}^{3}$, along with an associated feature vector $\bm{f}_{i}\in\mathbb{R}^{{\rm D}_{s-1}}$. The outcome of this stage encompasses ${\rm N}_{s}$ sampled points denoted by $\bm{\mathcal{P}}_{s}=\left\{(\bm{p}_{i},\bm{g}_{i})\ |\ i=1,\cdots,{\rm N}_{s}\right\}$. Within this set, each sampled point $i$ is distinguished by a feature vector $\bm{g}_{i}\in\mathbb{R}^{{\rm D}_{s}}$ that encapsulates the localized pattern surrounding it.

Our PointGL introduces a novel and efficient approach to learn local patterns through a dual-step process. The first step, known as Global Point Embedding, entails the execution of feature embedding for each point within the input set $\bm{\mathcal{P}}_{s-1}$. Diverging from the methodology of PointNet++, which generates spatially overlapping local point sets and rigorously encodes features for each point within each point set, PointGL undertakes feature embedding solely once for every input point. This strategic divergence substantially mitigates redundant computations. Subsequently, in the subsequent step known as Local Graph Pooling, points are selectively sampled from $\bm{\mathcal{P}}_{s-1}$, and the relationships between each sampled point and its neighboring points are meticulously modeled and collectively integrated to formulate localized representations. Remarkably, this entire sequence is accomplished exclusively through straightforward operations, effectively upholding computational efficiency. In the following sections, we proceed to provide an in-depth elaboration of both of these fundamental steps.

The employment of point-wise $\mathrm{MLPs}$ imparts invariance to point permutations upon the function $\bm{\Phi}(\cdot)$, which is capable of approximating a diverse array of continuous functions. Additionally, the point embedding procedure necessitates minimal operations, primarily reliant on meticulously optimized feed-forward $\mathrm{MLPs}$. Furthermore, given that point embedding is executed solely once for each input point, the efficiency of our PointGL remains unimpeded even with the integration of more intricate or deeper $\mathrm{MLPs}$.

To achieve this, we undertake the process through three distinct steps of local point association followed by feature aggregation: i) We initiate the process by sampling $\rm{N}_{s}$ points from the input data at the current stage utilizing the farthest point sampling (FPS) technique. Following this, we employ the k-nearest neighbors (kNN) algorithm to gather $\rm{K}$ neighboring points corresponding to each of the sampled points. ii) Subsequently, each sampled point is connected to its respective neighbors, effectively constituting a series of local graphs. These graphs encode the point-to-point relations through the incorporation of edge features. iii) In the final step, we subject the edge features of each local graph to a symmetric aggregation function. This operation yields the local feature representation.

The pivotal stride involves generating edge features capable of effectively encoding relationships among points while upholding efficiency. To accomplish this, we embrace a rudimentary yet remarkably efficacious strategy, encompassing the computation of the difference feature between a sampled point and each of its neighbors as the edge feature:

In this context, $\bm{v}_{j}\in\mathbb{R}^{{\rm D}_{s}}$ and $\bm{v}_{j,k}\in\mathbb{R}^{{\rm D}_{s}}$ signify the features of the $j$-th sampled point and its $k$-th neighboring point, respectively. The parameters $\bm{\alpha},\bm{\beta}\in\mathbb{R}^{{\rm D}_{s}}$ are subject to learning, with the initial value of $\bm{1}$ and $\bm{0}$, respectively, while $\odot$ signifies the Hadamard product.

The local output representation for the $j$-th sampled point, denoted as $\bm{g}_{j}$, is derived by aggregating the edge features linked with all the edges emanating from this point. This aggregation is defined as:

where $\rm{K}$ is the number of neighboring points. We opt for the employment of the max pooling operation as the aggregation function $\bm{\mathcal{A}}\left(\cdot\right)$ due to its efficacy and simplicity.

Beyond its simplicity and compactness, our approach to local graph pooling offers several salient advantages: i) it retains invariance to the ordering of neighbors, thereby ensuring resistance to variations in point permutations; ii) it refrains from involving intricate operations, thus ensuring commendable computational efficiency; iii) it is nearly parameter-free, facilitating its seamless integration in a plug-and-play manner.

As elucidated earlier, each learning stage within our methodology samples a subset of points from the input, endowing each sampled point with local geometric information. Through the accumulation of multiple stages, this design progressively furnishes a smaller number of points, each enriched with an extended contextual awareness.

The PointGL network is structured with $\rm S=4$ stages. These stages exhibit escalating embedding dimensions of ${\rm D}_{s}=\left\{128,256,512,1024\right\}$, complemented by a diminishing count of sampled points denoted as ${\rm N}{s}=\left\{512,256,128,64\right\}$. Each stage encompasses a residual MLP block, leveraging $\rm{K}=24$ nearest neighbors for the purpose of local feature aggregation.

To further optimize efficiency, drawing inspiration from [14], we have introduced an advanced iteration of PointGL, denoted as PointGL-elite. This variant refines the feature embedding dimensions based on the original PointGL framework. For a comprehensive overview of the architectural nuances, please consult Table I. In addition, Algorithm 1 shows the implementation of PointGL.

During the training phase, both the PointGL and PointGL-elite models underwent training for $250$ epochs utilizing the AdamW optimizer with a batch size of $32$. The optimizer utilizes a learning rate of $0.002$ and weight decay of 0.05. The CosineAnnealingLR [43] scheduler is employed to decrease the learning rate to the minimum value of 1e-4, and the warm up epochs is set to 0. The evaluation metrics encompass overall accuracy (OA) and class-average accuracy (mAcc) calculated on the test set.

In a specific context, PointGL showcased a cutting-edge overall accuracy of $86.9\%$, asserting its dominance over preceding methodologies such as PointMLP [14] and RepSurf [41]. Moreover, our self-contained PointGL demonstrated superiority over both PointMLP and RepSurf, even when employing a multi-scale inference strategy [35]. Of significant note is our approach’s substantial performance advantage over PointNet++ by a substantial margin ($86.9\%$ vs. $77.9\%$), thereby underscoring the supremacy of our local graph pooling mechanism for the abstraction of local patterns.

In contrast to the more contemporary RepSurf-U, our PointGL experiences a slight increase in FLOPs, yet manages to achieve $\bm{7.2}\times$ and $\bm{2.7}\times$ faster speeds during training and inference, respectively. These findings underscore that a reduction in model complexity doesn’t necessarily guarantee heightened efficiency. Our straightforward approach of global point embedding followed by local graph pooling effectively curtails redundant computations across local point sets, thereby serving as the fundamental basis for efficiency enhancement.

To further optimize efficiency, we introduce an accelerated version of PointGL termed PointGL-elite. With a mere $0.49\rm M$ parameters, PointGL-elite significantly reduces FLOPs to just $0.05\rm G$, accounting for only $\bm{5\%}$ of the FLOPs found in its PointMLP-elite counterpart ($0.91\rm G$). Despite maintaining a competitive accuracy level of $84.2\%$ OA, PointGL-elite achieves an exceptional inference speed of $2,035$ samples/s, surpassing the speeds of PointMLP-elite and PointNet++ by nearly $\bm{3}\times$ and $\bm{1.4}\times$, respectively. These outcomes firmly establish PointGL-elite as the point-based model with minimal model complexity and the swiftest inference rate.

Table IIIa confirms that PointGL outperforms the model variant, achieving higher accuracy while significantly reducing model complexity and improving efficiency. Specifically, while maintaining an equivalent parameter count, PointGL reduces FLOPs by an impressive factor of $\bm{11}\times$ and demonstrates approximately $\bm{2.7}\times$ faster training speed, coupled with an inference speed enhancement of approximately $\bm{2.5}\times$. These results effectively validate the critical importance of the global point embedding design within the PointGL framework.

To examine the effect of the number of residual MLP blocks on point embedding and its consequent impact on performance, we manipulated the quantity of these blocks within each learning stage, creating PointGL variants with $16$, $24$, and $32$ layers by adjusting the number of blocks to $1$, $2$, and $3$, respectively. As illustrated in Table IIIb, the addition of extra embedding layers does not consistently result in improved accuracy.

Furthermore, we represent $\bm{\alpha}\in\mathbb{R}^{{\rm D}_{s}}$ and $\bm{\beta}\in\mathbb{R}^{{\rm D}_{s}}$ as ${\rm D}_{s}$-dimensional learnable vectors, employed for channel-wise adjustment of responses in differential features. To assess the efficacy of our design, ablative experiments were conducted by substituting both $\bm{\alpha}$ and $\bm{\beta}$ with learnable scalars. The experimental results indicate a noticeable performance degradation due to this modification, resulting in an overall accuracy reduction of $0.7\%$. These experiments affirm the effectiveness of the channel-wise feature adjustment design.

Next, we proceed to further validate the impact of the values of ${\rm K}$, representing the number of neighboring points in the kNN algorithm. As presented in Table IIId, both excessively small (${\rm K}=12$) and large (${\rm K}=36$) values for ${\rm K}$ resulted in a decrease in predictive accuracy. This occurs because small ${\rm K}$ values may lead to a diminished receptive field for features, while large ${\rm K}$ values may attenuate local detailed features. Optimal predictive accuracy was achieved with a suitable value for ${\rm K}=24$. As a result, we adopted ${\rm K}=24$ in our experiments.

We have successfully implemented the local graph pooling as a versatile drop-in block, distinguished by its minimal parameter requirement and enhanced efficiency. To assess its adaptability in point-based models like PointNet++ and PointMLP, we replaced the native max pooling with our local graph pooling, excluding the sampling and kNN operations. The results shown in Table IIIe demonstrate that the straightforward incorporation of local graph pooling consistently enhances performance across a range of models. This underscores the universal nature of our pooling approach, highlighting its seamless applicability within existing point-based models to improve feature aggregation, all while incurring minimal additional parameters and computational overhead.

The versatility of our PointGL framework allows for its extension to a variety of tasks. To assess its performance in the context of 3D shape part segmentation, we conducted experiments on the ShapeNetPart dataset [51]. This dataset comprises a collection of $16,881$ shapes distributed across $16$ distinct classes, each annotated with $50$ parts. We randomly sampled $2,048$ points as inputs [8] and trained the model for $350$ epochs using the Adam [52] optimizer with a batch size of $32$. The optimizer is configured with betas=(0.9, 0.999) and learning rate of $0.003$. The stepLR scheduler is also employed to decrease the learning rate with step size of $40$ and gamma of $0.5$.

As demonstrated in Table IV, our PointGL achieved remarkable performance, boasting an instance average mean Intersection over Union (mIoU) of $85.6\%$, while also maintaining notable inference speed. Furthermore, our approach outperformed the PointNet++ baseline in the majority of categories. Fig. 4 visually reinforces how PointGL’s predictions closely align with the ground truth.

Moreover, our PointGL approach underwent evaluation on the S3DIS Area-5 dataset [53], which pertains to 3D semantic segmentation. In our experimental setup, we train the PointGL model with a batch size of 32 over 100 epochs, employing the AdamW optimizer. The optimizer is specifically configured with a learning rate of 0.01 and a weight decay rate of 1e-4. To manage the learning rate schedule, we utilize the CosineLRScheduler scheme, which reduces the learning rate progressively until it reaches a minimum value of 1e-5. As presented in Table V, PointGL achieved remarkably competitive performance, achieving an mIoU of $65.6\%$, surpassing established benchmarks like PointNet++ ($53.5\%$), while simultaneously performing on par with contemporary methods like RepSurf-U. Significantly, these achievements were reached with a lower FLOP count and maintained high-speed inference. These results underscore the adaptability of our PointGL approach and its potential for utilization across a spectrum of downstream tasks.

Real-world applications dealing with point cloud data often encounter challenges arising from sensor inaccuracies and complex scene structures, leading to data corruptions. Thus, the ability to handle point cloud corruptions effectively becomes crucial for practical applications. To assess the resilience and versatility of our proposed PointGL approach, we conducted a series of experiments on the ModelNet-C [34] and ShapeNet-C [56] datasets. Our evaluation covers tasks such as point cloud classification and part segmentation, conducted under various corruption scenarios.

Table VII presents a comprehensive summary of the evaluation results for point cloud classification models applied to the ModelNet-C dataset. The results highlight a significant observation - many state-of-the-art methods, known for their high Overall Accuracy (OA) on the clean ModelNet40 dataset, exhibit notable vulnerability when subjected to corruptions. In contrast, our proposed PointGL excels by achieving the lowest mean Corruption Error (mCE) of $0.77$ across all corruption scenarios. This accomplishment unquestionably demonstrates the robustness of our model. These findings underscore the substantial potential of PointGL in practical real-world scenarios.

The performance of our proposed PointGL on the ShapeNet-C benchmark is highlighted in Table VIII. The achieved class-wise mIoU score of $0.82$ significantly outperforms the state-of-the-art PointMAE method by a substantial margin of $0.11$. Notably, PointMAE is a masked autoencoder (MAE) model trained on a substantial volume of unlabeled point cloud data. This noteworthy achievement underscores that PointGL effectively harnesses vital local information while preserving the integrity of semantic information. The demonstrated performance not only speaks to the robustness of our approach but also underscores its broad applicability across different tasks.