MEDPNet: Achieving High-Precision Adaptive Registration for Complex Die Castings

Given two sets of point clouds $X=\{x_{i}\in\mathbb{R}^{3}\,|\,i=1,\ldots,N\}$ and $Y=\{y_{i}\in\mathbb{R}^{3}\,|\,i=1,\ldots,M\}$, the objective is to estimate rigid transformations $T_{i}=\{R_{i},t_{i}\}$ for $i=1,\ldots,N$ to align these two point clouds. In Fig 2, we present the architecture of MEDPNet. In brief, we embed the acquired die-casting part point cloud data into a high-dimensional space using DGCNN[10] and encode the contextual information with the Efficient Attention[14] module, finally estimating the alignment using a differentiable SVD layer][47]. In which, $x^{P}_{i}$ is the embedding of point $i$ in the $P$-th layer, and $h^{P}_{\theta}$ is a nonlinear function in the $P$-th layer parameterized by a shared multilayer perceptron (MLP). The forward mechanism is given by:

We input the collected unaligned input point clouds X and Y into the same space, where we embed each point of the two input point clouds individually, and iterate over the features of each point in the input point clouds, represented by the aggregation function as:

and

DGCNN constructs a k-NN (k-nearest neighbor) graph $M$, nonlinearly acquires edge values at edge endpoints, and performs per-vertex aggregation at each layer. Unlike PointNet[33], which extracts independent information from each point, DGCNN explicitly incorporates local geometric shapes into its representation. This is achieved through the forward mechanism:

where $N_{i}$ represents the set of neighbors of point $i$ in the k-NN graph, ensuring that local geometric features are considered during the aggregation process. In the task of die-casting part point cloud registration, DGCNN achieves higher quality registration performance by leveraging these local geometric information.

Before introducing Efficient attention, let’s first discuss the concept of dot-product attention. Dot-product attention is a fundamental attention mechanism commonly used in models like Transformers. For a given query vector $Q$, key vector $K$, and value vector $V$, the computation of dot-product attention is as follows:

Here, $d_{k}$ is the dimensionality of query/key vectors, $QK^{T}$ represents the dot product between query vector $Q$ and key vector $K$, and it is scaled by $\sqrt{d_{k}}$ to stabilize gradient magnitudes. The softmax function[46] normalizes the dot product results into attention weights, which are then used to weight the value vector $V$ to generate the final output. This mechanism allows the model to dynamically allocate attention based on the similarity between queries and keys, capturing relationships between different positions in the input sequence.

The principle of efficient attention is to optimize the traditional attention mechanism, particularly in addressing the challenges of high computational complexity and significant memory consumption when processing long sequence data. By reducing redundancies in computation and employing more efficient computational strategies, such as low-rank factorization and kernel techniques, it approximates the key operation $QK^{T}$ in the standard self-attention mechanism, as shown in Fig 3.

In the standard self-attention mechanism, the input sequence $X\in\mathbb{R}^{n\times d}$ undergoes linear transformations to obtain queries $Q$, keys $K$, and values $V$, where $Q=XW_{Q}$, $K=XW_{K}$, $V=XW_{V}$, and $W_{Q}$, $W_{K}$, $W_{V}\in\mathbb{R}^{d\times d_{k}}$ are the corresponding weight matrices, with $d_{k}$ being the feature dimension.

Traditionally, attention weights are obtained by computing $QK^{T}$ and applying the softmax function, i.e.,

This step has a computational complexity of $O(n^{2}d_{k})$, which for long sequence data, results in significant computational burden and memory requirements.

Efficient attention introduces an approximation technique to reduce this complexity, specifically, it uses a form

for approximation, where $\phi(\cdot)$ is a nonlinear function mapping to a lower-dimensional feature space, effectively reducing the required computational power and storage space. This mapping not only reduces the need for direct computation of $QK^{T}$ but also, by selecting an appropriate $\phi$ function, can lower the computational complexity of the attention mechanism from $O(n^{2}d_{k})$ to $O(nmd_{k})$, where $m$ is the dimension of the mapped lower-dimensional space, significantly smaller than the length of the input sequence $n$.

Precisely for these reasons, Efficient attention significantly enhances computational and storage efficiency in die-cast parts point cloud registration tasks by optimizing the attention mechanism. It effectively manages long-sequence dependencies, accurately captures changes and spatial relationships between point clouds, achieves high-precision registration, and broadens the application scope in resource-constrained environments.

First, let’s review the geometric point cloud registration method. Its core idea involves iteratively optimizing the rigid transformation {R, t}, where $R$ is the rotation matrix and $t$ is the translation vector. The objective is to minimize the distance $T$ between two point clouds, achieving their alignment. The expression for this can be given as follows:

our approach primarily adopts the ICP (Iterative Closest Point) method and the NDT (Normal Distributions Transform) method. The core idea of the ICP method is to achieve point cloud registration by iteratively optimizing the rigid transformation {R, t} to minimize the distance $T$ between point clouds. Its expression is:

Here, ICP primarily aligns point clouds through iterative optimization. In the process of minimizing the objective function, adjustments to $R$ and $t$ are made to bring the two point clouds as close together as possible in space.

The NDT method describes the local structure of point clouds by modeling the normal distribution of each point. Its optimization objective is formulated as:

$\mu_{x_{i}}$ and $\mu_{y_{i}}$ denote the mean of the normal distribution corresponding to points in two point clouds, and $\Sigma_{i}$ represents the covariance matrix of the normal distribution. The goal is to adjust $R$ and $t$ to make the two point clouds as consistent as possible in terms of normal distribution, minimizing the objective function.

Multi-scale Feature Fusion Module: In both ICP (Iterative Closest Point) and NDT (Normal Distributions Transform), we introduced a multiscale feature fusion module aimed at addressing key challenges in point cloud registration, such as local minima and sensitivity to initial values. Our approach first utilizes the Feature Pyramid Network (FPN) [17] method to generate multiple scales of point clouds through downsampling. At each scale, the ICP algorithm is independently applied to find the optimal rigid transformation. Subsequently, the coarse-scale registration results are propagated to finer scales, resulting in the final registration outcome and transformation matrix. We have successfully implemented feature fusion on die-cast point cloud pairs at $k$ different scales.

The multiscale ICP method aims to minimize the registration error across all scales $k$ and corresponding points $i$ by finding the optimal rigid transformation $R_{k}$ and $t_{k}$. This approach allows us to comprehensively tackle registration challenges at multiple scales, thereby enhancing the algorithm’s robustness in diverse scale environments. The expression for this method is given by:

similarly, multiscale Normal Distributions Transform (NDT) minimizes the cumulative error of the normal distribution for corresponding points at each scale, utilizing the Mahalanobis distance metric (measured by the difference between the inverse covariance matrix and the normal distributions). This optimization seeks to find the optimal rigid transformation $R_{k}$ and translation vector $T_{k}$ at each scale. This enables precise registration of point cloud $X$ with target point cloud $Y$ in the normal distribution across multiple scales through adjustments in rigid transformation and translation. It can be expressed as:

Dual Channel Fusion Module: Previous studies have primarily focused on enhancing the performance of the ICP (Iterative Closest Point) and NDT (Normal Distributions Transform) methods, yet they have overlooked the importance of ensuring stability under conditions of high precision. This issue becomes particularly evident when dealing with complex die-cast point cloud data, where the ICP algorithm may perform excellently on point cloud data ”a,” while the NDT algorithm shows superior performance on point cloud data ”b.” To overcome this challenge, we propose a novel dual-channel fusion module. Through multi-scale feature fusion, this module enables the ICP and NDT methods to obtain rigid transformation matrices $T_{1}$ and $T_{2}$, respectively. We input 300 pairs of rigid transformation matrices from the ICP and NDT methods into an MLP for learnable self-feedback weighting and iterate the weights of the optimal registration results by minimizing the registration error, as shown in Fig 4. The MLP consists of three fully connected layers, with neuron counts of 32, 64, and 32, respectively. Here, we opt for the Huber loss function, expressed as:

Where $a=l-\hat{l}$ represents the prediction error, i.e., the difference between the actual value $l$ and the predicted value $\hat{l}$. $\delta$ is a threshold parameter that determines the point at which the loss function transitions from squared error to linear error. When the absolute value of the error is less than or equal to $\delta$, the loss function behaves like the square of the error (similar to MSE), imposing a heavier penalty for smaller errors to encourage more precise fitting. Conversely, when the absolute value of the error exceeds $\delta$, the loss function becomes linear (similar to MAE), reducing the penalty for larger errors and enhancing the model’s robustness.

Although the merged matrix $T$ obtained at this point demonstrates certain reliability, repeated tests have shown that the model’s accuracy decreases when facing unfamiliar samples. We hypothesize that this instability might be due to the limited sample size, making it difficult for the model to learn the complete features of die-cast part point clouds. However, due to the irreplicability of die-cast samples and the industrial production cycle’s inability to accommodate the training duration for a large volume of samples, we introduced the hyperparameter $\varepsilon$. Initially, we hoped to directly obtain optimal weights and $\varepsilon$ through the MLP mechanism, but the limited training samples and excessive number of parameters to be optimized led to unsatisfactory results. To address this, we added a self-updating filtering mechanism on top of the MLP. With the determination of optimal weights $W_{1}^{*}$ and $W_{2}^{*}$, we use the root mean square error (RMSE) feedback from each iteration to determine the corresponding $\varepsilon$, choosing the $\varepsilon$ associated with the minimum RMSE as the input for the next iteration, as shown in Fig 4.The formula for RMSE is as follows:

Where $N$ is the number of samples, $l_{i}$ is the actual value of sample $i$, and $\hat{l}_{i}$ is the predicted value for sample $i$.Finally,we can obtain:

In this setup, we achieved learnable adaptive registration through a multilayer perceptron and a self-updating filtering mechanism, obtaining desirable results on the die-cast part point cloud dataset DieCastCloud.

This experiment utilizes Open3D[30] 1.2.0 and PCL 1.9.1 to implement algorithm execution in Python and C++. The experimental platform is Ubuntu 18.04 system, with PyTorch[37] version 1.8.1, CUDA version 11.1, GPU=RTX 3090 (24GB) * 1, CPU=15 vCPU AMD EPYC 7642 48-Core Processor. To test the generalization of different models, we will split DieCastCloud into training and testing sets.

Due to the complexity of the surface features of die cast parts, unlike the approach in PointNet[33] experiments of uniformly sampling 1024 points on the model’s outer surface, we opted to sample 4096 points. This decision was based on an understanding of the complexity of die cast surface features, aiming to more comprehensively preserve the point cloud’s feature information, thereby enhancing the accuracy and reliability of subsequent registration.

To ensure consistency and standardization in data processing, we performed a series of preprocessing steps on the collected point cloud data. Initially, the point cloud data was centered at the origin and scaled to fit within a unit sphere. Throughout this process, we only used the three-dimensional coordinates (x, y, z) of the points as input features, without introducing any additional attribute information, to accurately assess the model’s ability to recognize and process geometric shapes themselves.

The initial pose has a critical impact on point cloud registration. To better quantify the performance of coarse registration, we employed multiple error metrics, including Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and Mean Absolute Error (MAE), to ensure the reliability of the method. Moreover, considering practical applications in the die casting industry, we primarily focused on Root Mean Squared Error (RMSE) and registration time (s) during fine registration, where all angle measurements were made in degrees (°).

The DieCastCloud dataset contains 2,000 point cloud data, including 5 different types of die-cast parts. This dataset is randomly divided into a training set and a test set, with proportions of 0.8 and 0.2, respectively. In practical applications in the die-casting industry, the purpose of point cloud registration is to enhance the completeness of the point cloud data while preserving key features, to ensure high-precision 3D reconstruction and facilitate product quality control. Unlike other datasets[28][25], here, to ensure the practical feasibility of the method, the overlap rate of point cloud data in DieCastCloud is set to be greater than $85\%$.

We utilized the UR16e robotic arm equipped with the high-precision 3D laser scanner CIRRUS 3D 300 to collect point cloud data of die-cast parts, and we named the resulting dataset DieCastCloud. The point cloud data in DieCastCloud covers the main external surfaces of the die-cast parts, including complex geometric features such as pipes and holes. Additionally, we processed and filtered the collected raw point cloud data to obtain a richer sample set.

Finally, in the creation process of DieCastCloud, point cloud data was enhanced through techniques such as rotation, translation, scaling, and random erasure to increase the diversity of the data and enhance the model’s generalization capabilities. Specifically, we randomly rotated the point cloud data at random angles along any axis and translated it in any spatial direction, with the translation range controlled within [-800mm, 800mm]. The scaling ratio was constrained to between [0.95, 1.05] of the original point cloud size.

In this experiment, we use DCP-v2, which incorporates a Transformer, as our baseline. We compare the performance of Efficient DCP with other cutting-edge deep learning-based point cloud registration methods, including PointNetLK[32], GeoTransformer[18], PRNet[36], DeepGMR[9], and DCP[8]. We randomly split 2000 die-cast component point clouds from DieCastCloud into validation and test sets, utilizing different point cloud data during the training and testing periods.During training, we sampled the point clouds and applied a random rigid transformation along each axis, with rotations uniformly sampled within [0, 60°] and translations in the range of [-150mm, 150mm]. The source point cloud and the point cloud after the rigid transformation were used as the input to the network.

To ensure a fair comparison among these methods, we follow the convention and use performance metrics including Mean Squared Error (MSE) for rotation angles (MSE(R)), Mean Squared Error for translation directions (MSE(t)), Root Mean Squared Error for rotation angles (RMSE(R)), Root Mean Squared Error for translation directions (RMSE(t)), Mean Absolute Error for rotation angles (MAE(R)), and Mean Absolute Error for translation directions (MAE(t)), to guarantee the reliability of the experiments.

Table 1 assesses the performance of our method and its counterparts in this experiment. In this study, Efficient DCP adopts a structure that integrates DGCNN with Efficient Transformer, with a learning rate of 0.001, 200 epochs, train batch sizes of 32 and train batch sizes of 10, and employs Stochastic Gradient Descent (SGD) as the optimizer. Across all evaluated performance metrics, Efficient DCP showcases outstanding performance.

In this section, we conducted comparative experiments to validate our choice of the Iterative Closest Point (ICP) and Normal Distributions Transform (NDT) methods. In real industrial scenarios, registration time needs to be controlled within 60 seconds. To better evaluate the feasibility of the methods, we used Root Mean Square Error (RMSE) in millimeters and registration time in seconds as performance evaluation metrics, with the results shown in Table 2. Additionally, to more intuitively understand the impact of rotation angles (°) and translation distances (mm) on various methods, we conducted comparative experiments to test the performance of each method at specific rotation angles and translation distances, with results presented in Table 3. Finally, we elucidated the advantages of our dual-channel precision registration method based on multi-scale features.

We first evaluated the performance of SAC-IA[45], 4PCS[2], NDT[29], ICP[24], and MDR on the DieCastCloud dataset. To assess the robustness of these methods, we introduced noise with an intensity of 0.1 into the DieCastCloud dataset and then conducted comparative experiments to evaluate the stability of each method under noisy conditions. By incorporating a multi-scale feature fusion module, MDR possesses more comprehensive feature information compared to other methods, thereby achieving higher registration accuracy and noise resistance. As shown in Table 2, MDR demonstrates high performance while meeting the registration time requirements in practical applications.

Next, we tested the effects of rotation angles and translation distances on several algorithms. We rotated the point cloud data by 10°, 30°, and 90° around any spatial axis and translated it by 100mm, 500mm, and 1000mm in any spatial direction. As can be seen from Table 3, ICP showed a notable performance for different angles of rotation, while NDT performed better in facing translation issues and exhibited greater robustness in dealing with spatial position changes. Our method demonstrated the best registration performance compared to the other methods.

Overall, the MEDPNet method achieves high-quality registration results by initially applying Efficient DCP for coarse registration of unaligned point cloud pairs, followed by fine-tuning through MDR. To assess the accuracy and robustness of our method, we conducted tests on both clean and noisy samples, with our experimental results presented in Table 4. We selected root mean square error (RMSE) as the performance metric to comprehensively account for both variance and bias. The experimental outcomes indicate that our method not only ensures high accuracy but also maintains robustness across different conditions.

In this section, we present a visual comparison of the performance of various advanced methods against our MEDPNet approach, as illustrated in Fig 5. We selected three typical types of die casting samples and conducted experiments under both noise-free and noisy conditions to ensure the practicality of our method. Our comparison mainly includes PointNetLK[32], DCP[8], ICP[24], NDT[29], and our improved methods Efficient DCP, MDR, and MEDPNet, arranged in descending order according to root mean square error (RMSE). As can be observed in Fig 5, MEDPNet achieves state-of-the-art performance on both clean and noisy samples.