Necessity Feature Correspondence Estimation for Large-scale Global Place Recognition and Relocalization

In the fields of the robotics and computer vision, wide-ranging researches have been conducted on Simultaneous Localization and Mapping (SLAM) [1]. SLAM is widely used not only for mobile robots but also for unmanned technologies such as autonomous vehicles and underwater marine robots [2, 3, 4]. In traditional approaches, the pipeline of SLAM can be broadly divided into three parts: 1) front-end, 2) back-end, and 3) loop-closing [5]. Loop closing can optimize the robot’s pose when the robot detects a loop, also known as a Rendezvous, that connects to a previously visited location. It is important for the robot navigation task to accurately detect loop closing point, because failure to detect loops can result in incorrect pose optimization of the map and the robot, and lead the distorted map representation. [6, 7]. By successfully detecting previously visited locations, the robot’s poses can be corrected through relocalization even if the robot fails to estimate its past trajectories correctly due to accumulation of errors.

Place recognition is of significant importance for the loop closing detection, thereby many researches have been conducted. However, there are still difficulties in performing place Recognition for 3D LiDAR point clouds, because 3D LiDAR point clouds are sparse compared to 2D images and have complex distributions of 3D geometric structures [8]. In this case, information that can distinguish geometric structures may not be incorporated into the features, which can cause the failure to detect course-level previously visited places. It is also hard to achieve the fine-level pose estimation as well, because the estimation of the 6-DOF pose needs features that contain detailed structural information in 3D LiDAR point clouds.

To resolve these issues, in previous works, various registration methods based on ICP [9] have been widely used to estimate a fine pose after coarsely recognizing the global pose [10, 11, 12, 13]. However, methods like ICP can be influenced by the initial pose and can fall into a local minimum, resulting in misaligned 3D scans. Recent researches tackle these problems by estimating global place recognition and learning local descriptors for 6-DOF transform simultaneously or estimating initial pose for fine pose estimation [12, 14]. To accomplish this, it is crucial to establish point correspondences, leading to the adoption of various methods, including data augmentation [14] and unbalanced optimal transport [5]. Nevertheless, these methods do not consider the necessity of feature points for estimating true-positive correspondence and simply utilize all possible points. Using all feature points equally can result in incorrect correspondence estimation such as false-positive or false-negative one, as it tries to find correlations even if some of the feature points have no correspondence. This leads to a lack of discrimination in local feature learning for structural information. As a result, when using coarse place recognition, unnecessary features are aggregated, leading to limited accuracy in global place recognition. Furthermore, the inability of estimating feature correspondence also results in poorer relocalization performance.

To overcome these limitation, we propose a method considering the necessity of each feature point for both global place recognition and relocalization. By considering the necessity of each feature point based on the geometric characteristics before training, the network can estimate the key correspondences for each feature. Robust estimation of the correspondence can reduce the number of the outliers, thus make it easier to find correspondences at each local point and leading to better results in 3D point cloud registration. With fewer outliers, the accuracy of 6-DOF calculations during RANSAC or SVD will increase, enabling robust point cloud registration [15]. In addition, the better “local consistency” of features can also lead to improvement in global place recognition [8].

To determine the necessity of each feature point for learning, this paper employs two techniques that consider geometrical feature correlation. First, we weaken the influence of ground structures that have unnecessary geometrical features. The features on the ground can be considered outliers from a local feature perspective because they have structurally duplicated forms, which can cause inaccurate correspondence information. By weakening the influence of dominant ground we can remove features that can generate inaccurate correspondence. Second, we exploit the weights from probabilistic model based on geometric correlation to determine the necessity of the features. Since the feature points we want to find correspondences in two scenes have a geometric correlation, we found that it is possible to find and learn the strong necessity of inlier features by considering this geometric correlation. After finding the weights, the points corresponding to those weights are used to estimate correspondence according to the weights.

To summarize, we propose a method that enhances global place recognition and point cloud registration, by considering the geometric characteristics of 3D point clouds. We weaken unnecessary feature points and strengthen necessary ones while training by feature correspondence estimation. Our contribution is as follows.

• •

  We preprocess the ground to weaken unnecessary features identified through geometrical character. The ground has repetitive structures due to its characteristics, and these characteristics can become irrelevant in feature extraction or clustering [11, 16, 17]. Therefore, ground can act as outliers when estimating point correspondence between two scenes, so they are weakened through the preprocessing process.

• •

  We enhance learning features with better matching, considering probabilistic model based on geometric correlation through 3D alignment algorithms. Each overlapping points have corresponding relationships. We estimate correspondence using weights computed by the probabilistic model based on geometric correlation.

Our method shows better performance compared to the previous methods. We confirmed the high performance of our method through testing on the KITTI and KITTI-360 datasets. This demonstrates the critical role of considering probability-based geometric correlation in global place recognition and relocalization.

Research in place recognition in 3D point clouds can be divided into two main categories: global place recognition and point cloud registration [18]. Global place recognition is a coarse-level place recognition method that utilizes feature points of a scene to search topological locations on a large-scale map. Point cloud registration, on the other hand, is a fine place recognition method that calculates the 6-DOF of a scene to describe transformation in a metrical location perspective. Although accurate transformations within a scene can be described with point cloud registration, it is often used as a supplementary method in large-scale place recognition due to limitations in computing feature correspondences.

Global place recognition uses a method that compares the global descriptor of the current point clouds to that of the map to determine their similarity. The traditional approach is handcrafted-based method using BEV, generating a global descriptor [19, 20]. Recently, an end-to-end learning-based global place recognition approach has been proposed, showing promising results [21]. This method combines point cloud feature extraction [22] with vision-based global place recognition [23], demonstrating its viability for use in 3D point clouds. More recently, architectures have been proposed to generate global descriptors that better represent the place, thereby improving the performance of global place recognition [8, 24, 25].

Recent research has emerged with an architecture that combines point cloud registration and global place recognition [12, 13]. They aim for more accurate place recognition by not only learning global descriptors but also local features that can describe 6-DOF transforms. To learn local features, correspondence must be known, and methods have been proposed to augment training data to achieve [14]. In the case of [5] used sparse convolution network [26] to extract features and proposed unbalanced optimal transport to find correspondence between features. Our work also effectively estimates correspondence through the use of unbalanced optimal transport.

Suppose we have a set of points $L=\{l_{n}\}$ and the corresponding local feature set $F=\{f_{n}\}$. We aim to have the optimal feature set $\hat{F}$ which can be fully utilized both for global place recognition and relocalization. Here, let $C$ be the ground truth correspondence of $L^{A}$ and $L^{B}$ detected in scenes $A$ and $B$ respectively. In order to obtain $\hat{F}$, it is necessary to estimate $C$. However, as $C$ remains unknown in usual, estimating $C$ is the foremost thing that we need. Assume we have $H$, which is a function that finds correspondences between two point sets. Then $\tilde{C}$ that estimate $C$ of ${L^{A}}$ and ${L^{B}}$ is represented as the following:

where $F^{A}$ and $F^{B}$ is local feature extracted from $L^{A}$ and $L^{B}$ respectively. To solve this data association problem, unbalanced optimal transport can serve as one of the $H$ function for correspondence estimation [5, 27, 28]. The $\tilde{C}$ derived through unbalanced optimal transport is presented as the following:

Using Eqn. (2), existing methods obtain $\tilde{C}$ and estimate optimal local features subsequently. However, when estimating $\tilde{C}$ through unbalanced optimal transport, it is challenging to consider prior information such as geometric characteristics of $L$, because the existing approach simply applies uniform weights $U$ to every possible correspondenses of $L^{A}$ and $L^{B}$. This limitation makes it hard to consider true positive and false positive, which is crucial for relocalization, and furthur global place recognition as well.

Therefore, we propose a method that enhances both global place recognition and relocalization performance by strengthening the true-positive correspondences and weakening the False-positive ones by considering geometric correlation. By using robust correspondence estimation based on the necessity of feature points, it is easier to find the feature association for local points. Our approach can lead to fewer outliers and improved accuracy in 6-DOF when calculating RANSAC or SVD, reducing the error in metric pose and relocalization. Additionally, as the local feature correspondence was improved, “local consistency” can be maintained, leading to an improvement in the performance of global place recognition [8].

The overview of our proposed method is displayed in Fig. 2. First, when the current points are provided, we consider areas with repetitive and unnecessary geometrical features as the ground and preprocess those sections. Then, local features are extracted from the preprocessed data using the local feature extraction module. At the same time, overlapping points distance are found by performing ICP or a better align algorithm on the preprocessed data between the current point cloud and positive. The overlapping points found through the local features and the align algorithm are then estimated through weight unbalanced optimal transport for correspondence, and the pose is obtained through SVD. The obtained pose is then trained through correspondence auxiliary loss and pose loss. Additionally, the local features are used to generate a global descriptor through the aggregation module, NetVLAD [23]. The generated global descriptor can be trained through triplet loss.

In this paper, we presented a method of robust feature correspondence estimation for improving 3D place recognition and relocalization. We proposed a probabilistic technique to enhance the necessary features and suppress the unnecessary features simultaneously in the estimation of correspondence, by considering the geometric characteristics of the 3D scenes. To weakening the strong unnecessary correspondence, we first showed that features on the ground can act as outliers when estimating correspondence therefore should be preprocessed. We also found the key correlations through the geometric registration method to estimate robust correspondence by focusing on necessary features which can enhance both global place recognition and relocalization. We provided the evaluation results to show improvement performances in the place recognition problem on the KITTI odometry dataset and KITTI-360 dataset, an urban dynamic environment dataset. Our approach demonstrated significant improvement in both 3D point cloud registration and global place recognition performance.