N³-Mapping: Normal Guided Neural Non-Projective Signed Distance Fields for Large-scale 3D Mapping

A signed distance field represents the 3D scene by assigning a signed distance to each point in space relative to the closest surface. The sign of the distance indicates whether a point is inside or outside the surface. The signed distance function $f$ can be defined as: $f(\mathbf{x})=s$, which maps a 3D coordinate $\mathbf{x}$ to the signed distance value $s$.

Notably, the gradient of SDF $\nabla_{\mathbf{x}}f(\mathbf{x})$ points towards the closest surface and on the surface the gradient is equal to the surface normal: $\nabla_{\mathbf{x}}f(\mathbf{x})=\mathbf{n}$ (or $-\mathbf{n}$ depending on the normal’s orientation). This implies that normal priors can provide informative guidance for neural SDF learning.

In this paper, we utilize octree-based hierarchical sparse voxels to store learnable features at each node vertex and a shallow MLP to decode hidden features into SDF values. Following SHINE-Mapping [11], we build a hash table with Morton codes as keys to query features efficiently and facilitate map scalability. For an arbitrary point $\mathbf{x}$ within the map, its corresponding local feature vector at a given level $l$ can be obtained through trilinear interpolation function $TriLerp(\cdot)$ with its eight neighboring feature vectors $\{\mathbf{v}_{k}^{l}\}$. Then different levels of features are aggregated by summation and then fed into a shallow MLP $f_{\theta}$ with globally shared weights to decode the SDF value $s$:

We denote this function as $s=f_{\theta}(\mathbf{x})$ for brevity in the following sections. Since the entire process is differentiable, we can optimize the feature vectors and the parameters of the MLP jointly using the generated signed distance value as supervision.

Ideally, we aim to use true signed distance values to supervise the learning of our implicit map. As discussed in Section II, existing methods often resort to projective distance as SDF approximation, which is obtained by sampling points along the ray $\mathbf{r}$ and calculating the distance $s$ from the sampled point $\mathbf{x}_{r}$ to the endpoint $\mathbf{p}$:

This distance can be further projected along the surface normal $\mathbf{n}$ to reduce the approximation error [13, 14]:

However, this solution struggles to deal with curved or irregular surfaces, which are common in real-world scenarios. As illustrated in Figure 2, both the projective distance (blue line) and corrected distance (yellow line) still remain a substantial gap to the true distance (red line).

As discussed in Section III-A1, along the negative normal direction, the sampled points direct toward the closest surface, aligning with the gradient direction of the distance field. Therefore, we propose to directly sample points along the normal direction $\mathbf{n}$ and use the signed distance from the sampled point $\mathbf{x}_{n}$ to the surface point $\mathbf{p}$ as the supervision:

Such sampled SDF values are quite close to the ground truth. However, as the sampling distance increases, this approximation becomes less reliable because some sampled points may be closer to other surfaces. Hence, we set a truncation interval $[-tr,tr]$ around the surface and only perform normal-guided sampling inside this region according to the Gaussian distribution $\mathcal{N}(0,\sigma)$, where $\sigma$ is a hyperparameter that indicates the magnitude of the measurement noise. We also sample points in free space along the ray between the sensor and the truncation region. The SDF labels of these points are set to the truncation value $tr=3\sigma$. This primarily aims to eliminate potential artifacts caused by dynamic objects, without compromising the quality of surface reconstruction.

We apply the binary cross entropy (BCE) loss function like SHINE-Mapping [11] since it inherently enhances the importance of SDF labels that are close to zero. First, we map the SDF value to the occupancy probability within the range of (0, 1) through the sigmoid function: $S(x)=1/(1+e^{-\frac{x}{\beta}})$, where $\beta$ can control the reconstruction sharpness. Given sampled point $\mathbf{x}_{i}$ with SDF label $s_{i}$, forming a training pair, we use its corresponding occupancy value $o_{i}=S(s_{i})$ as supervision. Then, the BCE loss can be formulated as follows:

where $\hat{o}_{i}=S(f_{\theta}(\mathbf{x}_{i}))$ represents the predicted value of our implicit model after the sigmoid mapping.

Furthermore, if $\mathbf{x}_{i}$ is inside the truncation region, we add an eikonal term to ensure valid and continuous SDF values. This term serves as a regularization to encourage the SDF gradient w.r.t. 3D query point coordinates to have a unit length:

In summary, the final loss function is designed as follows:

where $\lambda_{e}$ represents the weight of the eikonal term. It is worth noting that we do not explicitly supervise the SDF gradient prediction using surface normals as done in [30, 31]. This is because the normal information has already been implicitly encoded in the SDF labels through the preceding normal-guided sampling process. Consequently, this simplified training process can lead to improved mapping efficiency.

Current feature grid-based methods [25, 10, 9] often select recently observed keyframes from the global set for efficient local optimization, similar to the sliding window method employed in traditional SLAM systems. However, as illustrated in Figure 3, this keyframe-oriented sliding window approach omits observations from frame $T_{n-3}$ for voxel $V_{0}$. It has been revealed that optimizing local features using such partial observations can lead to non-smooth reconstruction results [11].

To tackle this issue, we propose a voxel-oriented sliding window strategy, in which each observation is associated with a voxel instead of a keyframe. Historical data is accumulated in their corresponding voxels by hash querying. Benefiting from this design, each voxel within the local window can retain complete supervision signals for training. For simplicity, we define the sliding window as a cube within the octree structure of our implicit map (as detailed in Section III-A2). The local window is centered at the current sensor’s origin and its main axes are aligned with the global octree. Given the sensor origin coordinates $\mathbf{o}$ and the maximum perception range $r$, the 3D bounding box of this cube in voxel units can be derived as follows:

where $v$ is the leaf voxel size, $\lfloor\cdot\rfloor$ denotes the floor function, $\mathbf{o}^{v}$ is the voxel coordinate of the sensor origin, $r^{v}$ is the perception range in voxel units, and $[\mathbf{b}_{u}^{v},\mathbf{b}_{l}^{v}]$ represents the upper and lower bounds of the 3D bounding box in the voxel coordinate frame. This local window allows our approach to focus on mapping newly explored regions with limited memory usage while avoiding the forgetting issue. Moreover, to ensure global consistency, the MLP decoder will be frozen once convergence is achieved with a certain number of initial frames.

We note that the point cloud measurements tend to be unevenly distributed on the surface. If we simply employ random sampling at each training iteration, regions with higher point cloud density are more likely to be trained, while sparsely observed regions lack sufficient optimization to achieve complete convergence.

To address this problem, we adopt a hierarchical sampling strategy. In our implementation, each voxel maintains a block of training pairs, which includes query points and corresponding SDF labels, stored in an array. These arrays are organized in the list $L_{P}$. The association between each voxel and its index in $L_{P}$ is maintained through a look-up table $L$. As detailed in Algorithm 1, our sampling strategy involves two stochastic stages. In the first stage, we randomly sample $N_{v}$ voxels within the sliding window. This ensures a spatially uniform voxel-wise sampling, independent of the point cloud’s distribution. Then, within each sampled voxel, we further sample $N_{p}$ training pairs. If a voxel contains pairs fewer than a threshold $N_{t}$, we reduce the sampling number until it accumulates more observations. After collecting all sampled training pairs, we compute loss functions as described in Section III-B2. In practice, we perform hierarchical sampling in a parallel manner to accelerate this process.

For comparison, we choose two advanced TSDF integration-based methods, Voxblox [2] and Voxfield [20], along with two state-of-the-art implicit representation-based methods, SHINE-Mapping [11] and NeRF-LOAM [13], as our baselines. The odometry module of NeRF-LOAM is omitted since our focus lies in mapping performance. All methods run incremental mapping using their official open-source code with ground truth poses and the same voxel size.

To quantitatively evaluate mapping results, we adopt standard reconstruction metrics including Accuracy (Acc., $cm$), Completion (Comp., $cm$), Chamfer-L1 distance (C-L1, $cm$), Completion Ratio (Comp.Ratio, $\%$), and F-score ($\%$). Considering that the observed regions may not fully cover the ground truth model, the unobserved portions will be culled before evaluation. The final mesh used for evaluation is generated by marching cubes on the same fixed-size grid to ensure fairness.

Our method takes as input point-wise normals which are estimated using Open3D [32] with k-nearest neighbors set to 20. For implicit map representation, feature vectors are stored in the lowest three levels of the octree, each with a length of 8. The leaf voxel size is set to $0.2m$ and the truncation distance is $0.3m$. We sample 3 points near the surface and another 3 in free space. Our shared MLP decoder contains 2 fully-connected layers with ReLU activations, and each layer has 32 hidden units. For training, we set $Nv=1024$, $N_{p}=8$ and the sharpness parameter $\beta=0.1$. All experiments are conducted on a desktop PC with a 3.7 GHz Intel i9-10900X CPU and an NVIDIA RTX3090 GPU.

In our first experiment, we evaluate the mapping quality on the simulated MaiCity dataset [33], which provides both 64-beam noise-free LiDAR scans and a ground truth map of an urban scenario. Table I presents a quantitative comparison of our method and baselines. Our method achieves the best performance across all metrics. An intuitive qualitative demonstration is shown in Figure 4. It is evident that our approach produces the most accurate and complete reconstruction. For traditional methods, Voxblox [2] and Voxfield [20], the meshes obtained by them appear overly smoothed, lacking the preservation of details, like the tree and the pedestrian. SHINE-Mapping [11] uses a regularization-based continual learning strategy but still encounters the forgetting issue, resulting in an incomplete and non-smooth mesh. NeRF-LOAM [13] achieves good ground reconstruction, due to its ground separation strategy. Nonetheless, our method shows superior overall mapping accuracy, particularly in fine-grained structures and sparsely observed marginal regions.

We also choose the real-world Newer College dataset [34] for evaluation. This dataset comprises a hand-carried LiDAR sequence captured at Oxford University with noticeable measurement noise and motion distortion. A high-precision point cloud model observed by a terrestrial scanner is used as pseudo ground truth. In accordance with NeRF-LOAM [13], we take one out of every five scans for mapping in all methods. This makes the task more challenging. The quantitative results and qualitative results are shown in Table II and Figure 4, respectively. It can be seen that our approach outperforms all baseline methods on this noisy dataset. Voxblox and Voxfield struggle with sparse observations and dynamic objects, producing large holes on the ground, and wrongly eliminating the tree region. SHINE-Mapping and NeRF-LOAM can produce relatively complete maps. But the former shows non-smooth reconstruction and large errors, even on flat ground. The latter tends to produce fragmented artifacts in edge areas and small holes on the ground. Our approach effectively balances reconstruction accuracy, completeness, and smoothness, yielding the best mapping performance.

We compare the performance of our method with and without normal guided sampling. The results shown in Table III demonstrate that our normal guided sampling method significantly improves mapping accuracy and completeness. The corresponding visual evidence can be seen in Figure 5 (a1) and (b1). The area highlighted by the orange box illustrates that our method manages to avoid overestimating the signed distance with large incidence angles, leading to more valid distance fields. We further evaluate an alternative option that uses normals to correct the signed distance values for training [13], as depicted in Figure 5 (c1). The results show that it can provide some improvement over using the projective distance, but our reconstruction is more continuous and complete.

We adopt a naive replay-based method as the default baseline, which randomly samples data from all historical keyframes to train the implicit map together with current measurements. Table III shows that the reconstruction errors significantly decrease when using the voxel-oriented sliding window. This improvement is particularly evident in newly observed regions, as marked by the orange circle in Figure 5 (a2) and (b2). This is because our method can rapidly achieve convergence within the sliding window, while the replay-based baseline dilutes the training attention to newly observed regions by historical data. We also provide the results of applying a keyframe-oriented sliding window for comparison. As shown in Figure 5 (c2), this design can enhance the reconstruction quality of the current local map. However, it leads to inconsistent reconstruction in historically visited areas, as highlighted by the red circle, due to the forgetting issue. In contrast, our method avoids this problem since each voxel retains complete supervision from various viewpoints.

We compare the performance of our method with and without hierarchical sampling across different iteration settings to analyze its impact on training efficiency. The results in Figure 6 show that as the number of iterations increases, mapping performance gradually improves, reaching convergence at around 40 iterations. Notably, the mapping quality with hierarchical sampling is better even with fewer iterations. This illustrates the effectiveness of this strategy in enhancing training efficiency.