DSOR: A Scalable Statistical Filter for Removing Falling Snow from LiDAR Point Clouds in Severe Winter Weather

Over the past decade, several annotated datasets have been released with LiDAR scans to aid with the development of AV perception tasks [15]. Few include multi sensor winter weather data with extreme events necessary for training NN models. The lack of inclement weather data has been addressed in some literature by adding artificial corruption to existing datasets [16, 17]. Pfeuffer and Dietmayer [9] show that models trained on large datasets such as KITTI [18], fail to perform well when artificial rain or snow is introduced implying that availability of adverse data takes precedence over the size of the dataset.

The KITTI[18] and nuScenes[5] datasets provide LiDAR scans annotated with bounding boxes while SemanticKITTI [4] and nuScenes-lidarseg datasets provide point-wise annotations. These datasets, however, provide no data in inclement weather. The ApolloScape dataset [6] includes LiDAR scans with a semantic mask to extract point-wise annotations. The DENSE dataset [7] includes rain, fog, and snow but the conditions do not rise to the level of being severe. Moreover, annotations in DENSE are limited to bounding boxes and not point-wise labels. The CADC dataset [8] includes adverse weather data collected in Canada with bounding boxes around vehicles and pedestrians. Our dataset provides point-wise annotations and have been collected in consistent severe conditions. Table I provides an overview of relevant datasets compared with ours.

Noise introduced by snow particles in LiDAR scans can be filtered out in a number of ways; [1] and [14] introduce intensity based filters. These filters assume LiDAR returns from snowflakes will have a lower intensity and remove snow noise by intensity-value thresholding. However, intensity is dependant both on the laser wavelength and target reflectance and, as shown in [13], backscatter from snow particles near the sensor can be significant. To overcome this limitation, Park et al. [1] propose a second clustering stage based on point density to preserve objects of interest. However, in heavy snow with a high volume density clustering techniques will classify snow as objects of interest.

Another filtering technique is to apply image processing on LiDAR point clouds. Here, point clouds are converted to a 2D representation and techniques like Gaussian filters or their derivatives are applied to remove noise. However, as shown by Charron et al. [11], 3D point clouds are sparse and such approaches not only fail to effectively remove snow but also have the negative effect of smoothing edges in keypoint features. Duan et al. [19] use Principal Component Analysis (PCA) to convert 3D point clouds to 2D and apply DBSCAN clustering to find sparse regions and remove them. However, as mentioned previously, clustering techniques fail at high snowfall rates.

Filtering can also be applied to the 3D points directly. The Point Cloud Library (PCL) [20] includes general purpose filters such as the voxel grid filter, Statistical Outlier Removal (SOR) filter and Radius Outlier Removal (ROR) filter; these are, however, not designed for snow. They fail to account for the non-uniformity of the point clouds and hence Charron et al. [11] propose the Dynamic Radius Outlier Removal (DROR) filter for removing snow corruption. Their filter dynamically changes the search radius with range to effectively remove snow. In this work, we compare our DSOR filter with the DROR filter, which we consider to be the state of the art.

Bounding boxes provide vector annotations and often include undesired background objects which can be detrimental for AV perception tasks such as semantic segmentation. We have opted for point-wise labels as they are more precise and enable fine details in the environment to be highlighted, such as individual snowflakes. To maintain compatibility with existing systems and ensure adoption of inclement weather data into existing frameworks, we adopt the popular KITTI format [21]. We leverage the point-cloud labeling tool introduced by Behley et al. [4]. This is a manual labeling tool that can load point clouds either as multiple scans for quick labeling or as single scans for fine details. Fig. 2 shows a labeled scan from our WADS dataset.

Every point in a LiDAR scan has been labeled into one of 22 classes as shown in Fig. 3. Over 7 GB or 3.6 billion points have been labeled in all. Here, classes are grouped into categories for easy viewing. It can be seen that the majority of labeled points lie in urban driving scenarios with roads, buildings and various types of vehicles representing most of our labeled data.

In addition to these classes, we introduce two new labels to represent snow. active-snow captures falling snow particles and associated clutter noise in a LiDAR return. accumulated-snow captures snow that builds up on the sides of drive-able surfaces from vehicle traffic and snow removal. Accumulated snow often changes, sometimes through a day, enough to confuse feature-based algorithms. Access to such data will be useful for AV tasks like object detection, localization and mapping, and semantic and panoptic segmentation in adverse weather.

The DSOR filter is an extension of PCL’s SOR filter, designed to address the inherent non-uniformity in point clouds. Algorithm 1 helps visualize our approach. The point cloud is first loaded into a k-d tree and the k-nearest neighbor search is applied for every point to accumulate user provided k neighbors. The mean ($\mu$) and standard deviation ($\sigma$) of the mean distances are calculated to compute a global threshold given by equation 1.

To account for the non-uniform spacing of points, the distance of every point from the sensor is calculated and a new dynamic threshold ($T_{d}$) is computed using equation 2

Points with mean distances less than $T_{d}$ are considered to be outliers and removed. Here, r is a constant, a multiplicative factor for point spacing. Increasing r will increase $T_{d}$ and will result in the filter rejecting fewer points, whereas, reducing r will decrease $T_{d}$ resulting in a more aggressive filter. The DSOR filter, when applied to a snow corrupted point cloud, produces the cleanest point cloud as compared to the other approaches as shown in Fig. 4 (right most).

Fig. 4 shows a qualitative comparison of the original point cloud, corrupted by snow, passed through different filtering techniques. The SOR filter fails to remove most of the snow and removes key environmental features. The DROR filter does removes most of the snow while preserving environmental features. The DSOR filter removes even more snow detections from the point cloud while preserving key points removed by the SOR filter.

To understand the visual performance of the filters, the percentage of filtered points as a function of range have been plotted in Fig. 5.

DSOR vs SOR filter: Fig. 5(a) shows that the DSOR filter removes significantly more points than the SOR filter in the first 20 meters where most of the LiDAR interaction with snow particles occurs. Beyond 20 meters, the DSOR filter removes significantly fewer points than the SOR filter thus preserving key environmental features.

DSOR vs DROR filter: Fig. 5(b) shows that in the first 20 meters the DSOR filter also removes more points than the DROR filter. However, beyond 20 meters, the DSOR filter removes more points potentially leading to a lower precision at range.

In the context of this work, the aim of a filter must be to remove as much snow as possible (recall) and leave other points untouched (precision). It is possible to filter out all of the snow (recall of 100%) at the expense of environmental features (low precision) using any filter. This, however, is not the aim of this work. Both the DSOR filter and the DROR filter have been tuned to filter most of the snow and preserve as much of the environment as possible as shown in Fig. 4.

As shown in Fig. 6, the DSOR filter achieves a greater recall (95.6%) than the DROR filter (91.9%). This shows that the DSOR filter removes more of the snow corruption correctly than the DROR filter. The DROR filter, however, achieves a greater precision (71.5%) at filtering snow as compared to the DSOR filter (65.1%). This shows that the DSOR filter removes more non-snow points or environmental features than the DROR filter. From Fig. 5(b) it is evident that the loss in precision for the DSOR filter stems from removing points beyond 20 meters. At this range, we observe that features such as tree-tops, poles and overhanging power lines have been filtered out. Such features are sparse, and in the winter can return just a few points. Decrease in precision could also stem from removal of noisy measurements.

Table II compares the time taken to filter a point cloud by the DSOR, SOR and DROR filters. Results have been averaged over 100 point clouds collected by a 64 channel LiDAR with roughly 200k points each. The DSOR filter is, on average, 141 ms or 28% faster than the state of the art DROR filter.

As LiDAR’s continue to evolve, sensors with higher point density and number of lasers are expected. State of the art AV’s also concatenate returns from multiple LiDAR’s for increased visibility and dense spatial sampling. This increases the number of points being captured and processed in the pipeline. For this reason, filters need to scale with the size of point clouds to ensure adaptation on AV systems. To compare the time taken for filtering by the DSOR filter and DROR filter, multiple levels of sub-sampling were applied to a sequence of point clouds from WADS. Fig. 7 shows that the DSOR filter has a lower execution time as compared to the DROR filter as the size of a point cloud increases. The DROR filter uses a radius search and we speculate that the DROR filter has an approximate time complexity of $O(N^{2/3})$ [22]. A curve fit on Fig. 7 is consistent with this assessment. The DSOR filter uses a k-nearest neighbor search and has a lower time complexity of $O(Nlog(N))$ implying that our filter will scale better with the size of LiDAR point clouds.