Towards Better Performance and More Explainable Uncertainty for 3D Object Detection of Autonomous Vehicles

Unlike the organized pixel values in images, point cloud provides irregular data which prevents the direct application of classical convolutional neural networks (CNN) such as VGG, ResNet. To tackle this issue, many researchers preprocess the point cloud data by reorganizing it into 2D or 3D grids [2, 8, 15, 16, 17]. However, voxelization resulted in large size of the input and high computation cost until the 3D sparse convolution method was proposed [16, 18]. Moreover, the grid-based methods potentially cause information loss of point cloud and have limited receptive field. To make use of lossless point cloud information, PointNet-based architectures are proposed to extract point-wise and global features from the point cloud [5, 19, 20]. There are several 2-stage object detectors based on PointNet such as PointRCNN [7] and STD [21]. In this work, we propose a 2-stage method that utilize point-wise features for region proposal and refinement following PointRCNN.

To represent the uncertainty of neural networks, Bayesian modeling is proposed to estimate the epistemic and aleatoric uncertainties [22, 23, 24]. Epistemic uncertainty is model-based which arises from the uncertainty of model parameters. It reflects the limitation of the model on describing the biased training data and can be reduced by enlarging the dataset [24]. There are two main ways to predict the epistemic uncertainty: variational inference [25] and sampling [26]. The aleatoric uncertainty, on the other hand, is measurement-based which arises from the sensor noise, data representation and label noise, etc. [27]. It can be predicted by outputting the parameters of a distribution and the method is adopted in this paper. Recently, many efforts have been put on estimating the uncertainty of 3D object detection. [13] and [28] utilized Monte Carlo dropout to capture the epistemic uncertainty of the bounding box. While [4] and [14] predicted the aleatoric uncertainty by learning the parameters in the probabilistic distribution of bounding boxes. [10] and [11] further pre-estimated the uncertainty of the label and learned the probabilistic distribution by minimizing the KLD of predicted from preset ones. In this work, we estimate the aleatoric uncertainties of the corners from bounding boxes by learning the parameters of the probabilistic distribution and analyze how they represent the distributions of point clouds and are constrained by the cuboid geometry. Finally, we propose a Bayesian update method to recover the uncertainty of the original parameters in labels.

As shown in Fig. 1, we take raw point cloud data as input rather than voxelizing it. In the region proposal network (RPN) stage, we generate 3D regions of interest (ROIs) using the point-wise and global features extracted by a PointNet-like architecture. After ROI-pooling, we feed the pooled point features along with the original point coordinates and intensities to the point cloud encoder of PointNet++ [20] to learn the representation of the point cloud. The encoder is followed by three task-specified fully connected (FC) layers which output the classification scores, box residuals and the uncertainties of the locations of box corners, respectively. The FC layers only predict the residual of the bounding box. The final bounding box is recovered by combining the residual and the preset anchor of the ROI pooling layer together.

The labels of KITTI dataset describe the ground truth bounding box with 7 parameters: 3 for the center location, 3 for the dimensions and 1 for the orientation. The orientation is described by the yaw angle of the object. Accordingly, our ROI box and final refined bounding box are all parametrized by $[x,y,z,h,w,l,\psi]$ in which $[x,y,z]$ denotes the location, $[h,w,l]$ denotes the dimensions and $\psi$ is the yaw angle.

A corner-based equally weighted regression loss is proposed to enrich the representation capability of uncertainties. Rather than directly predicting the coordinate of each corner, we keep the original expression of the bounding box and transformed it to the corner coordinates in camera coordinate system while retaining the cuboid constraints of them.

Equation 1 is the transforming function from the original label parameters to the corresponding 8 corners.

$$\begin{bmatrix}x_{c}\\ y_{c}\\ z_{c}\end{bmatrix}=\begin{bmatrix}\sin\psi&0&\cos\psi\\ 0&1&0\\ -\cos\psi&0&\sin\psi\end{bmatrix}\begin{bmatrix}\pm l/2\\ \pm h/2\\ \pm w/2\end{bmatrix}+\begin{bmatrix}x\\ y\\ z\end{bmatrix}$$

(1)

in which $[x_{c},y_{c},z_{c}]^{T}$ is the location of the corner, $l,h,w$ are the dimensions of the bounding box, $\psi$ is the yaw angle and $[x,y,z]^{T}$ is the location of the box center. We transform both the label and recovered bounding box for loss calculation.

Denote the procedure from sensing to annotation as a measurement, and the predicted bounding box as the mean value, we assume the components of the coordinates of corners to be drawn from independent univariate Laplace distributions with a probability density function defined as follows:

$$p(x|\mu,b)=\frac{1}{2b}\exp{(-\frac{|x-\mu|}{b})}$$

(2)

in which $\mu$ is the predicted transformed component, $b$ is the predicted diversity of the Laplace distribution. The variance of the Laplace distribution equals to $2b^{2}$.

Then we take the negative log likelihood as the loss function for each single component of the corner:

$$\mathcal{L}(x,\mu,b)=-\log{p(x|\mu,b)}=\ln{2b}+\frac{|x-\mu|}{b}$$

(3)

Finally, we calculate the ensemble regression loss of the 3D bounding box by summing up the loss of all the components from all corners:

$$\mathcal{L}_{ens}=\sum_{i}\sum_{j}\mathcal{L}(x_{ij},\mu_{ij},b_{ij})$$

(4)

in which $i$ is the index of the corner, and $j$ is the index of the component of the corner.

Since the form of loss is negative log likelihood, summing up the losses is equivalent to multiplying the probability density defined in (2). In other words, by minimizing the ensemble loss, we are maximizing the likelihood of labeled corners under the Laplace distribution with parameters $\mu$, $b$.

We first compare the performance of our method with the state-of-the-art methods on KITTI test set. We pick some representative LiDAR-based methods listed in Tab. I. We highlight the comparison of performance between our method and PointRCNN which uses bin-based loss function and serves as the base for our method.

All results are evaluated by the average precision (AP) with a 3D intersection over union (IoU) threshold of 0.7 on easy, moderate, and hard difficulty levels respectively. The AP of our method on test set was calculated on official KITTI server.

As shown in Tab. I, our method achieves a comparable level of performance with the methods listed and surpasses most of them. When compared with the base network PointRCNN, our method has a comparable performance with it at easy difficulty level. With the increase of the difficulty, our method surpasses PointRCNN in terms of AP by 1.23% at moderate and 2.47% at hard difficulty levels which indicates not only a better performance but also higher robustness for less informative samples.

To find out how corner transformation and uncertainty modeling affect the performance of the model, we perform an ablation study in different cases on the val split. As shown in Tab. II, we set a baseline whose loss function was simply L1 form without corner transformation and uncertainty modeling. The baseline with corner transformation is proceeded with the L1 loss calculation on the components of transformed corners. The baseline with uncertainty adopts only the aleatoric uncertainty modeling without any other modifications following [14]. To make a fair comparison, the results of different cases share the same RPN result and follow the same training procedure. We also add the performance of PointRCNN on val split coming from [7] for comparison.

As shown in Tab. II, corner transformation and uncertainty modeling improve the performance of the model by 11.62% and 11.45% respectively on moderate difficulty level from the baseline. While our method with both the above features improves the AP by 16.23%, 14.60% and 18.27% on easy, moderate and hard difficulty levels respectively from the baseline. The corner transformation increases the performance of the model by re-weighting the original parameters of the bounding boxes and transforming them into equally weighted corners. That’s because the corners contribute equally in representing the bounding box such as IoU calculation, while the original parameters are not. In that case, even with simple L1 loss function, corner transformation considerably increases the performance of the model from the baseline. On the other hand, uncertainty modeling helps to increase the noise tolerance of the detector which means the noisy label affected less on updating the network weights due to the diversity of Laplace distribution. Combining these two methods, our method increases the performance by 0.4%, 1.99% and 1.75% on easy, moderate and hard difficulty levels respectively from PointRCNN with bin-based loss on val split.

In this section, we analyze the characteristics of the predicted uncertainty starting with its general behaviors. Then, we discuss the relationship between the corner uncertainty and the point cloud distribution. Finally, we introduce how cuboid constraint affects the distribution of the corner uncertainties.

To recover the yaw angle, we pick the edge that was parallel to the orientation of the car and utilized its two vertices as the corner pair. Denote the coordinates of the two corners in $x-z$ plane are $(x_{i},z_{i})$ and $(x_{j},z_{j})$. Then the yaw angle:

$$\psi=\arctan\frac{z_{i}-z_{j}}{x_{i}-x_{j}}$$

(9)

Applying the error transfer formula, we can obtain the variance of the yaw angle:

	$\displaystyle\sigma_{\psi}^{2}$	$\displaystyle=\left|\frac{\partial{\psi}}{\partial{x_{i}}}\right|^{2}\sigma_{xi}^{2}+\left|\frac{\partial{\psi}}{\partial{x_{j}}}\right|^{2}\sigma_{xj}^{2}+\left|\frac{\partial{\psi}}{\partial{z_{i}}}\right|^{2}\sigma_{zi}^{2}+\left|\frac{\partial{\psi}}{\partial{z_{j}}}\right|^{2}\sigma_{zj}^{2}$		(10)
		$\displaystyle=\frac{|z_{i}-z_{j}|^{2}(\sigma_{xi}^{2}+\sigma_{xj}^{2})+|x_{i}-x_{j}|^{2}(\sigma_{zi}^{2}+\sigma_{zj}^{2})}{\left[(x_{i}-x_{j})^{2}+(z_{i}-z_{j})^{2}\right]^{2}}$

Similarly, we can obtain the uncertainties of the dimensions of the box

$$\sigma_{d,k}^{2}=\frac{\sum_{k}(c_{i,k}-c_{j,k})^{2}(\sigma_{i,k}^{2}+\sigma_{j,k}^{2})}{\sum_{k}(c_{i,k}-c_{j,k})^{2}}$$

(11)

and the uncertainty of the location of the box

$$\sigma_{loc,k}^{2}=\frac{1}{2}(\sigma_{i,k}^{2}+\sigma_{j,k}^{2})$$

(12)

in which $c_{k},k\in\{1,2,3\}$ is the component of the corner.

With the variance obtained from the measurements discussed above, we can use Bayesian update method to approximate the final variance with (13) [30].

$$\sigma_{bayesien}^{2}=\frac{\prod_{i}\sigma_{i}^{2}}{\sum_{j}\prod_{i\neq j}\sigma_{i}^{2}}$$

(13)