LGmap: Local-to-Global Mapping Network for Online Long-Range Vectorized HD Map Construction

There are mainly two types of view transformation, forward projection and backward projection. Lift-Splat-Shoot (LSS)[4] takes advantage of the depth distribution to model the uncertainty of each pixel’s depth. But the drawbacks of forward projection is discrete and sparse BEV representation. BEVFormer [5] projects 3D points back onto 2D images. As a backward projection, one limitation of BEVFormer is false correlation between 3D and 2D space due to occlusion. To address these issues, we introduce a symmetric view transformation. The depth-map of each camera is generated from synchronized lidar point cloud. The LSS utilize depth supervision only at the training phase. Given the SD map of the scene, we evenly sample along each of the polylines for a fixed number of points. With sinusoidal embedding, BEVFormer apply cross-attention between the SD map feature representation with features from vision inputs on each encoder layer. In order to fuse BEV representations, we use the channel-attention-based fusion module.

In order to handle different map elements with distinct shape priors, we extend the instance-wise detection decoder with additional segmentation tasks. The unified transformer-based decoder for instance detection and segmentation benefits from both pixel-level classification task and region-level regression task. Additional segmentation branches accelerate the convergence performance of the instance-wise feature embedding.

The streaming strategy facilitates longer temporal association as the propagated hidden states encode all historical information. But a temporal fuser such as convGRU [6] may still face the problem of forgetting. The stacking strategy may integrate features from specific previous frames, offering flexibility in fusion of long-range information. The computational cost is linearly related to the number of fused frames. We propose a novel hierarchical temporal fusion (HTF). The hierarchical temporal fusion fully leverage local fusion capability of streaming strategy and long-range fusion capability of stacking strategy. And it minimizes memory and latency costs compared to the stacking strategy. Here we present two variants of HTF, streaming-streaming strategy and streaming-stacking strategy, as shown in Fig. 2. For streaming-stacking strategy, we random select N frames from the latest M previous frames for the stacking mode layer during the training phase. And select N frames by a certain distance strides during testing phase.

Firstly, we adopt classification loss, point2point loss and edge direction loss same as MapTR [8]. Secondly, we adopt image segmentation auxiliary dense prediction loss and depth prediction loss same as MapTRv2 [9]. Thirdly, we adopt BEV instance segmentation loss. Lastly, we adopt geometric 3D loss. Unlike the geometric loss of GeMap [10], which ignores the Z-axis, we extend the euclidean loss dimension from 2d to 3d.

We examine the efficacy of SVT component through ablation studies, utilizing the OpenlaneV2 dataset [1]. Starting with BEVFormer [5] and LSS [4] as baseline, the best score is 40.36% on the validation set, as shown in Table 1. Compared to the best baseline, the integration of BEVFormer and LSS increase 0.5% mAP. After adding image data-augmentation and BEV data-augmentation, the model performance has improved to 43.75%.

We build a single-frame baseline model by training 72 epochs with ResNet50 checkpoints. And then all experiments finetune the baseline model by 12 epochs. We use single-frame mode to finetune the baseline model, model can reach a score of 52.93% mAP, as shown in Table 2. For the streaming strategy, we use one convGRU [6] as dense fusion encoder. It has a performance improvement of 3.7%. And for the streaming-streaming strategy, two layers of convGRU are used instead of one. Unfortunately, the performance increase only 0.56% compared to single-frame. For the streaming-stacking mode, we select 4 frames out of latest 10 frames for the layer of stacking mode during training phase, and a certain distance strides of 5, 10, 15, 20 meters during testing phase. the performance reaches 57.13% mAP.

We use the hierarchical attention based decoder same as MapTR as baseline. The model performance reaches score of 33.6% DET-a, as shown in Table 3. Then we change the decoder to instance attention. The model performance increase 0.45%. Finally, we utilize ped crossing resampling to improve the performance to 35.42%

We utilize COCO pretrained model and finetune for 40 epochs as our 2D detector baseline. The dataset is resampled by a ratio from 5 to 20 times. The entire model is optimized by AdamW with a learning rate of 0.04 and resolution of $1568\times 2048$. And then we generate pseudo-labels by threshold of 0.3. YOLOv8-x with data augmentation can reach a score of 79.42% on DET-l, as shown in Table 4. The application of resampling has a performance improvement of 0.64%. TTA further improves 1.0% score. We utilize pseudo label to improve the performance to 81.81%. Finally, the model ensemble of YOLOv8 and YOLOv9 [3] improve the performance to 82.4%.

We train three versions of the models, using different backbones (ViT [14], InternImage-XL [16]) with different input image resolution scales (0.5, 0.75, 1). During the ensemble process, we utilize an ensemble strategy that incorporates predictions with low similarity. Initially, the models are sorted by their evaluation scores, the best model is the base model, and the other two models are subsequently integrated as proposal models. From the Table 5, it can be seen that the more models ensembled, the more remarkable performance improved.