MachMap: End-to-End Vectorized Solution for Compact HD-Map Construction

(1) inter-element direction consistency. The directions of elements in original map annotations are in a state of chaos, such as lane-dividers of moving forward from front-to-back or back-to-front as shown in Fig.1 (a). We noticed that the inconsistency of directions can negatively affect the training of the model. To reduce the discreteness of map organization, we follow a certain strategy to make the orientation of map elements as orderly as possible, and guarantee that this process does not lose any details of the map. Specifically, under the principle of conforming to the observation order of human eyes, a simple and intuitive strategy is to reorganize all polylines according to the rules from-front-to-back and from-left-to-right in bird-eye-view space.

(2) intra-element sequence compactness. Vectorized maps with evenly-distributed points have redundant semantic information, while compacted-points representation is sparse, which is more suitable for expression and storage of maps. To this end, we extract keypoints for all elements to supervise model training. Concretely, we adopt Douglas-Peucker algorithm [11] and Visvalingam algorithm [12] to condense a polyline composed of line segments to a similar polyline with fewer points. For these methods, points are removed in order of least to most importance, with importance related to the distance and triangular area respectively.

Backbone. Giving a list of $2$D images $\mathcal{I}\in\mathcal{R}^{N\times 3\times H\times W}$, extracting unified textures representation within images is a top-priority task. With regards to this, we utilize a shared InternImage [13] as strong backbone to extract image features, which employs deformable convolutions [1] as its core operator and has been meticulously designed. During the downsampling process, a series of feature maps in varying scales are generated and then aggregated by the Bi-directional Feature Pyramid Network, i.e. BiFPN [16].

Multi-view Encoder. Since the map vectors we ultimately need to predict lay in $3$D space, it is necessary to elevate surrounding features from camera-view to $3$D ego-view space. Rather than direct transformation to $3$D-view, we predefine a set of reference points and arrange them in a BEV raster. After that, we employ the camera intrinsics and extrinsics to project them onto several images and then aggregate the surrounding features. By averaging on the z-axis, we obtain the final bird-eye-view features $\mathcal{B}\in\mathcal{R}^{H_{B}\times W_{B}\times C}$.

Temporal Fusion Module. The provided dataset is collected and organized chronologically, with precise poses for each sample. This makes it possible to align current features with previous ones by poses, resulting in a larger real-world perception range beyond the current position. We follow the long-term fusion strategy proposed in VideoBEV [3], which affines the previous hidden state $\mathcal{H}_{t-1}$ of BEV feature into the current one $\mathcal{B}_{t-1}$ using vehicle ego pose. The latter is concatenated with the current BEV feature $\mathcal{B}_{t}$ in the channel dimension and fused by a $1\times 1$ convolutional layer as,

where $\oplus$ denotes the concatenation operator. The fused features are cached as the next hidden state and used as input for subsequent instance decoder. In practice, since the timestamp offset between adjacent frame is too small, we group the timestamps at specific intervals to expand the performance gain brought by this temporal-fusion module.

Instance Decoder. To benefit from multi-task loss, we opt for the MaskDINO  [6] framework, which conducts object detection and segmentation tasks simultaneously. Each query consists of content and position vectors, with the former is utilized to generate instance masks, while the latter undergoes iterative updates to yield normalized coordinates directly. Yet, due to the hierarchical relationship between map elements and their corresponding points sets, we adopt the query design paradigm in MapTR [7] for better adaptation to map element modeling. This implies that the query is point-wise, and a set of which can be aggregated to form a single instance and obtain its corresponding instance mask.

Output Head. Using only coordinates from point regression has some drawbacks. Firstly, there is a keypoint mismatching issue, where a well predicted instance may occur a mismatched point which belongs to other instance, as a result, a single bad apple spoils the whole bunch. Secondly, for $ped$-$crossing$, there exists a strong geometric prior, which is difficult to depict through vectors. However, masks not only can effectively constrain the geometry shape of instances, but it also impose a significant penalty on mismatched points during training. Empirically, we obtain $ped$-$crossing$ and $lane$-$divider$ through post-processing of instance masks, while point regression is employed only for $road$-$boundary$. As the common practice, we adopt cross-entropy and dice loss [9] for masks and L$1$ loss for point regression. In addition, we also add semantic loss to the BEV features as auxiliary supervision, and our final loss as,

where $\lambda_{\star}$ is the balance weight for different losses.

Training Setup. We adopt common data augmentation, e.g. random scaling, cropping, and flipping. At the same time, an IDA [4] matrix is updated to record view transformation to maintain spatial consistency. Then the final input shape is fixed at $896\times 768$, as this aspect ratio is close to the front view, i.e. $2048\times 1550$, which contains the most abundant visual map information. For BEV features, the default spatial shape of BEV queries is $64\times 32$, which corresponds to the perception ranges in lidar coordinate system are $[-30,30]m$ for the Y-axis and $[-15,15]m$ for the X-axis. Note all map masks are interpolated to $400\times 200$ to ensure that distinct elements can be easily distinguished without occupying too much memory. As for the hyperparameters of loss function, we set $\lambda_{\textit{cls}}$, $\lambda_{\textit{pts}}$, $\lambda_{\textit{mask}}$, $\lambda_{\textit{sem}}$ to $2$, $20$, $1$, and $3$ respectively.

Training Strategy. We train our model with a total batch of $8$ on $8$ GPUs. The AdamW [8] optimizer is employed with a weight decay of $5\times 10^{-2}$ and a learning rate of $3\times 10^{-4}$. Our training process consists of two stages: base training and fine-tuning. Firstly, we initialize the InternImage [13] with public pretrained weights [2, 15] and then train our model for $60$ epochs without any tricks except a multi-step schedule with milestone $[0.7,0.9]$ and $\gamma=\frac{1}{5}$. Afterward, we apply all proposed improving techniques to fine-tune the model for extra epochs with a learning rate of $1\times 10^{-4}$.

Table 1. Our statistical results show the compacted map can reduce more than $93\%$ points without expression performance losing under the threshold of $0.5m$, even it can still maintain more than $97\%$ performance under stricter $0.2m$.

Table 2. Comparing the results in row-$1\&2$, training more epochs brings a performance gain of more than $10$ points, which shows that accelerating the convergence speed is still a vital future work. Compared with row-$3\&4,5\&6,7\&8$, using the proposed improving techniques can always bring more than $5$ points of increase. Moreover, even starting with IN-$1$K as pretrained weights, our model still achieves 79.1.

Table 3. We succeed the championship with a performance advantage of $9.85$ mAP over the second place, demonstrating the effectiveness of our proposed MachMap method.