CANet: Curved Guide Line Network with Adaptive Decoder for Lane Detection

As an application of instance detection, CANet’s architecture is inspired by many existing techniques as in Figure 2. It uses ResNet [13] as backbone to extract image features, and uses PAFPN [14] with transformer [15] as neck to obtain multi-scale information. In the instance extraction branch, CANet proposes using curved guide line to obtain the key points (i.e. origins) identifying instances, then acquires instance-level features through RIM (Recurrent Instance Module) [4]. Under the guidance of these features, CANet calculates Gaussian mask and offset heatmap through the conditional convolution, and finally infers lane coordinates with adaptive decoder.

To the best of our knowledge, though guide lines have been implicitly used by some research [4] to mark the origins of lanes, we’re the first to name the boundary. If guide lines are not enforced, the origins will be randomly distributed, which is inefficient for learning. It is straightforward to consider the image border (a rectangle) as a guide line to restrict the freedom of lane origins. However, in some case it is observed that the response is very scattered, and the response peak is often indistinguishable from other non-response points.

Figure 3 helps to understand the relation between the response range and grazing angle. The black dotted line denotes the detected vectorized lane (hereinafter referred to as “vector line”), and the green dotted lines alongside indicate the range of pixels that generate a response on the heatmap. The greener the background is, the more positive the sample is. $d$ is the radius of the range. The tangents of two guide lines are illustrated in orange and magenta, with grazing angles as $\alpha 1$ and $\alpha 2$, respectively. Then, the relation between response range and grazing angle could be described in Equation (1). The bigger the angle (close to 90^∘), the tighter the response range. As the grazing angle gets smaller, the range scatters rapidly. This reflects the results in Figure 1a. Therefore, the guide line principle is to “reduce the number of lanes of small grazing angles”.

Figure 4 shows a typical lane image, sharing the notations as Figure 3. The bold blue dashed line represents a row of pixels, the red point is the intersection of the vector line and the current row, the yellow point is on the green dashed line, and the gray point is the pixel at a horizontal distance $d$ from the red one.

In traditional row-wise classification, all negative samples (yellow and gray pixels) are weighted with the same weight. UFAST [5] uses L1 distance to constrain expectations, making negative samples’ weights proportional to the distance from the positive sample in the same row, but it does not consider the influence of inclination on loss. Instead, CANet uses heatmap form supervision to construct a Gaussian distribution with the vector line as the center, to model that positive sample features decrease with distance.

Since lanes might be horizontal or vertical, it is crucial to choose either row or column-wise classification according to the lane shape. For traditional line classification, the network cannot dynamically adjust the anchor during inference because pixels from different rows are not comparable. CANet addresses this challenge with the aforementioned heatmap supervision, enabling the network to do global pixel comparison instead, which makes it possible to change the row-wise classification in training to a dynamic post-processing operation. Then, CANet is able to choose row or column-wise classification, two kinds of anchors, dynamically to the activation shape of the instance heatmap. Besides, the heatmap response range is the lane range, so there is no need to introduce additional classifiers that would introduce range-to-localization inconsistencies to indicate the range.

Extensive experiments were conducted on 3 widely used lane detection datasets, CULane [11], CurveLanes [18], and TuSimple [19]. For the sake of direct comparison, official metrics are used.

Table 1 presents the results of CANet compared to various models on different subsets of CULane. CANet achieves the state of the art overall with 79.86 F1-score (the current best result is 79.63). Particularly, CANet also delivers outstanding performance in Crowded, Dazzle, Cross and Night subsets, which are very difficult to detect.

CurveLanes contains more difficult scenarios such as curves. Table 2 illustrates the results of all models on CurveLanes. CANet-L achieves SOTA with 87.87 F1-score, 1.7 percentage points higher than the current best one. Actually, CANet’s small version CANet-S surpasses the current best F1 with only 13.1/44.9$\approx$29% computational cost. In terms of precision, CANet is a bit lower than the best result (91.69 vs. 93.58), however, CANet has the best recall rate, about 12.77 percentage points higher. This indicates that CANet does a better job on the trade-off between recall and precision.

Compared to the other datasets, TuSimple is simpler and Table 4 shows that all models perform very well, and CANet provides marginal improvements on most indicators.

For easy comparison, we divide lanes on CurveLanes into 3 groups according to grazing angles against rectangle guide line. Table 3 shows that curved guide line improves recall for all groups, especially for the smallest group. This is because curved guide line reduces the number of lanes of small grazing angles as illustrated in Figure 1b.

Table 5 presents how the two policies, curved guide line and adaptive decoder, affect each other with CurveLanes dataset. Baseline (Line 2) is derived from CondLaneNet-L with two updates: the downsampling factor for key points is reduced from 16 to 8, and the rectangle guide line is enforced. These updates make the baseline perform about 0.125 percentage points better than the original CondLaneNet-L. In Line 3, we change the rectangle guide line to curved guide line, and the F1-score increases by 1.036 to the baseline. Then, Gaussian mask supervision and adaptive decoder are solely applied in Line 4 and the model receives 1.221 increments in terms of F1. Finally, we combine both policies in Line 5 as CANet, and reach the best performance.