LEARNING HIGH-QUALITY PROPOSALS FOR ACNE DETECTION

Detection heads have been widely used in object detection and finely designed for different tasks. RPN ⁸, widely adopted in two-stage detection methods, is essentially a kind of fully-convolutional network(FCN) ¹³ to generate proposals at each feature map position. As shown in Fig. 1(a), RPN consists of two branches for classification and localization tasks, respectively. Law et al. ¹⁴ proposed a Cornernet to predict an object’s top-left corner and bottom-right corner points. Duan et al. ¹⁵ further extended Cornernet by predicting the center point of an object. Ma et al. ¹⁶ proposed to predict a matching degree score between the predicted bounding box and the ground truth as well as the IoU. Sun et al. ¹⁷ proposed to a universal relation exploring scheme to explore the relations of different entity level, such as objects, superpixels and pixels. Mask R-CNN ¹⁸ extended Faster R-CNN by adding an instance segmentation head. Wu et al. ¹⁹ conducted a thorough comparison between the fully connected head(fc-head) and the convolution head(conv-head) in R-CNN, and found an interesting phenomenon: fc-head is more suitable for the classification task, while conv-head is more suitable for the localization task.

The proposed SADH structure is related to the thought-provoking work ¹⁹. We both introduce a linear-style layer for the classification task. The difference is that ¹⁹ performs a fc-head on the features from RoIAlign, while the proposed method introduces a computationally efficient 1$\times$1 convolution in RPN, see Fig. 1(b) for example. However, our motivations are thoroughly different. Wu et al. are motivated by the fact fc-head has more spatial sensitivity than conv-head, while the proposed SADH is designed to improve the steepness of the classification confidence gradient.

Recently, localization confidence prediction has also drawn a growing attention. FCOS ²⁰ introduced a center-ness branch to predict the distance between a pixel and the center of its corresponding ground truth. Jiang et al. ²¹ proposed to predict the intersection-over-union(IoU) between the bounding box and the matched ground truth through an IoU-Net. Ma et al. ¹⁶ proposed to utilize the matching degree score of the corner points between the predicted bounding box and the ground truth to indicate the localization confidence. Mask Scoring R-CNN ²² proposed to predict mask IoU for better segmentation performance based on IoU-Net and Mask R-CNN.

FCOS, IoU-Net, and Mask Scoring R-CNN are all related to the proposed method. FCOS predicts a center-ness at each feature map position to down-weight the low-quality detections based on the center prior rule ²³. IoU-Net predicts the IoU and utilizes it in the non-maximum suppression(NMS) by preserving accurately localized bounding boxes. Mask Scoring R-CNN predicts a mask IoU for the instance segmentation task. In this paper, we also add a new branch to predict localization confidence. The difference is that we adopt the NWD as the metric to evaluate localization confidence, which is more suitable for detecting small acne lesions ¹². In addition, the NWD is learned by a novel SBCE loss and directly utilized to rectify the original classification scores.

As shown in Fig. 1(a), vanilla RPN performs a $3\times 3$ convolution on Feature Pyramid Networks(FPN) feature map, where the same six elements would be inputted to the calculation of adjacent elements on the following intermediate and final feature maps. As a result, the classification scores at adjacent positions on the final feature map are pretty close, leading to a flat classification confidence gradient. We argue that adjacent elements on the final feature maps for the classification task should be spatially independent for a steeper classification confidence gradient, while that for the localization task should be spatially correlated for similar regression results. From the perspective of spatial perception, we propose a Spatial Aware Double Head to improve the drawback of flat classification confidence gradient in the original RPN. As shown at the top of Fig. 1(b), the intermediate feature maps for the two tasks of RPN are disentangled into two parts, which are computed with a $1\times 1$ and $3\times 3$ convolution on the FPN feature map respectively.

For the classification task, the $1\times 1$ convolution is crucial to constrain the adjacent elements of final feature maps spatially independent. The red and blue elements on the final feature map are only related to the red and blue on the FPN feature map, respectively. As a result, the classification scores of proposals at adjacent positions are not necessarily too close, thus making the classification scores more discriminative. For the localization task, the intermediate feature map is computed by $3\times 3$ convolution, which is the same as RPN. As shown in Fig. 1(a), the six purple elements in the FPN feature map all participate in the computation of the red and blue elementsfine-grained. The spatial correlation benefits the regression results of proposals at adjacent positions. Moreover, following ²⁰, a group normalization(GN) is added before the ReLU layer to make the training more stable.

Label assignment in RPN selects positive and negative samples according to a label threshold and the anchors’ IoUs with the matched ground truths. The anchors with IoUs under the label threshold are regarded as negative samples, and the corresponding proposals are expected to yield extremely low classification scores, leading to a low correlation between the classification scores and the proposals’ IoUs. Jiang et al. ²¹ proposed to predict IoU to alleviate the issue caused by the low correlation. However, Wang et al. ¹² has demonstrated that NWD was a more suitable metric than IoU to evaluate the similarity of two bounding boxes in small object detection. In this paper, a new NWD prediction branch is proposed by performing a $1\times 1$ convolution on the intermediate feature maps for the localization task, as shown in the bottom part of Fig. 1(b). Besides, a sigmoid layer is performed on the final feature map to constrain the predicted NWDs within (0, 1).

We evaluate the proposed method on both the AcneSCU and ACNE04 datasets. For the AcneSCU dataset, we randomly split it into two parts, about $90\%$(248 images) are held out as the train set while the others are used as the test set. As some persons may exist in more than one image, all the persons in the test set are constrained not in the train set to avoid overfitting. For the ACNE04 dataset, it contains 1457 images with 18983 bounding boxes annotations of only one lesion category. Following ², the whole dataset is randomly split into $80\%$ train set and $20\%$ test set, consisting of 1165 and 292 images, respectively.

Our framework is built based on the Mask R-CNN implemented by MMDetection ³³ with one NVIDIA GeForce 3090. The reasons for choosing Mask R-CNN as the baseline are: 1) FPN ³⁴ is utilized to enhance the detection of various-sized objects; 2) RPN ⁸ is utilized for generating candidate regions efficiently, enabling the designing of more flexible and precise classifiers ¹⁰; 3) it could take advantage of instance annotations to improve the detection performance ¹⁸; 4) it’s robust and widely utilized for comparison in generic object detection.

We initialize the Mask R-CNN with parameters for COCO dataset in ¹⁸ and use a ResNet50 pretrained on ImageNet ³⁵ as the backbone. Deformable convolutional networks(DCN) ³⁶ is added to enhance the transformation modeling capability. To output feature maps with suitable scales for acne lesions, only P2, P3, P4, and P5 feature maps of FPN ³⁴ are preserved. 2000 proposals per feature map from FPN are performed with an NMS with IoU threshold of 0.7, and 1000 proposals per sub-image are outputted at last to R-CNN. The anchors and proposals are assigned as positive samples if their IoUs with the matched ground truth are greater than 0.5 when training both RPN and RCNN. In the inferring process, an NMS with IoU threshold of 0.5 is utilized for all the detection bounding boxes, and only 200 per sub-image at most are outputted as detection results at last. In this paper, we find the overall performance works well with the hyper-parameter $\lambda_{nwd}$ and $\omega_{nwd}$ range within (0.8, 1.2), and they are both set as 1 by default. The network is trained by an SGD optimizer with 15 epochs, where the learning rate, momentum, and weight decay are 0.002, 0.9, and 0.0001, respectively. There are two ways to merge detection results of sub-images: 1) follow  ³⁷ to perform an NMS on the detection results of sub-images; 2) directly input the whole image into the model and set a larger number of proposals in RPN and RCNN. As the post-processing techniques are not the focus of this paper, without loss of generality, we only investigate the detection performance on sub-images.

To efficiently evaluate the proposed methods, following ⁷, we evaluate Average Precision(AP) and AR at IoU=0.5 with 100 detection boxes at most. Please note that IoU=0.5 is already a harsh condition in consideration of the arbitrary boundary and pixel-level human label error of small acne lesions; see Fig. 1 for example. In addition, metrics with regard to different object sizes are also reported. Specifically, $AP$, $AP_{s}$, $AP_{m}$, and $AP_{l}$ denote the AP of all, small, medium, and large objects, respectively, and the same is true for AR. Please note AP is much more important than AR in object detection, because it evaluates the average precision under different recalls. However, AR is easy to improve by increasing the detection boxes. All this doesn’t conflict with our efforts to improve the quality and AR of the RPN proposals.

Comparison results with previous state-of-the-art methods on ACNE04 \topruleMethod AP \botruleFaster R-CNN ⁸ 0.103 R-FCN ³⁸ 0.140 Rashataprucksa et al. ⁶ 0.147 ACNet ⁷ 0.205 \botruleours 0.22 \botrule

To demonstrate the efficiency of the proposed method, state-of-the-art methods of both one-stage detection methods, e.g. FCOS ²⁰, FSAF ²⁴, and AutoAssign ²⁰, and the two-stage detection, e.g. Faster R-CNN ⁸, Cascade R-CNN ²⁵, PANet ²⁶, Cascade Mask R-CNN ²⁷, Mask Scoring R-CNN ²², HTC ²⁸, GHM ²⁹, Libra R-CNN ³⁰, OHEM ³¹, and PISA ³², are chosen as peer competitors on AcneSCU dataset. To make a fair comparison, we try our best to implement the compared methods with the same settings and hyper-parameters. For ACNE04, we compare the methods which have published results on it.

Table 3.3.1 shows the comparative results on AcneSCU. On the overall metric AP, our method outperforms all the compared state-of-the-art one-stage and two-stage generic object detection methods. Specifically, Mask R-CNN with masked crop outperforms the baseline with $2.4\%$ AP, which demonstrates the efficiency of the proposed masked crop. We argue that this is because masked crop could alleviate the influence caused by unavoidable partial lesions. Mask R-CNN with SADH achieves $1.6\%$ higher AP than that without SADH. This may benefit from a steeper classification confidence gradient, which could suppress the proposals of low IoUs. Compared to ”Mask R-CNN + SADH”, the NWD prediction branch could further improve the AP with $1.0\%$, which demonstrates the efficiency of NWD rectification, especially on medium and small lesions. In addition, we find AutoAssign achieves the greatest AR. That is because AutoAssign’s detection number is much greater than that of our method, about 5:1.

Table 4.3 shows the comparative results on ACNE04. For a fair comparison, we adopt the same data augmentation with previous method ⁷, which means our result doesn’t benefit from the proposed masked crop data preprocessing method. In addition, we calculate the overall performance on the whole images to ensure comparison under the same evaluation condition. Despite without masked crop, the proposed outperforms all the previous methods on the public ACNE04 dataset.

Moreover, we also evaluate the generalization performance of our method with different detection frameworks on AcneSCU. As shown in Table 4.3, it could be found our method achieves consistent improvement with different detection frameworks, demonstrating the generalization performance of our method. Specifically, our method could improve HTC, Faster R-CNN, and Mask R-CNN with $0.7\%$, $1.6\%$, and $2.6\%$, respectively. The best improvement of Mask R-CNN may be due to its best detection performance of the baseline.

Ablation study on ACNESCU \topruleGroup Convolution Loss Metric AP \botrule1 $3\times 3$ - - 0.488 $1\times 1$ - - 0.493 \botrule2 - $L_{1}$ loss - 0.487 - SBCE - 0.498 \botrule3 - - IoU 0.490 - - GIoU 0.480 - - DIoU 0.483 - - NWD 0.498 \botrule

To validate different components of the proposed method, three groups of ablation experiments are performed, and the results are shown in Table 4.4. Within each group, all the settings are same, including but not limited to DCNs in the backbone, GN in RPN head, and IoU threshold of NMS, except for the investigated component. Therefore, the influence of GN or DCNs on overall comparison is excluded. In addition, we try to make the smallest modification on the baseline. For example, we use the origin RPN for group 2 and 3.

The curve of the classification confidence gradient. The x-axis denotes the Manhattan distance between the feature map positions of the proposal and ground truth, while the y-axis denotes the classification confidence gradient, defined by $\Delta{s_{cls}}=s_{cls}(x)-s_{cls}(0)$, where $s_{cls}(x)$ denotes the classification score at the distance of x.

Specifically, the first group is conducted to analyze the kernel size of the convolution layer performed on the FPN feature map, emphasized with a bold font in Fig. 1(b), in the classification head of SADH. We find that $1\times 1convolution$ plays an important role in the success of SADH, which demonstrates that the kernel size indeed influences the independence of classification scores at adjacent positions on the final feature map. To valid the efficiency of SBCE loss, the second group compares the $L_{1}$ loss ³⁹ and SBCE loss used in the NWD prediction branch. The results show that the proposed SBCE achieves $1.1\%$ higher AP than $L_{1}$ loss. We argue SBCE loss had a steeper curve than $L_{1}$ loss, leading to a faster and better convergence. To analysis the effectiveness of the localization confidence metric, the third group is performed by learning 4 different metrics, i.e., IoU, GIoU ⁴⁰, DIoU ⁴¹, and NWD. It could be found that classification rectification with NWD is better than that with IoU, GIoU, and DIoU. This may be interpreted that NWD is a more suitable localization confidence metric for small acne lesions.

The visual view of some detection results is shown in Fig. 1. It could be found most skin lesions, especially those which have obvious visual features, could be well detected. The reasons for false detection results could be concluded as: 1) some lesions located on cheeks do not have apparent texture features on the front-view facial images; 2) several close tiny are detected as one lesion; 3) some tiny lesions are too tiny(with sizes less than 20) to be detected for the harsh IoU threshold; 4) some acne lesions with arbitrary boundaries are hard to detect; 5) noisy labels. These problems would be the subjects of our future works.

To intuitively demonstrate the efficiency of the proposed SADH, we plot the steepness of the proposals’ classification scores, which is referred to as classification confidence gradient in this paper, moving from the center of the ground truth to its boundary. As shown in Fig. 4.5, it could be found that SADH had a steeper classification confidence gradient than the vanilla RPN. We argue a steeper classification confidence gradient could generate more independent classification scores at adjacent positions on the final feature map, suppress the proposals with low IoUs but high classification scores, and then improve the detection of the hard samples.

To demonstrate the effectiveness of the NWD prediction branch, we visualize the relations between the proposals’ IoUs and classification scores $s_{cls}$, the predicted NWDs $p$, and the rectified final scores $s_{final}$, respectively. As shown in Fig. 2, there exists a large number of proposals with IoUs ranging from 0 to 0.4 and classification scores below 0.1. We argue that this is caused by the label assignment in RPN. The anchors whose IoUs below the label threshold, always set as 0.5, are regarded as negative samples. After the rectification of NWDs, these samples could be assigned with more appropriate classification confidences, i.e., the final scores $S_{final}$. As a result, the detection performance benefits from the higher relation between the classification confidences and IoUs.