DETDet: Dual Ensemble Teeth Detection

Given the 705 training images for quadrant-enumeration-disease data, we observed that there are only a few labels of enumeration in the disease data. To address this problem of small number of labels, HierarchicalDet[1] applied weight transfer to learn enumeration bounding boxes from the 634 enumeration-only data. Instead, we used the instance segmentation method to increase the mean average precision (mAP). Generally, instance segmentation yields better mean average precision (mAP) than detection algorithms, as the segmentation network learns additional mask information[2], enabling to understand finer-grained signals in images. Specifically, we trained Mask-RCNN with the SwinT backbone on the 634 enumeration-only data. As using Mask-RCNN with SwinT backbone outperformed SOTA methods(HTC[3] and Cascade Mask-RCNN[4]) in the given enumeration data, we adopted the former to predict quadrant and numerical index of each tooth, along with their corresponding bounding boxes. Finally, we only considered the bounding box predictions with scores higher than 0.7 to exclude low-scoring enumeration outputs.

In our preliminary experiment, two state-of-the-art methods, DiffusionDet[5] and DINO[7], a DETR variant with improved denoising anchor boxes, were evaluated on diagnosis dataset. While DiffusionDet yields a higher Average Precision than other object detection algorithms, DINO leads to a higher Average Recall than any other detection methods. Empirically, the large number of low-score DINO detections, approximately 3000 detections per image, contributes to the high Average Recall, whereas DiffusionDet predicts a substantial number of high-scoring true positives leading to the high Average Precision. Therefore, we employ an ensemble approach that combines both methods to achieve high scores in both Average Precision and Average Recall. Specifically, we utilize DiffusionDet predictions with bounding box scores higher than 0.05 and DINO predictions with scores lower than 0.05 to increase the Average Recall while maintaining precision. Additionally, it is noteworthy that, according to our test settings, DiffusionDet with the ResNet50 backbone outperformed SwinT. Hence, we use the ResNet50 backbone for both DINO and DiffusionDet in the diagnosis module.

In order to integrate the enumeration and diagnosis modules, we introduce a method called closest bounding box center matching. This method operates in two stages: it first predicts the enumeration bounding boxes and then matches these boxes to the nearest diagnosis bounding boxes. Through this integration, we are able to derive category IDs for all three elements: quadrant, enumeration, and disease. The combined bounding box score is computed by multiplying the scores from the enumeration and diagnosis modules. In summary, DETDet integrates enumeration and diagnosis through a dual-stage approach, ensembling two models to achieve high precision and recall.

Furthermore, the dual-stage integration offers the advantage of explicitly utilizing the 1571 unlabeled data dealing with the insufficient amount of original training data. The training data suffers from significant imbalance, as there is a notable disparity in the amount of caries data compared to other diseases. In particular, the dataset contains a scarcity of Periapical Lesion data. To address this data imbalance, we introduce a complementary module, as depicted in Fig. 2, which is designed to leverage unlabeled data for increasing the number of train samples. The complementary module is trained using a pseudo-labeling approach. Specifically, we segment all the dental images using the enumeration module and assign each tooth to its corresponding category. Subsequently, an EfficientNetB4[6] classifier is trained to classify teeth into four categories: normal, caries, deep caries, impacted, and periapical lesion. We apply this classifier to the unlabeled data, assigning pseudo labels to all teeth. Through pseudo labeling, we are able to augment the periapical lesion data, thereby achieving a more balanced dataset overall. While the complementary module generates fewer predictions compared to the diagnosis module, it effectively compensates for the predictions that the diagnosis module may miss.

The enumeration module, trained using Mask-RCNN with the SwinT backbone, is evaluated on our set of 50 test images. The outcomes are detailed in Table 1. The AP50 score stands at 0.987, indicating that the module adeptly classifies and detects teeth with a high level of precision. Nevertheless, the AP75 score is relatively lower. This discrepancy can be attributed to variations in the ground truth bounding boxes, as they were labeled by different individuals, resulting in potential differences in exact bounding box coordinates across all images.

The diagnosis module, trained using DiffusionDet and DINO with a ResNet50 backbone, is evaluated on our set of 50 test images. The outcomes are outlined in Table 2. The mAP, AP50, and AP75 scores are notably higher in DiffusionDet compared to DINO. However, the AR score is significantly superior in DINO, owing to its output of 32% more detections than DiffusionDet. Consequently, we combine these two models in an ensemble approach to capitalize on their strengths. The ensemble module selects the DiffusionDet model when the prediction score exceeds a certain threshold, otherwise it selects DINO. After conducting experimentation on the test data, we set the threshold at 0.05. This strategic choice yields a slight increase in mAP and AP50, along with a substantial improvement in AR. The results indicate a notable enhancement in overall performance.

The outcomes of the dual-stage integration are outlined in Table 3. We employed the closest bounding box matching method to integrate the enumeration and diagnosis modules, enabling the prediction of category IDs 1, 2, and 3.

The inclusion of the complementary module yields an overall enhancement in the metrics. The outcomes are consolidated in Table 4. Notably, there is a substantial increase in the AP50 scores for both disease and enumeration. The complementary module is trained using an expanded dataset, which encompasses unlabeled data. In particular, we augmented the periapical lesion and deep caries data by a factor of two, thereby mitigating the training data’s imbalance. Consequently, the complementary module contributes supplementary predictions to the dual-stage integration outputs, resulting in improved precision and recall.