Landmark Detection using Transformer Toward Robot-assisted Nasal Airway Intubation

DeTR is a state-of-the-art object detection model that treats object detection as a set prediction problem. It uses learnable object queries to form relations with extracted features using the transformer-based encoder and decoder modules. In the encoder, self-attention modules encode the extracted image features. In the decoder, two attention mechanisms - self-attention modules and cross-attention modules - are utilized to enable information exchange between object queries and encoded features. DeTR faces two challenges. Firstly, the computation complexity of the attention mechanism increases with the spatial size of the input, making it difficult to implement multi-level feature maps and improve small object detection accuracy [30]. Secondly, the random initialization of object queries makes them pair equally with all spatial locations of the image, rather than specific regions for detecting objects [13, 20, 30]. This prevents training object queries from focusing on specific regions. Therefore, long training iterations are necessary for DeTR to perform well.

To address the computation complexity challenge, Deformable DeTR was proposed. It combines deformable convolution with the attention mechanism to create Multi-scale Deformable Attention [30]. This novel approach can aggregate features from multi-level feature maps and decrease the computation complexity of the original attention mechanism used in DeTR. The deformable attention only samples key positions on the feature map to act as queries, rather than considering all possible spatial locations.

Another variant, SAM-DeTR, proposes the Semantic Aligner Module to accelerate DeTR’s convergence. To ensure semantic alignment, it aligns object queries and features within uniform embedding spaces. SAM uses learnable reference boxes to guide object queries to align with the semantics of encoded features. This process involves three sub-operations: extracting region features with reference boxes using ROIAlign; resampling salient points with $M$ points for each region to form new object queries and their corresponding positional embeddings; and evaluating re-weighting coefficients with current object queries to combine messages of old and new object queries. The operations related to object queries are formulated using the sigmoid function.

We propose integrating the semantic aligner module into Deformable DeTR, which combines deformable convolution with multi-scale deformable attention. To achieve this integration, we focus on adapting the decoder in Deformable DeTR to accommodate the SAM module. Specifically, we model the object queries and their positional embeddings as $query$, $query_{pos}\in\mathbb{R}^{N,256}$, following Deformable DeTR’s implementation, instead of directly modeling the object query of $\mathbb{R}^{N,4}$ used in the SAM-DeTR for modeling reference boxes where N represents the number of the object queries and 4 and 256 are hyperparameters for the channel of queries. We also use four feature levels in multi-scale deformable attention modules, consistent with the implementation in Deformable DeTR. In contrast, the SAM module of each layer in the decoder only uses encoded features from a single level to realize semantic alignment between object queries and encoded features. To achieve this, different decoder layers take encoded features from different levels as input.

The reference boxes used in the SAM module are obtained via a learnable linear projection over query position embeddings, while the reference points used in the cross-attention modules of the decoder are obtained from a learnable linear projection over reference boxes. To avoid inappropriate reference points being chosen, we do not directly regard the center points as the reference points, even though the reference boxes are represented as coordinates of the center points and the height and width of the bounding boxes.

To ensure interoperability between the cross-attention and SAM modules, we set the hidden dimension of cross-attention to $256\times 8$. Here, 256 is the dimension of the object queries, and 8 is the number of heads in multi-head attention. In the SAM module, each object query is assigned a reference box. During resampling, each reference box is resampled with $M$ salient points. $M$ representing the number of heads in multi-head attention (typically 8). Thus, after resampling, the dimension of the query changes to $Q^{new}\in\mathbb{R}^{N\times M\times 256}$. Before sending $Q^{new}$ into the cross-attention module, it is reshaped to $[N,M\times 256]$. Therefore, our approach integrates the SAM module into Deformable DeTR, providing a more effective and efficient landmark detection model. The overview architecture of our solution can be found in Fig. 1.

In the context of this research, we utilized two distinct datasets: one for nostril detection and another for glottis detection, with the aim of enabling airway intubation landmark detection.

The nostril dataset is derived from the publicly available BioID dataset [17], which was initially collected and annotated for human face keypoint detection. The dataset consists of 1521 grayscale images, all standardized to a resolution of 384x286 pixels. We manually use key point annotations of nostrils provided by the BioID dataset as the center points and expand the height and width to form bounding box annotations. To denote the nostril locations, we adopt the bounding box annotation format following the COCO format [19].

Similarly, the glottis dataset is based on the publicly available BAGALS dataset [14], initially designed for glottis segmentation. Collaboratively compiled by seven institutions, the BAGALS dataset comprises endoscope videos, detailed segmentation annotations, and frame-level information. From this collection, we extracted a subset of explicit annotations from nasal endoscopic videos, resulting in 881 images for glottis detection. As the videos originate from diverse sources, the frames within the subset exhibit varying resolutions. To form corresponding bounding box annotations, we use OpenCV to capture the contours of the segmentation annotations of BAGALS and annotate with their maximum and minimum coordinate values. We follow the COCO format for annotating the glottis dataset to maintain consistency.

In our experiments, the object detection models are implemented using MMdetection [8], a freely available toolbox for PyTorch that supports various deep learning-based detectors. All models are trained from scratch and evaluated using the abovementioned two datasets. The nostril dataset is partitioned into 1021 frames for training, 185 frames for validation, and 315 frames for testing. Similarly, the glottis dataset is divided into 377 frames for training, 88 frames for validation, and 416 frames for testing. For feature extraction, we opt for the ResNet50 backbone [16]. Besides, we employed the Adam optimizer [18] with a learning rate of $1\times 10^{-5}$ for the backbone and $1\times 10^{-4}$ for the detectors. We ensure standardized implementation and evaluation across all models by utilizing MMdetection and adhering to this experimental setup, facilitating reliable comparisons and analysis. We set the number of epochs to $24$ to observe the performance under limited training iterations.

Tables 1 and 2 present the results obtained from various models trained on two distinct datasets using a consistent 24-epoch training scheme. The baseline models consist of both CNN-based and Transformer-based architectures. The performance evaluation adheres to the standard COCO metric. We also visualize the qualitative results in Fig. 3.

As depicted in Table 1 and Figure 2(a), the test results on the glottis dataset reveal the advantages of combining the semantics aligner module with deformable attention, as evident from the performance of SAM Deformable DeTR and other baseline models trained from scratch. Notably, SAM-DeTR exhibits inferior results on the glottis dataset. This can be attributed to the inherent limitations of DeTR, despite the inclusion of a plug-and-play module in SAM-DeTR to expedite its convergence. SAM-DeTR encounters similar challenges as DeTR when trained on datasets with limited capacity. In contrast, the performance of Deformable DeTR surpasses that of SAM-DeTR, although it falls short compared to certain CNN-based detectors. These findings suggest that integrating the semantic aligner module into Deformable DeTR has the potential to enhance the model’s capabilities, an observation supported by the results of SAM Deformable DeTR. Specifically, the mAP increment demonstrates an 18-fold increase compared to SAM-DeTR and a 2-fold increase compared to Deformable DeTR.

In Table 2 and Figure 2(b), the performance of SAM Deformable DeTR on the nostril dataset achieves the highest capability. The Deformable DeTR and SAM-DeTR results are omitted from Table 2 since their mAP results are 0.000. This suggests that these models were not fully trained on the nostril dataset with limited training iterations. The nostril dataset lacks distinct appearances, and the features of the nostril are strongly correlated with the fixed locations and surrounding facial organs. SAM Deformable DeTR effectively leverages multi-scale features and spatial information in object detection, contributing to its superior performance on this dataset.