Simplify Implant Depth Prediction as Video Grounding: A Texture Perceive Implant Depth Prediction Network

We employ the widely used resblock to construct the encoder of IDPNet. Specifically, the encoder consists of two 3D resblocks [6] and two 2D resblocks [2]. The architecture of encoder and decoder are shown in the Fig. 3(c). The 3D resblock first takes the sub-volume as input and learns context information among slices. Then, these temporal features are fed into the 2D resblock to learn the texture feature in different slices. The output of encoder is a feature map $\mathbf{F}\in\mathbb{R}^{N\times C\times D\times H\times W}$. Considering that the regression of implant depth heavily relies on clearly neighboring tooth texture, which requires high-resolution feature representations. Hence, we adopt three deconvolution layers as decoder to consecutively upsamples the feature map.

Clinically, dentists determine the implant depth according to the texture of neighboring teeth, e.g., the bottom of the implant does not exceed the root of neighboring teeth. Therefore, IDPNet should possess the ability to perceive the texture variation among slices. In Fig. 2, we visualize 2D slices sampled with different sampling and compute the texture variation among these slices by standard deviation. We can observe from the figure that the larger the sampling interval, the more obvious such texture variations. This observation indicates that the neighboring 2D slices have a similar feature, while the distant slices have a big difference in features. Drawing inspiration from this observation, in this paper, we propose a Texture Perceive Loss (TPL), which assists the encoder learns more robust features.

The details of TPL is given in the Fig. 3(d). Specifically, we first reduce the channel $C$ of $\mathbf{F}$ to 1 and reshape the channel $D$ to $D^{{}^{\prime}}$, to restore the information of channel $D$. By this means, the pre-processed $\hat{\mathbf{F}}\in\mathbb{R}^{N\times D^{{}^{\prime}}\times H\times W}$ is obtained. Then, we apply the Canny operator for $\hat{\mathbf{F}}$ to extract textures along the channel $D^{{}^{\prime}}$. After obtaining a series of texture matrix $\mathbf{M}\in\mathbb{R}^{N\times D^{{}^{\prime}}\times H\times W}$, we perform the consistency loss $\mathcal{L}_{con}$ for the neighboring matrix to close these features, and the inconsistency loss $\mathcal{L}_{icon}$ for the distant matrix to distinguish these features. $\mathcal{L}_{TPL}$ is the summation of $\mathcal{L}_{con}$ and $\mathcal{L}_{icon}$, and we implement $\mathcal{L}_{con}$ and $\mathcal{L}_{icon}$ by L2 loss. In our implementation, we set the sampling interval $k$ of the distant matrix as 10.

The regression head is designed to predict the implant depth, which is implemented by two convolutions followed by the activation function, i.e., ReLU. We use the L1 loss to optimize the regression head:

where $j$ is the patient index in a mini-batch and $N_{p}$ is the total number of patient. $y_{j}=(s_{j},e_{j})$ and $\hat{y}_{j}=(\hat{s}_{j},\hat{e}_{j})$ is the predicted and ground-truth index of start and end implant slice, respectively.

As discussed in previous sections, we model implant depth prediction as the task of video grounding. Therefore, we follow the video grounding to introduce the temporal iou loss [16] to supervise the regression head:

The rationale of $\mathcal{L}_{tiou}$ is to maximize the overlapping between the predicted slice index and its ground truth. The overall training loss of IDPNet is:

As the IRD is used as pre-processing method to crop CBCT data for IDPNet, an approximate planting area is sufficient but requires quick inference speed. To evaluate the effectiveness of the proposed IRD, we conduct ablation experiments to investigate the effect of removing components in IRD, results are given in Table 1. $AP_{75}$ and FLOPs are used as evaluation metrics. We can observe from the table that removing both modules will result in a 2.2% performance decrease, but FLOPs is decreased by 11.27. This results meet with the requirement and demonstrate that the proposed IRD is effective and lightweight for clinical practice.

To demonstrate the effectiveness of the proposed loss function, we conduct ablation experiments to investigate the effect of each loss function in Table 2. We can observe from the second row of the table that using temporal iou loss alone will lead to regression failure. When combining both regression loss and temporal iou loss, the accuracy (m=0.8) improves by 1.6%. When the TPL loss is introduced, the improvement reaches to 5.0%. Although the accuracy of smaller IoU thresholds has decreased, high IoU threshold is required in clinical practice. This results demonstrate the effectiveness of TPL loss, which enables the encoder to perceive the texture variation among slices.

To further validate the effectiveness of the proposed TPL loss, in Fig. 4, we visualize the prediction result of TPNet with or without training by the TPL loss. From the figure we can observe that the introduction of TPL predict more precise start and end slices of the implant, due to the perception capability of texture variation.

As previously discussed, we model the task of implant depth prediction as video grounding. To demonstrate the superior performance of the proposed method, we compare TPNet with other state-of-the-art video grounding methods in Table 3. Specifically, we choose different visual feature based methods, e.g., the C3D-based method, TSP-PRL, and the I3D-based methods, MAN, VSLNet and DRN. From the table we can observe that, the C3D-based method perform better than the I3D-based networks in high iou threshold (e.g., TSP-PRL achieved 18.6% Acc, which is 1.5% higher than the best-performing I3D-based network - VSLNet). The proposed TPNet achieves the best accuracy of 20.3%, among all benchmarks. The experimental results proved the effectiveness of our method.