Trichomonas Vaginalis Segmentation in Microscope Images

To construct the dataset, we first collect 80 videos of Trichomonas samples with resolution 3088$\times$2064 from more than 20 cases over seven months. The phase-contrast microscopy is adopted for video collection, which captures samples clearly, even unstained cells, making it more suitable for sample imaging and microscopy evaluation than ordinary light microscopy. We then extract images from these videos to build our TVMI3K dataset, which finally contains 3,158 microscopic images (2,524 Trichomonas and 634 background images). We find that videos are more beneficial for data annotation, because objects can be identified and annotated accurately according to the motion of Trichomonas. Thus, even unfocused objects can also be labeled accurately. Besides, to avoid data selection bias, we collect 634 background images to enhance generalization ability of models. It should be noted that images are collected by the microscope devices independently, and we do not collect any patient information, thus the dataset is free from copyright and loyalties.

High-quality annotations are critical for deep model training [17]. During the labeling process, 5 professional annotators are divided into two groups for annotation and cross-validation is conducted between each group to guarantee the quality of annotation. We label each image with accurately object-level masks, object edges and challenging attributes, e.g., occlusions and complex shapes. Attribute descriptions are shown in Tab. 1.

$\bullet$ Image-level Attributes. As listed in Tab. 1, our data is collected with several image-level attributes, i.e., multiple objects (MO), small object (SO) and out-of-view (OV), which are mostly caused by Trichomonas size, shooting distance and range. According to statistics, each image contains $3$ objects averagely, up to $17$ objects. The size distribution ranges from $0.029\%$ to $1.179\%$, with an average of $0.188\%$, indicating that it belongs to a dataset for tiny object detection.

$\bullet$ Object-level Attributes. Trichomonas objects show various attributes, including complex shapes (CS), occlusions (OC), out-of-focus (OF) and squeeze (SQ), which are mainly caused by the growth and motion of trichomonas, highly background distraction (e.g. other cells) and the shaking of acquisition devices. These attributes lead to large morphological differences in Trichomonas, which increase the difficulty of detection. In addition, there is a high similarity in appearance between Trichomonas and leukocyte, which can easily confuse the detectors to make false detections. Examples and the details of object-level attributes are shown in Fig. 1 and Tab. 1 respectively.

$\bullet$ Dataset Splits. To provide a large amount of training data for deep models, we select 60% of videos as training set and 40% as test set. In order to be consistent with practical applications, we select data according to the chronological order of the sample videos from each case instead of random selection, that is, the data collected first for each case is used as the training set, and the data collected later is used as the test set. Thus, the dataset is finally split into 2,305 images for training and 853 images for testing respectively. Note that 290 background images in the test set are not included in our test experiments.

Fig. 2 illustrates the overall architecture of the proposed TVNet. Specifically, for an input Trichomonas image $I$, the backbone network Res2Net [9] is adopted to extract five levels of features $\{f_{i}\}_{i=1}^{5}$. Then we further explore the high-level features (i.e., $f_{3}\sim f_{5}$ in our model) to effectively learn object feature representation and produce object prediction. As shown in Fig. 2, we first design a high-resolution fusion (HRF) module to enhance feature representation by integrating high-resolution edge features and get three refined features ($\{f_{i}^{{}^{\prime}}\}_{i=3}^{5}$). Next, we introduce the neighbor connection decoder (NCD) [5], a new cutting-edge decoder component which improves the partial decoder component [33] with neighbor connection, to aggregate these three refined features and generate the initial prediction map $P_{6}$. It can capture the relatively coarse location of objects so as to guide the following object prediction. Specially, $\{P_{i}\}_{i=3}^{5}$ represents the prediction map of the $i$ layer, which corresponds to the $i$-layer in $\{f_{i}^{{}^{\prime}}\}_{i=3}^{5}$. Finally, we propose a foreground-background attention (FBA) module to excavate object critical cues by exploring semantic differences between foreground and background and then accurately predict object masks in a progressive manner.

With the deepening of the neural network, local features, such as textures and edges, are gradually diluted, which may reduce the model’s capability to learn the object structure and boundaries. Considering too small receptive field and too much redundant information of $f_{1}$ feature, in this paper, we design a high-resolution fusion (HRF) module which adopts the the low-level $f_{2}$ feature to supplement local details for high-level semantic features and boost feature extraction for segmentation. To force the model to focus more on the object edge information, we first feed feature $f_{2}$ into a $3\times 3$ convolutional layer to explicitly model object boundaries with the ground-truth edge supervision. Then we integrate the low-level feature $f_{2}$ with the high-level features ($f_{3}\sim f_{5}$) to enhance representation with channel and spatial attention operations [31, 2], denoted as:

where $\delta_{\downarrow}^{2}(\cdot)$ denotes $\times 2$ down-sampling operation. $\mathcal{G}(\cdot)$ is a $3\times 3$ convolution layer, and $Cat$ is concatenation operation. $M_{c}(\cdot)$ and $M_{s}(\cdot)$ represent channel attention and spatial attention respectively. $\otimes$ is the element-wise multiplication operation. After that, a 1$\times$1 convolution is used to adjust the number of channels to ensure the consistent output of each HRF.

To produce accurate prediction from the fused feature ($f_{3}^{{}^{\prime}}\sim f_{5}^{{}^{\prime}}$) progressively in the decoder, we devise a foreground-background attention (FBA) module to filtrate and enhance object related features using region sensitive map derived from the coarse prediction of the previous layer. For the fused feature $\{f_{i}^{{}^{\prime}}\}_{i=3}^{5}$, we first decompose the previous initial prediction $P_{i+1}$ into three regions, i.e., strong foreground region ($\mathcal{F}_{i+1}^{1}$), weak foreground region ($\mathcal{F}_{i+1}^{2}$) and background region ($\mathcal{F}_{i+1}^{3}$). The decomposition process can refer to [25]. Then each region is normalized into [0,1] as a region sensitive map to extract the corresponding features from $f_{i}^{{}^{\prime}}$. Noted that $\mathcal{F}^{1}$ provides the location information, and $\mathcal{F}^{2}$ contains the object edge/boundary information. The $\mathcal{F}^{3}$ denotes the remaining region.

After that, the region sensitive features can be extracted from the fused feature by an element-wise multiplication operation with up-sampled region sensitive maps followed by a 3$\times$3 convolution layer. Next, these features are aggregated by an element-wise summation operation with a residual connection. It can be denoted as:

where $\delta_{\uparrow}^{2}(\cdot)$ denotes $\times 2$ up-sampling operation. Inspired by [5], FBA can also be cascaded multiple times to gradually refine the prediction. For more details please refer to the Supp. In this way, FBA can differentially handle regions with different properties and explore region-sensitive features, thereby strengthening foreground features and reducing background distractions.

Baselines and Metrics. We compare our TVNet with 9 state-of-the-art medical/natural image segmentation methods, including U-Net++ [39], SCRN [34], U²Net [22], F³Net [30], PraNet [7], SINet [6], MSNet [37], SANet [29] and SINet-v2 [5]. We collect the source codes of these models and re-train them on our proposed dataset. We adopt 7 metrics for quantitative evaluation using the toolboxes provided by [7] and [29], including structural similarity measure ($S_{\alpha}$, $\alpha$ = $0.5$) [3], enhanced alignment measure ($E_{\phi}^{max}$) [4], $F_{\beta}$ measure ($F_{\beta}^{w}$ and $F_{\beta}^{mean}$) [20], mean absolute error (MAE, $\mathcal{M}$) [21], Sorensen-Dice coefficient (mean Dice, mDice) and intersection-over-union (mean IoU, mIoU) [7].

Quantitative comparison. Tab. 2 shows the quantitative comparison between our proposed model and other competitors on our TVMI3K dataset. It can be seen that our TVNet significantly outperforms all other competing methods on all metrics except $S_{\alpha}$ which is also on par with the best one. Particularly, our method achieves a performance gain of 2.2% and 2.3% in terms of $F_{\beta}^{w}$ and $F_{\beta}^{mean}$, respectively. This suggests that our model is a strong baseline for TVS.