Exploring Unsupervised Cell Recognition with Prior Self-activation Maps

We construct a semantic clustering module (SCM) which converts prior self-activation maps to pseudo masks. In SCM, the original information is included to strengthen the detailed features. It is defined as:

where ${I_{f}}$ denotes the fused semantic map, $I_{raw}$ is the raw input, $\beta$ is the weight of $I_{raw}$.

To generate semantic labels, an unsupervised clustering method K-Means is selected to directly split all pixels into several clusters and obtain foreground and background pixels. Given the semantic map $I_{f}$ and its $N$ pixel features $F=\{f_{i}|i\in\{1,2,...N\}\}$, $N$ features are partitioned into $K$ clusters $S=\{S_{i}|i\in\{1,2,...K\}\}$, $K<N$. The goal is to find $S$ to reach the minimization of within-class variances as follows:

where $c_{i}$ denotes the centroid of each cluster $S_{i}$. After clustering, the pseudo mask $I_{sg}$ can be obtained.

For the task of cell detection, a detection network is trained under the supervision of pseudo mask $I_{sg}$. In the inferring stage, the output of the detection network is a score map. Then, it is post-processed to obtain the detection result.

The coordinates of cells can be got by searching local extremums in the score map, which is described below:

where $T_{(m,n)}$ denotes the predicted label at location of $(m,n)$, $p$ is the value of the score map and $D_{(m,n)}$ indicates the neighborhood of point $(m,n)$. $T_{(m,n)}$ is exactly the detection result.

Due to the lack of instance-level supervision, the model does not perform well in distinguishing adjacent objects in the segmentation. To further reduce errors and uncertainties, the Voronoi map $I_{vor}$ which can be transformed from $I_{sg}$ is utilized to encourage the model to focus on instance-wise features. In the Voronoi map, the edges are labeled as background and the seed points are denoted as foreground. Other pixels are ignored.

We train the segmentation model with these two types of labels. The training loss function can be formulated as below,

where $\lambda$ is the partition enhancement coefficient. In our experiment, we discovered that false positives hamper the effectiveness of segmentation due to the ambiguity of cell boundaries. Since that, only the background of $I_{sg}$ will be concerned to eliminate the influence of false positives in instance identification.

We validated the proposed method on the public dataset of Multi-Organ Nuclei Segmentation (MoNuSeg) [13] and Breast tumor Cell Dataset (BCData) [11]. MoNuSeg consists of 44 images of size 1000 $\times$ 1000 with around 29,000 nuclei boundary annotations. BCData is a public large-scale breast tumor dataset containing 1338 immunohistochemically Ki-67 stained images of size 640 $\times$ 640.

In our experiments on MoNuSeg, F1-score and IOU are employed to evaluate the segmentation performance. Denote $TP$, $FP$, and $FN$ as the number of true positives, false positives, and false negatives. Then F1-score and IOU can be defined as: $F1=2TP/(2TP+FP+FN)$, $IOU=TP/(TP+FP+FN)$. In addition, common object-level indicators such as Dice coefficient and Aggregated Jaccard Index (AJI) [13] are also considered to assess the segmentation performance.

In the experiment on BCData, precision (P), recall (R), and F1-score are used to evaluate the detection performance. Predicted points will be matched to ground-truth points one by one. And those unmatched points are regarded as false positives. Precision and recall are: $P=TP/(TP+FP)$, and $R=TP/(TP+FN)$. In addition, we introduce MP and MN to evaluate the cell counting results. ’MP’ and ’MN’ denote the mean average error of positive and negative cell numbers.

Res2Net101 [7] is adopted as the activation network $U_{ss}$ with random initialization of parameters. The positive sample is augmented by rotation. The weights $\beta$ are set to $2.5$ and $4$ for training in MoNuSeg and BCData, respectively. The weight $\lambda$ is 0.5. The analysis for $\beta$ and $\lambda$ is included in the supplementary. Pixels of the fused semantic map will be decoupled into three piles by K-Means. . The following segmentation and detection are constructed with ResNet-34. They are optimized using CrossEntropy loss by the Adam optimizer for $100$ epochs with the initial learning rate of $1e^{-4}$. The function $diff(\cdot)$ is instantiated as the measurement of Manhattan distance.

In MoNuSeg Dataset, four fully-supervised methods Unet [20], MedT [24], CDNet [8], and the competition winner [13] are adopted to estimate the upper limit as shown in the first four rows of Table 1. Two weakly-supervised models trained with only point annotations are also adopted as the comparison. Compared with the method [22] fully exploiting localization information, ours can achieve better performance without any annotations in object-level metrics (AJI). In addition, two unsupervised methods using traditional image processing tools [21, 1] and two unsupervised methods [9, 10] with deep learning are compared. Our framework has achieved promising performance because robust low-level features are exploited to generate high-quality pseudo masks.

Following the benchmark of BCData, metrics of detection and counting are adopted to evaluate the performance as shown in Table 2. The first three methods are fully supervised methods which predict probability maps to achieve detection.

Furthermore, TransCrowd [16] with the backbone of Swin-Transformer is employed as the weaker supervision trained by cell counts regression. By contrast, even without any annotation supervision, compared to CSRNet [15], NP-CNN [23] and U-CSRNet [11], our proposed method still achieved comparable performance. Especially in terms of MP, our model surpasses all the baselines. It is challenging to realize multi-class recognition in an unsupervised framework. Our method still achieves not bad counting results on negative cells.

Ablation experiments are built on MoNuSeg. In our pipeline, the activation network can be divided into four layers which consists of multiple basic units including ReLU, BacthNorm, and Convolution. We exploit the prior self-activation maps generated from different depths in the model after training with the same proxy tasks. As shown in Fig. 3, the performance goes down and up with we extracting features from deeper layers. Due to the relatively small receptive field, the shallowest layer in the activation network is the most capable to translate local descriptions

We have also experimented with different types of proxy tasks in a self-supervised manner, as shown in Table 3. We can see that relying on the pre-trained models with external data can not improve the results of subsequent segmentation. The model achieves similar pixel-level performance (F1) when learning similarity or contrastiveness. But similarity learning makes the model performs better in object-level metrics (AJI) than contrastive learning. The high intra-domain similarity hinders the comparison between constructed image pairs. Unlike natural image datasets containing diverse samples, the minor inter class differences in biomedical images may not fully exploit the superiority of contrastive learning.