Cell Detection from Imperfect Annotation
by Pseudo Label Selection Using P-classification

We use a cell position heatmap that has shown a good performance [12]. The detection network $D$ is trained using the set of the annotated cell centroid positions to predict a heatmap so that a cell centroid becomes a peak with a Gaussian distribution. We use the U-net architecture for $D$. The method requires all cells in an image to be annotated as training data since the network is trained to produce zero values on the background area (non-annotated area). If we train the network using the imperfect training data, it affects the detection performance. Therefore, we use the masked loss function which gives the loss only around the given ground-truth of the cell centroids. A mask $M_{k}$ is generated for the $k$-th image so that a circular area within radius $r$ from an annotated position becomes the foreground (loss is calculated) and otherwise background. The radius of the mask should be large enough to cover a Gaussian distribution in the heat-map. Examples of the heatmap and mask are shown in the training data in Fig. 1.

Next, the trained model predicts the heatmap for all the original images $I$. Local maximum points that are larger than a threshold $th$ in the heatmap are detected as cell positions. The masked loss facilitates the reduction of the false negatives but produces many false positives. We have to select the reliable detection results from the noisy prediction to effectively perform pseudo labeling. The patch images are first generated by cropping the cell image based on the detected points. The patch image with the annotated positions ${C}$ can be considered positive labeled samples ${X}_{P}=\{x_{i}^{p}\}_{i=1,...,N_{p}}$, and the patch images except ${X}_{P}$ can be considered the unlabeled samples ${X}_{U}=\{x_{i}^{u}\}_{i=1,...,N_{u}}$.

The aim of the 3rd step is to extract optimal image features that separate the feature distribution of positive (cell) and negative label (background), which is useful to select reliable labels. To achieve this, we use PU learning [9] which uses positive labeled samples and unlabeled samples as training data, so the unlabeled samples can be classified as positive or negative.

Let $g$ be the classifier CNN that classifies an input patch image $x_{i}$ into positive (+1:cell) or negative (-1:background), where training is performed using positive and unlabeled data. In PU-learning, the following non-negative risk function [9] is used as the CNN loss:

where $\pi_{p}$ represents $prior$ probability of the positive samples (i.e., the proportion of the positive samples in the data), ${R}_{p}^{+}(g)=(1/N_{p})\sum_{i=1}^{N_{p}}{l(g(x_{i}^{p}),+1)}$, ${R}_{p}^{-}(g)=(1/N_{p})\sum_{i=1}^{N_{p}}{l(g(x_{i}^{p}),-1)}$ and ${R}_{u}^{-}(g)=(1/N_{u})\sum_{i=1}^{N_{u}}{l(g(x_{i}^{u}),-1)}$. $l$ is by default the zero-one loss, namely $l_{01}(t,y)=(1-\mathrm{sign}(ty))/2$.

After training, we obtain the feature extraction layers $\hat{g}$ by removing the FC classifier layer from $g$. We use the image features $\hat{g}(x_{i})$ in the next step.

Although PU learning aims to extract optimal features for distinguishing positive and negative samples, the proportion is unknown in real application and the distribution of the features depends on the dataset. Therefore, it is difficult to directly determine the discriminant plane in the feature space, and it may contain many incorrect labels. Instead, we take the pseudo labeling approach to select reliable samples from the unlabeled data $X_{U}$ based on the extracted features.

To minimize the risk of selecting incorrect labels as pseudo labels, we use P-classification [5] which is a technique for learning to rank. The advantage of using P-classification is that we can emphasize the classification performance at the top of the ranked list. More precisely, P-classification aims to learn a ranking (scoring) function which gives large scores to (possibly a limited number of) reliable positive samples rather than gives large scores for all positive samples. This indicates that we can reduce the risk that the high rank scored samples contain negative (incorrect) samples, (i.e., the precision on the higher-ranked samples becomes high).

Given the positive $\{x_{i}^{p}\}_{i=1}^{N_{p}}$ and negative samples $\{x_{k}^{n}\}_{k=1}^{N_{n}}$, P-classification optimizes the parameters of a scoring linear function $f_{\lambda}:=\sum_{j}{\lambda_{j}g_{j}}$, in which $g_{j}$ is the $j$-th feature of $\hat{g}(X_{i})$, using the following loss:

where $p~{}(\geq 1)$ is a hyper-parameter which controls the degree of concentration at the top of the list. Roughly speaking, when we set $p=1$, the method maximizes AUC (Area Under the receiver operating characteristic Curve), which is a standard scoring/ranking performance measure. When we set a larger $p$, the method maximizes the leftmost portion of the curve (i.e., performance at the top of the ranked list). This method only trains the discriminant layer, and thus we used $\hat{g}$ as the feature extractor network.

In this study, the negative samples are unknown and the objective is to reduce the risk of selecting incorrect labels as the pseudo-labels. Therefore, we use the unlabeled data $X_{U}$ as $\{x_{k}^{n}\}_{k=1,...,N_{n}}$, instead. Then we select the top $\alpha$ of the ranked unlabeled data is selected as the positive pseudo-labels. In addition, we also have to select the negative samples (background) from $X_{U}$ to reduce the false positives. To select bottom-negatives, we use P-classification by opposing the labels. Then, the bottom $\beta$ of the ranked unlabeled data is selected as the negative pseudo-labels.

From the selected pseudo-labels, we re-generate the pseudo ground-truth of the position heatmap and the mask that will be used in the next iteration. The center positions of the selected positive pseudo patches are additionally registered as pseudo labels $C_{P}$. It is used for both generating pseudo position heatmap and the mask for the training loss. The center positions of the selected negative pseudo patches are used for generating the mask, where a position in the pseudo heatmap has a value of 0 in the background. These steps are iteratively performed and improve the detection performance of $D$ using the reliable pseudo labels for both positive (cell) and negative (background) positions.

In the experiments, we used the five conditioned data from two datasets, BF-C2DL-HSC (HSC) from the Cell Tracking Challenge at ISBI [14, 18] and C2C12 [7]. HSC has a resolution of $1010\times 1010$ pixels. Since we assume that a large number of cells exist in a single image and only some of them can be annotated, we used frames with a large number of cells for our experiments. We used 264 images containing over 80 cells. Thirteen images with partially labeled cells were used for training, in which the total number of annotated cells is 125. The other unlabeled cells in the partially labeled images were used for unlabeled data. We used the other 251 images for test. In this data, noises appeared on the well, and many false positives were detected in the initial estimation. In C2C12 [7], myoblast cells were captured by phase contrast microscopy at a resolution of $1040\times 1392$ pixels and cultured on four different conditions: Control (no growth factor), where 10 images with partially labeled cells and 90 images are used as training and test data, respectively; FGF2 (fibroblast growth factor), with 38 images for training and 342 images for testing; BMP2 (bone morphogenetic protein), with 10 images for training and 90 images for test, 2-4) FGF2+BMP2, with 10 images for training and 90 images for test. These cells have more varied cell appearances compared with HSC. In detection, the heatmap was normalized from $0$ to $255$ and the threshold $th$ was set to $128$, and Mean squared error was used as loss function. The proportions of the labeled cells in a training image were 0.1 and the number of iterations was 2 in all experiments. The percentages of pseudo labels to be selected, $\alpha$ and $\beta$, were set to 5 % in all our experiments, in which we did not perform optimal parameter search for a fair comparison. These parameters work well as long as it is not set to an extremely large value. For $p$ in P-classification, we used 4, which was used in the reference paper[5]. This $p$-value will work if it is not too small or too large.

We used F-score as the detection performance metric. To compute F-score, we first assigned the positions in detection and ground-truth by one-by-one matching with a certain distance threshold, defined by the cell radius. We used 15 pixels for all data. Then we defined a detection position successfully assigned to a ground-truth as true positives, un-associated detected positions as false positives, and un-associated ground-truths as false negatives.

We evaluated the performance of our method using five data with three methods as ablation study; 1) Nishimura [12] that was a supervised detection method, it was simply trained using missing labels; 2) With mask, in which the detector [12] was trained with the masked loss; 3) Vicar [19] that is an image processing-based method using Yin [23] and distance transform [17]; 4) Yang [21] that uses PU-learning to directly determine the labels without pseudo labeling, in which the class prior was estimated by [4]. The idea was simply applied to our model, Tab. 1 shows the F-scores of these methods. The performance of our method was significantly better than the baseline method (Nishimura). The baseline method treated a lot of cells as background and thus degraded its performance. Supervised detection with masked loss is also worse. The performance from Vicar that is not a learning method is better than the baseline, however, it depends on the image characteristic and thus the performance was not well in some conditions. Yang [21] improved the performance in some data-sets. However, the performances in some data were significantly worse. In our method, detection results using the detection CNN trained by sparse supervised data is unstable, and thus the prior of the positive and negative samples in the test data is also unstable. Even if we use the correct prior in training, the prior in the test may be different from that in training. In this case, the PU-learning could not work well as the discriminator. For HSC, results of our method achived accuracy close to that of the fully-supervised. Our method further improved the performance in most of the data. For FGF2+BMP2, the further iteration improved the performance (0.750 at iteration 5). Fig. 2 shows the examples of our final detection results in each condition. In the results, there are several false positives (two peaks appear in a cell region) but few false negatives.

Fig. 3 shows the improvement process in iterations using HSC. In this experiments, the yellow points are given as the initial partial labels (Ground-truth). The green points are predicted to be cells by the detection network but not selected as pseudo-labeled points via P-classification. The image at iteration 0 indicates the prediction results, where the background region also has high values, so there are false positives. In iteration 0, the positive (red) and negative (blue) positions were successfully selected as pseudo labels. In iteration 1, the estimation of the background was improved. After the first iteration, the unselected detection points (green) are also good. We can observe that the positive pseudo positions (red) increase with the iteration.

Tables 3 and 3 show the precision, recall, and F-score in each iteration under train and test data , respectively. ’ittr.’ indicates the number of iteration for pseudo labeling. In the training phase, the recall of the initial estimation is very high but the precision is low. It supports our assumption that the results estimated using the masked loss have many false positives but few false negatives. We can observe that the precision was improved with keeping high recall by applying our method. Here, the improvements were saturated after the second iteration in both train and test data. This shows that our iterative pseudo learning approach improved the precision and F-score, and the few iterations worked enough. We consider that the small number of iteration does not affect the performance and improves the performance for many datasets, even though the additional iteration may improve the performance. It is our future work to find an optimal iteration using the relationship between the feature distribution and the selected pseudo labels.