An Investigation into Glomeruli Detection
in Kidney H&E and PAS Images using YOLO

For investigations of tissue morphology and, as a result, for making diagnostic conclusions, computational pathology approaches may offer fast and reliable solutions compared to conventional microscopy-based workflows. On the other hand, any manual evaluation of tissue samples can be time-consuming, costly, and subject to both inter- and intra-observer variability [1]. Consequently, researchers have recently focused their attention on automated solutions to detect and segment tissue structures in digital pathology whole slide images (WSIs). Many studies, such as determining tissue types, rely on the accuracy of tissue pattern segmentation, which is regarded as the foundation of automated image analysis. However, due to the complexity of tissue clustering into types, with architecture such as glands and organelles overlapping with each other, establishing precise segmentation is not a simple operation. This makes distinguishing these patterns from the tissue background, and mainly from each other a challenge. In addition, histopathological images may contain noise and artifacts created during image acquisition, as well as low contrast between foreground and background [1]. Segmentation models have been widely used in digital pathology to segment cells and other regions of interest [2]. However, training these models needs pixel-level object annotations made by an expert. Detailed labels (pixel-level) for histopathology images are expensive, time-consuming, and hard to achieve [3]. Moreover, in some of the applications in histopathology, only detecting the position of the specific tissue pattern without precisely outlining the borders may be sufficient [1]. These techniques are called tissue pattern detection, and are usually faster compare to segmentation methods. The main advantage of detection models is that they construct a bounding box around the tissue of interest rather than pixel-level labelling, making its training much more convenient.

Deep object detectors typically consist of two parts: a backbone trained on ImageNet and a head used to forecast object classes and bounding boxes. One-stage object detectors and two-stage object detectors are the most common head types [4]. Regions with convolutional neural networks (R- CNN) [5] series, is a good example of a two-stage object detection category. YOLO (you-only-look-once) is one of the examples for one-stage object detectors [4] which has been studied and explored on two different applications for detecting specific tissue patterns in this paper.

YOLO is a simple concept with several advantages. To begin with, YOLO is very fast as it does not require a complicated pipeline for a regression problem. Furthermore, the mean average accuracy of YOLO is higher than that of comparable real-time systems. Therefore, the network can be considered as a real-time object detector. Secondly, with YOLO, context information about object classes is encoded as well as their appearance during training and testing, unlike sliding window and region proposal-based approaches [6]. A popular object recognition approach, Fast R-CNN [7], may misidentify background patches in an image as objects. In comparison to Fast R-CNN, YOLO creates half the number of background errors [6]. And thirdly, YOLO learns to represent objects in a universally applicable way. YOLO surpasses the best detection algorithms like deformable parts models (DPM) and R-CNN when trained on natural images and evaluated on art. YOLO is less likely to fail when applied to new domains or unexpected inputs because of its high degree of generalizability [6].

In this paper, YOLO-v4 has been employed to find particular tissue patterns in WSIs. Comparisons on the same datasets, with segmentation approaches, will be performed. The histological evaluation of “glomeruli” is critical for identifying whether a kidney is transplantable [8]. The Karpinski score, which includes the ratio of sclerotic glomeruli to total number of glomeruli in a kidney segment, is critical for determining the necessity for a single or dual kidney transplant [9]. Clinical symptoms, immunopathology, and morphological abnormalities are all factors that go into classifying glomerular disorders. To classify glomerular diseases, these anatomic structures need to be detected. Automated glomeruli identification frameworks for kidney biopsies conducted by pathologists can be quite helpful because manual examination of kidney samples is time-consuming and error-prone [9, 8]. There are several segmentation methods to detect glomeruli in kidney images [10]. However, these methods require pixel-level annotation of the images. In detection methods, only determining the location of a given tissue pattern, the glomerulus, is required without the need to precisely delineating its borders.

In the field of histopathology, the lack of image data, annotation, and labels has always been a problem [11]. Hence, it is important to validate deep networks on their generalization capability. By training a network with public datasets, and then fine-tuning it with only limited data from a specific hospital or specific resource, we may be able to significantly improve the accuracy of the network on the validation set from the same resource.

In another application, YOLO-v4 as a detection network has been trained to recognize all glomeruli in a given kidney image. Multiple experiments were designed and carried out based on different training data from two public datasets to fine-tune the model, and tested on the private dataset from the University of Michigan as an external validation on two differently stained tissues, namely periodic acid–Schiff (PAS) staining and hematoxylin and eosin (H&E) staining.

The first dataset is a public collection of 31 tiled TIFF (SVS) WSIs. The annotation of the bounding boxes of these 31 WSIs has been performed by collaborating pathologists. This data is part of the WSI datasets generated within the European project AIDPATH (source: http://aidpath.eu/). The second dataset has been used for the HubMap competition (source: https://www.kaggle.com/c/hubmap-kidney-segmentation/overview). TIFF files ranging in size from 500MB to 5GB make up the dataset containing 8 WSIs for training, and 5 WSIs for subsequent testing. The segmentation annotation was provided for each of the WSIs in this competition. The generalization of the network has been tested by training on these two public datasets, followed by the external validation on the private dataset from University of Michigan. Another (private) dataset that has been used for training and fine-tuning the models has 7 PAS stained WSIs which has been collected from the University of Michigan annotated by an expert pathologist. In Figure 1 three samples of the training dataset for the network are shown.

The three datasets, two public and one private, have been used to design and conduct 14 experiments. These experiments have been trained on different combinations of public and private datasets. Results have been validated on the private dataset from the University of Michigan with two stains, 20 PAS stained WSIs and 16 H&E stained WSIs. YOLO served as the detector network which will be described in details in the following sections.

In Figure 2, two samples of the private validation dataset along with the annotated bounding boxes have been shown. On the top is a sample of tissue derived from a H&E stained WSI, and on the bottom is a sample of tissue from a PAS stained WSI. The results, average specificity and sensitivity for all experiments. Comparison of existing segmentation methods on the same datasets are discussed in the results section. In general, one could observe that the average specificity and sensitivity are higher on the PAS validation set, because all of the images in the training dataset are PAS stained. Also, there is an improvement in average specificity and sensitivity while fine-tuning the network with only 7 PAS WSIs from the University of Michigan.

A significant step in determining whether a kidney is transplantable is the histological examination of renal samples by experienced pathologists [9, 8]. The histopathology evaluation of the number of globally sclerosed glomeruli in relation to the overall number of glomeruli is essential for accepting or rejecting a donor’s kidneys [9]. Mutliple glomerulocentric pathology classification systems are employed for native kidney diseases [12, 13, 14] emphasizing the central role of glomerular injury. In Figure 3 samples of glomeruli in kidney images are shown.

Waste and excess fluids are expelled from the human body by glomeruli, which are clusters of capillaries responsible for expulsion. It is possible to group glomerular disorders according to their clinical symptoms, etymology, immunopathology, or morphological changes [9, 8]. A condition known as “glomerulosclerosis” is the result of the kidney lesion changing its morphology; this sclerosis can impact the kidney in many ways, depending on whether or not it is global or partial [10]. The number of glomeruli detected in each kidney biopsy should be counted in daily practice. Per kidney biopsy, about 20 to 30 cuts are made [10]. Additionally, glomeruli that are completely sclerosed must be noted (the entire glomerulus). Detection of localized sclerosis will provide further information regarding the patient’s condition. Each pathology report should include this information because the number of glomeruli assessed must be representative enough to determine a diagnosis [15]. On the other hand, if the sample has numerous sclerosed glomeruli, this may suggest that the patient has chronic kidney disease with dead glomeruli. As a result, the patient may not be suited for some medications, which will help to define adequate treatment [15]. This information is also entered into the national register for glomerulonephritis [10]. Time-consuming and tiresome, the count of glomeruli is a painstaking process. Because of this, image processing methods that can identify and categorize the glomerulus are needed.

With the emergence of deep learning networks, various options for computer vision tasks such as glomeruli object identification, semantic segmentation, and instance segmentation became available [10]. For instance, some works provide a detailed assessment of object identification and instance segmentation algorithms [16]. Others provide a complete review of semantic segmentation [17]. Several recent research efforts in digital pathology have used deep neural networks for glomeruli detection and segmentation [18, 19, 20, 21, 22, 8, 23, 24, 25, 10].

For glomeruli detection, YOLO has been applied on kidney images for the first time in this paper and compared with the existing segmentation method U-Net, using the same validation dataset. There are two different tissue stains in the validation dataset.

Immunopathology, clinical symptoms, and morphological abnormalities are all factors that go into classifying glomeruli disorders [33]. To classify the glomeruli diseases, these objects need to be detected first. Therefore, the average sensitivity and specificity of the detection matters.

By having True Positives as $TP$, False Positives as $FP$, False Negatives as $FN$, and True Negative as $TN$, The formula of sensitivity and specificity metrics [34] is as follows:

For computing true positives, false positives, false negatives, and true negatives, the IoU measure has been used (intersection over union) to determine the overlap between two boundaries divided by their union. Our dataset pre-defined an IoU threshold (i.e., 0.5) in classifying whether the prediction is a true positive or a false positive. Also, false negative would be those glomeruli objects that any predicted bounding boxes have not covered. Moreover, the true negatives were calculated based on the area of the whole slide tissue minus those predicted areas that were not containing any glomeruli.

As mentioned in the last section III, 7 training datasets with two public and one private datasets have been created and validated on two datasets with different stains from the University of Michigan. In this section, the average sensitivity and specificity have been calculated for all these experiments for all images, along with the comparison with the existing segmentation method.

There have been several technological advances across health care and digital pathology in recent years. Automated segmentation and pixel analysis of digital pathology images may identify diagnostic patterns and visual cues, leading to more reliable and consistent diagnostic categorization.

Glomeruli detection, as the first step of classifying the glomeruli diseases following by diagnosing different kidney diseases, is essential and critical in digital pathology. Because of the large number of these objects in the kidney, glomeruli detection could help pathologists save considerable time by computerized quantification. This paper trained YOLO-v4 with seven different training datasets consisting of two public datasets and a private dataset from the University of Michigan. Moreover, the networks were evaluated on 20 PAS stained images and 16 H&E stained images from the University of Michigan. By training YOLO-v4 on the first public dataset, and fine-tuning by only 7 PAS stained images from the University of Michigan, experiments achieved 85% average sensitivity and 89% average specificity while validating the network on 20 PAS stained images from the University of Michigan, which was the best result out of different training datasets. For evaluating the network on H&E stained images, 70% average sensitivity and 96% average specificity were obtained while training on both public datasets, followed by fine-tuning on the 7 PAS stained images. Also, final weights of a segmentation method based on U-Net have been used to evaluate the results on the same validation datasets. The model could achieve high specificity and lower sensitivity, making this method rather unreliable compared to YOLO with higher sensitivity. Moreover, obtaining pixel-level WSI annotations for the network is time-consuming. This makes the whole procedure for fine-tuning the model with limited data harder than detection methods like YOLO, which only requires a bounding box around the target objects.