PRMI: A Dataset of Minirhizotron Images for Diverse Plant Root Study

Plant root systems play important roles in supporting our natural ecosystems, adapting to changes in climate, and ensuring the sustainable production of plants (Aidoo et al. 2016; Alexander, Diez, and Levine 2015; Ettinger and HilleRisLambers 2017). Phenotyping RSAs is an important component of understanding plant root systems to support and enhance these roles along with advancing many aspects of plant science research (Pieruschka and Schurr 2019; Trachsel et al. 2011; Wasson et al. 2012).

Relative to above-ground plant phenotyping, below-ground field-based plant root phenotyping is challenging. Many common approaches for field-based root phenotyping, such as soil coring and “shovelomics” (Van Noordwijk and Floris 1979; Gregory 2008; Trachsel et al. 2011), are destructive, laborious, time-consuming, and cannot be carried out in real-time or over-time. Minirhizotron (MR) technology (Majdi 1996; Johnson et al. 2001) is one of the few most widely-used approaches for phenotyping RSA non-destructively over time. Generally, before planting, MR transparent tubes are installed in the field at an angle (e.g. $45^{\circ}$) to the soil surface in locations that should eventually be directly under or near to plants of interest. Then, as the plant’s root systems grow, a high-resolution camera can be inserted along the tube to capture root images at a variety of depths as shown in Figure 1. Since the MR tubes remain in the soil during the entire growing period, the camera is able to capture time-series RGB root images providing insight into RSA development. In addition to the development of the whole root structure, MR imagery can be used to observe changes of roots themselves throughout their life cycle such as color, diameter, angle, and length changes.

Precisely segmenting roots from the soil background in MR imagery is a critical step in studying root systems using MR technology. Traditionally, MR images are manually annotated in which users trace along each individual root to indicate the location and diameter. As can be imagined, manually tracing a large number of thin, hard-to-see roots is incredibly tedious and time-consuming. This annotation step is the primary bottleneck limiting the effectiveness of MR technology for large-scale RSA studies. Given the complexity of plant RSA, biological variation, and environmental- and management- impacts on RSA, a huge amount of data is needed for comprehensive RSA studies to lead to statistically reasonable conclusions. Thus, algorithms for high-throughput automatic root segmentation tasks are essential.

Deep learning, and convolutional neural networks specifically, have shown initial great success in plant root segmentation tasks (Yasrab et al. 2020; Smith et al. 2020). The features learned by well-trained deep neural networks are more representative and effective for segmenting roots as compared to earlier image processing-based approaches (Das et al. 2015; Galkovskyi et al. 2012; Pierret et al. 2013; Haralick, Sternberg, and Zhuang 1987; Lobregt, Verbeek, and Groen 1980). However, one limitation in applying these approaches is the need for large MR image datasets for training that include the soil characteristics and plant species of interest. Transfer learning has been investigated to help alleviate the need for collecting and annotating enormous MR image training sets for each soil type and plant species (Xu et al. 2020). However, much of these investigations have been limited by the lack of public sizeable public MR image datasets due to the fact that collecting and annotating images are extremely tedious, time-consuming and often requires some knowledge and expertise.

In order to help fill this gap and introduce the computer vision community to this problem domain, we compiled a dataset, named PRMI (Plant Root Minirhizotron Imagery), containing over 72 thousand MR RGB root images across six different species, namely, cotton, papaya, peanut, sesame, sunflower, and switchgrass. These images span a variety of conditions including varying root age, root spatial structures, soil types, and depths under the soil surface. We paired binary image-level labels for each image showing whether the image contains roots along with meta-data such as crop species, collection location, MR tube number, collection time, collection depth, and image resolution. In addition, over 63 thousand of the images have been manually annotated to generate pixel-level binary masks indicating whether the pixels correspond to root or not. Annotating MR root images is challenging, so learning from weak labels is essential, particularly when transitioning a trained method from one location (i.e., soil type and properties) and one plant species (i.e., root characteristics) to another. The primary use of the PRMI dataset is for supervised/weakly-supervised semantic segmentation as well as transfer learning. The RSA features (length, diameter, etc.) can be extracted for scientific purposes after segmentation. We hope this dataset can make contributions to facilitate the automatic segmentation of roots and the research of RSA with deep learning. The dataset is publicly available: https://gatorsense.github.io/PRMI/.

There are a few public MR image datasets available as summarized in Table 1. However, these datasets are relatively small (e.g., the largest collection for one species contains only 400 images) and only three of these datasets provide pixel-level ground truth masks. These datasets have been used to develop image processing-based approaches as well as deep-learning methods for root segmentation tasks. For example, early work on the Peach1 dataset (Zeng, Birchfield, and Wells 2006) proposed an image processing-based approach that first extracted a collection of features using linearly-shaped spatial filters. The soybean (Wang et al. 2019) and the chicory (Smith et al. 2020) datasets have been used for developing deep learning segmentation approaches.

There are several annotation tools that have been used for manually tracing roots. Zeng et. al., who provided the Peach1 dataset (Zeng, Birchfield, and Wells 2006), developed Rootfly for root tracing. Le Bot et. al. developed a software DART (Le Bot et al. 2010) to analyze RSA. Commercial tools are also available such as RootSnap (CID Bioscience) and WinRHIZO Tron (Regent Instrument) (Bauhus and Messier 1999). Furthermore, general image processing software such as Photoshop and ImageJ have also been used for root tracing (Abràmoff, Magalhães, and Ram 2004). Automated tools were also developed such as EZ-RHIZO (Armengaud et al. 2009), RootNav (Yasrab et al. 2019), and RootGraph (Cai et al. 2015).

In our PRMI dataset, MR technology was used to collect root images across cotton, papaya, peanut, sesame, sunflower, and switchgrass. For each species, root images were captured from multiple plants and multiple MR tubes across a variety of depths over different time periods and varying soil and environmental conditions. Many of the root images are related to each others in time or depth. For example, for a specific species, images were captured from the same MR tube and the same depth across times (i.e., a time series of MR imagery of particular plant root architectures). Thus, some of these images will contain the same root portions at different ages, sizes, shapes and colors. Although all the root images of different species were collected by MR technology, the specifics of the MR technology were different for each sub-collection. For example, the camera used for each species was distinct, resulting in different image resolutions, dots per inch (DPI), and color profiles. For each species, there were images containing plant roots as well as images containing purely soil and background. Some example images for each species are shown in the Figure 2.

The details of the sub-collections of each species are shown in Table 2, including the dpi, resolution, number of locations, number of collection tubes, number of root images, number of non-root images, the number of images in train/val/test sets, and the total number of images. To maintain the original appearance of the root images, we compiled the dataset using raw root images in their original format without any pre-processing such as re-sizing or any normalization.

Each MR image in this collection is paired with image-level annotations and more than 63 thousand of the images are also paired with pixel-level annotation. Next, we will introduce the annotation details.

For each species, the root images were collected from different locations, dates, tubes, and depths. To ensure that the training, validation, and testing data share a similar data distribution while ensuring that testing data does not contain the same tubes and roots used for training, we split the data into train, validation and test sets based on the tubes.

We randomly selected 60% of tubes for training, 20% for validation, and the remaining 20% for test for each species other than switchgrass. In terms of switchgrass root images, to ensure confidence in the accuracy of segmentation masks in the test set, all the images with manually generated ground-truth were placed in the test set. The training set was generated by randomly selected 80% of tubes with AI-guided annotation with the remaining compiled in the validation set. In addition, all the images with only image-level annotations were placed in the training set to support weakly-supervised segmentation. The total number of images for train, validation and test sets are shown in Table 2.

The dataset could also be split according to other factors such as dates, depths, and locations for specific research questions. For example, researchers can split the dataset based on dates to study the changes of roots over time. We paired image-level labels with meta-data for each image, which can be easily used to sort and split dataset based on different attributes.

The primary proposed use of this PRMI dataset is for root segmentation tasks. To generate baseline benchmark results and show the challenges of this dataset, we trained a U-net (Ronneberger, Fischer, and Brox 2015) for supervised learning and IRNet (Ahn, Cho, and Kwak 2019) for weakly-supervised learning on each sub-collection species to show the segmentation results. All the models were trained on training set for 300 epochs using a single Nvidia A100 GPU. For each sub-collection, the model performing best on the validation set will be used to evaluate the segmentation performance on the test set. The Intersection-Over-Union (IoU) and F1 score were calculated as evaluation metrics. More details about training can be found in the appendix D.

Table 3 shows the F1 and IoU scores for each sub-collection. In general, the dataset is quite challenging as the supervised model got around 35% average IoU for all species. This dataset is even more challenging for CAM-based weakly-supervised model since the CAM methods focus on the most different parts in the images which could be complicated soil conditions underground instead of roots when roots are rare and sparse. In addition, the number of root pixels and the number of soil pixels are highly imbalanced resulting in the fact that model could be easily biased towards soil features instead of root features. Some species, such as cotton and sunflower, have relative less images having roots compared with other species. With these facts, we believe this dataset can be helpful for researchers to build and develop more robust and advanced algorithms for automatic root segmentation tasks.

In this work, we present PRMI, a large-scale dataset of diverse plant root images captured by MR technology across different species in a variety of conditions. This dataset is very well suited to study supervised/weakly-supervised semantic segmentation, transfer learning, one- or few-shot learning, and data distribution shifts. Through this work, we hope the PRMI dataset can bring more attention of computer vision community to this problem domain and facilitate the research of weak-learning mechanisms and/or transfer-learning mechanisms that can be applied with very limited data and significant changes to the data distribution.