DDD: Discriminative Difficulty Distance for plant disease diagnosis

Metric learning has emerged as a prominent research area in recent years, particularly in computer vision. Computer vision has gained significant attention due to the considerable variation in visual features observed among data samples from the same class. Wang et al. introduced the Triplet loss  (Wang et al. 2014), a loss function designed to learn similarity distances from images directly. The main goal of this approach is to construct a separable embedding space that effectively captures subtle visual similarities between images within the same category. Building on this foundation, Sohn et al. expanded the Triplet loss by introducing the multi-class N-pair loss  (Sohn 2016), offering a more flexible framework for learning representations. Additionally, Chen et al. developed baseline++  (Chen et al. 2019), a method that reduces intra-class variation by leveraging the cosine distance between input features and the weight vectors of each class. These approaches share a common focus on designing loss functions to optimize feature space distances, thereby enhancing discrimination and improving model performance.

Researchers have widely adopted contrastive learning to automate the expensive manual annotation of large amounts of data and to investigate the relationships between different forms of data for multi-modal applications. In addition, contrastive learning has achieved state-of-the-art results through its discriminative learning framework. Chen et al. introduced SimCLR  (Chen et al. 2020), a self-supervised learning approach that integrates certain supervised elements. Chechik et al. trained a large-scale image similarity model for retrieval using triplet loss  (Wang et al. 2014), based on the concept of invariant mapping and its application to metric learning  (Chechik et al. 2009). Similarly, Radford et al. developed CLIP  (Radford et al. 2021), which utilizes contrastive learning to model the relationship between images and text. These contrastive learning methods focus on the distances within the latent space, similar to distance learning, and these distances are optimized and used as part of the learning objective to improve the representation.

Measuring the distance between datasets is an essential indicator of the transferability of knowledge. Calderon-Ramirez et al. introduced DeDiM  (Calderon-Ramirez, Saul and Oala, Luis and Torrents-Barrena, Jordina and Yang, Shengxiang and Elizondo, David and Moemeni, Armaghan and Colreavy-Donnelly, Simon and Samek, Wojciech and Molina-Cabello, Miguel A. and López-Rubio, Ezequiel 2023), a metric designed to assess dataset similarity to evaluate semi-supervised learning in the context of distributional discrepancies between labeled and unlabeled datasets. Similarly, Alvarez-Melis et al. developed OTDD  (Alvarez-Melis and Fusi 2020), a method that utilizes optimal transport to compute the distance between datasets. While both approaches effectively quantify the distance between datasets, they fall short in providing targeted improvements for tasks that require handling fine-grained features or addressing significant domain gaps.

In this paper, we propose the concept of Discriminative Difficulty Distance (DDD) as a new metric for objectively quantifying the differences between image data or image datasets using ML models. DDD is a pseudo-distance metric between datasets calculated based on low-dimensional representations (embeddings) of data. It aims to quantify the classification difficulty of one dataset relative to another, as perceived by a ML model trained on the first dataset. As a specific application, we explore effective implementation strategies for plant disease diagnosis, a fine-grained task characterized by a significant domain gap. Calculating the DDD between the training and evaluation datasets serves as a crucial measure for assessing the diversity of the training dataset and aids in identifying challenging-to-classify classes. Moreover, it facilitates the construction of a more desirable, broader, and robust training dataset necessary for the task.

In this paper, we consider a more appropriate method for calculating DDD. Specifically, we propose that the distance between two datasets, calculated using low-dimensional representations generated by a suitably designed image encoder ($M_{E}$), serves as a viable candidate for DDD. We further evaluate its effectiveness in this same context. When we assume that the ML model $M_{C}$ has been adequately trained for the target task, its ability to correctly identify disease $a$ with high probability suggests that diagnosing disease $a$ is relatively straightforward for $M_{C}$. This implies that the characteristics of disease $a$ in the training data are similar to those in the test data. Conversely, if $M_{C}$ frequently miss-classifies disease $a$ as disease $b$, it suggests that the characteristics of disease $a$ in the training data are similar to those of disease $b$ in the test data. Based on this assumption, we can verify whether the diagnostic similarity $S_{ij}$ between disease $i$ in one dataset and disease $j$ in the other, as calculated using $M_{E}$, reflects the confusion matrix $P_{ij}$ which can be calculated from the estimated probability of data classification by the ML model. If these values are highly consistent, we can determine that the distance calculated using the low-dimensional representation constructed by $M_{E}$ is an effective DDD.

Let $X_{t}(t={\mathrm{train},\mathrm{test}})$ be the dataset to be used to calculate the similarity between datasets, $M_{E}$ be the image encoder that converts images into low-dimensional representations, and $M_{C}$ be the plant disease identifier constructed by training on a sufficiently large $X_{\mathrm{train}}$ (a total of 244,063 images in Table 1 below). The objective is to devise a construction strategy for the image encoder $M_{E}$ that yields a dataset distance closely aligned with the probability distribution of the predictions made by $M_{C}$ for each disease in $X_{\mathrm{test}}$. The following are the steps we propose for implementation.

Suppose $M_{C}$ and $M_{E}$ use the same architecture and share the training data. In that case, $R$ is necessarily higher, as the low-dimensional representations obtained for a given image will be very similar for both $M_{E}$ and $M_{C}$. Therefore, ensuring that the training data for $M_{E}$ differs from that of $M_{C}$ is crucial. For comparison and discussion, we also evaluated under these conditions in this experiment. These results are marked with †  in the Results column, as described below.

Table 1 shows an overview of the training and test data, which consists of 244,063 leaf images from 34 classes of the four crops used in this experiment. Between 2016 and 2020, experts individually cultivated the plants, inoculated them with pathogens, photographed them, and labeled the images under strict disease control at 27 agricultural experiment stations across 24 prefectures in Japan. The images generally focus on a single leaf near the center, although many also feature multiple leaves at various distances from the subject. In our experimental setup, we separated the training and test images by ensuring they came from different locations, and we strictly evaluated the test data as entirely unknown.

In this study, we adopted EfficientNetV2-m (EfficientNetV2)  (Tan and Le 2021), which was pre-trained on ImageNet21k  (Deng et al. 2009), for $M_{C}$ and $M_{E}$ with primitive data augmentation techniques including random rotation, flipping, cropping, and rectangular masking for data augmentation. As an optimizer, SGD with a learning rate of $1.0\times 10^{-4}$ and a momentum of $0.9$ was used. As will be introduced later, $M_{C}$ trained all the Training data for all four crops, whereas each of the four $M_{E}$s trained the training data for one crop.

In this study, we discussed and evaluated the optimal configuration of the encoder $M_{E}$, which acquires low-dimensional representations that calculate the distance between images to achieve DDD. To do this, we compared five models that used the same ML model as mentioned above but were trained on different images. For the evaluation of $M_{E}$, we used $R$ and $R_{d}$ defined in “Implementation and evaluation of a reasonable distance as DDD in the plant disease diagnosis task” according to the policy described in ”Implementation Policy”.

• baseline is pre-trained by large open dataset ImageNet21k.

• +cucumber has been additionally trained using the baseline cucumber training image dataset.

• +tomato has been additionally trained using the baseline tomato training image dataset.

• +eggplant has been additionally trained using the baseline eggplant training image dataset.

• +strawberry has been additionally trained using the baseline strawberry training image dataset.

Table 2, 3 shows that the $M_{E}$ fine-tuned for each plant has achieved higher $R$ than the baseline for unknown crops. This result means that in plant disease diagnosis, even for unknown crops, it is possible to obtain low-dimensional representations that show more accurate diagnostic potential than the baseline from the $M_{E}$ fine-tuned for each plant. These results suggest there are clues to plant image diagnosis other than the universal image features learned using large-scale data. Furthermore, even when a researcher fine-tunes the model on other crops,this knowledge can still be partially leveraged, serving as a valuable tool that provides an effective indicator for seeking diagnostic potential.

Let us compare the $S$ values for eggplant, obtained by +cucumber, which showed a high $R$ in Figure 1, to the $P$ values for eggplant alone. The areas with a significant tendency for diagnostic errors, beyond just the diagonal components, align closely. This result suggests that DDD can also help address bottlenecks in diagnostic ability.

Figure 2 shows that almost all $M_{E}$ models reach their maximum $R$ when $\alpha$ is between 0.1 and 5.0. This observation suggests that the most suitable similarity measure $S$ for DDD can be achieved by appropriately adjusting $\alpha$. It is worth noting that the $R$ for the +cucumber dataset is exceptionally high in the case of strawberry. Strawberry is a relatively simple task with a smaller domain-gap than other crops. So, the knowledge used to identify the disease class in the cucumber training dataset encompasses much of the knowledge used to identify the disease class in strawberry. As a result, the knowledge from the cucumber training dataset is highly transferable to the strawberry dataset, leading to a higher $R$ for the +cucumber dataset when applied to strawberry.