Predicting the Reliability of an Image Classifier under Image Distortion

Many image classification models have assisted humans from daily ordinary tasks like shopping (Google Lens) and entertainment (FaceApp) to important jobs like healthcare (Calorie Mama) and authentication (BioID).

A well-known weakness of image-classifiers is that they are often vulnerable to image distortions i.e. their performance significantly drops if the input images are distorted [1]. As illustrated in Figure 1, a ResNet model achieved 99% accuracy on a set of CIFAR-10 images. When the images were slightly rotated, its accuracy dropped to 82%. It predicted wrong labels for $20^{\circ}$ rotated images although these images were easily recognized by humans. In practice, input images can be distorted in various forms e.g. rotated images due to an unstable camera, dark images due to a poor lighting condition, noisy images due to a rainy weather, etc.

For a quality control purpose, we need to evaluate the image-classifier under different distortion levels to check in which cases it is unreliable/reliable. As this task is very time- and cost-consuming, it is important to perform it automatically using a machine learning (ML) approach. In particular, we ask the question “can we predict the model reliability under an image distortion?”, and form it as a binary classification problem. Assume that we have an image-classifier $T$ and a set of labeled images ${\cal D}$ (we call it verification set). We define the search space of distortion levels ${\cal C}$ as follows: (1) each dimension of $\mathcal{C}$ is a distortion type (e.g. rotation, brightness) and (2) each point $c\in\mathcal{C}$ is a distortion level (e.g. {rotation=20^∘, brightness=0.5}), which is used to modify the images in ${\cal D}$ to create a set of distorted images ${\cal D}^{\prime}_{c}$. The model $T$ is called “reliable” under a distortion level $c$ if $T$’s accuracy on ${\cal D}_{c}^{\prime}$ is above a stipulated threshold $h$, otherwise “non-reliable”. Recall the earlier example, if we choose the threshold $h=95\%$, then the ResNet model is non-reliable under $20^{\circ}$-rotation as its accuracy is only 82%. In other words, the distortion level $20^{\circ}$-rotation has a label “non-reliable”. Our goal is to build a distortion-classifier $S$ that receives a distortion level $c\in{\cal C}$ and classifies it as $0$ (“non-reliable”) or $1$ (“reliable”). For simplicity, we treat “non-reliable” as negative label whereas “reliable” as positive label.

The process to train the distortion-classifier $S$ consists of three steps. (1) Construct a training set: a typical way to create a training set for $S$ is to randomly sample distortion levels from the search space ${\cal C}$ and computing their corresponding labels. Given a $c_{i}\in{\cal C}$, we compute the accuracy $a_{i}$ of the model $T$ on the set of distorted images ${\cal D}^{\prime}_{c_{i}}$. If $a_{i}\geq h$, we assign $c_{i}$ a label “1”, otherwise a label “0”. As a result, we obtain a training set ${\cal R}=\{c_{i},\mathbb{I}_{a_{i}\geq h}\}_{i=1}^{I}$, where $\mathbb{I}_{a_{i}\geq h}$ is an indicator function and $I$ is the sampling budget. We illustrate this procedure to construct the training set ${\cal R}$ in Figure 2. (2) Rebalance the training set: using random distortion levels often leads to an imbalanced training set ${\cal R}$ as a majority of them fall under negative class, especially when the threshold $h$ is high or model performance under distortion is generally poor. Thus, we need to rebalance ${\cal R}$ using an imbalance handling technique like SMOTE [2], NearMiss [3], or generative models [4]. (3) Train the distortion-classifier: we use the rebalanced version of ${\cal R}$ to train a ML predictive model e.g. neural network to classify unseen distortion levels.

Although current imbalance handling methods can rebalance the training data, they often suffer from generating false positive samples, leading to a sub-optimal training set for the distortion-classifier $S$. In this paper, we improve the training set ${\cal R}$ of $S$ with two contributions: (1) a Gaussian process (GP) based sampling technique to create a training set ${\cal R}$ with a higher fraction of real positive samples and (2) a GP-based imbalance handling technique to further rebalance ${\cal R}$ by generating more synthetic positive samples.

GP-based sampling: we consider the mapping from a distortion level $c$ to the model’s accuracy on the set of distorted images ${\cal D}^{\prime}_{c}$ as a black-box, expensive function $f:{\cal C}\rightarrow[0,1]$. The function $f$ is black-box as we do not know its expression, and $f$ is expensive as we have to measure the model’s accuracy over all distorted images in ${\cal D}^{\prime}_{c}$. We approximate $f$ using a GP [5] that is a popular method to model black-box, expensive functions. We use $f$’s predictive distribution to design an acquisition function to search for distortion levels that have a high chance to be a positive sample. We update the GP with new samples, and repeat the sampling process until the sampling budget $I$ is depleted. Finally, we obtain a training set ${\cal R}=\{c_{t},\mathbb{I}_{f(c_{t})\geq h}\}_{t=1}^{I}$.

GP-based imbalance handling: we use SMOTE on ${\cal R}$ to generate synthetic positive samples. But, SMOTE often generates many false positive samples [6]. To solve this problem, we assign an uncertainty score for each synthetic positive sample via the variance function of a GP. We filter out synthetic samples whose uncertainty scores are high. This helps to reduce the false positive rate of SMOTE.

To summarize, we make the following contributions.

1. 1.

   We are the first to define the problem of Prediction of Model Reliability under Image Distortion, and propose a distortion-classifier to predict if the model is reliable under a distortion level.

2. 2.

   We propose a GP-based sampling technique to construct a training data for the distortion-classifier with an increased fraction of real positive samples.

3. 3.

   We propose a GP-based imbalance handling method to reduce the false positive rate when generating synthetic positive samples.

4. 4.

   We extensively evaluate our method on six benchmark image datasets, and compare it with several strong baselines. We show that it is significantly better than other methods.

5. 5.

   The significance of our work lies in providing ability to predict reliability of any image-classifier under a variety of distortions on any image dataset.

The remainder of the paper is organized as follows. In Section 2, related works on image distortion, model reliability, and imbalance classification are reviewed. Our main contributions are presented in Section 3, where we describe two GP-based methods for addressing class imbalance. Experimental results are discussed in Sections 4 while conclusions and future works are represented in Section 5.

Image distortion. Most deep learning models are sensitive to image distortion, where a small amount of distortion can severely reduce their performance. Many methods have been proposed to detect and correct the distortion in the input images [7, 1], which can be categorized into two groups: non-reference and full-reference. The non-reference methods corrected the distortion without any direct comparison between the original and distorted images [8, 9]. Other works developed models that were robust to image distortion, where most of them fine-tuned the pre-trained models on a pre-defined set of distorted images [10, 11, 12]. While these methods focused on improving the model quality, which is useful for the model development phase, our work focuses on predicting the model reliability, which is useful for the quality control phase.

Model reliability prediction. Assessing the reliability of a ML model is an important step in the quality control process [13]. Existing works on model reliability focus on defect/bug prediction [14], where they classify a model as “defective” if its source code has bugs. A typical solution has three main steps [15, 16]. First, we collect both “clean” and “defective” code samples from the model repository to construct a training set. Second, we rebalance the training set. Finally, we train a ML predictive model with the rebalanced training set.

Some works target to other reliability aspects of a model such as relevance and reproducibility [17]. However, there is no work addressing the problem of model reliability prediction under image distortion.

Imbalance classification. As the problem of classification with imbalanced data has been studied for many years, the imbalance classification has a rich literature. Most existing methods are based on SMOTE (Synthetic Minority Oversampling Technology) [2], where the synthetic minority samples are generated by linearly combining two real minority samples. Several variants have been developed to address SMOTE weaknesses such as outlier and noisy [18, 19, 20, 21, 22]. Other approaches for rebalancing data are under-sampling techniques [3], ensemble methods [23, 24], and generative models [25, 26, 27].

Predicting model reliability is an important task in the quality control process. In this paper, we solve this task in the context of image distortion i.e. we predict if an image-classifier is unreliable/reliable under a distortion level. We form this task as a binary classification process with three main steps: (1) construct a training set, (2) rebalance the training set, and (3) train a distortion-classifier. As the training set is highly imbalanced, we propose two methods to handle the imbalance: (1) a GP-based sampling and (2) SMOTE-GP.

In the GP-based sampling, we approximate the black-box function mapping from a distortion level to the model’s accuracy on distorted images using GP. We then leverage the GP’s mean and variance to form our sampling process.

In the SMOTE-GP method, we compute an uncertainty score for each synthetic positive sample. We then filter out ones whose uncertainty scores are higher than a threshold.

We demonstrate the benefits of our method on six image datasets, where it greatly outperforms other baselines.