Post-hoc Models for Performance Estimation of Machine Learning Inference

We first describe the problem setup and our general framework for designing our post-hoc model. Let $f:\mathcal{X}\rightarrow\mathbb{R}^{k}$ be the machine learning model of interest for which we want to estimate the performance; for example, a deep neural network that maps input space into a $k$ dimensional output. Let $(\bm{x},\bm{y})\sim{\cal{D}}$ be the distribution of our dataset. Given an input $\bm{x}\in\mathcal{X}$, the model outputs a prediction $\hat{\bm{y}}=f(\bm{x})$. For image recognition, $\hat{\bm{y}}$ is a probability distribution over the classes. However, in more practical scenarios, $\hat{\bm{y}}$ can contain more complex features. For instance, for object detection, $\hat{\bm{y}}$ contains the likelihoods of multiple objects, as well as their locations and sizes in the input image. During inference time, it is desirable to know the performance of the classification. This performance can be assessed through a performance metric $m(\bm{y},\hat{\bm{y}})$. For example, $m$ could be defined as the F1 score, recall, mean Average Precision (mAP), BLEU score etc. We also define a score function $s^{m}(\cdot)$ which is a processed version of the raw performance metric $m$ via

In simple use cases, we set the processing function as $s\leftarrow\text{\emph{identity}}$, and in more complex use cases, $s^{m}$ is allowed to be a sophisticated function of $m$.

Our goal is to build a post-hoc model $g$ that estimates the performance $\hat{\bm{s}}$ of the model $f$ during inference. Let $\ell(\bm{s},\hat{\bm{s}})$ be a loss function that takes the true performance $\bm{s}$ and the estimated performance $\hat{\bm{s}}$ as inputs. Let $\mathcal{S}$ be a training dataset for post-hoc modeling. Typically $\mathcal{S}=\{(\bm{x}_{i},y_{i})\}_{i=1}^{n}$ is a secondary training dataset of size $n$ independent of the one used to build $f$. When training multiple models – as in the example of dataset shift use case (Section 3.4) – we will split $\mathcal{S}$ into disjoint sets (e.g., train-validation). We use $\mathcal{S}$ to fit the model $g$ via

An overview of our approach is shown in the top row of Fig. 1, with details on each of the use cases given in Table I.

Generalized model calibration. A standard performance metric is the binary classification error $m(\bm{y},\hat{\bm{y}})={\mathbf{1}}_{\bm{y}\neq\arg\max_{j}\hat{\bm{y}}_{j}}$, where $\hat{\bm{y}}_{j}$ is the $j^{\text{th}}$ element of $\hat{\bm{y}}$. In this scenario, $\hat{g}$ aims to predict the accuracy of $f$. To accomplish this, (3.1) can be optimized with a Bayes-consistent loss function such as cross-entropy as discussed in [8] to output an estimate of the correct probability $\mathbb{P}({\mathbf{1}}_{\bm{y}=\arg\max_{j}\hat{\bm{y}}_{j}}{~{}\big{|}~{}}\bm{x})=\hat{g}\circ f(\bm{x})$.

More generally, suppose the metric $m$ takes values in $[0,1]$ and we output an estimate $\hat{m}=\hat{g}\circ f$ by solving (3.1). We can then introduce calibration errors for $m$ similar to confidence. For instance, the continuous Expected Calibration Error takes the form $\text{ECE}(\hat{m})=\operatorname{\mathbb{E}}_{(\bm{x},\bm{y})}[|\operatorname{\mathbb{E}}[m(\bm{x},\bm{y})|\hat{m}(\bm{x})=\alpha]-\alpha|]$, which is the average mismatch between the predicted and actual performance. Section 4 describes how solving problem (3.1) leads to refined calibration outputs for metrics specific to object detection.

We next show how our framework can incorporate multiple models for the model selection use case. Besides $f\triangleq f_{0}$, suppose we have multiple candidate models $F=(f_{k})_{k=0}^{K}$. Understanding the best model for an input $\bm{x}$ can be useful in a variety of scenarios, for example if the inference runs in the cloud with a library of models available [27, 2], and we need to choose the best model to run. Specifically, we define the index of the best model as

In this use case, we set the score function to be $s(\{m_{i}\}_{i=0}^{K})=\arg\max_{0\leq k\leq K}m_{i}$ with $m_{i}=m(\bm{y},f_{k}(\bm{x}))$. We solve the index prediction problem by setting $s^{m}\leftarrow s^{m}_{\text{model}}$ in (3.2)

The predicted $\hat{s}$ obtained from $\hat{g}$ is the index of the (predicted) best model.

Another use case is for a resource-constrained end device (e.g., a smartphone) that wants to choose between a smaller local DNN model, or the library of larger models available in the cloud [28]. Here, let $f_{0}$ be the lightweight inference-time efficient model that is run on the client device. $f_{1}$ is a more accurate model that is more computationally intensive and hence run on in the cloud. If $f_{0}$ on the mobile device gives an inaccurate result, ideally we would like to offload the data to the cloud for inference by the stronger models. Thus the question is: Can we predict when $f_{0}$ fails, and if the data should be offloaded; and if so, which server model $f_{k}$ should be used? Note that the difference between this offloading use case and the previous model selection use case (Section 3.2) is that we only have the result of $f_{0}$ as input to $\hat{g}$, whereas for model selection $\hat{g}$ has knowledge of all the models $f_{0},f_{1},\ldots,f_{K}$.

In our final example, we consider a dataset shift use case, where the model $f$ has been trained on one domain with data distribution ${\cal{D}}_{\text{source}}\in\mathcal{X}\times\mathcal{Y}$, and is used in a related domain with distribution ${\cal{D}}_{\text{target}}\in\mathcal{X}\times\mathcal{Y}_{\text{target}}$. The motivation is that while existing techniques [30] can improve the accuracy on the new domain ${\cal{D}}_{\text{target}}$, they may result in mis-calibrated confidence, hence necessitating post-hoc adjustment.

We consider two post-hoc models $f_{c}$ and $\hat{g}$, which are applied sequentially to the outputs of the black box model. First, model $f_{c}$ minimizes the test error in the new domain by using the prediction error as a performance metric, i.e., by solving (3.1) with $s^{m}(\bm{y},f(\bm{x}))=\bm{y}$. In many instances, this can be accomplished through logit adjustment to account for changes in class priors [30], although more sophisticated strategies might be needed depending on the amount of dataset distribution shift. $f_{c}$ is not the focus of this work, as adapting to dataset shift is a well-studied problem [24]; rather, we focus on performance estimation after dataset shift through model $\hat{g}$.

Following this adaptation to the new domain, we ask the question: Can we accurately predict the inference performance on the new domain using a post-hoc model $\hat{g}$? To do this, we define the score function for use in (3.2):

Datasets and pre-trained models: We evaluate our framework using two types of machine learning tasks: image classification and object detection.

Image classification: We use CIFAR-10 [35], which consists of 60,000 32x32 color images in 10 classes, with 6000 images per class. We split the dataset into training, two validation, and test sets, with 4500/100/300/1000 images per class, respectively. The target dataset ${\cal{D}}_{\text{target}}$ for the dataset shift use case (V1, V2) is a modified version of the two validation sets. Essentially, to model the distribution shift on the new domain, we randomly select 3 out of the original 10 classes and sample these classes with frequencies 3:3:1), then V1 is used to train the dataset shift calibrator $f_{c}$, and V2 is used to train the post-hoc model. The black-box machine learning model is a ResNet model [36].

Object detection: We use the VOC and COCO datasets. VOC [37] contains 11,540 images with objects from 20 target classes. We randomly pick 1000/500 images for validation and test, respectively. COCO [38] contains 91 classes, with 2.5 million labeled instances in 328k images in the dataset. We use 4000/1000 images for validation and test, respectively. We evaluate 9 different pre-trained models on these datasets, ranging from compressed models designed for mobile devices to more powerful models, as summarized in Table II.

Post-hoc model: For the post-hoc model, we experimented with two model designs: a 3-layer fully connected neural network and gradient boosting using XGBoost [39]. We label the former as Post-hoc-NN throughout our experiments, and the latter as Post-hoc-XGB. The neural network has 2 hidden layers. The number of neurons is two thirds of the input size for each layer (e.g., for dataset shift, the input size is 20, so the number of neurons per layer is 13 and 9 respectively). The activation function is a ReLu layer. The learning rate is set to 0.03. The gradient boosting method uses trees with maximum depth 5, subsampling ratio 0.7, learning rate 0.1 and numer of epoch 300. We varied these hyperparameters and chose the values above based on their overall performance.

Per-image features: The inputs to the post-hoc model $\hat{{\bm{g}}}$ include model features (outputs of the black-box machine learning model) and image features (features pertaining to the original input image), as shown in Table III. We created handcrafted features to summarize these inputs. The model features require summarization because the black-box model outputs might be very high-dimensional and their dimension is not fixed. The image features similarly require summarization because the high image resolution results in high-dimensional features. The intuition behind per-image features, considering the multiple models use case and number of bounding boxes feature as an example, is that knowing that there are many bounding boxes suggests a more complex image, suggesting that the post-hoc model should predict that a more powerful machine learning model is needed. In Section 4.2.1, we systematically investigate which of the per-image features correlate best with the performance metrics.

We note that, different from previous work that uses per-object features[12, 13], we use per-image features (i.e., an aggregation of the features of all the objects in an image). This allow us to overcome the influence of undetected ground truth. In other words, using per-object features alone results in undetected objects, whose information can’t be incorporated into the performance estimate (since they are false negatives and never detected). Per-image features, as we use, contain general information of the whole image, so it contains information about those undetected objects and can help us estimate per-image performance metrics such as recall and F1-score.

Metrics: The overall evaluation metrics include:

• •

  Expected calibration error (ECE) [9, 8]: The ECE measures the calibration accuracy, by sorting the predictions into $J$ bins ($J=10$ in our experiments), and counting how many samples were put into the correct bins, weighted by the empirical probability of that bin: $\mathrm{ECE}=\sum_{j=1}^{J}\frac{\left|B_{j}\right|}{n}\left|\operatorname{true}\left(B_{j}\right)-\operatorname{predict}\left(B_{j}\right)\right|$, where $n$ is the number of samples, $\operatorname{true}(B_{j})$ is the number of samples that fall in bin $B_{j}$, and $\operatorname{predict}(B_{j})$ is the number of samples that were predicted to fall into bin $B_{j}$. ECE are used in generally throughout Section 4.2, as it is one of the most popular metrics to evaluate calibration accuracy.

• •

  Coefficient of determination ($R^{2}$): The $R^{2}$ value is a measure of correlation, defined as $1-\frac{\sum_{i}\left(y_{i}-\hat{y}\right)^{2}}{\sum_{i}\left(y_{i}-\hat{y}_{i}\right)^{2}}$, where $y_{i}$ is true value, $\bar{y}$ is the mean value of y, and $\hat{y}_{i}$ is the predicted value. This metric is used in Figure 2 to show the correlation between features and performance metrics.

• •

  Spearman’s rank correlation coefficient: The Spearman correlation coefficient is defined as the Pearson correlation coefficient between two rankings. In our case, we compute the Spearman correlation coefficient between the ground truth ranking and the ranking produced by the post-hoc model (e.g., convert the predicted F1 scores to a ranking). This is used in Section 4.2.1 and Section 4.2.2 for the basic evaluation and the dataset shift use case.

Different evaluation metrics are appropriate for different scenarios. Although the ECE is widely used to evaluate calibration, it has several drawbacks, leading us to consider the other additional metrics listed above. First, when the prediction output is continuous, the ECE computation requires bins, which are artifically introduced just for evaluation purposes. Second, ECE is sensitive to arbitrary bin settings, (e.g., the more number of bins we use, the larger the error). Finally, we sometimes care more about whether one sample performs better than another, rather than the exact value of the performance metric. In the offloading use case, for example, we want to know which sample has better performance on the server compared to its peers, rather than exactly how much better it will perform. Rank correlation gives us the accuracy of these rankings, without being influenced by arbitrary settings, such as the number of bins for the ECE calculation.

The metrics we use to evaluate object detection and image classification include:

• •

  Intersection over Union (IoU) [40]: IoU measures how well a detected object’s location matches with the ground truth (for those objects correctly classified), and is defined as: $\text{IoU}=\frac{|A\cap B|}{|A\cup B|}$ where A and B are areas of the predicted and ground truth bounding boxes, respectively.

• •

  F1-score, recall, and precision: F1-score is a measure of accuracy of the classification task, defined in terms of recall ($\frac{\text{true positives}}{\text{true positives + false negatives}}$) and precision ($\frac{\text{true positives}}{\text{true positives + false positives}}$) as $2\cdot\frac{\text{ precision }\cdot\text{ recall }}{\text{ precision }+\text{ recall }}$. We compute the F1-score per image.

• •

  Accuracy: The classification accuracy is defined as $\frac{1}{n}\sum_{i}1_{\hat{y}_{i}=y_{i}}$.

Baselines: The baseline methods that we compare our post-hoc model against include:

• •

  Confidence: For image classification, we use the regular confidence values output by the machine learning model. For object detection, the model may output a both a location confidence and class confidence/probability for each object in an image. For such models (e.g., YOLO), we compute the combined confidence by multiplying the class probability (first term) and the location confidence (second and third terms) as: $P\left(\text{class}\mid\text{object}\right)*P(\text{object})*IOU$. For models without location confidence, such as SSD, we just use the class confidence.

• •

  Calibrated confidence: Confidence calibration methods include temperature scaling and vector scaling [8]. Briefly, temperature scaling scales all the logits by a scalar parameter $T$, while vector scaling is a multi-class extension that applies a linear transformation to the logits. To extend this to object detection, which also cares about an object’s location, when training the calibration model we label a detected object as having 0 confidence if there is no object actually present.

In this section, our goal is to evaluate how well post-hoc models can estimate the performance of a combination of black-box machine learning models.