Enhancing Out-of-Distribution Detection with Multitesting-based Layer-wise Feature Fusion

Many deep learning systems have achieved state-of-the-art recognition performance when the training and testing data are identically distributed. However, neural networks make high-confidence predictions even for inputs that are completely unrecognizable and outside the training distribution [47], leading to a significant decline in prediction performance or even complete failure. Therefore, the detection of out-of-distribution testing samples is of great significance for the safe deployment of deep learning in real-world applications. This detection process determines whether an input is In-Distribution (ID) or Out-of-Distribution (OOD). OOD detection has been widely utilized in various domains, including medical diagnosis [43] , video self-supervised learning [51] and autonomous driving [6].

Recent advancements in representation learning have led to the development of distance-based out-of-distribution (OOD) detection methods. These methods map a testing input into a suitable feature space and utilize a distance-based score function to determine if the testing input belongs to the in-distribution (ID) or OOD category based on its relative distance to the training data [33, 52, 58, 56]. These methods commonly depend on a pre-trained encoder, which maps the test input to an embedding space while preserving the dissimilarity between the test input and the training data. Typically, the pre-trained encoder is a sub-network extracted from a pre-trained classifier, with most existing methods employing feature mapping from the input layer to the penultimate layer. These extracted features are generally considered as high-level semantic features that exhibit strong relevance to the corresponding labels.

However, existing methods tend to overlook the feature representations extracted in shallow layers. In this work, we argue that these low-level features, which capture local and background information, might contain valuable and crucial information for reflecting the dissimilarity between the test input and the training data. We formulate the OOD detection task as a hypothesis testing problem:

Here $P_{\mathbf{x}}$ is the training distribution, $\mathcal{Q}$ is a set of distributions and $P_{\mathbf{x}}$ is not included in $\mathcal{Q}$. In the open world scenario, the distributions within the set $\mathcal{Q}$ exhibit diversity, and changes in the distribution between $P_{\mathbf{x}}$ and $Q\in\mathcal{Q}$ can occur at any level of features, including both high-level semantic features and low-level localized features. Consequently, fully leveraging the features extracted from different layers of the neural network can provide a wealth of comprehensive signals to aid the out-of-distribution detection method in identifying distributional shifts.

Several studies have highlighted the effectiveness of utilizing multi-scale features extracted from different intermediate layers for OOD detection [37, 18, 69]. For example, MOOD [37] adaptively selects intermediate classifier outputs for OOD inference based on the complexity of the test inputs. However, in the case of MOOD, the primary motivation for adaptively selecting the optimal exit is to reduce computational costs rather than enhance the OOD detection accuracy. In another recent study [18], the authors propose treating the scores computed on the features of each layer as a type of functional data and identifying out-of-distribution samples by integrating changes in functional trajectories. Nevertheless , a potential issue with this approach is that not all features extracted from the intermediate layers are relevant for OOD detection. There are multiple options available for aggregating the feature scores extracted from each layer. These options include selecting feature similarity scores for each layer and determining the metric for score trajectory differences. However, the selection and determination of these aggregation methods are currently open issues in the field. The Multi-scale OOD detection (MODE, [69]) is an attention-based method that utilizes both global visual information and local region details of images to enhance OOD detection. It introduces a trainable objective called Attention-based Local Propagation, which utilizes a cross-attention mechanism to align and emphasize the local regions of the target objects in pairwise examples. However, the aforementioned methods necessitate modifications in the pre-training method or the backbone network, as well as the selection of similarity metrics for the score trajectories.

In this paper, we propose a novel and general framework, called Multitesting-based Layer-wise Out-of-Distribution Detection (MLOD) to enhance the performance of detecting samples that the MLOD has not encountered during training. Our proposed approach utilizes commonly used pre-training methods and models and leverages p-values to normalize the detection output at each layer. It aims to determine whether there exists a layer of features in a multi-layered pre-trained MLOD that can effectively detect distributional shifts between the test sample and the training data. To accomplish this, our approach calculates p-values based on the empirical distribution of the score function across different layers and employs multiple hypothesis testing techniques to control the True Positive Rate (TPR). Additionally, our framework can identify the layer that can detect the presence of distributional shifts between the test sample and the training data. Considering the potential high correlation between features extracted from different layers of a pre-trained neural network, we adopt five multiple hypothesis testing methods to adjust the p-value. These methods include the Benjamini-Hochberg procedure [8], adaptive Benjamini-Hochberg procedure [9], Benjamini-Yekutieli procedure [10], Fisher’s method [16], and Cauchy combination test [39]. We conduct systematic experimental comparisons to illustrate the practical advantages of MLOD on several benchmarks. On CIFAR10, the MLOD-Fisher method significantly reduces the False Positive Rate (FPR) from 24.09% to 7.47% on average and consistently outperforms the other methods on five OOD datasets.

Our main contributions are summarized as follows:

• •

  We propose a novel OOD detection framework from the perspective of the multi-layer feature of deep neural networks, namely Multitesting-based Layer-wise Out-of-distribution Detection.

• •

  We provide a comprehensive evaluation of the effectiveness of MLOD through both theoretical understanding and experimental verification, focusing on multiple combinatorial tests. The main multiple test methods that we consider in our evaluation include BH, adaptiveBH, BY, Fisher method, and Cauchy method.

• •

  Extensive experiments demonstrate that MLOD outperforms post-hoc methods that solely rely on the feature of the final output layer, as well as enhance the performance of various existing OOD scores. These experiments were conducted using the current benchmarks, and the results indicate a significant improvement in performance.

The primary aim of out-of-distribution (OOD) detection is to ascertain whether a given input is sampled from the training distribution or not. Let $\mathcal{X}$ and $\mathcal{Y}$ denote the input and label space, respectively. The training distribution over $\mathcal{X}\times\mathcal{Y}$ is denoted as $\mathcal{P}_{id}$, while the marginal distribution on $\mathcal{X}$ is denoted as $\mathcal{D}_{id}$. After training a neural network on the training data derived from $\mathcal{P}_{id}$, the feature extractor is represented as $\phi(\mathbf{x})$, which is a sub-network of the pre-trained neural network. The feature-based OOD detector uses a decision function to determine whether a test input belongs to the ID or OOD sample. The decision function is defined as follows:

Here, $\mathbf{x}^{*}$ denotes the test input, and $S(\mathbf{x}^{*},\phi)$ is a scoring function to quantify the similarity between the test input $\mathbf{x}^{*}$ and the training data on the embedding space derived by the feature extractor $\phi$. The threshold $\lambda_{\phi}$ acts as a tuning parameter that regulates the probability of misclassifying an ID sample, also known as the True Positive Rate (TPR). To maintain TPR a desired level of $1-\alpha$, the threshold $\lambda_{\phi}$ is selected based on the $\alpha$-quantile of the empirical distribution of $\{S(\mathbf{x}_{i},\phi)\}_{i=1}^{n}.$ This is given by

where $\mathbb{I}\{\cdot\}$ represents the indicator function, and $\{\mathbf{x}_{i}\}_{i=1}^{n}$ corresponds to a validation set consisting of $n$ ID inputs. Therefore, $\lambda_{\phi}=\hat{F}^{-1}(\alpha;\phi)=\inf_{s\in{\mathbb{R}}}\{s:\hat{F}(s;\phi)\geq\alpha\}.$

Consider a pre-trained neural network $f$ with $m$ layers. The network can be represented as the composition of functions:

In this equation, $h$ represents the top classifier which operates on the features extracted from the $m$-th layer. Similarly, $g_{1}$ denotes the feature mapping function from the input to the first layer of features. For $2\leq i\leq m$, $g_{i}$ is the transformation function from the $(i-1)$-th layer to the $i$-th layer. We introduce the notation $\phi_{1}(\mathbf{x})=g_{1}(\mathbf{x})$, and define the mapping function from the input layer to the output of the $i$-th layer as:

In the case of OOD samples, distribution shifts can arise at any feature layer. Consequently, we can leverage the hierarchical structure of the MLOD $f$ across its layers to maximize its potential for OOD detection. This allows us to reformulate OOD detection as a layer-wise assessment of similarity between the test input and the training data.

A naive approach to implement layer-wise detection is expressed as follows:

In this approach, the test input $\mathbf{x}^{*}$ is classified as ID only if all detectors $G(\mathbf{x}^{*},\phi_{i})$ agree that $\mathbf{x}^{*}$ is an ID sample. Conversely, if there exists a layer of features $\phi_{i}(\mathbf{x}^{*})$ for which $G(\mathbf{x}^{*},\phi_{i})=\text{OOD}$, then $\mathbf{x}^{*}$ is determined to be an OOD sample.

However, this simple approach suffers from a significant drawback: the True Positive Rate (TPR) deteriorates, rendering the final outcome of the layer-wise OOD detection unreliable. It is important to note that each detector $G(\mathbf{x}^{*},\phi_{i})$ has a probability $\alpha$ of misclassifying an ID sample as an OOD sample. When aggregating the results from multiple layers, the probability of committing this error accumulates. Specifically, the probability can be expressed as $1-{(1-\alpha)}^{m}$, assuming that the feature mappings $\{\phi_{i}\}_{i=1}^{m}$ are independent. As the number of layers in the pre-trained neural network increases, the TPR can approach zero. This observation indicates that the detector presented in Equation (6) fails to maintain the TPR at the desired level.

To address the aforementioned issues and challenges, our proposed detection framework aims to achieve the following objectives:

• 1.

  Applicability to general pre-trained models: The framework should be applicable to a wide range of pre-trained models without the need for re-training or the use of specialized model architectures.

• 2.

  Standardization of layer scores: The framework should ensure that the scores from each layer are standardized, avoiding any bias in decision-making caused by the varying ranges of score distributions in the intermediate layers.

• 3.

  Fusion of layer results while maintaining desired TPR level: The framework should be able to effectively fuse the results from each layer while ensuring that the TPR remains within the desired range.

To achieve Objective 1, our framework is designed to be compatible with various types of detection scores, including output-based scores, logits-based scores, and feature-based scores. By accommodating these different types of detection scores, our framework ensures its applicability to a wide range of pre-trained models without the need for re-training or the use of specific model architectures.

To achieve Objective 2, we employ the p-value [2] to standardize the distribution of scores across different layers. For a test input $\mathbf{x}^{*}$, its p-value is defined by

In fact, when the sample size $n$ is large enough, the decision rule $\{\mathbf{x}:S(\mathbf{x},\phi)<\lambda_{\phi}\}$ in Equation (2) is equivalent to the decision rule $\{\mathbf{x}:\text{p-value of }\mathbf{x}<\alpha\}$.

To achieve Objective 3, we employ the technique of multiple hypothesis testing in statistics to adjust the p-value in order to make TPR within the desired target range. We consider five specific methods for multiple hypothesis testing:

• •

  The Benjamini-Hochberg procedure [8]: This procedure controls the false discovery rate (FDR) while controlling the proportion of false positives among the rejected hypotheses.

• •

  The adaptive Benjamini-Hochberg procedure [9]: This procedure is an adaptive version of the Benjamini-Hochberg procedure that provides a more powerful control of the FDR when the number of hypotheses tested is large.

• •

  The Benjamini-Yekutieli procedure [10]: This procedure is a modification of the Benjamini-Hochberg procedure that controls the false discovery rate under arbitrary dependency structures.

• •

  The Fisher’s method [16]: This method combines the p-values from multiple hypothesis tests using Fisher’s combining function to obtain an overall p-value.

• •

  The Cauchy combination test [39]: This method utilizes the Cauchy combination test to combine p-values from multiple hypothesis tests, providing a robust and powerful approach for multiple hypothesis testing.

We combine the proposed framework, MLOD, with these five methods, which are denoted as MLOD-BH, MLOD-adaBH, MLOD-BY, MLOD-Fisher, and MLOD-Cauthy respectively.

Recall the difinition of $\phi_{i}$ in Equation (5). Given a test input $\mathbf{x}^{*}$, we compute the score value $S(\mathbf{x}^{*};\phi_{i})$ and obtain corresponding p-values denote as $p_{i}$. After going through all $\phi_{i}$, we obtain $m$ p-values: $\{p_{1},...,p_{m}\}$. Let the desired level of TPR is $1-\alpha$. The details of the five methods are described below:

The investigation into out-of-distribution (OOD) detection within deep neural networks has been undertaken through diverse perspectives, primarily spanning density-based, distance-based, and classification-based methods. Density-based methods explicitly model in-distribution samples using probabilistic models, flagging test data in low-density regions as OOD. As exemplified in the study by [33], class-conditional Gaussian distributions are utilized to model the distributions of multiple classes within in-distribution samples. [72] introduces a more expressive density function based on deep normalizing flow. Recent approaches have delved into novel OOD scores, with likelihood regret [63] proposing a score applicable to variational auto-encoder (VAE) generative models.

Distance-based methods operate on the premise that OOD samples should exhibit relatively greater distances from the centers of in-distribution samples. In [33], OOD detection is achieved by evaluating the Mahalanobis distance between test samples and their nearest class-conditional distributions. Another non-parametric approach [56] involves computing the k-nearest neighbors (KNN) distances between the embeddings of test inputs and training set embeddings. Due to its lack of imposed distributional constraints on the underlying feature space, this method exhibits enhanced flexibility and generality. In addition, several studies make use of the spatial separation between the embedding of the input and the centroids of respective classes[59, 25, 17]. Other works such as SSD+ [50] have adopted off-the-shelf contrastive losses for OOD detection. However, this approach results in embeddings that exhibit insufficient interclass dispersion. CIDER [40] specifically addresses this issue by optimizing for substantial inter-class margins, thereby yielding more favorable hyperspherical embeddings.

In the domain of classification-based methods, a baseline for OOD sample detection was established by using maximum softmax probability (MSP) as the indicator score from a pre-trained network [20]. The ODIN method [35] in early research improved upon this benchmark, employing temperature scaling and input perturbation to amplify the separability of ID and OOD samples. The progression continued with Generalized ODIN [22], an extension of ODIN [35], which introduced a specialized network for learning temperature scaling and a strategy for selecting perturbation magnitudes. Recognizing challenges associated with overconfident posterior distributions for OOD data when employing softmax confidence scores, [38] proposed a novel approach utilizing the energy score derived from logit outputs for OOD detection. To address the issue of high-confidence predictions for OOD samples within pre-trained models, ReAct [54] proposed a simple and effective technique aimed at mitigating model overconfidence on OOD data. Another avenue of OOD detection, denoted as OE [21] methods, involves the utilization of a set of collected OOD samples during training to assist the learning of ID/OOD discrepancy. However, such outlier exposure approaches hinge on the availability of OOD training data. In scenarios where no OOD samples are accessible, certain methodologies endeavor to synthesize OOD samples to enable ID/OOD separability. A recent study, VOS [15], proposed the synthesis of virtual outliers from the low-likelihood region in the feature space, which is more tractable given lower dimensionality. Noteworthy contributions from MOS [27] have advocated for OOD detection in large-scale settings, aligning more closely with real-world applications.

In this work, we introduces the MLOD framework, which leverages multitesting-based layer-wise feature fusion for OOD detection. The proposed framework is applicable to various pre-trained models and is supported by a comprehensive experimental analysis that evaluates the performance of various methods. This analysis demonstrates the efficacy of MLOD in improving OOD detection.