Low-Dimensional Gradient Helps Out-of-Distribution Detection

The framework of OOD detection can be described as follows. We consider a classification problem with $C$ classes, where $\mathcal{X}$ stands for the input space and $\mathcal{Y}$ for the label space. The joint data distribution over $\mathcal{X}\times\mathcal{Y}$ is denoted as $D_{\mathcal{X}\mathcal{Y}}$. Let $f_{\theta}$ : $\mathcal{X}\mapsto\mathcal{Y}$ be a model trained on samples drawn i.i.d. from $D_{\mathcal{X}\mathcal{Y}}$ with parameter $\theta$. Then the distribution of ID data is denoted as $D_{in}$, which is the marginal distribution of $D_{\mathcal{X}\mathcal{Y}}$ over $\mathcal{X}$. The distribution of OOD data is presented as $D_{out}$, whose label set has no intersection with $\mathcal{Y}$. The goal of OOD detection is to decide whether a test input $x$ is from $D_{in}$ or $D_{out}$. For post hoc methods, the decision is made through a score function $S$ as follows:

where $\lambda$ is a threshold and samples with scores higher than $\lambda$ are classified as ID data. The threshold is usually set based on ID data to guarantee that a high fraction of ID data (e.g. 95$\%$) is correctly identified as ID samples.

In this section, we progressively answer two questions: 1) how to calculate gradients in the label-agnostic condition? 2) how to reduce the gradient dimension and simultaneously retain as much information as possible? In the context of OOD detection, input labels are inaccessible to users, and thus the normal cross-entropy loss used in the training process is not applicable to calculating gradients for test data. Borrowing the idea from JEM [39], we induce a label-free energy function that has the same conditional probability $p_{\theta}(y|x)=\exp(f^{y}_{\theta}(x))/\sum_{y}\exp(f^{y}_{\theta}(x))$ as the original model:

It essentially reframes a conditional distribution over $y$ given $x$ to an induced unconditional distribution over $x$. With the energy-based model $E(x;\theta)$, we can calculate the gradient given any input $x$ without $y$ as follows:

It is equivalent to a weighted average of the derivative of outputs for all labels. To eliminate the scale discrepancies among different dimensions, we normalize it using the mean and variance matrix calculated from the training data:

where $\mathcal{M}$ and $\mathcal{I}$ are the mean and variance matrix, and $G(x)$ is the normalized gradient of sample $x$. After obtaining the label-agnostic gradient, the next crucial problem is reducing its dimension to facilitate its practical utilization. Recent empirical studies, such as LoRA [17] and DLDR [15], have shown that the gradient of ID data resides in a low-dimensional subspace. The low-rank spectrum analysis [18, 19, 20] of neural tangent kernel (NTK, [40]) has also provided theoretical evidence explaining the above phenomenon. Therefore, a simple linear dimension reduction method is sufficient for extracting a low-dimensional representation of the gradient. The most commonly used algorithm is principal component analysis (PCA, [41]), which employs the top-K eigenvectors of the gradient covariance matrix as its dimension reduction matrix. Denote the eigendecomposition as follows:

where $C$ is the gradient covariance matrix computed on all the training data. Based on the diagonal values of $\Sigma$, PCA selects $K$ column vectors of $V$ corresponding to the first $K$ large eigenvalues. Then the low-dimensional representation is calculated as follows:

Because of the high dimensionality of gradients, the above eigen decomposition cannot be calculated directly in practice. Hence, we adopt a power iteration method [42, 43] to obtain the eigenvectors under reasonable computation costs, following the idea in NFK [21]. Specific algorithms are in Appendix A.

In addition to the PCA algorithm, in this paper, we also propose an alternative approach to estimate the low-dimensional subspace encompassing principal components. Drawing upon the observation that the parameter update induced by a sample has a more pronounced influence on samples belonging to the same class while exerting minimal influence on samples from other classes [44, 45, 46], it is reasonable to infer that parameter gradients from samples within the same class tend to align in similar directions. Previous empirical studies [19, 22] support the above inference by demonstrating that parameter gradients from samples of the same class exhibit a high degree of direction similarity. Moreover, our empirical experiments, detailed in Appendix B, further validate the above claim by revealing a high cosine similarity between the gradients of samples from the same class and their corresponding gradient centers. Based on these findings, we propose utilizing the average gradient (AG) of each class as an alternative subspace estimation:

In this way, the extraction of low-dimensional subspace becomes easy to implement. However, this subspace is inaccurate compared to the PCA one. Specifically, we measure the reconstruction error of gradients of ID training and test data to the PCA and AG subspace, which reflects the amount of information lost of gradients after projecting them into the low-dimensional subspace. The results in Table II indicate that the gradient information is partially lost in the AG subspace while almost completely retained in the PCA subspace. Furthermore, the similar error of AG subspace and 10-dimensional PCA subspace implies that the average gradient direction is consistent with the top-C (C is the number of classes of ID dataset) principal components of PCA, which demonstrates the rationality of using AG subspace to reduce the gradient dimension. Considering the computational cost of PCA, the AG subspace is more suitable for scenarios involving large volumes of data and complex models.

Based on the above two subspace extraction methodologies, the overall algorithm for calculating the low-dimensional gradient is shown in Alg 1. It is worth noticing that although the subspace extraction process takes some computing time, the follow-up projection process can be fast with efficient parallel calculation. Specifically, the subspace can be partitioned into multiple lower-dimensional subspaces, each of which can be stored on different GPUs. Then, the projection process can be independently executed on each GPU. By subsequently concatenating the outcomes obtained from each GPU, the final low-dimensional gradient can be obtained efficiently. Specific analysis about the computation time can be seen in Sec 5.4.

This section elucidates the application of low-dimensional gradients in OOD detection. To design effective score functions based on the gradients, we first analyze their properties to figure out the divergence between ID and OOD data. The visualization of our low-dimensional gradients presented in Figure 2 reveals a discernible phenomenon: the low-dimensional gradients derived from ID data tend to exhibit clustering, forming cohesive groups, while simultaneously manifesting a distinctive separation from the gradients originating from OOD data. This observation demonstrates a striking parallel to the clustering and separation patterns observed in forward features. Therefore, prior distance-based methods such as Maha [6] and KNN [7] can be seamlessly adapted to operate on our gradients. Furthermore, since our gradients show the separability between samples of different classes, they can serve as a distinctive form of data representation and be used to classify samples. Thus, output-based methods [1, 2] can also be combined with our gradients. Apart from the visualization analysis, we also investigate the distribution of our gradients to dig up their distinctions between ID and OOD data. As Figure 3 shows, the values of OOD gradients fall within the range of ID gradients in the top-n dimensions (n is a number close to the number of classes $C$), while in the rest dimensions, their values are noticeably larger. Hence, we can employ previous feature rectification methods [4, 5] on the rest dimensions of our gradients to obtain improved detection performance compared to the output-based methods.

Subsequently, we provide detailed explanations regarding the integration of our gradients with output-based methods [1, 2], feature rectification methods [4, 5], and distance-based methods [6, 7]. Besides, inspired by the ensemble strategy proposed in Vim [48], we also explore a simple ensemble technique that combines forward and backward information to achieve better OOD detection performance.

We first evaluate our method on the large-scale ImageNet benchmark, which poses significant challenges due to the substantial volume of training data and model parameters. The comparison results are presented in Table III, where we evaluate six different detection methods on four networks using either features or our low-dimensional gradients as inputs. Given the poor performance of the Mahalanobis method [6] on the ResNet50 model, we do not evaluate it on other networks. Additionally, since ViT uses layer normalization instead of batch normalization, we exclude the BATS method [5] from testing on the ViT model. To control our computational overhead, we randomly select 50000 training samples as the training dataset for our detection algorithms. We train the linear network in Eq.(5) with $0.01$ learning rate, $512$ batch size, and $3$ epochs, achieving a test accuracy of $74.24\%$ on ResNet50, $71.07\%$ on MobileNet-v2, $73.98\%$ on DenseNet121 and $82.78\%$ on ViT. For ReAct and BATS, we choose to truncate the last 50 dimensions of the 1000-dimensional gradients with $p=0.7$ and $\lambda=0.1$. Regarding the KNN method, we set $K=10$ for both forward and backward embeddings.

It is worth noticing that our reduced gradient integrated with the simplest detection algorithm MSP always achieves remarkable performance. For example, on the ResNet50 architecture, our method has FPR95 of $32.38\%$ and AUROC of $91.87\%$, reducing the FPR95 by $32.38\%$ and improving the AUROC by $9.05\%$ compared to the original MSP. Moreover, combing with feature rectification methods like ReAct [4] and BATS [5], our performance can be further improved. For instance, on ResNet50 model, our approach achieves SOTA average performance with FPR95 of $23.03\%$ and AUROC of $95.45\%$ based on ReAct method, surpassing the best baseline by $7.67\%$ in FPR95 and $2.15\%$ in AUROC. One expectation is for SUN dataset, where ReAct works better for features, indicating that the choice of embedding is indeed task-dependent. But overall our low-dimensional gradients are quite promising for distinguishing ID and OOD data.

There is an impeccable performance of our method on the Places OOD dataset with ResNet50 model, which implies a significant disparity in the gradient space between the ImageNet and Places datasets. To gain further insight into this disparity, we randomly choose an OOD sample in the Places dataset and an ID sample in the ImageNet dataset and analyze the density distribution of their forward features and reduced gradients. The result presented in Figure 4 demonstrates that their distinctions are marginal in the feature space but pronounced in the gradient space, which aligns with the performance we achieved on the Places dataset.

To further demonstrate the effectiveness of our low-dimensional gradients, we also evaluate our method on the CIFAR10 benchmark. The comparison results are presented in Table IV. The hyper-parameters of baseline methods are set to the recommended values, i.e., $T=1$ for Energy, $p=0.9$ for ReAct, $\lambda=0.1$ for BATS, and $K=5$ for KNN. The Mahalanobis method we tested solely relies on the original distance formulation in Eq.(9) without incorporating any input pre-processing techniques or feature ensemble approaches. Thus, its performance is expected to be inferior to what has been reported in previous works where such enhancements were employed. Comparing all the results, our proposed gradients integrated with the KNN [7] method demonstrate superior performance on diverse models, e.g., achieving the smallest FPR95 of $21.55\%$ and the highest AUROC of $96.33\%$ on ResNet18. Notably, the Mahalanobis method shows a terrible performance on the MobileNet-v2 model, which implies its high sensitivity to network architectures. In contrast, the performance of our method is consistently outstanding across different models.

In addition to the commonly-used OOD datasets, we further conduct experiments under the near-OOD setting proposed in [65]. Specifically, in the setting of CIFAR10 as ID dataset, we evaluate our method on two hard OOD datasets including CIFAR100 [66] and Tiny-ImageNet [67] with ResNet18 model. We adopt the reduced dimension $K=200$ and KNN basic score function [7], the same as our common settings on CIFAR10 benchmark. The results are shown in Table V, where our method still achieves surpassing performance compared to other approaches. Specifically, our method exceeds the best baseline in terms of FPR95 by $2.06\%$ on CIFAR100 dataset and $1.52\%$ on Tiny-ImageNet dataset.

In addition to the aforementioned approaches, there are other detection algorithms, such as ODIN [3], GradNorm [9], and Vim [48], that demonstrate outstanding performance but are not suitable for integration with our low-dimensional gradients. In this section, we compare our method with these algorithms to demonstrate our superior performance in OOD detection.

ODIN is an output-based method that incorporates the gradient of inputs as a data enhancement mechanism to obtain good detection performance. However, since that our low-dimensional gradients are not normalized to a scale of 0-1, it is difficult to determine an appropriate enhancement hyper-parameter. Hence, we do not combine ODIN with our reduced gradients.

GradNorm leverages the norm of parameter gradients as its scoring function to effectively identify OOD samples, exhibiting outstanding performance on the ImageNet benchmark dataset. Nevertheless, in our specific setting, where gradients are used as inputs to our linear network, GradNorm is not suitable.

Vim, a novel approach, capitalizes on the low dimensionality of forward features and introduces a fusion strategy that combines the reconstruction error and energy score function as its detection function. However, the high dimensionality of gradients makes it computationally intensive to calculate the reconstruction error, thereby rendering this method inappropriate to apply to the gradients. Instead, we draw inspiration from the fusion concept and propose an ensemble strategy to detect OOD samples in Section 4.5.

The comparison results are presented in Figure 5, where we use ResNet18 and ResNet50 respectively on CIFAR10 and ImageNet. Our method demonstrates comparable performance on CIFAR10 benchmark and superior performance on ImageNet benchmark, surpassing the best baseline by $5.24\%$ in AUROC.

To further improve our detection performance, we explore strategies for integrating forward and backward information, as depicted in Eq.(11). We select the best-performing baseline method using our reduced gradients to conduct this ensemble approach. The experimental results are presented in Table VI, revealing the effectiveness of information ensemble on both CIFAR10 and ImageNet benchmarks. Particularly noteworthy is the further improvement in our performance on the ImageNet dataset, achieving an FPR95 of $19.55\%$ and an AUROC of $96.12\%$.

In this section, we provide further analysis of the influence of hyper-parameters on our detection performance to demonstrate the stability of our approach. Specifically, we study the influence of the reduced dimension $K$ on our detection performance and accordingly propose the strategy of choosing $K$. Besides, since our reduced gradients are applied in different methods, we also investigate how the hyper-parameters in these methods impact our detection performance.

Effect of K. In Figure 6, we systematically analyze the effect of $K$ on six baseline methods. We vary the number of $K=\{$10, 30, 50, 70, 90, 110, 130, 150, 170, 190, 210, 230, 250$\}$ for CIFAR10 and $K=\{$50, 100, 150, 200, 250, 300, 350, 400, 450, 500, 550, 600, 650, 700, 750, 800, 850, 900, 950, 1000$\}$ for ImageNet. Our analysis reveals that the performance of our gradient-based methods improves as the dimension $K$ increases, both for the CIFAR10 and ImageNet datasets, except that the FPR95 of the KNN method reaches its lowest value when $K=230$ and exhibits a slight increase when $K=250$ on CIFAR10 benchmark.

Choice of $K$. The exception result of KNN score on $K=250$ is actually consistent with our ratio analysis presented in Appendix A, where we observe that the first 230 principal components account for nearly $100\%$ of the total variance. If the dimension is lower than $230$, the information of ID gradients cannot be covered. If the dimension is higher than $230$, the information on residual dimensions may be unexpectedly included, which accounts for a large proportion of OOD gradients while negligible in ID gradients. Since the difference between ID and OOD gradients can be more significant if magnitudes of OOD gradients are filtered out while magnitudes of ID gradients are retained, we do not expect to induce larger values for OOD gradients as the dimension increases. Therefore, the optimal reduction dimension should be the least dimension that can account for the total variance of ID gradients. Our experiments on CIFAR10 benchmark demonstrate that $200$ is a suitable dimension that is widely applicable on different architectures.

Effect of Hyper-parameters in Baseline Methods. We analyze how the following hyper-parameters influence our detection performance on their corresponding methods:

• •

  The temperature $T$ in Energy [2]

• •

  The percent $p$ in ReAct [4]

• •

  The tuning parameter $\lambda$ in BATS [5]

• •

  The sort $k$ in KNN [7]

Our experiments are conducted on ImageNet with a fixed reduced dimension of $K=1000$. The results are presented in Figure 7. For the BATS and KNN methods, we observe that our performance remains stable with the change of hyper-parameters. In the case of the Energy method, the performance deteriorates with an increase in temperature, aligning with the observation in the original Energy [2] paper. Regarding the ReAct method, we observe a notable performance improvement when the percent changes from 0.5 to 0.6, which is expected since the gradients of ID data undergo a substantial impact when $p$ is small. These results provide valuable insights into the sensitivity and stability of our approach with respect to specific hyper-parameters in different baseline methods.

Beyond addressing the curse of dimensionality [14], in this part, we take a deeper look at the benefits of dimensionality reduction for OOD detection. Firstly, we evaluate the reconstruction error of gradients to analyze the influence of dimensionality reduction on ID and OOD data. The reconstruction error is defined as $\|G-GVV^{T}\|_{2}/\|G\|_{2}$, which reflects the ratio of the gradient component in the residual space to the whole gradient magnitude. Experiments are conducted on CIFAR10 benchmark with ResNet18 model, using SVHN [53] as OOD dataset. The result is reported in Table VIII, which reveals that gradients of ID data approximately cluster in the low-dimensional space while gradients of OOD data do not fall into it. This difference induces a larger distance between ID and OOD gradients in the low-dimensional space compared to the whole space, which is verified by our empirical results in Table IX. Therefore, the benefits of our method are not limited to utilizing complete gradient information, but also to widening the difference between ID and OOD gradients in the low-dimensional space.

In this part, we compare the detection performance of using PCA subspace and the alternative cheaper one (average gradient subspace, AG) on the CIFAR10 benchmark with ResNet18 model. The results are shown in Table VII, where we adopt the KNN [7] as our basic score function. From it, we can find that the 200-dimensional PCA subspace significantly outperforms the AG subspace. Meanwhile, the performance of the 10-dimensional PCA subspace slightly exceeds that of AG, consistent with our analysis in Sec 3.2. According to the result, using PCA subspace for dimensionality reduction is a better choice. But in practice, we can flexibly choose the subspace considering the trade-off between detection performance and computational overhead, since our analysis in Sec 5.4 reveals that computing the average gradient only requires little time overhead.

In this part, we report the specific time cost of our approach. All the experiments are conducted on four NVIDIA GeForce RTX 2080Ti GPUs. Generally, our method consists of two stages: offline subspace extraction and online score calculation. The offline stage can be pre-computed and saved using the in-distribution (ID) training dataset, meaning it only needs to be performed once. The results presented in Table XI and Table XII exhibit the computation time of each stage. We can observe that the PCA-based subspace extraction takes a longer time compared to the average gradient approach (Table XI). However, as to the inference cost, the computation time of our method is almost equal to the time required for a single backward propagation as shown in Table XII, which is considered acceptable for practical applications.