What is Wrong with
One-Class Anomaly Detection?

With the rapid increase in the performance of deep learning-based methods, the demands for applying this technology are emerging in recent. However, simply adopting it may not be ideal due to the mismatched label information between the training and test set. A self-driving system, for example, should make the best decision on the abnormal scene or status even though it never observed this condition before. Under this scenario, it is required to detect whether the given data is unseen (abnormal) or not, to build a reliable and secure machine learning system.

The anomaly detection (AD) task focuses on to identify suspicions or abnormal events. We can categorize this task into supervised (liang2017enhancing) or unsupervised (chalapathy2017robust; oza2018one) by the training strategy. The former train the model with both normal and abnormal samples, while the latter use normal data only. Since collecting the abnormal cases is time-consuming or impossible in the real-world, unsupervised learning gets more attention despite the superior performance of the supervised scheme. In an unsupervised approach, most of the methods formulate the task as an one-class classification problem (i.e. classify as normal or not). While such simplicity works on the well-refined scenarios, they potentially fail when the given normal dataset is composed of samples from diverse classes (Figure LABEL:fig:overview_prev). Since conventional methods ignore the latent class information, they tend to draw a single binary decision boundary that loosely covering a wide range. This could be problematic when the normal dataset contains multiple latent class information. Can we leverage this semantic knowledge to guide the AD methods to judge the anomalies more accurately? We believe a different approach is needed.

Base on this assumption, we introduce a latent class-condition-based AD scenario and its benchmark datasets. This simulates the circumstance where both normal and abnormal data have multi-class samples (Figure LABEL:fig:overview_ours). We would like to emphasize that our proposed scenario is realistic; in the real-world, various (object) classes could be normal, and the abnormal cases could consist of diverse classes as well (e.g. street signs of different countries). Because of the nature of our scenario, the optimal solution needs to generate semantic-aware tight decision boundaries (as in Figure LABEL:fig:overview_ours). However, neither supervised nor unsupervised approaches can handle the scenario properly. For example, supervised models cannot be applied since no label is given. Conventional unsupervised AD methods create a single and loose decision region (since they ignore latent class information). At this point, one natural question can arise: How can we use latent semantic information? Since the latent label is accessible to the oracle (or human) only, we may detour this limitation, for example, by utilizing the pseudo-latent class label.

To tackle this, we propose a Confidence-based self-Labeling Anomaly Detection (CLAD) framework to bridge the gap between the supervised and unsupervised approaches. We model the latent class information using self-labeling so that supervised learning can be adapted (Figure 2). To do this, we first train the feature extraction network following xie2016unsupervised. Then, we cluster the training samples using the extracted latent feature and allocate pseudo labels via self-labeling (lee2013pseudo). Now the classification network can learn to predict pseudo labels by standard supervised learning. At the inference, we decide whether the test sample is abnormal or not by the confidence-based AD method (liang2017enhancing). Since our method leverages the hidden class information, it successfully avoids the generation of undesirable loose decision regions typically suffered by one-class methods. Several experiments on latent multi-class scenarios demonstrate that the proposed method substantially outperforms recent one-class AD methods.¹¹1Our code is available at https://github.com/JuneKyu/CLAD

Traditional one-class AD methods learn a decision boundary based on the given normal samples in the training dataset (Figure LABEL:fig:overview_prev). Such one-class strategy is effective when a single class label is treated as normal alone (and the rest of them are abnormal). However, this assumption may not hold in a real environment since the normal category can consist of heterogeneous semantic labels. With this circumstance, conventional one-class AD generates a loose decision boundary to cover all samples drawn from diverse classes and thus vulnerable to a false negative.

To reduce the gap between the real-world and the one-class AD scenarios, we simulate the scenario environment where the latent sub-classes exist implicitly (Figure LABEL:fig:overview_ours). With this environment, it is crucial to learn a decision boundary by seeing not only the normality of the data samples but also its semantics. Note that such class information is not observable, thus the AD framework may require learning the semantic representation in an unsupervised or self-supervised manner.

Before going into the details, let us define the problem statement. We have a training set $X_{tr}=\{\mathbf{x}_{i}\}^{N}_{i=1}$ and a test set $(X_{te},Y_{te})$. $X_{te}$ contains both normal/abnormal samples with corresponding label $Y_{te}$; $0$ if normal and $1$ otherwise. The core idea of our framework is to consider the concealed semantic information. To do that, we generate the pseudo-label $y^{*}\in\mathcal{Y}^{*}=\{1,...,L\}$, where the cardinality of $\mathcal{Y}^{*}$ is assumed pseudo-class counts. Then, our AD framework inferences a true $y\in Y_{te}$ based on the classifier $F$ that is trained with generated pairs $(X_{tr},Y^{*}_{tr})$, where $Y^{*}_{tr}=\{\mathbf{y}^{*}_{i}\}^{N}_{i=1}$.

(1) Feature extraction. First, we train the autoencoder ($E$ and $D$) with reconstruction loss on training dataset $X_{tr}$ and calculate the latent feature $\mathbf{z}$ using the trained encoder $E$.

(2) Self-labeling via clustering. We further fine-tune the encoder $E$ to increase the possibility of assigning a sample $\mathbf{x}$ to the given cluster $C_{i}$ by the Kullback-Leibler divergence between the latent feature $\mathbf{z}$ and soft-alignment of $C_{i}$ (xie2016unsupervised). We simultaneously update the encoder and cluster assignment and this makes the clustering algorithm to be robust. Then, we assign a label $y^{*}$ by referring to the allocated cluster $C_{i}$. Now the $X_{tr}$ have associated pseudo-labels ${Y}_{tr}^{*}$.

(3) Supervised classification. With (pseudo) latent class labels $Y^{*}_{tr}$ and the normal sample $X_{tr}$, a classifier $F$ is trained with a standard supervised learning scheme. By referring to the class information that is intrinsic in the latent labels, our framework can generate the semantic-aware tight decision boundaries that envelop each relevant class sample only.

(4) Confidence-based AD. Because of the tight decision boundary by the pairs $(X_{tr},Y^{*}_{tr})$, we argue that the AD can be viewed as the out-of-distribution (OOD) task. In detail, the OOD sample is defined as the data where the class label is not included in the training dataset. We assume the label set $\mathcal{Y}^{*}$ is a multi-classes label set from $Y^{*}_{tr}$. Then, we treat $(\mathbf{x},y^{*})$ as an OOD sample when the $y^{*}\not\in\mathcal{Y}^{*}_{tr}$. Because the confidence-based algorithms predict the samples as OOD when the classifier outputs a small probability for all classes, we can safely convert anomaly detection into an OOD task. (i.e., it is an abnormal sample when $y^{*}\not\in\mathcal{Y}^{*}_{tr}$).

Under these assumptions, we measure anomaly detection scores by adapting the scoring scheme in ODIN (liang2017enhancing). We define the score $s(\mathbf{x};T,\delta)=max_{i}p(\tilde{\mathbf{x}};T)_{(i)}$, where $p(\tilde{\mathbf{x}};T)_{(i)}$ is output of the classifier $F$ in each class $i$. $T$ is the parameter for the temperature scaling and $\tilde{\mathbf{x}}$ is the input term $\tilde{\mathbf{x}}=\mathbf{x}-\epsilon(\mathbf{x})$ perturbed by the reverse FGSM method (goodfellow2014explaining) that makes the OOD samples more separable. If the score $s(x;T,\delta)$ is greater than a given threshold $\delta$, then we predict the $\hat{y}$ as normal.

Baselines. We compare with one-class AD methods: OCSVM (scholkopf2001estimating), OCNN (chalapathy2017robust), OCCNN (oza2018one), SVDD (tax2004support), and DeepSVDD (ruff2018deep). We follow the implementation setups based on the official codes.

Datasets. We use following datasets: MNIST (lecun-mnisthandwrittendigit-2010), CIFAR-10 (krizhevsky2009learning), GTSRB (Stallkamp2012), and Tiny-ImageNet (ILSVRC15).

We devise the super-categories by merging the semantic labels to simulate our AD scenario. For example, MNIST is as {Curly, Straight, Mix} and GTSRB based on the semantic meanings of traffic signs. Note that both datasets share a similar domain prior which represents the text or symbols. For the natural scene datasets such as CIFAR-10 and Tiny-ImageNet, we set as {Thing, Living} and {Animal, Insect, Instrument, Structure, Vehicle}, respectively. When we evaluate the models, we pick one super-category for training and the rest of the subsets as a test dataset. Please see Appendix D for the detailed settings of the scenario.

Results. Table 1 shows the AUROC of the MNIST and GTSRB datasets. Our framework surpasses all the one-class AD methods in MNIST. Among them, it is notable that CLAD outperforms others on Mixed in a huge margin. This scenario is very challenging since it carries complex information due to the mixed shape of digits such as ‘2’ or ‘6’. For the GTSRB dataset, our CLAD reaches the best performance for SPDL, DIRC, REGN and second-best for the rest. The only method comparable to ours is DeepSVDD. However, it suffers the inconsistent performance (worst score in SPEC) while our framework shows stable and high performances for all the cases.

To demonstrate the superior performance of CLAD in the complex image domain, we evaluate it on the CIFAR-10 and Tiny-ImageNet datasets (Table 2). Our framework achieves the best performance for all the scenarios in CIFAR-10 and four of five cases in Tiny-ImageNet. Similar to GTSRB, the scores of DeepSVDD are inconsistent; we claim that DeepSVDD is sensitive to the latent class labels. It maps all the (normal) samples into a single-modal hypersphere (Figure LABEL:fig:overview_prev), making it vulnerable to the anomalies that close to the normals in terms of the class information.

We introduced a new aspect of the AD task and proposed a confidence-based AD framework. We assume that (ab)normal categories can have (unobservable) multi-class samples in contrast to the one-class AD scenarios. We believe that our scenario and method are practical in real world. One possible usage is for the industrial environment. When the normal sensor signals with various range can be altered by the surroundings, conventional one-class AD methods suffer spurious detection.

Since our framework trains a classifier using self-labeling, extracting the right representation of the data sample is crucial. However, most of the feature extraction methods have difficulty with handling a dataset bias (bahng2020learning). As future work, we will focus on the disentanglement of the scene context with a key concept of the object discovery (burgess2019monet).

This research was supported by the National Research Foundation of Korea grant funded by the Korea government (MSIT) (No. NRF-2019R1A2C1006608), and also under the ITRC (Information Technology Research Center) support program (IITP-2020-2018-0-01431) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation).