Multi-view Deep One-class Classification:
A Systematic Exploration

Given a multi-view datum $\{\mathbf{x}_{train}^{(v)}\}_{v=1}^{V}$ with $V$ views for training, fusion based solutions first introduce a set of DNN based encoders to encode the input observation of each view into their latent embeddings. For the $v_{th}$ view, an encoder $Enc^{(v)}:\mathbb{R}^{d^{(v)}_{1}\times d^{(v)}_{2}\times\cdots d^{(v)}_{M_{v}}}\mapsto\mathbb{R}^{d^{(v)}_{l}}$ encodes $\mathbf{x}_{train}^{(v)}$ into a latent embedding $\mathbf{h}^{(v)}$ with a dimension $d^{(v)}_{l}$:

where $\mathbf{h}^{(v)}$ is a $d^{(v)}_{l}$-dimensional column vector. In this way, embeddings from different views can be collected as a set $\{\mathbf{h}^{(v)}\}^{V}_{v=1}$. Subsequently, fusion based methods select a fusion function $F_{f}:\mathbb{R}^{d^{(1)}_{l}\times d^{(2)}_{l}\times\cdots d^{(V)}_{l}}\mapsto\mathbb{R}^{D}$ to fuse the embeddings of different views into a $D$-dimensional vector ${\mathbf{h}}$ as the joint embedding of the multi-view datum:

Since only very weak supervision is available (i.e. all training data share a common positive label), discriminative information is unavailable for guiding the representation learning of encoders in multi-view deep OCC. Therefore, as a baseline, we propose to leverage the frequently-used reconstruction paradigm to guide the model training. To this end, a set of DNN based decoders are introduced to decode the input data of each view from the joint embedding $\mathbf{h}$: For the $v_{th}$ view, an decoder $Dec^{(v)}:\mathbb{R}^{D}\mapsto\mathbb{R}^{d^{(v)}_{1}\times d^{(v)}_{2}\times\cdots d^{(v)}_{M_{v}}}$ intends to map $\mathbf{h}$ back to $v_{th}$ view’s original input ${\mathbf{x}}_{train}^{(v)}$:

where $\hat{\mathbf{x}}_{train}^{(v)}$ is the reconstructed input of $v_{th}$ view. To train the DNN based model, one can simply minimize the differences between original inputs and reconstructed inputs:

In addition to the commonly-used mean square errors (MSE) above, other types of reconstruction loss $\ell(\cdot)$ are also applicable, such as $L_{1}$-norm reconstruction loss. During testing, an incoming multi-view datum $\{\mathbf{x}_{test}^{(v)}\}_{v=1}^{V}$ is fed into the network to obtain the reconstructed datum $\{\hat{\mathbf{x}}_{test}^{(v)}\}_{v=1}^{V}$ by Eq. 3 - 5. As the DNN based model is trained with only data from the positive class $\mathcal{C}_{t}$, one can follow the standard practice in OCC to assume that lower reconstruction errors indicate a higher likelihood that the testing datum belongs to $\mathcal{C}_{t}$. In other words, a baseline score for the $v_{th}$ view can be directly obtained by $\mathcal{S}^{(v)}(\mathbf{x}^{(v)}_{test})=-\ell(\mathbf{x}_{train}^{(v)},\hat{\mathbf{x}}_{train}^{(v)})$. Finally, we can obtain a score function by the reconstruction errors of all views:

where $F_{l}(\cdot)$ is a late fusion function that combines scores of different views into a final score, which is discussed later. An intuitive illustration of the framework is given in Fig. 2.

The key to realize a fusion based multi-view deep OCC method is the implementation of fusion function $F_{f}(\cdot)$. Thus, we design five specific ways to realize $F_{f}(\cdot)$:

1) Summation based fusion (abbreviated as SUM, and the abbreviation of other methods are similarly given). Summation based fusion combines latent embeddings from different views by summing them up. Specifically, it assumes that embeddings of all views share the same dimension $d_{l}^{(v)}=D$, and a joint embedding $\mathbf{h}$ can be yielded by:

When the dimensions of embeddings are different, we can introduce a linear mapping parameterized by a learnable matrix $\mathbf{P}^{(v)}$ to map the $v_{th}$ embedding to the shared dimension $D$: $\textstyle\hat{\mathbf{h}}^{(v)}=\mathbf{P}^{(v)}\cdot\mathbf{h}^{(v)}$. Since neural networks can flexibly map a datum into an embedding with any dimension with a linear mapping layer, we simply assume that all embeddings $\mathbf{h}^{(v)}$ share the same dimension $D$ here to facilitate analysis in the rest parts of this paper.

2) Max based fusion (MAX). Similar to summation based fusion, max based fusion also assumes a shared dimension $d_{l}^{(v)}=D$ and use the maximum of embeddings of different views as the joint embedding ${\mathbf{h}}$:

3) Network based fusion (NN). It is easy to notice that both summation based fusion and max based fusion assume a shared embedding dimension across different views. To make the fusion more flexible, it is also natural to map all latent embeddings into the joint embedding ${\mathbf{h}}$ by a fully-connected neural network with learnable parameters:

where $\mathbf{W}$ and $\mathbf{b}$ are learnable weights and biases of corresponding neurons, and $Cat(\cdot)$ and $\sigma(\cdot)$ denote the concatenation operation and the activation function respectively. Note that we can also leverage a multi-layer fully-connected network to perform DNN based fusion.

4) Tensor based fusion (TF). Tensor based fusion [41] is an emerging method in multi-view deep learning. The core idea of tensor based fusion is to combine the embeddings of different views by the tensor outer product $\textstyle\mathcal{Z}=\bigotimes_{v=1}^{V}\mathbf{h}^{(v)}$, where $\mathcal{Z}$ is a $d^{(1)}_{l}\times d^{(2)}_{l}\times\cdots d^{(V)}_{l}$ tensor. Afterwards, $\mathcal{Z}$ is fed into a linear layer with weight tensor $\mathcal{W}\in\mathbb{R}^{D\times d^{(1)}_{l}\times d^{(2)}_{l}\times\cdots d^{(V)}_{l}}$ and bias vector $b\in\mathbb{R}^{D}$ to obtain the unified representation $\mathbf{h}$:

Note here we slightly abuse the notation of matrix-vector multiplication by considering $\mathcal{W}$ and $\mathcal{Z}$ as $D\times K$ matrix and $K$-dimensional vector, where $K=\prod^{V}_{v=1}d^{(v)}_{l}$. However, a severe practical problem is that tensor based fusion requires computing the tensor $\mathcal{Z}$ and recording $\mathcal{W}$, which incurs exponential computational cost. To address this problem, we leverage the low-rank approximation technique in [42] by considering the calculation of the unified representation $\mathbf{h}$’s $k_{th}$ element, $\mathbf{h}(k)$. Suppose that the weight $\mathcal{W}$ is yielded by stacking $D$ tensors $\mathcal{W}=[\mathcal{W}_{1};\mathcal{W}_{2}\cdots;\mathcal{W}_{D}]$, where $\mathcal{W}_{k}\in\mathbb{R}^{1\times d^{(1)}_{l}\times d^{(2)}_{l}\times\cdots d^{(V)}_{l}}$ and $k=1,\cdots,D$. Thus, we have:

where $b(k)$ is the $k_{th}$ element of $b$. Then, $\mathcal{W}_{k}$ can be approximated by a set of learnable vectors as follows:

where $\mathbf{w}_{r,k}^{(v)}\in\mathbb{R}^{d^{(v)}_{l}}$ and $R$ is the rank of low-rank approximation. Since $\mathcal{Z}=\bigotimes_{v=1}^{V}\mathbf{h}^{(v)}$, tensor based fusion can be computed in a highly efficient manner by rearranging the order of inner product and outer product [42], which enables tensor based fusion to be computationally tractable.

In a training batch with $N$ multi-view data, we denote the embeddings of the $n_{th}$ multi-view datum $\{\mathbf{x}^{(v)}_{n}\}^{V}_{v=1}$ by $\{\mathbf{h}_{n}^{(v)}\}^{V}_{v=1}$, which are learned by a set of encoder networks $\{Enc^{(v)}\}^{V}_{v=1}$. Then, an alignment function $F_{a}$ is defined to compute a quantitative measure of alignment across learned embeddings of different views:

where $\mathcal{F}_{a}$ is the set of available alignment functions. As shown in Eq. 14, a key difference between alignment based solutions and fusion based solutions is that fusion usually occurs within one multi-view datum, while the alignment of two views can involve multiple multi-view data. To maximize the alignment across different views, we can equivalently minimize the alignment loss $\mathcal{L}_{a}=-\mathcal{A}$. Similar to fusion based solutions, we also resort to the reconstruction paradigm and a set of decoder networks $\{Dec^{(v)}\}^{V}_{v=1}$ to guide the training of DNNs. As a result, alignment based solutions minimize the following loss function:

where $\mathcal{L}_{r}$ is the reconstruction loss defined in Eq. 6 and $\alpha$ is the weight of alignment loss. Given a testing datum $\{\mathbf{x}_{test}^{(v)}\}_{v=1}^{V}$, we also leverage the reconstruction errors as baseline scores, which is the same as Eq. 7.

The core issue for alignment based solutions is the design of alignment function $F_{a}$. Inspired by the literature of multi-view deep learning, we propose to implement the alignment function by the following ways:

1) Distance based alignment (DIS). A commonly-seen technique to align two embeddings is to minimize their distance, as a smaller distance usually indicates better alignment. Therefore, we propose to adopt the widely-used pair-wise $L_{p}$-norm distance of all embeddings to measure alignment:

where $||\cdot||$ denotes the $L_{p}$-norm and $p$ is a non-negative integer. As can be seen from Eq. 16, it requires the embeddings of different views to share a common dimension. Note that the alignment function in Eq. 16 is equivalent to the correspondent autoencoder proposed in [48], which leverages multi-view alignment for cross-modal retrieval. A drawback of such an alignment function is that it only performs the view alignment within one multi-view datum.

2) Similarity based alignment (SIM). In addition to distance, similarity is another intuitive way to measure the degree of alignment. Given a similarity function $s(\cdot)$, we can similarly define an alignment function like Eq. 16. Nevertheless, such an alignment function only considers the view similarity within one multi-view datum. To consider the view similarity across different datum, we are inspired by [47] and propose to adopt a more sophisticated similarity measure between different views: For $i_{th}$ and $j_{th}$ view, a similarity loss $Sim(i,j)$ is computed based on $s(\cdot)$ and a hinge loss:

where $m$ is a margin. The above similarity loss encourages the embeddings from the same multi-view datum to be similar, while embeddings from two different different multi-view data to be dissimilar. The similarity function $s(\cdot)$ can be realized by multiple forms, such as inner product and cosine similarity. Then, the final alignment function can be calculated by:

3) Correlation based alignment (DCCA). Canonical correlation analysis (CCA) is a classic statistical technique for finding the maximally correlated linear projections of two vectors. Thus, a natural way for us to align different views in deep learning is the CCA’s deep variant, deep CCA [44]. To conduct correlation based alignment, we intend to maximize the correlation between two views. Specifically, we stack $v_{th}$ view’s embeddings of $N$ multi-view data in a training batch into a $d^{(v)}_{l}\times N$ embedding matrix: $\textstyle\mathbf{H}^{(v)}=[\mathbf{h}^{(v)}_{1},\cdots,\mathbf{h}^{(v)}_{N}]$, while $\mathbf{H}^{(v)}$ can be centered by $\textstyle\bar{\mathbf{H}}^{(v)}=\mathbf{H}^{(v)}-\frac{1}{N}\mathbf{H}^{(v)}\cdot\mathbf{1}$, where $\mathbf{1}$ is a $N\times d^{(v)}_{l}$ all-1 matrix. With the embedding matrix $\mathbf{H}^{(i)}$ and $\mathbf{H}^{(j)}$ for the $i_{th}$ and $j_{th}$ view, we first estimate the covariance matrices $\textstyle\sum_{ii}=\frac{1}{N-1}\bar{\mathbf{H}}^{(i)}\cdot\bar{\mathbf{H}}^{(i)\top}+r\mathbf{I}$, $\textstyle\sum_{ij}=\frac{1}{N-1}\bar{\mathbf{H}}^{(i)}\cdot\bar{\mathbf{H}}^{(j)\top}$ and $\textstyle\sum_{jj}=\frac{1}{N-1}\bar{\mathbf{H}}^{(j)}\cdot\bar{\mathbf{H}}^{(j)\top}+r\mathbf{I}$, where $r$ is the coefficient for regularization and $\mathbf{I}$ is an identity matrix. With estimated covariance matrices, we compute an intermediate matrix $T_{ij}=\textstyle\sum^{-{1}/{2}}_{ii}\cdot\sum_{ij}\cdot\sum^{-{1}/{2}}_{jj}$. It can be proved that the correlation of view $i$ and $j$ is the matrix trace norm of $T_{ij}$ [44]:

The final alignment function can be calculated by:

Suppose that the deep OCC model $\mathcal{M}^{(v)}$ is trained with data from the $v_{th}$ view. Given a newly incoming multi-view datum $\{\mathbf{x}^{(v)}_{test}\}^{V}_{v=1}$, the OCC result for the $v_{th}$ view is given by:

The final score for the multi-view datum $\{\mathbf{x}^{(v)}_{test}\}^{V}_{v=1}$ is computed by a late fusion function $F_{l}(\cdot)$:

The choice of deep OCC model plays a center role in designing tailored deep OCC solutions. This paper introduces two representative deep OCC methods in the literature to construct baseline models for multi-view deep OCC: Standard deep autoencoders (DAE) and the recent deep support vector data description (DSVDD) [3]:

1) DAE based solution (DAE). DAE leverages DNNs as the encoder $Enc^{(v)}$ and the decoder $Dec^{(v)}$ to reconstruct input data from a low-dimensional embedding. Formally, given $N$ training data $\{\mathbf{x}^{(v)}_{n}\}^{N}_{n=1}$ from the $v_{th}$ view, DAE requires to minimize the reconstruction loss:

where $\bm{\theta}^{(v)}_{E}$ and $\bm{\theta}^{(v)}_{D}$ represent the learnable parameters of the encoder network $Enc^{(v)}$ and decoder network $Dec^{(v)}$ respectively, and $\lambda$ is the weight for the $L_{2}$-norm regularization term. For inference, the reconstruction errors are often directly used as scores:

DAE based baseline is also viewed as the most fundamental baseline for multi-view deep OCC.

2) DSVDD based solution (DSV). In general, DSVDD intends to map embeddings of data from the positive class $\{\mathbf{h}^{(v)}_{n}\}^{N}_{n=1}$ to a hyper-sphere with minimal radius. To be more specific, DSVDD can be implemented by a simplified version or a soft-boundary version [3]. Since the simplified version enjoys less hyperparameters and better performance in practice, we choose it to perform multi-view deep OCC. Specifically, simplified DSVDD encourages embeddings of all data to be as close to a center $\mathbf{c}^{(v)}$ as possible. Formally, simplified DSVDD requires to solve the following optimization problem:

where $\bm{\theta}^{(v)}$ are the learnable parameters of DSVDD, and $\lambda$ is the weight for the $L_{2}$-norm regularization term. The encoder $Enc^{(v)}$ can be pre-trained in a DAE fashion. The non-zero center $\mathbf{c}^{(v)}$ is initialized before training and can be adjusted during training. The above optimization problem can be efficiently solved by gradient descent. During inference, one can score a test datum $\mathbf{x}^{(v)}_{test}$ by calculating the distance between its embedding $\mathbf{h}_{test}^{(v)}$ and the center:

The basic intuition for generative pretext tasks is to generate data from some views based on data from other views. Formally, given a multi-view datum $\{\mathbf{x}^{(v)}\}_{v=1}^{V}$, we partition the view indices into two subsets $\mathcal{P}$ and $\mathcal{Q}$, which satisfy:

Note that the intersection of $\mathcal{P}$ and $\mathcal{Q}$ may not be empty. By $\mathcal{P}$ and $\mathcal{Q}$, we can partition the multi-view data into two sets of data, $\{\mathbf{x}^{(i)}\}_{i\in\mathcal{P}}$ and $\{\mathbf{x}^{(j)}\}_{j\in\mathcal{Q}}$. The goal of generative pretext tasks is to generate $\{\mathbf{x}^{(j)}\}_{j\in\mathcal{Q}}$ by taking $\{\mathbf{x}^{(i)}\}_{i\in\mathcal{P}}$ as inputs. To fulfill this task, we propose to introduce $|\mathcal{P}|$ encoder networks and $|\mathcal{Q}|$ decoder networks, where $|\cdot|$ denotes the number of elements in the set. $\{\mathbf{x}^{(i)}\}_{i\in\mathcal{P}}$ is first mapped to the embedding set $\{\mathbf{h}^{(i)}\}_{i\in\mathcal{P}}$ by encoders, and a joint embedding is then obtained by:

where $F_{f}(\cdot)$ can be any fusion function defined in Sec. 4.1.2. The decoder networks use $\mathbf{h}^{\mathcal{P}}$ as the input to infer the data $\{\mathbf{x}^{(j)}\}_{j\in\mathcal{Q}}$, which aims to learn:

where $\bm{\theta}_{D}^{(j)}$ is the set of learnable weights for decoder $Dec^{(j)}$. In this way, $\{\mathbf{x}^{(j)}\}_{j\in\mathcal{Q}}$ is used as supervision signal to guide the training of encoders and decoders. Similarly, the generation errors $\ell(Dec^{(j)}(\mathbf{h}^{\mathcal{P}}),\mathbf{x}^{(j)})$ can be used for scoring during inference.

There are many ways to divide $\mathcal{P}$ and $\mathcal{Q}$, we select two of them to build our baseline solutions here:

1) Plain prediction (PPRD), where $\mathcal{Q}=\{v\}$ and $\mathcal{P}=\{1,2,\cdots,V\}-\mathcal{Q}$. It means that we predict data from the $v_{th}$ view by data from the rest of views. Due to the lack of standard to select a specific $v$, we vary $v$ from $1$ to $V$, and alternatively use each view as learning target, which results in multiple rounds of prediction. To avoid excessive computational cost, we introduce $V$ encoders and $V$ decoders in total, and the $v_{th}$ encoder and decoder are specifically responsible for data of the $v_{th}$ view in each round of prediction. The final score is yielded by averaging the results of all rounds of prediction.

2) Split prediction (SPRD), where $\mathcal{P}=\{v\}$ and $\mathcal{Q}=\{1,2,\cdots,V\}$. It means that we predict data of all views by a data from the $v_{th}$ view. Likewise, we also alternatively use data of each view to predict data of all views, and introduce $V$ encoders and $V$ decoders that are shared in different rounds of prediction. As shown above, generative pretext tasks aim to maximally capture the inter-view correspondence during representation learning, which cannot be realized by previous baseline solutions.

Image data are the most fundamental data type in deep learning. We process image data into multi-view data by the following means:

1) Multiple image descriptors. Many image descriptors have been proposed to depict different attributes of images, such as texture, color and gradient. Therefore, it is natural to convert a single image into a multi-view data by describing it with different image descriptors. In this paper, we choose several popular image benchmark datasets with comparatively small images (e.g. $32\times 32$ images), which are less complex for image descriptors to depict: MNIST³³3http://yann.lecun.com/exdb/mnist/, FashionMNIST⁴⁴4https://github.com/zalandoresearch/fashion-mnist/, CIFAR10⁵⁵5https://www.cs.toronto.edu/~kriz/cifar.html, SVHN⁶⁶6http://ufldl.stanford.edu/housenumbers/, CIFAR100⁵ and nine image datasets from the MedMNIST dataset collection⁷⁷7https://medmnist.github.io/. To obtain multi-view data, we extract six types of features, i.e. color histogram, GIST, HOG2x2, HOG3x3, LBP and SIFT, which are implemented by a feature extraction Toolbox⁸⁸8https://github.com/adikhosla/feature-extraction.

2) Multiple pre-trained DNN models. Classic image descriptors often find it hard to describe high-resolution images effectively. Therefore, to convert high-resolution images to multi-view data, we propose to describe them by multiple pre-trained DNN models with different network architectures. Those DNN models are usually pretrained on a large-scale generic image benchmark dataset like ImageNet [56], while different network architectures enable them to acquire image knowledge from different views. We extract the outputs from the penultimate layer of each pretrained DNN model as the representations of the image. For high-resolution image data, we collect image data from the Cat_vs_Dog⁹⁹9www.diffen.com/difference/Cat_vs_Dog/ dataset and MvTecAD¹⁰¹⁰10https://www.mvtec.com/company/research/datasets/mvtec-ad/ dataset collection that contains fifteen datasets. As to DNN architectures, we select VGGNet [57], Inceptionv3 [58], ResNet34 [59] and DenseNet121 [60] pretrained on ImageNet. In addition, it is worth mention that the MvTecAD dataset collection is specifically designed for evaluating OCC models, which makes it even more favorable for multi-view Deep OCC.

Compared with image data, video data contain both spatial and temporal information, so it is even more natural to transform them into multi-view representation. Since anomaly detection is a representative application of OCC, we simply collect video data from benchmark datasets that are designed for the video anomaly detection (VAD) task [61]. To yield video data from a different view, we calculate the optical flow map of each video frame by a pretrained FlowNetv2 model [62]. In this way, each video frame is represented from the view of both RGB and optical flow, which depict videos by both appearance and motion. Afterwards, we leverage the joint foreground localization strategy from [63], so as to localize both daily and novel video foreground objects by bounding boxes. Based on those bounding boxes, we can extract both corresponding RGB and optical flow patches from the original video frame and optical flow map respectively, which serve as a two-view representation of each foreground object in videos. Extracted patches are then normalized into the same size ($32\times 32$ patches). As to VAD datasets, we select UCSDped1/UCSDped2¹¹¹¹11http://www.svcl.ucsd.edu/projects/anomaly/dataset.htm, Avenue¹²¹²12http://www.cse.cuhk.edu.hk/leojia/projects/detectabnormal/dataset.html, UMN¹³¹³13http://mha.cs.umn.edu/proj_events.shtml#crowd and ShanghaiTech¹⁴¹⁴14https://svip-lab.github.io/dataset/campusdataset.html. For VAD datasets that provide pixel-level ground-truth mask for abnormal video foreground (UCSD ped1/ped2, Avenue and ShanghaiTech), those patches that are overlapped with any anomaly mask are labelled as $-1$, other patches are labelled as $1$. Although UMN dataset does not provide pixel-level mask, its anomalies happen at a certain stage, and all foreground objects exhibit abnormal behavior at that stage. Therefore, we simply label each foreground patch in that stage as $-1$, otherwise labelled as $1$. In this way, we can yield video based multi-view datasets, which are readily applicable to evaluate multi-view deep OCC with real-world application background.

To enable a better insight to devised multi-view deep OCC baselines, we conduct an experiment to compare best baselines’ performance and the best single-view performance on each benchmark dataset in terms of AUROC. The best single-view performance is obtained by training a deep autoencoder with data from one single views and selecting the best performer among the obtained deep autoencoders. In particular, it should be noted that the best single-view performance is actually hindsight, i.e. it is usually not practically accessible due to the absence of the negative class in multi-view deep OCC. Therfore, it is merely used as a reference to reflect how existing baselines exploit multi-view information. The results are shown in Fig. 5, and we can come to an interesting conclusion: Despite of a systematic exploration, current baselines still suffer from insufficient capability to exploit multi-view information for multi-view deep OCC. Specifically, on 12 out of the total 26 datasets, the performance of best baseline is still inferior to the best single-view performance. However, an ideal multi-view learning model is supposed to be superior or comparable to the best single-view performance. Such results imply two facts: First, it is discovered that existing baselines are still unable to find a perfect way to exploit the contributing information embedded in each view. Second, redundant information in multi-view data could be detrimental to the multi-view deep OCC performance. As a consequence, there is a large room for developing improved multi-view deep OCC solutions.

In this section, we will discuss the impact of typical hyperparameters for the devised baselines: The rank number $R$ for tensor based fusion, the margin $m$ for similarity based fusion and the weight of alignment loss $\alpha$ for alignment based baselines (DIS, SIM and DCCA). We choose the $R$, $m$ and $\alpha$ value from $\{4,8,16,32,64\}$, $\{0,1,3,5,7\}$ and $\{0.01,0.1,0.5,0.9,0.99\}$ respectively, and show the corresponding performance on representative datasets in Fig. 6. Surprisingly, we notice that the performance under different hyperparameter settings remains stable in the majority of cases. The performance fluctuations are usually within the range of $1\%$, except for the case of DCCA on UCSDped1. Consequently, we can speculate that a breakthrough of performance requires progress on model design, and tuning hyperparameter may not produce a performance leap.

As a common component for almost all baselines, late fusion has a major influence on the performance multi-view deep OCC. Since we assume that no datum from negative classes are available for validation, it is hard to apply many existing late fusion solutions here. As a preliminary effort, we take DAE for an example and explore three simple strategies for late fusion: Averaging strategy (LF-AVG, used by default), max-value strategy (LF-MAX) and min-value strategy (LF-MIN), which compute the final score by the mean, maximum and minimum of all views’ scores. For simplicity, we test them on existing multi-view datasets and show the results in Table IV. As it is shown by Table IV, the averaging strategy almost constantly outperforms max-value and min-value strategy (except for BDGP and Wiki). The min-value strategy also achieves acceptable results in most cases, which is consistent with our intuition that any abnormal view should signify an abnormal datum. However, it is noted that the max-value strategy can produce very poor fusion results, e.g. on Citeseer and Cora dataset. Therefore, the averaging strategy could still be an informative baseline late fusion strategy for multi-view deep OCC, which is somewhat similar to the case of multi-view learning.