Anomaly detection (…)

A modified version of VGG-16 [Simonyan2015] is used as the encoder, in which the dense layers and the last convolutional block were removed, leaving the current adaptation with 4 blocks. The first 3 blocks contain 2 convolutional layers, each block followed by a MaxPooling layer. The last block holds 3 convolutional layers, each layer followed by Dropout [Cicuttin2016]. The architecture is depicted in Fig. 2.

After comparing this configuration with others, i.e., Inception v_3, Xception and Inception ResNet v_2, we concluded that VGG-16 has given the best results so far, as seen in Section 4.2.3 . This allows the extraction of the input image features with high resolution, using these features as input to the FPM.

The Feature Pooling Module receives the features from the encoder in order to correlate them at different scales, making it easier for the decoder to reconstruct the input image.

Lim  [Lim2019] proposed a Feature Pooling Module, in which the features from a 3 X 3 convolutional layer are concatenated with the Encoder features, resulting as input to a next convolutional layer with dilation rate of 4. The features from this layer are then concatenated with the Encoder features, hence used as input to a convolutional layer with dilation rate of 8. The process repeats itself to a convolutional layer with dilation rate of 8. The output of every layer is concatenated with a pooling layer, resulting in a 5 X 64 multi-features layers. These will finally be passed through InstanceNormalization and SpatialDropout.

After numerous experiments and tests, an improvement to this configuration was made. A convolutional layer with dilation rate of 2 is introduced, followed by the removal of the pooling layer. The output of the last layer, i.e., convolutional with dilation rate of 16, proceeds to BatchNormalization [Ioffe2015] (ensuring better results than InstanceNormalization) and SpatialDropout [Cicuttin2016] (Fig. 3).

Due to the removal of the multi-features layers, Dropout is used instead of SpatialDropout. The latter promoted independence between the resultant feature maps by dropping them in their entirety, correlating the adjacent pixels.

The proposed method receives a raw image and its corresponding mask, i.e., ground-truth, as input, both with 512 width and height. A pre-trained VGG-16 is being used. The first three layer blocks of the VGG-16 are being frozen in order to apply transfer learning to our model, since the feature extraction has been learned in a different context, the current objective is to learn the new layers and relate their features with the new problem.
The binary cross entropy loss is used, assigning more weight to the foreground in order to alleviate the imbalancement in the data, i.e., number of foreground pixels much higher than the background ones, and also taking in consideration when there are not any foreground pixels. The optimizer used is Adam [Kingma2015], proposing better results amongst other tested optimizers, i.e., RMSProp, Gradient Descent, AdaGrad [Duchi2012] and AdaDelta [Zeiler2012], and the batch size is 4.
A maximum number of 80 epochs is defined, stopping the training if the validation loss does not improve after 10 epochs, this can be achieved by using the early stopping callback. The learning rate starts at 1e-4, and is reduced after 5 epochs by a factor of 0.1 if the learning stagnates, by applying a reduce on plateau callback.

In order to fully understand the viability of the model, three different metrics are the focus throughout the evaluation. The first one is the AUC ( Area Under Curve ) [Flach2011], it is equal to the probability that the classifier will rank a randomly chosen positive example higher than a randomly chosen negative example. This metrics allows the perception of the model detecting an object correctly, i.e., if the AUC is equal to 100% then the model detected every object in it. Nevertheless, this metric only focus on classification successes, failing to consider classification errors and not providing information regarding the segmentation.

The Accuracy and the F-Measure could be potential candidates, since they are highly used and usually produce reliable results, however, since the dataset is highly imbalanced when regarding the foreground and the background pixels, these two metrics are not an option, since they are highly sensitive to imbalanced data, therefore MCC (Mathews Correlation Coefficient ) is used as an alternative [Chicco2020]. This metric ranges from $[-1,+1]$ where $+1$ represents a perfect segmentation according to the ground truth, and $-1$ the opposite.

$MCC=\frac{\left(TP\times TN\right)-\left(FP\times FN\right)}{\sqrt{\left(TP+FP\right)\left(TP+FN\right)\left(TN+FN\right)\left(TN+FP\right)}}$

To complement the previous metric, the mIoU ( mean intersection over union ) is also used. IoU is the area of overlap between the predicted segmentation and the ground truth divided by the area of union between the predicted segmentation and the ground truth, and when applied to a binary classification, a mean is calculated by taking the IoU of each class and averaging them. This metric ranges from $[0,1]$ where $1$ represents a perfect overlap with the groundtruth and $0$ no overlap at all.

$IoU=\frac{\text{Area of Overlap}}{\text{Area of Union}}$

Nevertheless, metrics such as Recall, Specificity, PWC ( Percentage of wrong classifications ), Precision and F-Measure are also going to be used in some cases, in order to compare this proposed method with other state of the art.

Using [Lim2019] as a starting point, some ablation tests were made in order to understand the importance of the different components in the architecture and what could be improved. The metrics used throughout these evaluations are stated in 4.1, in order to compare each one from the previous analysis.

With the previous configurations established, some results applied to the full datasets are compared with the state of the art.

Regarding the CDNet2014 dataset, the proposed method outperforms the top state of the art technique in this dataset, i.e., FgSegNet_v2, when using 200 frames for training, improving by a long margin the LowFrameRate class, going from 0.8897 to 0.9939 in the F-Measure, more details in Tab. 6. Some visual results can also be seen in Fig. 4, presenting results close to the ground truth, even when dealing with LowFrameRate images.

In the SBI2015 dataset, the overall F-Measure decreased from 0.9853 to 0.9447 when compared with the FgSegNet_v2, but increased from 0.8932 to 0.9447 when compared with the CascadeCNN, [Wang2017-2]. Nevertheless, the results still confirm a good overall evaluation on this dataset, compensating the higher results assembled in the CDNet2014 dataset. More details can be seen in Tab. LABEL:tab:sbi.

Last, in the CityScapes dataset, a test was made in three different classes (Road, Citizens and Traffic Signs). As seen in Tab. LABEL:tab:cityscapes and in Fig. 5, the proposed method is able to detect almost every object, confirmed by the AUC, making a good segmentation on them.