Unconstrained Face Recognition using ASURF and Cloud-Forest Classifier optimized with VLAD

The input image sent to the model is first used for detection of faces from it. For the purpose of face detection from the original image, we have used Haar-Cascade detector. [11] has the ability of detecting faces rapidly by keeping only information present in grayscale images. The major aspects of Haar-Cascade include an integral image representation of the original image which accounts for quick evaluation of Haar-like features, introduction of an efficient classifier that is modeled using a subset of features from Adaboost and finally a method for integrating more complex classifiers such that they form a cascading architecture. The integral image is computed by sum of pixels above and to the left of the location x,y and given by

where $i(x,y)$ is the original image and $ii(x,y)$ is the integral image. Finally the detected faces are sent further into the model for image preprocessing and subsequently for classification.

After receiving the detected image, it must be preprocessed to reduce the noise present in it. Noise is defined as random fluctuations in brightness or color information in images. Filtering is generally used for reducing noise in the nearby pixel values which are distant from the signal values.

The method proposed in this paper for filtering uses Bilateral filter, a non-linear, non-iterative and simple technique for blurring an image while taking care of strong edges. [2] smoothen the image by utilizing a non-linear combination derived by averaging the smooth regions of the nearby image values, preserving edges. Other filtering techniques such as Gaussian blurring [3] which performs linear operation, neglecting edges are determined by sigma and non-linear filters like median filter [4], which replace the pixel values with the median value available in the local neighborhood are outperformed by Bilateral filters. In Bilateral Filtering, weight of the intensity values of the surrounding pixels replace the intensity value of a pixel that is based on Gaussian distribution. This system loops over each pixel and simultaneously adjust weights of the neighboring pixels, retaining sharp edges. Traditional filters generally operate on three separate bands of color resulting in different levels of contrast and smoothing patterns which causes perturbation in the balance of colors and order in which they appear. The method used here uses three bands at once resulting into the average of similar colors and minimizing the artifacts highlighted above. The pixel at value $x$ is replaced with the mean of similar and nearby pixel values. The preprocessed image is sent for keypoint detection further in the model. Bilateral filtering combines the range and spatial domain filters,

where n(x) is the normalization factor calculated as,

in which c($\xi$, x) computes the geometric proximity between a nearby point $\xi$ and the neighboring center $x$. s(g($\xi$), g(x)) calculates the photometric similarity between $x$ and point $\xi$.

Smooth apparent deformations are caused in images with smooth boundaries when cameras click these image when placed in varying positions. Affine transforms of an image plane play a good role to locally approximate these deformations. Calculation of affine invariant image local features are the main cause for issues in the domain of object recognition. Speeded-up robust features (SURF) is invariant to in-plane rotation, contrast, scale and brightness. The key point point detector in SURF interpolates the highly discriminative facial points. For the purpose extracting the physical characteristics from these key points detected it is sent for description of these interest points by constructing feature vectors. In order to achieve faster computation time, fast Hessian matrix approximation is used.

The proposed method performs well when the object has similar illumination conditions and suffers from extreme changes in angle. The method implemented in this paper - Affine-SURF(ASURF) - not only combines the advantages of affine invariance but is also computationally efficient as SURF [33]. The method reliably finds features that have large affine distortions. After this, SURF is applied on all the images. Thus, ASURF effectively overcomes all six mentioned constraints.

After computing the descriptors from ASURF, we have gathered enough information which can be used to distinguish relevant changes in the image parts. For creating the feature vector, a codebook $C$ of clusters is being created. Subsequently each cluster has a set of descriptors which are computed using k-means clustering technique. For every descriptor in an image, it is associated with its nearest centroid of its cluster, $c_{i}$ in the vocabulary.

Bag-of-words (BoW) model involves counting the number of descriptors in each cluster in the vocabulary. Subsequent steps involve the creation of sparse histogram over the vocabulary, to represent the image in the compact form. Such histograms can be used by classification algorithms to categorize them into different classes.

Similar to Bag of Words (BoW) model, Vector of Locally Aggregated Descriptors (VLAD) is used for representing images in compact form based on the locality criteria. In Bag of words a codebook $C$ is constructed of $k$ visual words with the help of k-means. Here, each local descriptor $x$ is related to its closest visual word. In case of VLAD descriptor, the main idea is to accumulate the differences between the descriptor and its associated cluster. The component of $v$ is computed to obtain the sum of all image descriptors which is given by,

where $v_{i,j}$ represents the vector over visual words and local descriptor component, x is the descriptor taken for visual word $c_{i}$. Further, the vector v is L2-normalised by

The quantized vector is sent further for multi-class image classification.

CloudForest is a classification algorithm that uses an ensemble of decision trees and is written in Go programming language. Its merit lies in its efficient implementation which fully utilizes the multi-threading potential of modern day machines thus making it fast and flexible. Apart from classification, this method also performs feature selection, regression and structure analysis on heterogeneous data with missing values.

Decision trees are often used for many machine learning and data mining applications. According to [14], it is unaltering under many feature transformations and is also robust to addition of irrelevant features. In particular, trees that grow deep tend to learn exceedingly irregular patterns, thereby overfitting training datasets.

Random forests finds its use in all sorts of machine learning applications, ranging from classification to regression. It constructs multiple decision trees with the goal of reducing the variance. It does so by taking the mean of the predictions calculated by the individual trees. This comes at the cost of an increase in bias and loss of interpretability, but greatly boosts the operation of the final model.

• 1.

  FACES95[30] contains 1440 images of 72 individuals with 20 images for each individual being taking using a fixed camera with a delay of 0.5 seconds between successive frames in the sequence. Significant head movement and lighting variations were introduced between images of the same individual. Minor variations in head turn, tilt and expression is present.

• 2.

  FACES96[31] is a significantly larger facial dataset which was constructed in a manner similar to FACES95. In total, there are 3040 images for 152 individuals with complex backgrounds.

• 3.

  The ORL[32] Database is relatively small with 40 classes and 400 images. The pictures vary with respect to time and light conditions, facial expression and details.

The model proposed in the paper has been executed on three standard datasets, namely, FACES95, FACES96 and ORL faces. The datasets considered pose variations in head rotation, expression, illumination, scale and affine distortions in the facial images. The model was ran for two different feature aggregators including VLAD and Bag of words with Cloud Forest as classifier. Cloud Forest was further compared with other ensemble classification algorithms like Random Forest and the results are tabulated in Table 1. By setting the classifier as Cloud Forest and running the model with VLAD as the quantization method, it achieves an accuracy of 97.69%, 89.94%, 86.67% on FACES95, FACES96 and ORL respectively in contrast to Bag of words where 97.22%, 94.08%, 77.50% are the accuracies when tested for the same datasets as mentioned before. Also we can conclude that VLAD performs better than Bag of words in most cases. Now upon changing the classifier from Cloud Forest to Random Forest, it can be seen from Table 1 that it could not keep up with the performance in most datasets as that of Cloud Forest except for ORL faces. It gave an accuracy of 95.37% on FACES95 whereas Cloud Forest proved to be 97.69% accurate on the same. Similar trend can be seen in case of FACES96 with 77.24% on Random Forest and 89.94% on Cloud Forest. The experimentation has been extended for other classifiers like k-nearest neighbours which when tested on the above mentioned datasets like FACES95, FACES96 and ORL achieves an accuracy of 64.14%, 61.54%, 63.89% with Bag of Words as the aggregator and 62.36%, 57.21%, 71.62% with VLAD as the aggregator.

In addition to testing the model on parameters like accuracy, analysis on training and testing time has been performed for deeper insights into the efficiency of the proposed system. On training the model with Cloud Forest as the classifier, it takes much lesser time as compared to Random Forest in all the datasets considered in our case. Such a decrease in time in Cloud Forest can be seen because of the multi-threaded ensemble nature of the classifier. On datasets like FACES95, FACES96, ORL the Cloud Forest algorithm proposed in the paper takes 11.88 seconds, 103.66 seconds, 1.75 seconds to train as compared to 139.46 seconds, 733.33 seconds, 15.82 seconds for the same respectively. Thus, if the application being considered requires training large facial datasets, Cloud Forest classifier is the more appropriate candidate to be considered. However, for time-constrained applications like real time surveillance systems, Random Forest Classifier is more appropriate as testing time is more efficient in this method. The metrics for the time taken for classification is shown in Table 2.