Stacked Autoencoder Based Feature Extraction and Superpixel Generation for Multifrequency PolSAR Image Classification

Synthetic Aperture Radar (SAR) has been popularized in recent years as a technique that captures high resolution microwave images of the earth surface. With SAR technique an image can be taken regardless of weather conditions or time of the day unlike optical sensors. The other major reason to use SAR is its operability over multiple frequency bands, viz, from the X band to P band. The penetrability of the L and the P bands allows SAR to capture data from even below ground level. In case of PolSAR it draws information of the target in four polarization states which makes it an information rich technique. These are some of the reasons that establish the superiority of SAR over optical data capturing techniques.

One of the very early approaches of classification of multifrequency PolSAR data was attempted using DNN (Dynamic Neural Network) [1]. Deep learning based multiplayer autoencoder network was also proposed [2]. It uses Kronecker product of eigenvalues of coherency matrix to combine multiple bands information. Recently, an Optimized Wishart Network (OWN) for classification of multifrequency PolSAR data is reported [3].

Superpixel algorithm in conjunction with deep neural networks has gained popularity for capturing spatial information of a PolSAR image. Hou et al. presented [4] a way of using Pauli decomposition of PolSAR image to generate superpixels and autoencoder to extract features from the coherency matrix of each PolSAR pixel. Prediction of the network was then used to run a KNN (k nearest neighbours) algorithm in each superpixel to determine the class of the complete superpixel. Guo et al. introduced a method to apply Fuzzy clustering algorithm over PolSAR images to generate superpixels [5]. This method considered only those pixels which are similar to their neighbours, pixels which are in all probability badly conditioned were ignored. In another work [6] Cloude decomposition features were used to generate superpixels and CNN (Convolutional Neural Network) to perform the classification. Adaptive nonlocal approach for extracting spatial information was also proposed [7]. It uses stacked sparse autoencoder to extract robust features.

In this paper we propose a classification algorithm for multifrequency PolSAR images. For doing so, we extracted 33 features from each band of PolSAR image to construct a feature vector for each pixel. After that we used two-layer stacked autoencoder to reduce dimensionality of the input vector. The output of autoencoder which contained information of all features of all bands was used to generate superpixels. Next, each pixel along with its corresponding superpixel was used to construct robust feature vector. Finally softmax classifier was used to perform the classification task.

In this paper we propose a classification algorithm for multifrequency PolSAR images. This involves extraction of 33 features from each band which are then used by a two-layer stacked autoencoder to reduce dimensionality of the input vector. This is used to generate superpixels whose features are used to generate a feature vector including information regarding the pixel and the superpixel the pixel belongs to. The merging of this information ensures that the effect of any noise or error can be reduced on the pixels. After this, the softmax layer to used for classification.

This paper is organized as follows: Section II discusses the proposed network architecture; Section III explains the experiments conducted on the Flevoland dataset; Section IV concludes with our observations and the discussions based on the results.

The raw data of a PolSAR image is a 2x2 scattering matrix where each element denotes the phase and magnitude information of the received waves in vertical and horizontal polarization after scattering of vertically and horizontally polarized waves as 4 complex numbers:

Due to the reciprocity condition, it is assumed that $S_{hv}=S_{vh}$. So we have 3 complex numbers where each complex number has 2 elements-real and imaginary. If a neural network were to be given the 6 elements as input, it wouldn’t be able to compute the phase information since it considers the real and imaginary components as separate features. This brings the need to give crafted features to the network instead of raw data. We have created 33 dimensional feature vector extracted from one frequency band of a PolSAR image. It contains 6 features of coherency matrix [8], 3 features of Freeman decomposition [9], 4 features of Krogager decomposition [10], 4 features of Yamaguchi decomposition [11], 9 features of Huynen decomposition [12] and 7 features of Cloude decomposition [13] as shown in Table 1. Hence combining information of all three bands we get 99 dimensional feature vector corresponding to each PolSAR pixel.

The Coherency Matrix contains the 6 elements from the scattering matrix. Freeman decomposition has 3 features which describe the power of single-bounce(odd-bounce), double-bounce and volume scattering of the transmitted waves. The Huynen decomposition aims to form a single scattering matrix to model the scattering mechanism of the surface. The Cloude decomposition was proposed in 1997 and it aims to model the surface scattering using the parameters such as Entropy, Anisotropy and other angles. The Krogager decomposition aims to factorize the scattering matrix as the combination of a sphere, a diplane and a helix. The Yamaguchi decomposition adds a Helix scattering component to the Freeman decomposition to model complicated manmade structures.

As shown in Fig. 1 our proposed network architecture contains three modules. First module is a two layer autoencoder network. The purpose of this module is to reduce the dimensionality of the input vector by learning efficient representation of combined PolSAR frequency bands information.

Let $\mathbf{X_{i}}$ be the input feature vector of $i^{th}$ PolSAR pixel. Let $\mathbf{W_{11}}$, $\mathbf{W_{12}}$, $\mathbf{b_{11}}$ and $\mathbf{b_{12}}$ be weights and biases of two encoder layers. A hidden representation of input feature vector $\mathbf{X_{i}}$ can be calculated as $\mathbf{h_{i}}=f(\mathbf{W_{12}}^{T}f(\mathbf{W_{11}}^{T}\mathbf{X_{i}}+\mathbf{b_{11}})+\mathbf{b_{12}})$, where $f$ is a $tanh$ activation function. Let $\mathbf{W_{21}}$, $\mathbf{W_{22}}$, $\mathbf{b_{21}}$ and $\mathbf{b_{22}}$ be weights and biases of two decoder layers. A reconstructed input vector can be calculated as $\mathbf{X_{i}}^{{}^{\prime}}=f(\mathbf{W_{22}}^{T}f(\mathbf{W_{21}}^{T}\mathbf{h_{i}}+\mathbf{b_{21}})+\mathbf{b_{22}})$. We have used mean square error to train the autoencoder. The cost function of first module of the proposed network is given as follows:

Here $\beta$ is a regularization parameter, $\alpha$ is the learning rate, $\gamma$ is the sparsity parameter, N is total number of training samples, $U_{1}$ is a size of reduced dimensional feature vector $\mathbf{h_{i}}$, $\rho$ is a sparsity parameter, $\rho_{t}$ is the average activation value of the $t^{th}$ hidden unit and $KL(\rho\|\rho_{t})$ is a Kullback-Leibler divergence which encourages sparsity in the hidden representation $\mathbf{h_{i}}$. For our application we have set the value of $U_{1}=5$. Once the training of the first module is complete we disconnect the decoder layers.

Next, we feed entire PolSAR image as an input to this network to obtain hidden representations of all pixels of the PolSAR image. We will use this hidden representation of all pixels of the PolSAR image to generate superpixels. This process has two advantages: i) it contains feature information of all bands and ii) its dimensionality is substantially reduced in comparison to the input feature vector. To generate superpixels we have used algorithm similar to simple linear iterative clustering (SLIC) [14]. Instead of the RGB input image we will use hidden representation of PolSAR image obtained from using first module of the proposed network. Using SLIC we measure the distance between any two pixels which is given by Eq. 2:

where $D_{s}=\sqrt{(x_{i}-x_{j})^{2}+(y_{i}-y_{j})^{2}}$ and $D_{h}=\lVert\mathbf{h_{i}}-\mathbf{h_{j}}\rVert_{2}$. Here $m$ is a parameter controlling the relative weight between $D_{s}$ and $D_{h}$ and $s$ is a size of search space [14]. $(x_{i},y_{i})$ and $(x_{j},y_{j})$ are the positions of $i^{th}$ and $j^{th}$ PolSAR pixels on Euclidean plane. This distance measure was finally used for superpixel generation.

The second module of our proposed architecture combines each pixel and corresponding superpixel information to construct robust feature vector. This is done by letting $\mathbf{S_{j}}$ be the $j^{th}$ superpixel and $\mathbf{h_{i}}\in\mathbf{S_{j}}$. Let $\mathbf{c_{j}}$ be the cluster center of $\mathbf{S_{j}}$. To extract robust feature using both pixel and superpixel information input for second autoenocder $\mathbf{H_{i}}$ can be constructed as $\mathbf{H_{i}}=[\mathbf{h_{i}};\mathbf{c_{j}}]$ [7]. Since the dimensionality of the hidden representation $h_{i}$ is low, a single layer autoencoder is sufficient for an effective reconstruction. Let $\mathbf{r_{i}}=f(\mathbf{W_{13}}^{T}\mathbf{H_{i}}+\mathbf{b_{13}})$ be the activation value obtained at hidden layer of second autoencoder. Let $\mathbf{H_{i}^{{}^{\prime}}}=f(\mathbf{W_{23}}^{T}\mathbf{r_{i}}+\mathbf{b_{23}})$ be the reconstructed input. The cost function for the second autoencoder can now readily be described by:

Here $U_{2}$ is the size of feature vector $\mathbf{r_{i}}$. Once the training of the second module of the autoencoder is complete we again disconnect the decoder layer. Output of the second module of autoencoder contains both pixel and superpixel information. Finally we use softmax classifier to obtain predicted probability distribution.

With this theoretical model we conduct the following experiments. We start with the details of dataset chosen for our experiments. After that we analyze the performance of the proposed network for different band combinations. Finally we will perform the analysis of performance for different feature decompositions. The complete experiment on the proposed network architecture was implemented using Python 3.6 and it was executed on a 1.60 GHz machine with 8GB RAM for all the experiments.

Experiments have been conducted on a dataset of Flevoland [15], an agricultural tract in the Netherlands, whose data was captured by the NASA/Jet Propulsion Laboratory on 15 June, 1991. This dataset has often been viewed as the benchmark dataset for PolSAR applications. The intensities after Pauli decomposition of the dataset have been used to form an RGB image as shown in Fig. 2(a-c). The ground truth of the data set shown in Fig. 2(d) identifies a total of 15 classes of land cover.

We have evaluated the accuracy of each class with respect to all possible combinations of the data acquired in the three frequency bands. Please note that all the Overall Accuracy mentioned in the paper is the precision obtained. From Table 2 it can be observed that while the network with just the C band information recognizes Onions and Lucerne with high accuracy, it fails in the case of classes such as Beet or Oats. While in the case of the C band, the network fails to recognize Wheat accurately. It can be noted that the lack of information to recognize Wheat is compensated by the help of the L band. In the case of the P band, the network fails to accurately identify Onions and Peas but correctly identifies the majority of the Rapeseed and Fruit. It can also be observed that the L band performs better individually than the C or the P bands.

A total of six PolSAR decomposition methods have been applied to the PolSAR data and their features have been given as input to the proposed network. The significance of each decomposition technique is evident from the Table 3. It can be seen that the Krogager and Freeman decompositions fail to provide enough information to recognize Beet and Onions. On the other hand Cloude decomposition and Huynen decomposition provide sufficient information respectively.

The proposed method for classification of multifrequency PolSAR image is also compared with other methods available in the literature. Comparison of overall accuracies is reported in Table 4. ANN [2] and OWN [3] have used small subset of Flevoland dataset containing 7 classes. On the other hand Stein-SRC [16] used ground truth with 14 classes. For parity in comparison with our results we have calculated the overall accuracy of the proposed method using ground truth of 7, 14 as well as 15 classes. We observe from Table 4 that the proposed method outperforms all the three methods [3, 2, 16].

In this paper a classification network for multifrequency PolSAR data is proposed. Proposed network involves three modules. First module reduces the dimensionality of the input feature vector. Output of the first module has been used to generate superpixels. The second module constructs robust feature vector using each pixel and its corresponding superpixel information. Finally the last module of the proposed network conducts classification using softmax classifier. It is observed that combining multiple frequency bands information improves overall classification accuracy. We validated our proposed network on Flevoland dataset resulted in 99.59% overall classification accuracy. Experimental result shows that this proposed network outperforms other reported methods available in the literature.