Multiview Variational Deep Learning with Application to Scalable Indoor Localization

Machine learning applications with a multiview embodiment rendered with multiple sources can exploit feature correlations among various views to attain the best model for inference. Multiview data can be efficiently generated by variational inference hoffman2013stochastic from a single view, as reported in zhao2018multi . Variational inference is also adopted in broad discriminative models such as clustering dilokthanakul2016deep , classification xu2017variational , and regression girolami2006variational to utilize the probabilistic latent feature space. From the perspective of a deep network, variational deep learning (DL) wilson2016stochastic can jointly optimize objectives of a Gaussian process marginal likelihood to train the deep network. Reparameterization of variational inference wainwright2008graphical derives mean-field latent feature vectors to represent the posterior of an input. Compared to stand-alone deep networks, support vector machines (SVMs), and other recent systems, the variational DL can substantially improve classification and regression performance wainwright2008graphical . In this paper, we apply the variational DL to localization of transmit devices based on radio signal data in a practical WiFi environment.

Similarly to an example that generates a single image view from multiple image views tulsiani2017multi , radio signals retrieved in multiple groups of receivers at distributed locations can form multiview information for machine learning to locate the transmit device. By the nature of variational DL that the elements of the extracted latent vector have no correlations, we can map between a multivariate standard normal distribution and each view. As the variational DL is forced to ensure its latent vector approximated to the standard normal distribution with a reduced number of independent Gaussian processes, we can mitigate the signal uncertainties. Compared to the global positioning system (GPS), which leads to 5 m to 10 m outdoor localization accuracy, indoor localization requires more accurate positioning of the transmit devices. Recently, a channel state information (CSI) of WiFi has emerged as the strong candidate for indoor localization rather than a scalar-valued received signal strength indicator (RSSI) li2018indoor . The subcarrier CSIs of WiFi orthogonal frequency division multiplexing (OFDM) channel form complex vector information, providing much rich information on radio localization with an excellent localization accuracy.

In a WiFi network, device localization can be achieved by analyzing the CSIs of a radio packet arriving at multiple receiver antennas of an access point (AP), complying with the IEEE 802.11a/g/n/ac standards for the multi-input multi-output (MIMO) air interface. In the experiment with an Intel WiFi link (IWL) 5300 network interface controller (NIC), the physical layer API reports a complex CSI vector of 30 selected subcarriers for an antenna receiving a WiFi packet halperin2011predictable . The phase difference of CSIs of multiple antennas provides angle of arrival of a received packet, which is the key information to localize the transmit device.

The received CSI of packet at subcarrier $i\in\{1,\dots,I\}$ of antenna $m\in\{1,\dots,M\}$ with nominal CSI $H_{m,i}$ and noise $N_{m,i}$ is represented as $\hat{H}_{m,i}=|H_{m,i}|e^{j2\pi\angle{H_{m,i}}}+N_{m,i}$. The nominal CSI amplitude is $|H_{m,i}|$, and the nominal CSI phase is represented as

$$\angle{H_{m,i}}=s_{i}~{}\delta_{f}~{}\tau+(m-1){f_{c}}\frac{d\sin\theta}{c},$$

(1)

where a subset of subcarriers $\{s_{i}\}$ is selected among available subcarriers indexed between $-S$ and $+S$. The constant $\delta_{f}$ is the frequency difference between subcarriers, $f_{c}$ is the center frequency of the channel, $d$ is the distance between adjacent receiver antennas, and $c$ is the speed of light. Here, the phase $\angle{H_{m,i}}$ is a function of the time of flight $\tau$ and the angle of arrival $\theta$, which implies that subcarrier frequencies and the geometry of the antenna array cause the relative phase difference due to different radio arrival times. Many of the previous localization techniques aimed to find the nominal time of flight and angle of arrival schmidt1986multiple . But in real 802.11 communication, several offsets and noise accompany them, and the measured phase $\angle{\hat{H}}_{m,i}$ is represented as $\angle{\hat{H}}_{m,i}=\angle{H_{m,i}}+s_{i}~{}\lambda+\mu_{m}+\beta+Z_{m,i},$ where $\lambda$ and $\mu_{m}$ denote the subcarrier-dependent offset coefficient and the receiver antenna-dependent offset, respectively, and $\beta$ and $Z_{m,i}$ denote packet-dependent offset and noise, respectively tzur2015direction . Empirically, these offset and noise cause a large fluctuation to the CSI phase and thus make it hard to be solved by the analytical methods. In our approach with variational inference, the CSI is mapped to an unit phasor complex that measures phase difference of CSIs among different antennas:

$$x_{m,i}=\frac{\hat{H}_{m,i}/|\hat{H}_{m,i}|}{\hat{H}_{M,i}/|\hat{H}_{M,i}|},~{}~{}~{}~{}m\in\{1,\dots,M-1\}.$$

(2)

Here, we adopt variational DL to mitigate the noise and signal fading problems in the CSI training samples, defined as $\mathbf{x}=[x_{m,i}]$, with $m\in\{1,\dots,M-1\}$ and $i\in\{1,\dots,I\}$.

In order to construct the localization system scalable in a complex area, we should consider exclusion of non-informative CSI views. One can deploy multiple APs over the area consisting of $K$ sub-areas, where a collection of APs in each sub-area forms a view of the CSI sample. We apply deep learning to classify the dominant views from multiple sub-areas.

We apply a variational DL for regression to be trained with input as a pair of $\mathbf{x}$ (i.e., CSI in our case) and a true label $\mathbf{y}$ (i.e., Cartesian coordinate in our case). Let us assume a latent feature vector $\mathbf{z}$ consisting of $z_{j}=f_{j}(\mathbf{x}),j\in\{1,\dots,J\}$ of independent Gaussian processes (GPs) with probability of $z_{j}\sim\mathcal{GP}(\mu_{j},\sigma_{j}^{2})$, where the mean-field feature vectors $\boldsymbol{\mu}=[\mu_{1},\dots,\mu_{J}]$ and $\boldsymbol{\sigma}=[\sigma_{1},\dots,\sigma_{J}]$ indicate mean and standard deviation, respectively. The latent vector is used to estimate the regression output $\mathbf{\hat{y}}$ where the true $\mathbf{y}$ is supposed to be represented as $\mathbf{y}(\mathbf{x})|\mathbf{z}\sim\mathcal{N}(\mathbf{y}(\mathbf{x});\mathbf{z},\boldsymbol{\sigma}^{2}\odot\boldsymbol{I})$. In order to make the learning variables differentiable for gradient descent based back-propagation, it requires reparameterization. The weights and biases of a neural network (NN) to obtain the latent vector $\mathbf{z}$ from the input $\mathbf{x}$ is updated by sampling of noise vector $\boldsymbol{\epsilon}$:

$$\mathbf{z}=\boldsymbol{\mu}+\boldsymbol{\sigma}\odot\boldsymbol{\epsilon},~{}~{}~{}\boldsymbol{\epsilon}\sim\mathcal{N}(0,\mathbf{I}).$$

(3)

Then with the variational posterior over the estimated distribution $q(\mathbf{z})$, Jensen’s inequality can be applied to give evidence lowerbound (ELBO) of the marginal log-likelihood of regression wilson2016stochastic :

$$\log p(\mathbf{y})\geq\mathbb{E}_{q(\mathbf{z}|\mathbf{y})}[\log p(\mathbf{y}|\mathbf{z})]-\mathbf{KL}[q(\mathbf{z}|\mathbf{y})||p(\mathbf{z})]\triangleq\mathcal{L}(q)=\mathbf{ELBO}.$$

(4)

In (4), the first term at the right hand side is cross entropy between the label $\mathbf{y}$ and the regression output from the latent vector $\mathbf{z}$, which is equivalent to the regression loss. The second term is the Kullback–Leibler (KL) divergence between the estimated posterior $q(\mathbf{z}|\mathbf{y})$ and the distribution $p(\mathbf{z})$. Our aim is to approximate the estimated posterior $q(\mathbf{z}|\mathbf{y})$ to be close to the posterior $p(\mathbf{z}|\mathbf{y})$; in other words, to minimize $\mathbf{KL}[q(\mathbf{z}|\mathbf{y})||p(\mathbf{z}|\mathbf{y})]$. Again the log-likelihood can be represented as the following equation:

$$\begin{split}\log p(\mathbf{y})&=\int\log\big{(}p(\mathbf{y})\big{)}q(\mathbf{z}|\mathbf{y})dz\\ &=\int\log\frac{p(\mathbf{y},\mathbf{z})}{q(\mathbf{z}|\mathbf{y})}q(\mathbf{z}|\mathbf{y})dz+\int\log\frac{q(\mathbf{z}|\mathbf{y})}{p(\mathbf{z}|\mathbf{y})}q(\mathbf{z}|\mathbf{y})dz\\ &=\mathbf{ELBO}+\mathbf{KL}[q(\mathbf{z}|\mathbf{y})||p(\mathbf{z}|\mathbf{y})].\end{split}$$

(5)

Since the log-likelihood $\log p(\mathbf{y})$ is bounded, in order to minimize the KL divergence $\mathbf{KL}[q(\mathbf{z}|\mathbf{y})||p(\mathbf{z}|\mathbf{y})]$, we have to maximize the term ELBO. The KL divergence term in (4) with the normal distribution $q(\mathbf{z}|\mathbf{y})$ and the standard normal distribution $p(\mathbf{z})$ can be simplified to

$$\mathbf{KL}[q(\mathbf{z}|\mathbf{y})||p(\mathbf{z})]=\frac{1}{2}\sum_{j=1}^{J}(\mu_{j}^{2}+\sigma_{j}^{2}-\ln(\sigma_{j}^{2})-1).$$

(6)

By updating the NN parameters to jointly reduce the cross entropy and the KL divergence $\mathbf{KL}[q(\mathbf{z}|\mathbf{y})||p(\mathbf{z})]$, we can both approximate the posterior and estimate the desired regression output.

We introduce novel learning system, or the VSDL, with which the relative degree of importance of each view among the multiple views is used to improve the regression accuracy. A type of selective sampling, referred to as co-testing, was introduced muslea2000selective to efficiently extract features from multiview data, where its basic idea is to inject divided views to multiple independent learning networks. However, this model had a strict assumption that each data sample might have a strong correlation among the views. On the other hand, we focus on a situation where views are not correlated to each other nor informative, but there is a given information whether or not a view is informative. In our case, we require localizing a target in a two-corridor environment, as in Figure 1.

As you can see the Figure 1, the WiFi radio signal can propagate not only through a line of sight (LoS) but also non-line of sight (NLoS) paths. Subsequently the received signal consists of multi-path fading as well as the signal noise. The CSI data view at APs only from NLoS paths are non-informative, since the multi-paths are very unpredictable. In this kind of case, it must be inefficient if all views are included in training. However, we can judge whether or not a view is informative so as to utilize such given information for supervised training. In our VSDL model, the learning parameters are much effectively updated by excluding non-informative data views. Looking into the big picture, VSDL is designed as a two-stage learning network for regression, consisting of a view-oriented variational deep network and a view-classified regression network.

We apply the VSDL system for indoor localization on a two-corridor real building environment with 43 training points and 9 test points, as in Figure 3(a). Each corridor is 7 m long, where training and test points are spread by 0.5 m spacing. We install seven APs consisting of 3-antenna IWL 5300 NIC in laptop computers, which are placed at the corners of the corridors. For a transmitter, the same laptop with a single antenna is used to transmit WiFi packets on channel 36 at 5.18 GHz. The APs receive packets from the transmitter at the same time using the monitor mode and we combine them as a multiview input at the server side. Each AP receives the WiFi packet with three antennas to form three Tx-Rx radio channels. Each channel produces a CSI vector consisting of 30 subcarrier CSIs ($I=30$). We then take one of the three CSI vectors as the reference to produce two relative CSI vectors. In this way, we obtain an input sample $\mathbf{x}$ consisting of 420 $(=7\times(3-1)\times 30)$ relative CSIs. We collect 100 sample packets for every training and test points in a noisy environment, which manifests high data fluctuation that makes it difficult to estimate the location by other analytical methods.

In this scenario, we divide input $\mathbf{x}$ into two views $\mathbf{x_{1}}$ and $\mathbf{x_{2}}$ ($K=2$), which represent AP associations with corridors 1 and 2, respectively. Therefore, $\mathbf{x_{1}}$ has information of AP1 to AP5 and $\mathbf{x_{2}}$ has that of AP3 to AP7. Along with the CSI data, the location label $\mathbf{y}$ in the Cartesian coordinate and the given corridor label $\mathbf{u}$ are utilized for training. There are three cases for $\mathbf{u}$ depending on the training location: The location belongs, 1) only to corridor 1 ($\mathbf{u}=[1,0]$), 2) only to corridor 2 ($\mathbf{u}=[0,1]$), and 3) to both corridors 1 and 2 ($\mathbf{u}=[1,1]$). The NNs for view-oriented learning update parameters only when the view is informative ($u_{k}=1$). Here, the information from AP3 to AP5 are common in both views and considered to be informative for every sample.

Along the entire system, we set the number of hidden layers $L$, $P$, $Q$, and $R$ to be three. The number of nodes of each layer decreases from 1000 to 500. For the feature output of every layer except for the last one, the Relu activation is used. Adam optimizers are used to update the parameters with learning rate of $10^{-5}$. First, we aim to verify if our joint optimization works well. As seen in Figure 4, the losses of regression and view classification are minimized together for any given number of latent variable $J$. This trend implies that our system properly operates to obtain an advanced regression result assisted by the view selection. In addition, we obtain the best regression result with $J=120$ rather than keeping the size of the input CSI vector, which corresponds to a mean-field feature compression ratio of $2/7$. There should be a sufficiently large number of variables to estimate the posterior of the CSI data, while too many variables cause overfitting of the network. Obviously, $J$ depends on the scenario as well as the application requirement.

In our VSDL system, the trade-off parameter $\alpha$ can influence the performance of regression as seen in Figure 5. As $\alpha$ approaches 0, the network becomes too sensitive on view classification, and it makes both losses worse. On the other hand, as $\alpha$ approaches 1, high view classification loss occurs, resulting in poor regression (localization) accuracy. We obtain the best regression accuracy with $\alpha=0.5$.

Figures 3(b) and 3(c) show comparisons of the regression results from a simple variational DL and our VSDL with $J=120$ and $\alpha=0.5$. We take three test locations as the representative cases. The locations A and B are near the end of corridors and location C is in the intersection of the two corridors. The test results for the location A, B, and C are shown in blue, red, and green, respectively. In terms of multiview data learning, the variational DL extracts features from all corridor views including non-informative views. Therefore, as seen in Figure 3(b), the regression results for many cases are located somehow outside the topology of training. In contrast, the VSDL system updates the learning parameters only for informative views to classify the dominant view and hence to achieve better localization, as seen in Figure 3(c).

Further, we compare the proposed VSDL with several existing machine learning systems. To discriminate the CSI data, both classification and regression methods were introduced in previous papers to improve the localization accuracy. In our experiment environment, as well as simple variational DL, we implemented RBM based classification BiLoc wang2017biloc , CNN based classification CiFi wang2017cifi , SVM based regression SVR zhou2017csi , and stand-alone DNN based regression. Figure 6 and Table 1 show the comparison results. We do not plot the results of CiFi in this scenario, since the convolutional analysis from batch information cannot extract proper features and results in a very poor localization accuracy of 1.97 m. First, the variational DL, whose results are described in 3(b), outperforms other existing systems due to the usage of variational inference. Here, we observe that the introduction of the variational inference brings the key advantage for WiFi CSI localization in a noisy radio channel. In addition, the VSDL system with novel two-stage view-selective learning on the variational inference base significantly improves the localization accuracy by 30 $\%$, from 1.10 m to 0.77 m. As the VSDL is very scalable by the nature of its design, we expect further performance improvement in environments with more corridors.

WiFi device localization has been a very attractive area of study as WiFi networks are omnipresent to provide network application services to anonymous users in these days, anticipated to open new business opportunities as well as new technical challenges. The technical performance of WiFi localization has been improved disruptively by the use of channel state information measured at multiple receive APs.

In this paper, we introduce a machine learning design that combines the variational deep learning very effectively in multiview learning architecture. We report an observation that the latent vectors generated at the intermediate layer of variational deep learning form strong feature behaviors to provide classification of effective view selection that enhance the accuracy of localization by a great deal. Our system, the view-selective deep learning, or the VSDL, achieves a localization accuracy of 0.77 m, which manifests a more than 30 % improvement in a two-corridor field experiment compared with the best known system based on SVM. The VSDL is completely scalable as exploiting the benefit of multiview-based regression, and hence the WiFi localization network can be expanded with no limit, such as in complex building structure. Our design of extracting features in the latent space to deal with informative and non-informative views in a multiview variational deep learning network is very powerful so that it can be applied to various applications with no limit on scalability.

The indoor localization with radio signals, for example, WiFi radio signals, which finds the location of a mobile device very accurately can create a great deal of impact in mobile service application. This can be a benefit to off-line stores and services such as in a shopping mall, hospitial, and public buildings, where location-based services can directly improve quality of experiences, especially when associated with social network services. Of course, such network features of localization can also harm the privacy of people in public. Radio localization may fail in a hot spot in the sense of crowded radio network traffic. However, such a failure may not cause critical problems except for some frustration with internet-based applications.