Towards a Robust WiFi-based Fall Detection with Adversarial Data Augmentation

In a MIMO wireless configuration, a typical CSI matrix (or transmission matrix) is formed by transmitting a symbol from each of the transmitting antennas, and its response on the multiple receiving antennas are noted. As the physical space constraints, the propagation of wireless signals, the received signals, in turn, contain information that characterizes the environment they pass through. If a person presents in the environment, additional signal paths are introduced by the scattering of the human body (Fig. 1).

In MIMO, the system configuration typically contains M antennas at the transmitter and N antennas at the receiver. The channel response is then expressed as a transmission matrix $H$ of dimension $N\times M$. The matrix contains elements that are complex and describe both the amplitude and phase variations of the link. The direct path formed between receiving antenna $i$ and transmitting antenna $j$ is expressed as $h_{ij}$.

The received vector $y$ is expressed in terms of the channel transmission matrix $H$, the input vector $x$ and noise vector $n$ as

Current WiFi standards (e.g., IEEE 802.11n/ac) use orthogonal frequency division modulation (OFDM) in their physical layer. OFDM splits its spectrum band into multiple frequency sub-bands, called subcarriers, and sends the digital bits through these subcarriers in parallel. The CSI reveals a set of channel measurements depicting the amplitude and phase of every OFDM subcarrier. The CSI of a single subcarrier is in the following mathematical format $H_{i}=|H_{i}|e^{j\angle H_{i}}$ where $|H_{i}|$ and $\angle H_{i}$ denote the amplitude and the phase of the CSI at the $i^{th}$ subcarrier respectively.

Most of the works in WiFi sensing employ either the Intel 5300 WiFi NIC card with the corresponding CSI tool [15] or the Atheros WiFI NIC 9k series with the Atheros CSI Tool [16]. These are used to record signals from COTS WiFi devices with multiple antennas for MIMO communication and at the physical layer adopt OFDM that supports IEEE 802.11/ac standard. In comparing, the Intel 5300 CIS tool only provides CSI for 30 subcarriers (out of the total 56 subcarriers for 20 MHz bandwidths and 114 subcarriers for 40 MHz). This limit by is addressed by the Atheros CSI Tool, which provides CSI data on all 114 subcarriers for 40 MHz bandwidth.

Various techniques have been proposed and studied to address the need for an automatic fall detection system [4, 5]. Wearable sensor-based approaches are among the first since the proposal Lord and Colvin for an accelerometer-based approach in 1991 [6]. Following the work are methods that employ numerous kinds of sensors, ranging from gyroscopes, barometric pressure sensors, RFID, to smartphones with combinations of sensory information[7, 8]. However, these systems only work when carried by users while the always-on-body requirement [17] for the elders is not always applicable[18].

Ambient device-based approaches try to make use of ambient information caused by falls to detect the accident. The ambient information being used includes audio noise, floor vibration, and infrared sensing data.[9] Computer vision-based approaches use cameras to monitor the environment, either by capture images or video sequences[10]. However, these approaches either come with an on-body requirement, which is ineffective in many situations, a high rate of false alarm due to ambient noise, or privacy intrusion issues.

Among Wi-Fi-based methods, one of the most popular approaches is WiFall by Han et al[19]. Their process consists of the following three phases: Data Processing, Anomaly Detection, and Activity Classification. Firstly, in the Data Processing phase, the authors perform weighted moving average method on CSI in order to reduce environmental noise. In the Anomaly Detection phase, static human bodies create no turbulence in the CSI domain and thus create no effect in CSI, while activities such as walking, running, falling are can generate a variance in CSI. As a result, human activities can be perceived as anomalies compared to stationary humans and recognized by an anomaly detection algorithm. After the Anomaly Detection phase, any detected action would go through the Activity Classification phase. This phase involves extracting 7-feature vectors manually engineered and putting these vectors through a binary Support Vector Machine classifier to recognize fall from other human activities. On average, WiFall manages to achieve an 87% precision detection rate with an 18% false alarm rate when tested on the same domain used for training. Similarly, Anti-Fall [20] and RT-Fall [21] perform manual feature engineering techniques but on both phase and amplitude information. With the additional use of phase shifts, the performance appears to improve with Anti-Fall reach a precision of 89% and a false alarm rate of 13%, while RT-Fall reaches a true positive rate of 91% and false alarm of 8%. One out-of-the-box approach is employed in WiSpeed [22] where estimation models are built to estimate the speed and acceleration of different activities. This information is then used to detect falls, with the rationale being the acceleration and speed patterns of falling down and other activities are different. WiSpeed is able to achieve an impressive true positive rate of 95% and zero false alarms.

Besides traditional methods using complex signal processing techniques like [19, 20, 21, 22], deep-learning-based approaches are also popular since deep learning models are able to extract non-linear features in the hidden representation space automatically. Consequently, these methods eliminate the need for a signal processing procedure. One such deep-learning-based approach is conducted by Yousefi et al. [23]. Following CARM proposed by Wei et al.[24], the authors treated fall detection problem as a sequential data processing problem with CSI being the object of interest. While Wei et al. use hidden Markov Model for CARM, Yousefi et al. choose a Long-Short Term Memory (LSTM) network, a common approach to handle sequential data. However, this approach is more of an activity recognition system than merely a fall detection system as they classify 6 different activities, one of which is falling. With this scheme, Yousefi et al. manage to reach an average accuracy of 91% among all 6 classes.

The most successful attempt to address the robustness issue is FallDeFi by Palipana et al.[14], which employs a feature engineering pipeline to extract robust features for training with simple learning procedures. The method can be factorized into three modules. The first module, named Data Collection and Preprocessing, aims at obtaining clear CSI by using linear interpolation, Discrete Wavelet Transform (DWT) based noised filtering as well as a Principal Component Analysis (PCA) based stream decorrelation and selection algorithm. The second module is the Feature Extraction module. This module receives the stream selection passed from the first module and extracts the features from the stream. Firstly, the Short-Time Fourier Transform (STFT) of the original stream is collected. The authors then use a Power Burst Curve (PBC) to detect events. When an event has been detected, the features are extracted from both the STFT Spectrogram and the PBC. After this, the features are passed to the final module for classification with an SVM classifier. Palipana et al. achieve an accuracy of 80% when the domains used for training and testing are different.

Recently, domain adaptation method with the adversarial learning paradigm of Jiang et al. [25] can automatically disentangle environment-dependent features, though still rely on spectrogram images [25]. However, this paradigm consists of two limitations. First, adversarial domain adaptation requires us to have data of many domains (a domain is a pair of human and environment). Secondly, domain adaptation is not realistic. For this method to work properly, it must have both data in the source domains (these data are labeled) and the target domains (these data are unlabeled and we are trying to make a prediction in these domains). Therefore, in this research, we are concerned with situations in which we do not have access to the data of the target domain. Shu et al. [26] has also experimented with adapting a vanilla CNN with Virtual Adversarial Domain Adaptation (VADA). However, the performance is not exciting, with only 53.0% of accuracy for adapted models and 35.7% for the source-only model for the 6-class dataset released in [23]. The Decision-boundary Iterative Refinement Training with a Teacher (DIRT-T) in the same paper [26] follows the domain adaptation scheme where the model is refined on the known target domain but does not yield further improvement over VADA. Based on the work of [13], we explore adapting the Adversarial Data Augmentation (ADA) method to remove these limitations.

First, we need to convert the original binary data into numerical matrices. The numerical data is investigated through visualization for better insights. The amplitude heat map of a raw data sample can be found in Fig. 3. For the sake of information loss tolerance in downsampling, we then concatenate signals from two antennas to create a set of $10000\times 60$ matrices, each represents the CSI in 10 seconds, matching the source sampling rate of the device at $1000Hz$. According to the experiments conducted by [24], there is a correlation between the phase of CSI and human interactions. As we observe in the visualization step, there is information redundancy in overlapped sampling periods. Hence, we can downsample the data to $500\times 60$ matrices to prevent memory error.

Recall that Shu et al. has employed a vanilla $18$-layer CNN for the WiFi activities recognition task [26]. We first evaluate ADA on top of a simple $6$-layer CNN. Firstly, a sample is resized to the shape of $500\times 60$ to be the input. After that, the input is passed through a convolution layer with $64$ feature maps and a $5\times 5$ kernel. Subsequently, the tensor goes through a max-pooling layer to reduce the size of the model. In addition to that, there is another convolution layer consisting of $128$ feature maps and a $5\times 5$ kernel, followed by a supplementary max-pooling layer. Finally, there are two fully connected layers, followed by a softmax layer for classification.

Although CSI can be evaluated as image data, CSI seems to be more relevant to the category of sequential data [11]. Therefore, we suggest that we adopt a more reasonable approach, such as Recurrent Neural Network (RNN). As RNN is vulnerable to exploding and vanishing gradient, we use Long-Short Term Memory (LSTM) cell for our RNN. First, the original input is put through a series of transpose, reshaping, and splitting operations. After these steps, the resulting tensor $X_{1}\in R^{b\times t}$, with $b$ is the number of samples in a batch, $t$ is the number of time steps, is passed to the next component - which is an LSTM cell. For our approach, we are using an LSTM cell with the size of the hidden state vector $h=200$. The output of the LSTM cell is a tensor $X_{2}\in R^{b\times h}$, which is then pushed through a softmax layer of size $h\times c$, with $c$ is the number of classes in the classification task. In our problem, $c=2$, corresponding to falling and not falling.

Among two publicly available datasets, Stanford WiFi Activity Recognition [23] and FallDeFi [14], we employ the FallDeFi dataset as it comes with a clearer description of domains.

The activities in FallDeFi is recorded in 5 environments:

• •

  Bathroom and toilet of size $2.9m\times 4m$

• •

  Two consecutive bedrooms of size $3.1m\times 5.2m$

• •

  A corridor of size $1.1m\times 9m$

• •

  A kitchen of size $4.2m\times 4.5m$

• •

  A lab with a size of $4.6m\times 7.3m$

Particularly, the transceiver and receiver for recording are two Linux laptops equipped with the Intel 5300 WiFi NIC card and corresponding CSI tool [15]. These laptops are placed with different distances and Line of Sight (LoS) settings for each environment as follow:

• •

  Bathroom and toilet: 5m, through two walls with plaster partitions.

• •

  Two bedrooms: 5m, through wall

• •

  Corridor: 9m, LoS uncluttered

• •

  Kitchen: 4m, LoS cluttered

• •

  Lab: 7m, non-LoS cluttered

Although each laptop is equipped with two external omnidirectional antennas, only the CSI from two antenna pairs (i.e., from 1 transceiver antenna to 2 receiver antennas).

The dataset is collected in two different ways into two subsets A and B. The two subsets are different by the date of collection along with natural chronological changes of involved compartments, the number of actors, and LoS variation created by moving furniture and recording devices. Hence, the dataset can be grouped into 10 domains in total.

As we are aiming to achieve environment-independence, the experiments are designed to accomplish independence between the training and testing data distributions. Among the $10$ domain employed, we use $9$ domains for the training procedure and $1$ domain for testing the final result. As presented in III-B, the optimal distance between the source distributions and the distribution of the generated samples $\rho$ is unknown. Therefore, we experiment with different values of $\rho$ to determine which value would give the best performance.

From the evaluation, we got a very thought-provoking result (Fig. 5). There is a downward trend in the accuracy of the fall detection task with CNN-ADA when we increase the value of $\rho$, and the model achieves the highest accuracy of $64.12\%$ at the $\rho$ value of $0.001$. In contrast, as the value of $\rho$ increases, LSTM-ADA accuracy sees a steady escalation and reaches the highest value of $66.03\%$ at the $\rho$ value of $4$.

Furthermore, we adapt the VADA method of Shu et al. [26] as a binary classification model on FallDeFi. Our VADA adaptation uses the ”small” CNN settings described in [26] with an 18-layer vanilla CNN trained using virtual adversarial training. After the training in 10000 iterations on the train set, VADA achieves a $52.7\%$ on the test set (Table I).