Smartphone Impostor Detection with Behavioral Data Privacy and Minimalist Hardware Support

A good impostor detection solution needs to be able to detect suspicious impostors while not affecting the usability of the smartphone owner. At the center of this trade-off is selecting an appropriate algorithm for impostor detection. One of our key takeaways is that choosing the right algorithm (and model) is more important for achieving security and performance goals than increasing the model size or adding hardware complexity to accelerate a model.

Previous work on implicit smartphone user authentication mostly leverage (user, non-user) binary classification techniques (Frank et al., 2012; Zheng et al., 2014; Lee and Lee, 2017). This scenario requires both data from the real user and other users for training, and we call it the Impostor Detection-as-a-Service (IDaaS) scenario in this paper. For a specific customer, the data from him/herself are labeled as benign or negative (the user), while all data from other customers are labeled as malicious or positive (non-users). We select certain classification-based algorithms that give the best accuracy for impostor detection (security) and legitimate user recognition (usability) in the non-privacy-preserving IDaaS scenario. We treat them as benchmarks when comparing with our privacy-preserving impostor detection algorithms.

Among the many Machine Learning (ML) algorithms we investigated, we report the results on Support Vector Machine (SVM) and Kernel Ridge Regression (KRR). SVM is a powerful and commonly used linear model, which can establish a non-linear boundary using the kernel method. KRR alleviates over-fitting by penalizing large parameters and also achieves the highest detection rate in the literature (Lee and Lee, 2017) while requiring the computation of 14 heuristically chosen features. Surprisingly, we show that even a simple Deep Learning algorithm (Multi-layer perceptron, MLP) without heuristic and tedious hand-crafting, can outperform it. MLP is a family of feed-forward neural network models, consisting of an input layer, an output layer and one or more hidden layers in between.

Eq (1) gives the formula for TPR and TNR, as well as for the other metrics commonly used in comparing the ML/DL models: accuracy, recall, precision and F1. Accuracy is the percentage of correctly identified samples over all samples. Recall, i.e., TPR, is the percent of all attacks that are detected whereas precision is the percent of all reported attacks that are real attacks. F1 is the harmonic mean of recall and precision.

The above binary classification approaches can only be applied to the IDaaS scenario where the data from other users are available. However, smartphone users may not be willing to send their sensor data to a centralized service for training, due to behavioral data privacy concerns. Therefore, we need to consider another important scenario where the smartphone user only has his/her own data for training. We call this the local anomaly detection (LAD) scenario.

We consider two representative algorithms for one-class learning, i.e. One-Class SVM (OCSVM) and Long Short-Term Memory (LSTM), an RNN algorithm. We propose enhancing the LSTM-based deep learning models with the comparison of reference and actual Prediction Error Distributions (PEDs). We show that generating and comparing the prediction error distributions is the key to a successful detection for this LAD scenario.

We use an LSTM-based model as an outlier detector (Malhotra et al., 2015), by training it to predict the next sensor reading, and investigating the prediction errors. The intuition is that an LSTM model trained on only the normal user’s data predicts better for his/her behavior than for other users’ behavior. The deviations of the actual monitored behavior from the predicted behavior indicate anomalous behavior. Typically, a threshold value is used to decide if the prediction error is normal or not.

As we do not need to assume the prior distribution of PED, non-parametric tests are powerful tools to determine if two distributions are the same. The Kolmogorov-Smirnov (KS) test is a statistical test that determines whether two i.i.d sets of samples follow the same distribution. The KS statistic for two sets with $n$ and $m$ samples is:

where $F_{n}$ and $F_{m}$ are the empirical distribution functions of two sets of samples respectively, i.e. $F_{n}(t)=\frac{1}{n}\sum_{i=1}^{n}1_{x_{i}\leq t}$, and $sup$ is the supremum function. The null hypothesis that the two sets of samples are i.i.d. sampled from the same distribution, is rejected at level $\alpha$ if:

where $c\left(\alpha\right)$ is a pre-calculated value and can be found in the standard KS test lookup table.

We evaluate the algorithms for impostor detection using the WALK subset in the Human Activities and Postural Transitions (HAPT) dataset (Reyes-Ortiz et al., 2016) at UCI (Dheeru and Karra Taniskidou, 2017). The HAPT dataset contains smartphone sensor readings. The smartphone is worn on the waist of each of 30 participants of various ages from 19 to 48. Each reading consists of the 3-axial measurements of both the linear acceleration and angular velocity, so it could be treated as a 6-element vector. The sensors are sampled at 50Hz. We select 25 out of the 30 users in the HAPT dataset as the registered users while the other 5 users act as unregistered users. To investigate the feasibility of user versus impostor classification, the samples from the correct user are labeled negative for impostor detection while all the data from the other 24 registered users and the 5 unregistered users are labeled as positive.

In the IDaaS scenario, each data sample used in both training and testing contains 64 consecutive readings from the same user. At 50Hz sampling frequency, 64 readings correspond to 1.28 seconds which is the latency to detect an impostor. Models are trained for each registered user using his/her data and randomly picked sensor data of the other 24 registered users. We make sure that the training samples have no overlap with the testing samples. The samples from unregistered users are used to examine whether unseen attackers can be successfully detected.

In the LAD scenario, the training data is only from the real user. The testing samples still include the data from the real user and the other users. We test for window sizes of 64, which is the same size as the IDaaS scenario, and 200, which corresponds to a longer detection latency of 4s but shows an improvement in the detection accuracy (Table 2). An LSTM-based model is trained to minimize its average prediction error for each registered user. In the testing of LSTM-based models, prediction errors for consecutive readings in each sample form a testing PED.

We evaluate each of the 25 registered users against each of the 30 users, i.e. 750 test pairs in total, and we report the average metrics of all pairs. Table 1 and Table 2 show the results of different algorithms in the IDaaS and the LAD scenarios, respectively. Table 1 shows that in the IDaaS scenario, the SVM model outperforms the other models, including KRR with 14 manually designed features (Lee and Lee, 2017), on all evaluated metrics. A simple deep learning model, MLP, performs almost as well, achieving accuracy $>97\%$. Larger models, e.g. MLP-500 and models with more layers, e.g. MLP-200-100, also slightly improve the accuracy.

Table 2 shows the approaches we evaluated for the LAD scenario, for 2 window sizes of 64 (left) and 200 (right) sensor measurements. For each LSTM algorithm, we also tested different hidden state sizes, from 50 to 500. LSTM-th compares the average prediction error in a window with a threshold obtained from the validation set, while PED-LSTM-Vote compares the empirically-derived PEDs. We randomly choose 20 samples of prediction errors from the validation set and use them to represent the reference PEDs. In the testing phase, the prediction error distribution of each testing sample is compared to all the reference distributions. The PED-LSTM-Vote models consider a sample as abnormal if more than half of the D statistics of the KS tests are larger than the fixed threshold in Eq (4). Table 2 shows the results for the $\alpha$-values that give the best detection accuracy, i.e. $\alpha=0.10$ for a 64-reading window and $\alpha=0.05$ for a 200-reading window.

In Table 2, the one-class SVM (OCSVM) achieves an average accuracy of 62-69%, thirty percent worse than the 2-class SVM model trained with positive data involved. The LSTM-th models (64-reading window) have an accuracy between 72% and 75%, only slightly better than the one-class SVM model, regardless of the hidden state size. If PED and statistical KS test are leveraged, we see a significant improvement in the detection accuracy up to 87.1% and 90.2% for a 64-reading window and a 200-reading window, respectively.

However, the overhead in execution time may increase. In Section 5, we discuss such security-performance trade-offs, which are essential to algorithm selection in practice.

The results in Section 3.4 show that in the IDaaS scenario, detection in 1.28s with very high sensitivity levels (95%-99%) can be achieved for accuracy, security (TPR) and usability (TNR) when SVM or MLP models are used. In the data-privacy preserving LAD scenario, the detection accuracy, using our LSTM-based models enhanced by collecting error distributions, can reach 87.13% for the same detection latency of 1.28s. If the user allows a detection latency of 4 seconds which is usually not long enough for an impostor to perform malicious operations on the smartphone after stealing it, the accuracy can be increased to 90.24%. Although the accuracy is not perfect, it is comparable to the state-of-the-art one-class smartphone authentication using various handcrafted features and complex model fusion (Kumar et al., 2018) in the literature. Also, our privacy-preserving one-class model (PED-LSTM-Vote-200) achieves better detection accuracy, F1 score, TPN and TNR results than the state-of-the-art two-class KRR model with hand-crafted features (Lee and Lee, 2017) for this data set when both are using a 64-reading window.

A key contribution we make is to show that it is the Prediction Error Distributions and KS test that provide the significant increase in detection capability. While tuning the size of deep learning models, e.g. LSTM, has little impact on accuracy, the KS test for PED comparison increases the overall accuracy by +12.4% for the 64-reading window and +19.2% for the 200-reading window. Therefore, we provide the hardware support for generating empirical PEDs and computing the KS statistic in Section 4.4.

We first consider what operations are needed by the DL (and ML) algorithms we want to implement. These are first the PED-LSTM algorithm, and also the MLP and SVM algorithms, which are the best impostor detection algorithms for the two scenarios considered in the previous section.  Table 3 shows the operations needed for these different DL/ML algorithms. The instructions from Vargmax to Vsqrt (at the bottom of Table 3) are needed only by the KRR algorithm (Lee and Lee, 2017), the previous highest performing method, to compute the 7 features for the accelerometer and the gyroscope each, i.e., the average, maximum, minimum and variance of the sensor data and three frequency domain features: the main frequency and its amplitude and the second frequency. We decide not to implement these operations, since they are not needed by the other higher-performing algorithms, while needing significant extra hardware.

The programming model of SID  is to execute macro-instructions, where each macro instruction is a whole vector or matrix operation. The number of iterations is automatically determined by the hardware, in a Finite State Machine (FSM), based on the hardware’s knowledge of the number of parallel data tracks available in the implementation. The macro-instruction supplies the type of operation needed, and the dimensions of the vector or matrix.

The format of a SID macro instruction is shown in Table 4. The Mode field specifies one of the operation modes in Table 3. Three memory addresses can be specified in a macro-instruction: Addr_x and Addr_y for up to two operands, and Addr_z for the result. Instead of implementing vector machines with vector registers of fixed length, we use memory to store the vector or matrix operands and results. This design is less expensive than vector registers. It is also more efficient since it supports the flexible-size inputs and outputs and operates seamlessly with our automatic hardware control of the execution of a vector or matrix operation. This is one way (memory not vector registers) to use minimal hardware and achieve scalability (macro-instruction with FSM control).

Each macro-instruction initializes the FSM state of the control unit to indicate the number of iterations of the specified operation. Each cycle, the FSM updates the number of uncomputed iterations, according to the number of parallel tracks, to decide when a macro-instruction finishes (details in the following paragraphs). Thus, the same SID  software program can run on SID hardware modules with a different number of parallel tracks, without modification. This achieves our performance scalability goal.

The Length and Width fields can initialize three state registers, reg_length, reg_width and reg_width_copy, which define the control FSM state. The FSM can be configured by instructions in two ways: the one-dimension iteration and the matrix-vector iteration. For the one-dimension iteration in a vector operation, the value of reg_length is initialized by the length field of the instruction. During execution, reg_length is decreased every cycle by N(track), which is the number of parallel datapath tracks, until reg_length is no larger than N(track) and the next instruction will be fetched.

When an instruction for a matrix-vector operation is fetched, the length field initializes reg_length and the width field initializes both reg_width and reg_width_copy. A matrix-vector multiplication macro-instruction computes the product of a matrix of size width-by-length with a vector containing length elements. The SID module performs loop tiling by computing a tile of width rows by N(track) columns in the matrix before moving to the next tile. When the last tile of columns in the matrix is computed, the next instruction can be fetched.

Figure 1 shows a SID  implementation with four parallel datapath tracks. Each track consists of a Look-up Table (LUT), a multiplier (MUL) and an adder (ADD), which are put into three consecutive pipelined execution stages (EXE0, EXE1 and EXE2). We also have a small local scratchpad memory in the last execution stage (EXE2) for faster access to intermediate results during a macro-instruction. The control path shows 6 pipeline stages: fetch, decode instruction, 3 execution stages and write the result back to memory.

Each macro-instruction is decoded into the FSM control mechanism in the Decode stage of SID ’s pipeline. This design is scalable since the hardware is aware of the number of parallel datapath tracks that are implemented and can perform automatic control of the FSM for vector and matrix operations, and any loop iterations required. For performance, the control by an FSM avoids using branch instructions for frequent jump-backs in loop-control, as is needed in general-purpose processors, which can take up a large portion of processor throughput for the simple loops needed to implement vector and matrix computation.

We discuss two optimizations: we reduce the number of memory accesses with local storage and we minimize the hardware design cost by reusing functional units.

When computing the matrix-vector multiplication in the MVmul mode, we use a local scratchpad memory and loop tiling to save the latency of storing and accessing partial sums from memory and also reduce the memory traffic. In the Vmaxabs mode which finds the maximum absolute value in a vector by doing a comparison of input elements in the EXE2 stage, a temporary maximum is stored in the local scratchpad to reduce external memory accesses; it gets updated every cycle. In the Vsqnorm mode which computes the squared L2-norm of a vector, the local scratchpad memory stores the partial sum of x[i]² (x is the input vector), which are computed by the multipliers and adders in the EXE1 and EXE2 stages.

When computing non-linear functions like sigmoid (Vsig), tanh (Vtanh) and exponential (Vexp), we avoid implementing complex non-linear functional units, but instead use the flexible look-up table (LUT) to look up the slope and intercept for linear approximation. An added benefit of our approach over prior work, e.g. (Chen et al., 2014), is that we place the LUTs before the multipliers and adders in the three consecutive execution stages so that no extra multipliers or adders are needed. The ELE0 (LUT) stage of SID outputs a slope, k[i], and an intercept, b[i], for each input value. The interpolation is then computed in the later two stages as z[i] = k[i] $\times$ x[i] + b[i] with z[i] being the value of the non-linear function for input x[i]. Also, instead of having another adder tree stage for the MVmul mode, we save on hardware cost by reusing the adders in the EXE2 stage to sum the products computed in the EXE1 stage and the partial sum read from the local scratchpad. The new partial sum is written back to the local scratchpad memory if the computation is not finished.

Another novel contribution of this work is to show that empirical probability distributions can be collected and compared efficiently using the multipliers and adders already needed for the ML/DL algorithms. To the best of our knowledge, we are the first to describe the following simple and efficient hardware support for collecting error distributions and comparing them with the KS test.

We add two operations for this KS test, but these general-purpose operations can be useful for other ML/DL algorithms and statistical tests as well. The first is a vector-scalar comparison (VSsgt described in Table 3). The second operation is Vmaxabs, which can be used to find the maximum absolute value in a vector.

We illustrate with an example in Figure 2, showing a five-step workflow. The grey dotted boxes represent inputs, which include the reference prediction error distribution (PED), the test PED, the threshold and the output. The reference PED is collected in the training phase and is represented by reference histogram bin boundaries and a reference cumulative histogram. The test PED is collected online and represented by a series of observed test errors.

Step ①: compare an observed error with reference bin boundaries. The output of this step is a vector of “0”s and “1”s. “1” means that the corresponding bin boundary is greater than the observed error, and “0” otherwise. This uses the VSsgt operation. Step ②: accumulate all binary vectors from ①. The accumulated vector, namely the “test cumulative histogram”, represents the cumulative histogram of the observed test errors using the reference bins. Step ③ and step ④: find the largest difference in the reference and test cumulative histograms. Step ③ is a vector subtraction and step ④ is the Vmaxabs operation, to find the maximum absolute value in a vector. This gives us the right hand side of Eq (3) without dividing by $n$, the number of data points in the testing error distribution. Hence, we have computed $n\times D_{n,m}$ at the end of step ④. Step ⑤: compare the largest difference with a threshold. We do the equivalent comparison as Eq (4) by multiplying both sides by $n$. In our experiments, $m$, which is the number of data points in the reference error distribution, and $n$ are hyper-parameters that are decided during training and always set to be the same. In the example of Figure 2, both $n$ and $m$ are 5. The test histogram is considered abnormal if the largest difference is larger than the threshold, $T$.

New sensor measurements, e.g., motion sensors like the accelerometer and gyroscope, are available every 20 ms in most smartphones. Hence, we cannot detect any impostor in less than this time.

Figure 3 compares different machine learning and deep learning algorithms with respect to accuracy and execution time on SID. The algorithms to the left of the black dashed line are used in the IDaaS scenario. Although the SVM algorithm achieves slightly higher accuracy than MLP-200-100 (98.4% versus 97.1%), it needs a longer execution time for doing inferencing.

The algorithms used in the LAD scenario are to the right of the dashed line. We measure the execution time of the best algorithm in Section 3.4, i.e. PED-LSTM-Vote, and the baseline algorithm, LSTM-th, for comparison. We choose the best LSTM size, which is 200, and consider the cases of both the 64-reading window and the 200-reading window. Figure 3 shows that while the KS test technique increases the detection accuracy, it also needs additional execution time. However, the execution time (less than 4 ms) of all algorithms is always much smaller than 20ms, so the performance of SID  is more than adequate to achieve impostor detection with the highest accuracy of PED-LSTM-Vote. Depending on the size of the window (64-reading or 200-reading), the detection time from attack to detection by PED-LSTM-Vote is 1.28s or 4s (see Section 3.5) plus the last 4 ms execution time for the LSTM-PED-Vote inference algorithm. Hence, the execution time of SID is negligible (and more than sufficient) compared to the time to collect sufficient consecutive sensor readings.

Figure 4 compares models used in the IDaaS and LAD scenarios, in terms of their accuracy and the model size. In the IDaaS scenario (left), we see that a 2-layer MLP-200-100 can achieve a slightly higher detection accuracy with a smaller model size than MLP-500. The SVM model has a very small improvement on the accuracy over MLP-200-100 but incurs the highest cost in terms of memory usage as it has to store support vectors. Hence, MLP-200-100 appears to be the best for the cost (execution time + memory usage) versus accuracy trade-off for the IDaaS scenario.

In the LAD scenario (right) which is better for protecting the privacy of a user’s sensor data, we evaluate the additional memory usage of the KS test technique compared to the baseline LSTM-th algorithm where the size of the LSTM cell is still 200. We find that PED-LSTM-Vote uses only 1.6% and 4.8% more space, for the 64-reading window and 200-reading window, respectively. They are better choices in the memory-versus-accuracy trade-off as the improvement in accuracy is significant.

We implement an FPGA prototype of the SID module, in order to show how small a hardware module can be to support impostor detection. Our implementation has four parallel tracks and a 256-byte scratchpad memory. The size of the datapath RAM is 1.75MB and the size of the instruction RAM is 128KB. We use 32-bit fixed-point numbers since prior work (Liu et al., 2015) has shown significant accuracy degradation with 16-bit numbers for certain models. SID would be even smaller if we use lower-precision numbers. The platform board is Xilinx Zynq-7000 SoC ZC706 evaluation kit. The hardware implementation is generated with Vivado 2016.2.

In Table 5, we compare SID  implementing LSTM-PED-Vote to two FPGA implementations of Recurrent Neural Network (RNN) accelerators, C-LSTM (Wang et al., 2018) and DeltaRNN (Gao et al., 2018). The C-LSTM algorithm represents some matrices in LSTM as multiple block-circulant matrices, to reduce the storage and computation complexity. DeltaRNN ignores minor changes in the input of the Gated Recurrent Unit (GRU) RNN to reuse the old computation result and thus reduce the computation workload. These accelerators are capable of RNN inference, which is found to be important for the LAD scenario, and consist of multiple submodules or pipeline stages. More hardware resources are required because each of the submodules is designed to optimize and compute one specific operation in the inference data flow, e.g., one matrix-vector multiplication or non-linear activation, and the functional units are not reused for different operations. Also, the accelerators have a higher level of parallelism than the SID module. However, they lack the support for generating and comparing empirical prediction error distributions, which we have shown is indispensable to achieve better accuracy.

Table 5 shows a major difference between our minimalist SID module and the performance-oriented accelerators. The FPGA resource usage of Slice LUTs, Slice Flip-flops and DSPs of SID are one or two orders of magnitude less than the other two RNN implementations. We measure the FPGA power consumption using the TI Fusion Digital Power Designer tool while the power of C-LSTM is also measured with a TI Fusion Power device and DeltaRNN with a Voltcraft energy monitor. SID’s power consumption is an order of magnitude less, making it more suitable for a smartphone.

While we have used an FPGA implementation to prototype SID, and enable comparisons with existing FPGA accelerators, further power reduction is achievable using an ASIC implementation in real smartphone products.