Privacy Preserving in Non-Intrusive Load Monitoring: A Differential Privacy Perspective

NILM and DP are both well investigated. Since the seminal work by Hart [5], diverse techniques for NILM have been proposed to solve NILM for improved inference performances. Our work focuses on the disaggregation process in NILM. The classical algorithm is Combinatorial Optimization (CO), proposed in Hart’s seminal work. This algorithm combines heuristic methods with prior knowledge on the switching events. Zulfiquar et al. have furthered this research by introducing the aided linear integer programming technique to speed up CO in [11]. Probabilistic methods have also been proposed recently. For example, Kim et al. have applied Factorial Hidden Markov Model (FHMM) to NILM, and have used Viterbi algorithm for decoding [12]. The key challenge in this line of research is to model the appropriate states for the hidden Markov model. Makonin et al. have used machine learning techniques to solve this issue and have proposed the notion of super-state HMM [13]. The advance in deep learning warrants exploiting more temporal features in NILM. For example, Kelly et al. have deployed the long short-time memory (LSTM) framework for NILM in [14].

There is a recent interest in designing the privacy preserving NILM, and our work also falls into this category. However, most literature have focused on discussing thwarting the privacy attacks on the meter data [15]. The major technique is to utilize the storage system to physically inject noises into the meter data, which does offer the end users a certain privacy guarantee. Backes et al. have theoretically examined the way to achieve different levels of privacy using storage injection manipulation in [16]. More recently, a variety of methods have been exploited to achieve privacy preservation.

Chen et al. have utilized the combined heat and privacy system to prevent occupancy detection in [17]. Cao et al. have investigated the practical implementation from a fog computing approach [18]. Rastogi et al. have further proposed a distributed implementation in [19].

To the best of our knowledge, we are the first to exploit the physical meaning of privacy parameters in the electricity sector.

In establishing the connection between DP parameters and the inference accuracy of NILM, our principal contributions can be summarized as follows:

• •

  NILM Formulation with DP: We use the compressive sensing framework to formulate NILM inference and incorporate the parameters of DP into the formulation.

• •

  Theoretical Characterization of NILM Inference: Based on the compressive sensing formulation, we theoretically characterize the asymptotic upper and lower bounds for the one-shot NILM inference accuracy.

• •

  Hierarchical Multi-shot NILM: We generalize the one-shot NILM solution to more practical multi-shot scenarios, and propose an effective hierarchical algorithm.

We imagine our proposed framework has at least three early adopters.

• •

  The first adopter could be the consumers themselves. To preserve privacy, the consumers could utilize the local storage devices (provided by electric vehicles, photovoltaic panels, etc.) to inject noises to achieve certain level of differential privacy. Our theoretical understanding provides an inference accuracy bound to decipher the privacy preserving guarantee.

• •

  The other adopter could be the ISOs or the utility companies. When recording the meter data from the consumers for potential behavior analysis, the ISOs or utility companies could directly inject noises into the recorded meter data. Due to the Law of Larger Numbers, the injected noise will incur minimal impact on auditing (in fact, auditing before injecting the noise could solve this issue) and most classical tasks of utility companies. However, the injected noise could affect behavior analysis (see detailed discussion in Appendix -A). This is another way to decipher the physical meaning of DP parameters.

• •

  The last adopter could be the third-party privacy preserving entities. They could collect the data from the consumers for different purposes. And for different privacy preserving requirements, the consumers are paid for different compensations. This could enable new business models for the privacy preserving industry, and our theoretical results could give the first cut understanding on the cost benefit analysis for the emerging business models.

The rest of the paper is organized as follows. Section II formulates NILM as a sparse optimization problem. Using the compressive sensing framework, we establish the lower and upper bounds of inference accuracy for NILM, and relate the $\epsilon$-differential privacy to these asymptotic bounds in Section III to better understand the physical implication of DP. Then, in Section IV, we generalize the compressive sensing framework to the multi-shot scenario and introduce our hierarchical algorithmic framework. Numerical studies verify the effectiveness of our theoretical conclusion in Section V. Concluding remarks and several interesting future directions are given in Section VI. We provide all the necessary proofs and justifications in Appendix. We visualize the paradigm of our paper in Fig. 1, which highlights the underlying logic between sections.

In essence, the goal of NILM is to infer user behaviors from meter data. Mathematically, for each end user (or a building complex, a campus, etc.), it may own $N$ appliances, denoted by a set $\mathcal{A}=\{A_{1},..,A_{N}\}$.

For simplicity, we assume the state space for each appliance is binary, i.e., on or off. For appliance $i\in\mathcal{A}$, when its state is on, its energy consumption at each time $t$ is a random variable $Z_{i}^{t}$, with mean of $P_{i}$. Define the vector of mean energy consumption over all appliances $\mathbf{P}$ as follows:

Hence, at each time $t$, the energy consumption of the end user is this aggregate energy consumption of its appliances whose states are on. We denote this subset by $S_{t}\subseteq\{1,...,N\}$, and denote the meter data at time $t$ by $y_{t}$. Thus, by definition of $y_{t}$, we know

where $T$ stands for the length of observation period. If we consider the end user’s hourly operation with a resolution of $6$ seconds, then $T$ is $600$. The advanced metering technology has made it possible to collect data at much finer resolution, e.g., in a sub-second scale [20]. Such data could allow us to exploit more unique energy consumption patterns for each appliance. However, in this paper, we choose to focus on a stylized model to better clarify the connection between DP and NILM, without specifying the resolution.

The classical NILM seeks to infer the on/off switch events of all appliances. We denote $X_{t}\in\{0,1\}^{N}$ the on/off indicator of the $N$ appliances at time $t$ (with zero indicating off and one indicating on). NILM aims at inferring $X_{t}$ from $y_{t}$.

A variety of NILM algorithms have been proposed as we have reviewed in Section I. In this work, we choose the compressive sensing framework for mathematically exploiting the sparsity structure in the event vector $X_{t}$. However, conducting NILM inference can be delicate as the energy consumptions in $\mathcal{A}$ can be quite diverse, and some ‘elephant’ appliances (that consume a huge amount of power) may dominate the period of interest. In this case, it is almost impossible to accurately distinguish the on/off states of small appliances. In the following formulation, we devise a sufficient condition to enable accurate NILM inference, based on which, we propose the hierarchical framework for more practical situations in the subsequence analysis.

To better characterize the relationship between differential privacy and NILM, we focus on the inference for a particular time $t$ and assume the knowledge of $X_{t-1}$. We term this problem the one-shot NILM inference.

To exploit the sparse structure, the compressive sensing formulation requires the following transforms:

With these transforms, we can mathematically characterize the sparsity assumption.

This sparsity assumption motivates us to formulate the following optimization problem for the one-shot NILM inference at each time $t$:

where $\delta$ is the parameter to characterize the sensitivity of $y_{t}$.

The notion of sensitivity is defined for random variables:

Back to our problem (P1), the sensitivity constraints require the meter data are bounded, which yields our second assumption:

This assumption is straightforward to justify: it simply suggests that when the appliances are running, their energy consumption levels are within the specific range. And such range corresponds to the sensitivity parameter $\delta$.

Note that, (P1) is similar to the classical problem in the compressive sensing framework [25] but not exactly the same. We denote the optimal solution to (P1) by $\Delta_{t}^{0}$ and regard it as the ground truth. Deciphering this ground truth is quite challenging: the binary constraints in addition to the $l_{0}$-norm objective function make the problem intractable.

To tackle these challenges, we propose to first relax the binary constraints and then use the standard compressive sensing technique to handle the non-convexity induced by the $l_{0}$-norm objective function.

Specifically, we make the following technique alignment assumption to enable exact relaxation:

This assumption allows us to relax the binary constraints without changing the structure of the optimal solution set:

We formally state the equivalence between (P1) and (P2) in the following Lemma.

We show detailed proof in Appendix -C. This lemma allows us to focus on solving a more tractable problem, (P2). This is due to the key theoretical basis of compressive sensing: we could effectively use $l_{1}$-norm to approximate $l_{0}$-norm in (P2) [26], which yields (P3),

Denote its optimal solution by $\Delta_{t}^{*}$. To map $\Delta_{t}^{*}$ onto $\{0,1\}^{N}$, we conduct rounding. Denote the solution after rounding by $\bar{\Delta}_{t}$. The rounding process is as follows: we first set all elements in $\bar{\Delta}_{t}$ to be $0$; and then, for each non-zero element $\Delta_{t}^{*}(j)$ in $\Delta_{t}^{*}$, with probability $\Delta_{t}^{*}(j)$, we set the corresponding element $\bar{\Delta}_{t}(j)$ to be $1$. To characterize the inaccuracy induced by the approximation of $l_{0}$-norm, we compare $\mathbf{E}[\bar{\Delta}_{t}]$ and the ground truth, $\Delta_{t}^{0}$. The comparison yields the following theorem:

We characterize the specific form of $C(\mathbf{P})$ in Appendix -E. To prove Theorem 1, we mainly make use of the important conclusion in compressive sensing proposed by Candes et al. in [26].

We adopt the notion of $\epsilon$-differential privacy [9] to connect NILM with DP. It uses the parameter $\epsilon$ to denote the probability that we are unable to differentiate the two datasets that have only one piece of data difference.

To put it more formally, we define a mapping $\mathscr{B}(D)$ from a dataset $D$ to $\mathbb{R}$. This mapping enables a query function $q:D\to\mathbb{R}$. To measure the difference between two datasets $D$ and $D^{\prime}$, we adopt the distance metric $d(D,D^{\prime})$, measuring the minimal number of sample changes that are required to make $D$ identical to $D^{\prime}$. When $d(D,D^{\prime})=1$, we say $D$ and $D^{\prime}$ are neighbor datasets. This allows us to characterize the notion of $\epsilon$-DP: if for all neighbor datasets $D_{1}$ and $D_{2}$, and for all measurable subsets $Y\subset\mathbb{R}$, the mapping $\mathscr{B}$ satisfies,

we say the mechanism $\mathscr{B}$ achieves $\epsilon$-DP [9].

To achieve $\epsilon$-DP, a simple mechanism is Laplace noise injection [9]. We show the detailed noise injection mechanism in the following theorem.

Thus, the Laplace noise injection mechanism helps us establish the connection between $\epsilon$-DP and NILM inference.

Next, we construct a performance bound related to the privacy parameter $\epsilon$ and show how the level of privacy preservation connects to the performance bound of the NILM inference.

Recall that $\Delta f$ is the sensitivity for meter data $y_{t}$. After injecting the noise, instead of inferring $X_{t}$ using $y_{t}$, now we can only rely on the noisy data $y_{t}+n_{t}$, which yields optimization problem (P4).

Denote the solution to (P4) by $\Delta_{t}^{\epsilon}$, and the solution after rounding by $\bar{\Delta}_{t}^{\epsilon}$. Then, it is straightforward to apply Theorem 1 to (P4), and obtain the following proposition.

This proposition not only dictates that the error’s upper bound increases almost linearly with the magnitude of noises, but also allows us to connect the DP parameters with the NILM inference accuracy.

Mathematically, we define inference accuracy $\alpha$ as follows:

To evaluate $\mathbf{E}[\alpha]$ over all possible injected noises $n_{t}$ and $n_{t-1}$, we denote

We can now establish the lower bound for NILM inference accuracy as Theorem 3 suggests.

Note that a smaller $\epsilon$ implies a higher level of differential privacy. In proving the lower bound of inference accuracy, the key is to identify that for $\|\mathbf{E}[\bar{\Delta}_{t}^{\epsilon}]-\Delta_{t}^{0}\|_{2}$ in Proposition 1, there is a trivial upper bound, $N$. Hence, we can refine Proposition 1 to construct the lower bound for $\mathbf{E}[\alpha]$.

As indicated by Theorem 3, the derived lower bound decreases at the rate of $\epsilon^{-1}$ when $\delta$ is small enough (i.e. the impact of the exponential term is neglectable).

To construct the upper bound, we need to exploit the structure of problem (P4). Specifically, standard mathematical manipulations of constraints in (P4) yield the following upper bound characterization:

Theorem 4 implies that the upper bound decreases at the rate of $o({\epsilon}^{-1}e^{-\epsilon})$. That is, the upper bound would also decrease when $\delta$ becomes larger. Hence, there is a certain gap between the lower bound and the upper bound. We want to emphasize that although the proof (in Appendix -G) is based on the compressive sensing framework, it can be generalized to many other NILM algorithms for the upper bound construction.

The special difficulty in multi-shot NILM inference, compared with one-shot NILM, is the temporal coupling between inferences. This issue, if not well addressed, could lead to large cumulative errors, and ultimately lead to the cascading inference failure²²2See the cascading effect over network in [27] for more detailed discussion.. Therefore, in this part, after the problem formulation, we intend to design an effective algorithm to contain this error.

The multi-shot NILM inference problem is a straightforward generalization of the one-shot NILM inference. It seeks to infer the appliance switch events over a period of $T$ from the meter data sequence $\boldsymbol{Y}=[y_{1},..y_{T}]$. The inference also relies on the vector of mean energy consumption $\mathbf{P}$, defined in Eq. (1). We denote $\boldsymbol{X}=[X_{1},...,X_{T}]$ the state probability matrix, where each $X_{t}$ captures the probability of appliances states at time $t$. To establish the asymptotic bounds for our multi-shot NILM inference accuracy, we denote $\boldsymbol{X}^{0}=[X^{0}_{1},...,X^{0}_{T}]$ the ground truth of switching events over $T$, and $\bar{\boldsymbol{X}^{\epsilon}}=[\bar{X_{1}^{\epsilon}},..,\bar{X_{T}^{\epsilon}}]$ the inference results returned by our proposed algorithm.

It is worth noting that the situation becomes even more complicated if we conduct rounding at each time slot. We submit that it is valuable to keep the inference results obtained in solving problem (P3) without rounding as these results can be interpreted as the probability of switching. This allows us to derive the state probability matrix $\boldsymbol{X}=[X_{1},...,X_{T}]$. Note that, after obtaining the matrix $\boldsymbol{X}$, we can no longer conduct the rounding naively: there is more information about the actual meter readings. We could utilize the simple error correcting methods to decipher the embedded information.

Specifically, the error correcting aims to make the binary adjustments to reduce the approximation error according to the actual meter data. Though it could be possible for us to consider more complex methods for error correcting rather than direct adjustment, they make little difference in terms of the worst case bound analysis.

We describe the classical treatment to multi-shot NILM inference in Algorithm 1, where we adopt the notion of the Hadamard Product, which is defined for two matrices $\boldsymbol{A}$ and $\boldsymbol{B}$ as follows:

There is a simple way to construct rough inference accuracy bounds directly from Theorem 3 and Theorem 4. The trick is to examine the rounding procedure in Algorithm 1 and to connect it with the bounds for one-shot NILM inference. Formally, we define the accuracy of the classical algorithm for multi-shot NILM inference by $\alpha_{m}$:

Denote the lower bound and upper bound for the one-shot NILM inference accuracy by $b(\delta,\epsilon)$ and $B(\delta,\epsilon)$, respectively (see Theorem 3 and 4). Recall that $\epsilon$ is the DP’s parameters (see Theorem 2) and $\delta$ describes the sensitivity of the meter data for each period $y_{t}$ (see Eq. (7)).

With these definitions, we can characterize the lower and upper bounds for $\mathbf{E}[\alpha_{m}]$:

Albeit that we have deployed the error correction method, the two bounds in Theorem $5$ are still rather loose. For example, as $T$ increases, the lower bound could quickly approach $0$ while the upper bound would quickly approach $1$. This is because we still need to take into account the cascading effects in the union bound though we have made some efforts to contain the error. Another reason is due to Assumption 3. We may relax this assumption for improved bounds.

The idea is simple and straightforward. Intuitively, to relax Assumption 3, we need to conduct a hierarchical decomposition for different proper appliance sets. To obtain the proper sets, we first sort all the appliances in $\mathcal{A}$ according to their mean energy consumption levels. Then, starting with the least energy-consuming appliance, we could form a number of proper sets that satisfy Assumption 3.

To construct the sets, we use the greedy policy. For a certain set with $S$ appliances, the condition that makes the $(S+1)^{th}$ appliance join the current set instead of creating a new set is as follows:

where the subscript $i$ in $P_{i}$ also illustrates its ascending order in the set.

We call each set a hierarchy, which establishes the basis for our hierarchical multi-shot NILM inference. We apply Algorithm 1 to each hierarchy in the descending order. We characterize the whole procedure in Algorithm 2.

We submit that the hierarchical decomposition helps refine asymptotic bounds for $\mathbf{E}[\alpha_{m}]$ in that it characterizes the connection between hierarchies. This allows us to derive a recursive formulation to estimate the overall inference accuracy. Define inference accuracy for hierarchy $i$ as $\alpha_{i}$:

where $N_{i}$ denotes the number of appliances in hierarchy $i$; $\boldsymbol{X}^{\epsilon}[i]$ and $\boldsymbol{X}[i]^{0}$ denote the inferred result and the ground truth of the appliances in hierarchy $i$, respectively. We also define the mean power matrix for hierarchy $i$ as $\mathbf{P}_{i}$ and the minimum power in hierarchy $i$ as $P^{i}_{m}$. For simplicity, we also define the maximal summation of $U$ appliances whose power is smaller than $P^{i}_{m}$ as $P^{i}_{U}$, in which $U$ is the sparsity of the switching events as we defined in Assumption 1.

Now we can formally introduce our bounds characterization theorem for hierarchical multi-shot NILM inference, with the help of Theorem 5. For simplicity, define

Then, we can show that

Specifically, suppose we have $K$ hierarchies in the problem, the overall accuracy $\alpha_{H}$ can be recursively obtained as follows:

The associated bounds for our multi-shot compressive sensing algorithm with hierarchical decomposition could also be derived straightforwardly with the help of Theorem 6.

In this paper, we use three publicly available datasets.

• •

  The UK-DALE dataset [28] records the electricity consumption from five households in UK, with a sampling rate of 6 Hz for each individual appliances. For performance evaluations, we choose the data segment from 2013/05/26 21:03:14 to 2013/05/27 0:54:50.

• •

  The TEALD dataset [20] is collected by Rainforest EMU2 in Canada. With a sampling rate of 1 Hz, we use the data segment from 2016/02/09 16:00:00 to 2016/02/10 15:59:59 for performance evaluation.

• •

  The Redd dataset [29] is also commonly used for NILM performance evaluation. We use the data from 2011/04/18 21:22:13 to 2011/05/25 03:56:34 with a sampling rate of 3 HZ.

We use the dataset of building 4 in UK-DALE dataset [28] to conduct the first numerical analysis. In this building, there are 8 appliances.

Using the one-shot NILM inference based on compressive sensing, we evaluate the inference performance with increasing level of DP. Fig. 2 plots the theoretical lower bound, the actual performance traces, and the theoretical upper bound to evaluate our theorems. The lower bound is not tight mainly due to the sensitivity and parameter $C(\mathbf{P})$. The sensitivity is not tight in that it is time-varying. In some time slots, some appliances are simply idle. However, they all contribute to the sensitivity. The parameter $C(\boldsymbol{P})$ is not tight due to exactly the same reason. The meter data suggests $\delta$ be $2$, and $C(\boldsymbol{P})$ should be $0.015$. Nevertheless, our lower bound is tight in terms of magnitude. The mean of the actual inference accuracy indeed decreases at the rate of $\epsilon^{-1}$.

On the other hand, our proposed upper bound seems tight for this case study. We want to emphasize that this upper bound may not always be tight in that we assume the existence of an oracle in the upper bound construction. Hence, this upper bound would be more ideal to serve as a benchmark for all possible NILM algorithms. Hence, from Fig. 2, we can conclude that compressive sensing is a good approach for one-shot NILM inference.

We then use the UK-DALE dataset to validate our Algorithm 1 which relies on Assumption 3. The inference accuracy is shown in Fig. 3. In fact, our proposed approach performs remarkably good, and the inference accuracy trend shows a similar decreasing rate as Theorem 6 dictates. We want to emphasize that, all of our figures are plotted on a half log scale, which is ideal to verify the decreasing rate of $\epsilon^{-1}$.

We use the TEALD dataset to further demonstrate the effectiveness of our hierarchical decomposition. There are 13 appliances in the dataset, with diverse energy consumptions. We group them into four hierarchies and conduct the NILM inference. Fig. 4 plots its inference accuracy trend in the DP parameter, which again highlights the remarkable performance of our proposed algorithm, and verifies the effectiveness of our derived bounds. During the simulation, we observe that our hierarchical algorithm seems to be quite robust against noises, which may lead to interesting theoretical results, though a detailed discussion is beyond the scope of our work.

We examine the robustness of our conclusion through numerical studies. Specifically, to understand if our conclusion only holds for Laplace noise injection. We consider other noise injection methods, which also achieves $\epsilon$-DP. Here, we employ the staircase mechanism, described in Algorithm 3, which achieves $\epsilon$-DP [30].

We follow the same routine in Section V-C to conduct the numerical study and compare the results in Fig. 5. It is clear that different noise injection methods lead to similar relationships between the DP parameters and the inference accuracy. This further illustrates the robustness of our conclusion against noise injection methods.

We can now safely conclude that, in the compressive sensing framework, the inference accuracy heavily relies on the DP parameter $\epsilon$. Specifically, we observe the inference accuracy decreases linearly with $\ln{(\frac{1}{\epsilon})}$ as our figure shows when $\ln{(\frac{1}{\epsilon})}$ is large. We now try to generalize this conclusion to other common frameworks in NILM literature. Specifically, we conduct numerical studies to empirically compare how DP’s parameter affects the accuracy of common NILM algorithms, including sparse Viterbi algorithm for super-state HMM (Sparse-HMM), aided linear Integer Programming (ALIP), linear Integer Programming (IP), Recurrent Neural Network (RNN), CO, FHMM, and our proposed algorithm (Multi-shotCS).

We conduct numerical comparison on Building 1 in the Redd dataset. For supervised learning models, we divide the dataset for training (before 2011-04-03) and for validation (after 2011-04-03). Then, we inject noises to the validation set for performance evaluation. For unsupervised model (including our approach), we directly inject noises to the meter data.

We explain the details about the hyperparameter settings as follows. In fact, most of the settings follow the seminal works.

• •

  When implementing the SparseHMM algorithm, we follow the hyperparameters in the seminal Redd dataset numerical study in [13]. Specifically, the number of the maximal super state is set to be $4$; and the state number choice parameter $\epsilon$ is set to be $0.00021$.

• •

  As for ALIP and IP algorithms, we follow all the parameters in [11], including states, specific ratings and the state transient values. We choose the zero initialization for these two algorithms.

• •

  For RNN implementation, we follow the RNN structure in [14]. Specifically, we first adopt the 1D convolution layer to handle the input data. And then use two bidirection LSTM model and a fully connected layer to derive the output. We adopt the Adam optimizer and the initial learning rate of $0.01$ for training. The number of training epochs is set to be $5$ and the batch size is set to be $128$.

• •

  For CO and FHMM, we again adopt the classical settings in NILMTK [31]. The detailed settings are also provided in the online document [32].

• •

  For our proposed Multi-shot compressive sensing algorithm, we evaluate the meter data and suggest $\delta$ should be $20$. This is derived by observing the maximal fluctuations in the historical meter data during the period without any switch event.

As indicated by Fig. 6, among all the algorithms, SparseHMM achieves the best performance as it makes full use of the sparsity assumption and exploits the temporal correlations through the hidden Markov chain. They are also the underlying guarantee for the remarkable performance of our proposed framework. However, the utilization of temporal correlation makes both algorithms vulnerable to the noise injection due to the cascading inference failure. While ALIP outperforms the conventional IP approach, both of them show strong robustness to noise injection. This is because they do not rely on the sparsity assumption, and the median filter process in the two algorithms contributes to smooth out the fluctuations incurred by noise injection. The ordinary performance of RNN is due to the limited training data and the overfitting issue. The fewer assumption utilized by classical methods (CO and FHMM) makes them more robust to noise injection at the cost of low inference accuracy.

The key observation is that the inference accuracy of each of the seven algorithms exhibits piecewise linear relationship with $\ln(\frac{1}{\epsilon})$, though with different breaking points and slopes. The slopes can serve as the indicators to evaluate each algorithm’s robustness to Laplace noise (and hence its difficulty to achieve privacy preserving).

Nonetheless, our proposed bounds provide the correct magnitude to explain the physical meaning of DP parameters in most common frameworks for NILM, which could be inferred from Fig. 6 empirically. Therefore, in the practical utilization of DP, our theoretical performance bounds could offer magnitude estimation for the effects of the different DP levels. Moreover, these empirical results also inspire us to derive a performance bound for these NILM algorithms for further discussion.