Corruption-tolerant Algorithms for Generalized Linear Models

Generalized linear models (GLMs) [17] are effective models for a variety of discrete and continuous label spaces, allowing the prediction of binary or count-valued labels (logistic, Poisson regression) as well as real-valued labels (gamma, least-squares regression). Inference in a GLM involves two steps: given a feature vector ${{\mathbf{x}}}\in{\mathbb{R}}^{d}$ and model parameters ${{\mathbf{w}}}^{\ast}$, a canonical parameter is generated as $\theta:=\left\langle{{{\mathbf{w}}}^{\ast}},{{{\mathbf{x}}}}\right\rangle$ then the label $y$ is sampled from the exponential family distribution

where the function $h(\cdot)$ is specific to the GLM and $\psi(\cdot)$ is a normalization term, also known as log partition function. It is common to use a non-canonical link such as $\theta:=\exp(\left\langle{{{\mathbf{w}}}^{\ast}},{{{\mathbf{x}}}}\right\rangle)$ for gamma distribution. GLMs also admit vector valued label ${{\mathbf{y}}}\in{\mathbb{R}}^{n}$ by substituting the scalar product by inner product $\left\langle{{{\mathbf{y}}}},{\text{\boldmath$\mathbf{\eta}$}}\right\rangle$ where $\text{\boldmath$\mathbf{\eta}$}:={{\mathbf{X}}}{{\mathbf{w}}}^{\ast}$ is the canonical parameter and ${{\mathbf{X}}}\in{\mathbb{R}}^{n\times d}$ is the covariate matrix.

Problem Description: Given data $\{{({{\mathbf{x}}}^{i},y_{i})}\}_{i=1}^{n}$ generated using a known GLM but unknown model parameters ${{\mathbf{w}}}^{\ast}$, statistically efficient techniques exist to recover a consistent estimate of the model ${{\mathbf{w}}}^{\ast}$ [14]. However, these techniques break down if several observed labels $y_{i}$ are corrupted, not just by random statistical noise but by adversarially generated structured noise. Suppose $k<n$ labels are corrupted i.e. for some $k$ data points $i_{1},\ldots,i_{k}$, the actual label $y_{i_{j}},j=1,\ldots,k$ generated by the GLM are replaced by the adversary with corrupted ones say $\tilde{y}_{i_{j}}$. Can we still recover ${{\mathbf{w}}}^{\ast}$? Note that the learning algorithm is unaware of the points that are corrupted.

Breakdown Point: The largest fraction $\alpha=k/n$ of corruptions that a learning algorithm can tolerate while still offering an estimate of ${{\mathbf{w}}}^{\ast}$ with bounded error is known as its breakdown point. This paper proposes the SVAM algorithm that can tolerate $k=\Omega\left({{n}}\right)$ corruptions i.e. $\alpha=\Omega\left({{1}}\right)$.

Adversary Models: Contamination of the training labels $y_{1},\ldots,y_{n}$ by an adversary can misguide the learning algorithm into selecting model parameters of the adversary’s choice. An adversary has to choose (1) which labels $i_{1},\ldots,i_{k}$ to corrupt and (2) what corrupted labels $\tilde{y}_{i_{1}},\ldots,\tilde{y}_{i_{k}}$ to put there. Adversary models emerge based on what information the adversary can consult while making these choices. The oblivious adversary must make both these choices with no access to the original data $\{{({{\mathbf{x}}}^{i},y_{i})}\}_{i=1}^{n}$ or true model ${{\mathbf{w}}}^{\ast}$ and thus, can only corrupt a random/fixed subset of $k$ labels by sampling $\tilde{y}_{i_{j}}$ from some predetermined noise distribution. This is also known as the Huber noise model. On the other hand, a fully adaptive adversary has full access to the original data and true model while making both choices. Finally, the partially adaptive adversary must choose the corruption locations without knowledge of original data or true model but has full access to these while deciding the corrupted labels. See Appendix B for details.

Contributions: This paper describes the SVAM (Sequential Variance-Altered MLE) framework that offers:
1. robust estimation with a breakdown point $\alpha=\Omega\left({{1}}\right)$ against partially and fully adaptive adversaries for robust least-squares regression and mean estimation and $\alpha=\Omega\left({{1/\sqrt{d}}}\right)$ for robust gamma regression. Prior works do not offer any breakdown point for gamma regression.
2. exact recovery of the true model ${{\mathbf{w}}}^{\ast}$ against a fully-adaptive adversary for the case of least squares regression,
3. the use of variance reduction technique (see §3.1) in robust learning, which is novel to the best of our knowledge,
4. extensive empirical evaluation demonstrating that despite being a generic framework, SVAM is competitive to or outperforms algorithms specifically designed to solve problems such as least-squares and logistic regression.

In the interest of space, we review aspects of literature most related to SVAM and refer to others [6, 15] for a detailed review.

Robust GLM learning has been studied in a variety of settings. [2] considered an oblivious adversary (Huber’s noise model) but offered a breakdown point of $\alpha={\cal O}\left({{\frac{1}{\sqrt{n}}}}\right)$ i.e. tolerate $k\leq{\cal O}\left({{\sqrt{n}}}\right)$ corruptions. [25] solve robust GLM estimation by solving M-estimation problems. However, they require the magnitude of the corruptions to be upper-bounded by some constant i.e. $\left|{y_{i}-\tilde{y}_{i}}\right|\leq{\cal O}\left({{1}}\right)$ and offer a breakdown point of $\alpha={\cal O}\left({{\frac{1}{\sqrt{n}}}}\right)$. Moreover, their approach solves $L_{1}$-regularized problems using projected gradient descent that offers slow convergence. In contrast, SVAM offers a linear rate of convergence, offers a breakdown point of $\alpha=\Omega\left({{1}}\right)$ i.e. tolerate $k=\Omega\left({{n}}\right)$ corruptions and can tolerate corruptions with unbounded magnitude introduced by a partially or fully adaptive adversary.

Specific GLMs such as robust regression have received focused attention. Here the model is ${{\mathbf{y}}}=X{{\mathbf{w}}}^{\ast}+{{\mathbf{b}}}$ where $X\in{\mathbb{R}}^{n\times d}$ is the feature matrix and ${{\mathbf{b}}}$ is $k$-sparse corruption vector denoting the adversarial corruptions. A variant of this, studies a hybrid noise model that replaces the zero entries of ${{\mathbf{b}}}$ with Gaussian noise ${\mathcal{N}}(0,\sigma^{2})$. [18, 24] solve an $L_{1}$ minimization problem which is slow in practice. (see §5). [1] use hard thresholding techniques to estimate the subset of uncorrupted points while [15] modify the IRLS algorithm to do so. However, [1, 15] are unable to offer consistent model estimates in the hybrid noise model even if the corruption rate $\alpha=k/n\rightarrow 0$ which is surprising since $\alpha\rightarrow 0$ implies vanishing corruption. In contrast, SVAM offers consistent model recovery in the hybrid noise model against a fully adaptive adversary when $\alpha\rightarrow 0$. [21] also offer consistent recovery with breakdown points $\alpha>0.5$ but assume an oblivious adversary.

Robust classification with $y_{i}\in\left\{{-1,+1}\right\}$ has been explored using robust surrogate loss functions [16] and ranking [8, 19] techniques. These works do not offer breakdown points but offer empirical comparisons.

Robust mean estimation entails recovering an estimate $\hat{\text{\boldmath$\mathbf{\mu}$}}\in{\mathbb{R}}^{d}$ of the mean $\text{\boldmath$\mathbf{\mu}$}^{\ast}$ of a multivariate Gaussian ${\mathcal{N}}(\text{\boldmath$\mathbf{\mu}$}^{\ast},\Sigma)$ given $n$ samples of which an $\alpha$ fraction are corrupted [11]. Estimation error is known to be lower bounded $\left\|{\hat{\text{\boldmath$\mathbf{\mu}$}}-\text{\boldmath$\mathbf{\mu}$}^{\ast}}\right\|_{2}\geq\Omega\left({{\alpha\sqrt{\log\frac{1}{\alpha}}}}\right)$ for this application even if $n\to\infty$ [5]. [6] use convex programming techniques and offer ${\cal O}\left({{\alpha\log^{\frac{3}{2}}{\frac{1}{\alpha}}}}\right)$ error given $n\geq\tilde{\Omega}\left({{\frac{d^{2}}{\alpha^{2}}}}\right)$ samples and a $\text{poly}\left({n,d,\frac{1}{\alpha}}\right)$ runtime. [3] improve the running time to $\frac{\widetilde{\cal O}\left({{nd}}\right)}{\text{poly}(\alpha)}$. The recent work of [4] uses an IRLS-style approach that internally relies on expensive SDP-calls but offers high breakdown points. SVAM uses $n={\cal O}\left({{\log^{2}\frac{1}{\alpha}}}\right)$ samples and offers a recovery error of ${\cal O}\left({{\textit{trace}(\Sigma)(\log\frac{1}{\alpha})^{-1/2}}}\right)$. This is comparable to existing works if $\textit{trace}(\Sigma)={\cal O}\left({{1}}\right)$. Moreover, SVAM is much faster and simpler to implement in practice.

Meta algorithms such as robust gradient techniques, median-of-means [12], tilted ERM [13] and maximum correntropy criterion [9] have been studied. SEVER [7] uses gradient covariance matrix to filter out the outliers along its largest eigenspace while RGD [20] uses robust gradient estimates to perform robust first-order optimization directly. While convenient to execute, they may require larger training sets, e.g., SEVER requires $n>d^{5}$ samples for robust least-squares regression whereas SVAM requires $n>\Omega\left({{d\log(d)}}\right)$. In terms of recovery guarantees, for least-squares regression without Gaussian noise, SVAM and other methods [1, 15]) offer exact recovery of ${{\mathbf{w}}}^{\ast}$ so long as the fraction of corrupted points is less than the breakdown point while SEVER’s error continues to be bounded away from zero. RGD only considers an oblivious/Huber adversary while SVAM can tolerate partially/fully adaptive adversaries. SEVER does not report an explicit breakdown point, RGD offers a breakdown point of $\alpha=1/\log d$ (see Thm 2 in their paper) while SVAM offers an explicit breakdown point independent of $d$. SVAM also offers faster convergence than existing methods such as SEVER and RGD.

A popular approach in robust learning is to assign weights to data points, hoping that large weights would be given to uncorrupted points and low weights to corrupted ones, followed by weighted likelihood maximization. Often the weights are updated, and the process is repeated. [2] use Huber style weighing functions used in Mallow’s type M-estimators, [15] use truncated inverse residuals, and [22] use Mahalanobis distance-based weights.

SVAM notes that the label likelihood offers a natural measure of how likely the point is to be uncorrupted. Given a model estimate $\hat{{\mathbf{w}}}^{t}$ at iteration $t$, the weight $s_{i}={\mathbb{P}}\left[{{y_{i}\,|\,\eta^{t}_{i}}}\right]=\exp(y_{i}\cdot\eta_{i}^{t}-\psi(\eta_{i}^{t})-h(y_{i}))$ can be assigned to the $i\text{${}^{\text{th}}$}$ point where $\eta^{t}_{i}=\left\langle{\hat{{\mathbf{w}}}^{t}},{{{\mathbf{x}}}^{i}}\right\rangle$. This gives us the weighted MLE¹¹1Recall that for gamma/Poisson regression we need to set $\eta^{t}_{i}=\exp(\left\langle{\hat{{\mathbf{w}}}^{t}},{{{\mathbf{x}}}^{i}}\right\rangle)$ given the non-canonical link for these problems. $\tilde{Q}({{\mathbf{w}}}\,|\,\hat{{\mathbf{w}}}^{t})=-\sum_{i=1}^{n}s_{i}\cdot\log{\mathbb{P}}\left[{{y_{i}\,|\,\left\langle{{{\mathbf{w}}}},{{{\mathbf{x}}}^{i}}\right\rangle}}\right]$ solving which gives us the next model iterate as

However, as §5 will show, this strategy does not perform well. If the initial model $\hat{{\mathbf{w}}}^{1}$ is far from ${{\mathbf{w}}}^{\ast}$, it may result in imprecise weights $s_{i}$ that are large for the corrupted points. For example, if the adversary introduces corruptions using a different model $\tilde{{\mathbf{w}}}$ i.e. $\tilde{y}_{i_{j}}\sim{\mathbb{P}}\left[{{y_{i}\,|\,\left\langle{\tilde{{\mathbf{w}}}},{{{\mathbf{x}}}^{i_{j}}}\right\rangle}}\right],j\in[k]$ and we happen to initialize close to $\tilde{{\mathbf{w}}}$ i.e. $\hat{{\mathbf{w}}}^{1}\approx\tilde{{\mathbf{w}}}$, then it is the corrupted points that would get large weights initially that may cause the algorithm to converge to $\tilde{{\mathbf{w}}}$ itself.

Key Idea: It is thus better to avoid drastic decisions, say setting $s_{i}\gg 0$ in the initial stages no matter how much a data point appears to be clean. SVAM implements this intuition by setting weights using a label likelihood distribution with very large variance initially. This ensures that no data point (not even the uncorrupted ones) gets large weight (c.f. the uniform distribution that has large variance and assigns to point a high density). As SVAM progresses towards ${{\mathbf{w}}}^{\ast}$, it starts using likelihood distributions with progressively lower variance. Note that this allows data points (hopefully the uncorrupted ones) to get larger weights (c.f. the Dirac delta distribution that has vanishing variance and assigns high density to isolated points).

This section adapts SVAM to robust least-squares/gamma/logistic regression and robust mean estimation and establishes breakdown points and LWSC/LWLC guarantees for their respective $\tilde{Q}_{\beta}$ functions (see Defn 1). We refer the reader to §1 for definitions of partially/fully adaptive adversaries.

Robust Least Squares Regression. We have $n$ data points $({{\mathbf{x}}}^{i},y_{i})$, ${{\mathbf{x}}}^{i}\in{\mathbb{R}}^{d}$ sampled from a subGaussian distribution ${\mathcal{D}}$ over ${\mathbb{R}}^{d}$. We consider the hybrid corruption setting where on the $G=(1-\alpha)\cdot n$ “good” data points, we get labels $y_{i}=\left\langle{{{\mathbf{w}}}^{\ast}},{{{\mathbf{x}}}^{i}}\right\rangle+\epsilon_{i}$ with Gaussian noise $\epsilon_{i}\sim{\mathcal{N}}(0,\frac{1}{\beta^{\ast}})$ with variance $\frac{1}{\beta^{\ast}}$ added. On the remaining $B=\alpha\cdot n$ “bad” points, we get adversarially corrupted labels $\tilde{y}_{i}=\left\langle{{{\mathbf{w}}}^{\ast}},{{{\mathbf{x}}}^{i}}\right\rangle+b_{i}$ where $b_{i}\in{\mathbb{R}}$ is chosen by the adversary. Note that $b_{i}$ can be unbounded. We also consider the pure corruption setting where clean points receive no Gaussian noise (note that this corresponds to $\beta^{\ast}=\infty$). SVAM-RR (Alg. 2) adapts SVAM to the task of robust regression.

Note that in the pure corruption setting, SVAM assures exact recovery of ${{\mathbf{w}}}^{\ast}$ simply by running the algorithm long enough. This is not a contradiction since in this case, the LWSC/LWSS properties can be shown to hold for all values of $\beta<\infty$ since we effectively have $\beta^{\ast}=\infty$ in this case. Thm 2 holds against a partially adaptive adversary but can be extended to a fully adaptive adversary as well but at the cost of a worse breakdown point (see Thm 3 below). Note that SVAM continues to assure exact recovery of ${{\mathbf{w}}}^{\ast}$.

Establishing LWSC/LWLC: In the appendices, Lemmata 15 and 16 establish LWSC/LWLC properties for robust least squares regression while Theorems 14 and 21 establish the breakdown points and existence of suitable increments $\xi>1$. Handling a fully adaptive adversary requires making mild modifications to the notions of LWSC/LWLC, details of which are present in Appendix G.1.

Model Recovery and Breakdown Point: For pure corruption, SVAM-RR offers exact model recovery against partially and fully adaptive adversaries as it assures $\left\|{\hat{{\mathbf{w}}}^{T}-{{\mathbf{w}}}^{\ast}}\right\|_{2}^{2}\leq\epsilon$ for any $\epsilon>0$ if SVAM-RR is executed long enough. For hybrid corruption where even “clean” points receive Gaussian noise with variance $\frac{1}{\beta^{\ast}}$, SVAM-RR assures $\left\|{\hat{{\mathbf{w}}}^{T}-{{\mathbf{w}}}^{\ast}}\right\|_{2}^{2}\leq{\cal O}\left({{\frac{1}{\beta^{\ast}}\sqrt{\frac{d\log(n)}{n}}}}\right)$ as $\alpha\rightarrow 0$ i.e. $\left\|{\hat{{\mathbf{w}}}^{T}-{{\mathbf{w}}}^{\ast}}\right\|_{2}^{2}\rightarrow 0$ as $n\rightarrow\infty$ assuring consistent recovery. This significantly improves previous results by [1, 15] which offer ${\cal O}\left({{\frac{1}{\beta^{\ast}}}}\right)$ error even if $\alpha\rightarrow 0$ and $n\rightarrow\infty$. Note that SVAM-RR has a superior breakdown point (allowing upto 18% corruption rate) against an oblivious adversary. The breakdown point deteriorates as expected (still allowing upto 0.36% corruption rate) against a fully adaptive adversary. We now present analyses for other GLM problems.

Robust Gamma Regression. The data generation and corruption model for gamma regression are slightly different given that the gamma distribution has support only over positive reals. First, the canonical parameter is calculated as $\eta_{i}=\exp(\left\langle{{{\mathbf{w}}}^{\ast}},{{{\mathbf{x}}}^{i}}\right\rangle)$ using which a clean label $y^{i}$ is generated. To simplify the analysis, we assume that $\left\|{{{\mathbf{w}}}^{\ast}}\right\|_{2}=1$, $\phi=0.5$, ${{\mathbf{x}}}^{i}\sim{\mathcal{N}}({\mathbf{0}},I)$. For the $G=(1-\alpha)\cdot n$ “good” points, labels are generated as $y_{i}=\exp(\left\langle{{{\mathbf{w}}}^{\ast}},{{{\mathbf{x}}}^{i}}\right\rangle)(1-\phi)$ i.e. the no-noise model. For the remaining $B=\alpha\cdot n$ “bad” points, the label is corrupted as $\tilde{y}^{i}=y^{i}\cdot b_{i}$ where $b_{i}>0$ is a positive real number (but otherwise arbitrary and unbounded). A multiplicative corruption makes more sense since the final label must be positive. SVAM-Gamma (Algorithm 4) adapts SVAM to robust gamma regression. Due to the alternate canonical parameter used in gamma regression, the initialization requirement also needs to be modified to $\beta_{1}\cdot\left({\exp\left({\left\|{\hat{{\mathbf{w}}}^{1}-{{\mathbf{w}}}^{\ast}}\right\|_{2}}\right)-1}\right)^{2}\leq 1$. However, the hyperparameter tuning strategy discussed in §3 continues to apply.

Model recovery, Consistency, Breakdown pt. It is notable that prior results in literature do not offer any breakdown point results for gamma regression. We find that Thm 4 requires $\beta_{1}\cdot\left({\exp\left({\left\|{\hat{{\mathbf{w}}}^{1}-{{\mathbf{w}}}^{\ast}}\right\|_{2}}\right)-1}\right)^{2}\leq 1$ and $\beta\geq 1$ which imply $\left\|{\hat{{\mathbf{w}}}^{1}-{{\mathbf{w}}}^{\ast}}\right\|_{2}\leq\ln 2$. This is in contrast to Thms 2 and 3 that allow any initial $\hat{{\mathbf{w}}}^{1}$ so long as $\beta_{1},\xi$ are sufficiently small. SVAM-Gamma guarantees convergence to a region of radius ${\cal O}\left({{\alpha\sqrt{d}}}\right)$ around ${{\mathbf{w}}}^{\ast}$ whereas Thms 2 and 3 assure exact recovery. However, these do not seem to be artifacts of the proof technique. In experiments, SVAM-Gamma did not offer vanishingly small recovery errors and did indeed struggle if initialized with $\beta_{1}\ll 1$. It may be the case that there exist lower bounds preventing exact recovery for gamma regression similar to mean estimation.

Robust Mean Estimation. We have $n$ data points of which the set $G$ of $(1-\alpha)\cdot n$ ”good” points are generated from a $d$-dimensional spherical Gaussian ${{\mathbf{x}}}^{i}\sim{\mathcal{N}}(\text{\boldmath$\mathbf{\mu}$},\Sigma)$ i.e. ${{\mathbf{x}}}^{i}=\text{\boldmath$\mathbf{\mu}$}+\text{\boldmath$\mathbf{\epsilon}$}^{i}$ where $\text{\boldmath$\mathbf{\epsilon}$}^{i}\sim{\mathcal{N}}({\mathbf{0}},\Sigma)$ and $\Sigma=\frac{1}{\beta^{\ast}}\cdot I$ for some $\beta^{\ast}>0$. The rest are the set $B$ of $\alpha\cdot n$ ”bad” points that are corrupted by an adversary i.e. $\tilde{{\mathbf{x}}}^{i}=\text{\boldmath$\mathbf{\mu}$}^{\ast}+{{\mathbf{b}}}^{i}$ where ${{\mathbf{b}}}^{i}\in{\mathbb{R}}^{d}$ can be unbounded. SVAM-ME (Algorithm 3) adapts SVAM to the robust mean estimation problem. For notational clarity we use, $\text{\boldmath$\mathbf{\eta}$}=\text{\boldmath$\mathbf{\mu}$}$, in this problem.

Model recovery, Consistency, Breakdown pt. Note that for any constant $\alpha>0$, the estimation error does not go to zero as $n\rightarrow\infty$. As mentioned in §2, an error of $\Omega\left({{\alpha}}\right)$ is unavoidable no matter how large $n$ gets. Thus, the best hope we have is of the estimation error going to zero as $\alpha\rightarrow 0$ and $n\rightarrow\infty$. The error in Theorem 5 does indeed go to zero in this setting. Also, note that the error depends only on the trace of the covariance matrix of the clean points, and thus for $\text{trace}(\Sigma)={\cal O}\left({{1}}\right)$, the result offers an estimation error independent of dimension. SVAM-ME offers a large breakdown point (allowing upto 26% corruption rate).

Establishing LWSC/LWLC for Gamma Regression and Mean Estimation: In the appendices, Lemmata 28, 29, Lemmata 23, 24 establish the LWSC/LWLC properties for the $\tilde{Q}_{\beta}$ function for gamma regression and mean estimation and Theorems 27 and Theorem 22 establish the breakdown points and existence of increments $\xi>1$.

Robust Classification. In this case the labels are generated as $y_{i}=\text{sign}(\left\langle{{{\mathbf{w}}}^{\ast}},{{{\mathbf{x}}}^{i}}\right\rangle)$ and the bad points in the set $B$ get their labels flipped $\tilde{y}_{i}=-\text{sign}(\left\langle{{{\mathbf{w}}}^{\ast}},{{{\mathbf{x}}}^{i}}\right\rangle)$. SVAM-LR (Algorithm 5) adapts SVAM to robust logistic regression.

We used a 64-bit machine with Intel® Core™ i7-6500U CPU @ 2.50GHz, 4 cores, 16 GB RAM, Ubuntu 16.04 OS.

Benchmarks. SVAM was benchmarked against several baselines (a) VAM: this is SVAM executed with a fixed value of the scale $\beta$ by setting the scale increment to $\xi=1$ to investigate any benefits of varying the scale $\beta$, (b) MLE: likelihood maximization on all points (clean + corrupted) without any weights assigned to data points – this checks for benefits of performing weighted MLE, (c) Oracle: an execution of the MLE only on the clean points – this is the gold standard in robust learning and offers the best possible outcome. In addition, several problem-specific competitors were also considered. For robust regression, STIR [15], TORRENT [1], SEVER [7], RGD [20], and the classical robust M-estimator of Tukey’s bisquare loss were included. Note that TORRENT already outperforms $L_{1}$ regularization methods while achieving better or competitive recovery errors (see [1, Fig 2(b)]). Since SVAM-RR was faster than TORRENT itself, $L_{1}$ regularized methods such as [18, 24] were not considered. For robust mean estimation, popular baselines such as coordinate-wise median and geometric median were taken. For robust classification, the rank-pruning method RP-LR [19] and the method from [16] were used.

Experimental Setting and Reproducibility. Due to lack of space, details of experimental setup, data generation, how adversaries were simulated etc are presented in Appendix C. SVAM also offered superior robustness than competitors against a wide range of ways to simulate adversarial corruption (see Appendix D for details). Code for SVAM is available at https://github.com/purushottamkar/svam/.

The authors thank the anonymous reviewers of this paper for suggesting illustrative experiments and pointing to relevant literature. B.M. is supported by the Technology Innovation Institute and MBZUAI joint project (NO. TII/ARRC/2073/2021): Energy-based Probing for Spiking Neural Networks. D.D. is supported by the Research-I Foundation at IIT Kanpur and acknowledges support from the Visvesvaraya PhD Scheme for Electronics & IT (FELLOW/2016-17/MLA/194). P.K. thanks Microsoft Research India and Tower Research for research grants.