Constrained regret minimization for multi-criterion multi-armed bandits

The multi-armed bandit (MAB) problem is a fundamental construct in online learning, where a learner has to quickly identify the best option (a.k.a., arm) among a given set of options. In the stochastic MAB problem, each arm is associated with an (a priori unknown) reward distribution, and a sample from this distribution is revealed each time an arm is chosen (a.k.a., pulled). The classical goal is to use these samples to quickly identify the arm with the highest mean reward. The most popular metric to evaluate the performance of a learning algorithm is regret, which captures how often a suboptimal arm was pulled by the algorithm.

While the classical stochastic MAB formulation has been applied in various application scenarios, including clinical trials, portfolio optimization, anomaly detection, and telecommunication Bouneffouf and Rish [2019], it ignores a key aspect of most real-world decision-making problems—namely, that they have multiple criteria of interest. For example, when comparing testing kits in a clinical trial, one would want to keep track of the false-positive rate as well as the false-negative rate of each kit. Similarly, choosing the best financial portfolio involves balancing risk and reward. A wireless node deciding which channel to transmit on, or what transmission rate to use has to balance several criteria, including throughput, delay, and energy consumption. This multi-criterion nature of decision making is not always adequately captured by the classical MAB approach of optimizing a one-dimensional reward signal.

The most common approach for incorporating multiple arm attributes into MAB formulations is to define the reward as a suitable function (say a linear combination) of the attributes of interest. For example, risk-aware portfolio optimization can be cast as an MAB problem where the best arm is one that optimizes a certain linear combination of mean value and a suitable risk measure (such as standard deviation or Conditional Value at Risk) (see, for example, Sani et al. [2012], Vakili and Zhao [2016], Kagrecha et al. [2019]. However, this approach assumes that the different attributes of interest can be expressed and compared on a common scale, which is not always reasonable. For example, how does one ‘equate’ the impact a certain increment in risk to that of an another increment in the mean return of a portfolio?

A more natural approach for multi-criterion decision making is to instead pose the optimal choice as the solution of a constrained optimization. In other words, optimize one attribute subject to constraints on the others. However, despite the modest literature on multi-criterion MABs (surveyed below), little attention has been paid to formulating, and designing algorithms for constrained multi-criterion MABs. This paper seeks to fill this gap. Formally, we assume that each arm is associated with a $D$-dimensional probability distribution, and the best arm is the one that minimizes a certain arm attribute subject to constraints on $m$ other attributes. For this setting, we pursue regret minimization, i.e., we seek to minimize the average number of pulls of non-optimal arms.

To simplify the presentation, and to highlight the key aspects of this problem, we first consider a single constraint (i.e., $m=1$). Each arm is associated with a probability distribution, and we consider the problem of optimizing an objective attribute, subject to a single constraint attribute satisfying a user-specified constraint. The algorithm design and performance evaluation for this special case (with $m=1$) generalize readily to the multi-criterion problem; see Section 5. An example that fits this template that is of significant interest in the finance community: the optimization of the expected return, subject to a risk constraint. Here, the objective attribute is the mean of an arm distribution, while the constraint attribute measuring risk could be Conditional Value at Risk (CVaR).

For this problem, we propose an algorithm, called Constrained Lower Confidence Bound (Con-LCB), that guarantees logarithmic regret, i.e., the average number of plays of all non-optimal arms (including those that violate the constraint) is at most logarithmic in the horizon. If Con-LCB is presented with an infeasbile instance, i.e., an instance where all arms violate the specified risk constraint, the algorithm in effect relaxes this constraint just enough to make at least one arm compliant. Another feature of Con-LCB is that at the end of the horizon, it outputs a boolean flag that correctly identifies with high probability, whether or not the given instance was feasible.

Finally, we establish fundamental lower bounds on the performance of any algorithm on this constrained regret minimization problem. Our results demonstrate a fundamental tradeoff between regret minimization and feasibility identification, similar to the well-known tradeoff between regret minimization and best arm identification in the classical (unconstrained) MAB problem Bubeck et al. [2009].

The remainder of this paper is organized as follows. A brief survey of related literature is provided below. In Section 2, we introduce some preliminaries and formulate the constrained mean minimization problem. We present our algorithm (Con-LCB) and its performance guarantees in Section 3. Information-theoretic lower bounds on performance are discussed in Section 4. Finally, the general formulation for multi-criterion MABs is introduced in Section 5. All proofs are deferred to an appendix, which is part of the ‘supplementary materials’ document uploaded separately.

For multi-objective MAB problems with vector rewards, different notions of optimality are considered. For example, Drugan and Nowe [2013], Yahyaa and Manderick [2015] consider the notion of Pareto optimality. In these papers, all dimensions are considered equally important and the aim is to play all Pareto-optimal arms an equal number of times. Another important notion of optimality is lexicographic optimality (see Ehrgott [2005]). Here there is an order of importance among different dimensions. Tekin and Turğay [2018], Tekin [2019] consider the notion of lexicographic optimality for contextual bandit problems. In this line of work, the goal is to obtain higher reward in an important dimension and for tie-breaking, use rewards obtained in dimensions of lower importance.

Turning now to the literature on risk-aware arm selection, Sani et al. [2012], Galichet et al. [2013], Vakili and Zhao [2016], David and Shimkin [2016], Bhat and Prashanth [2019], Prashanth et al. [2020], Kagrecha et al. [2019] consider the problem of optimizing a risk metric alone or consider a linear combination of mean and a risk metric. Zimin et al. [2014] looks at the learnability of general functions of mean and variance, and Maillard [2013] proposes an optimization of the logarithm of moment generating function as a risk measure in a regret minimization framework. Cassel et al. [2018] look at path dependent regret and provides a general approach to study many risk metrics.

None of the above papers considers the constrained MAB problem, which frames the optimal arm as the solution of a constrained optimization problem. There has been some recent work for the constrained MAB problem. A constrained linear bandit setting is considered in Pacchiano et al. [2021] under the assumption that there is at least one arm which satisfies the constraints. The papers Amani et al. [2019], Moradipari et al. [2019] consider the problem of maximizing the reward subject to satisfying a linear constraint with high probability. Constrained setting was also considered in David et al. [2018] and Chang [2020]. Both these papers consider a single constraint on arm selection; David et al. [2018] considers a constraint on VaR, and Chang [2020] considers an average cost constraint (each arm has a cost distribution that is independent of its reward distribution). All the papers above implicitly assume that the instance presented is feasible, whereas we address the issue of encountering an infeasible instance.

In this section, we describe the formulation of our constrained stochastic MAB problem. To keep the exposition simple, we consider a single constraint (i.e., $m=1$) for now; the general formulation with multiple constraints (i.e., $m\geq 1$) is discussed in Section 5. Informally, under the regret minimization objective considered here, the goal is to play, as often as possible, the arm that optimizes the objective, subject to a constraint on another attribute.

Formally, consider a multi-armed bandit problem with $K$ arms, labeled $1,2,\cdots,K.$ Each arm is associated with a (possibly multi-dimensional) probability distribution, with $\nu(k)$ denoting the (joint) distribution corresponding to arm $k\in[K]$¹¹1For a positive integer $n,$ we denote $[n]:=\{1,2,\cdots,n\}.$. Suppose that $\nu(k)\in\mathcal{C},$ the space of possible arm distributions. The objective and the constraint attributes are defined via functions $g_{0}$ and $g_{1}$ respectively, mapping $\mathcal{C}$ to $\mathbb{R}.$ Additionally, the user provides a threshold $\tau\in\mathbb{R}$ which specifies an upper bound on the attribute $g_{1}.$ An instance of the constrainted MAB problem is defined by $(\nu,\tau),$ where $\nu=(\nu(k),\ 1\leq k\leq K).$ The arms which satisfy the constraint $g_{1}(\nu(\cdot))\leq\tau$ are called feasible arms; the set of feasible arms is denoted by $\mathcal{K}(\nu).$ The instance $(\nu,\tau)$ is said to be feasible if $\mathcal{K}(\nu)\neq\emptyset,$ and is said to be infeasible if $\mathcal{K}(\nu)=\emptyset.$

Consider first a feasible instance. An optimal arm in this case is defined as an arm that minimizes $g_{0}(\nu(\cdot)),$ subject to the constraint $g_{1}(\nu(\cdot))\leq\tau$. Denote the optimal value as $g_{0}^{*}=\min_{k\in\mathcal{K}(\nu)}g_{0}(\nu(k)).$ Arms which have $g_{0}(\nu(\cdot))$ larger than $g_{0}^{*}$ (whether or not feasible) are referred to as suboptimal arms. Note that there can also exist infeasible arms with a smaller objective $g_{0}$ than $g_{0}^{*}.$ We refer to such arms as deceiver arms; the set of deceiver arms is denoted by $\mathcal{K}^{d}(\nu),$ where $\mathcal{K}^{d}(\nu)=\{k\in[K]:g_{1}(\nu(k))>\tau,~{}g_{0}(\nu(k))\leq g_{0}^{*}\}.$ For a suboptimal arm $k,$ the suboptimality gap is defined by $\Delta(k):=g_{0}(k)-g_{0}^{*}>0.$ (Note that the suboptimality gap is also defined for infeasible, non-deceiver arms). Finally, for an infeasible arm $k,$ the infeasibility gap is defined as $\Delta_{\tau}(k)=g_{1}(k)-\tau>0.$ Figure 1(a) provides a visual representation of a feasbile instance.

Next, consider an infeasible instance. The optimal arm in this case is defined in as the one with the smallest value of $g_{1}(\nu(\cdot)).$ Let the optimal value be denoted by $g_{1}^{*}=\min_{k\in[K]}g_{1}(\nu(k))>\tau.$ This is equivalent to requiring that if the algorithm is faced with an infeasible instance, it must ’relax’ the constraint just enough, until at least one arm satisfies the constraint. The constraint gap for an arm $k$ that is not optimal is defined as $\Delta_{\text{con}}(k)=g_{1}(\nu(k))-g_{1}^{*}>0.$ Figure 1(b) provides a visual representation of an infeasbile instance.

For any (feasible or infeasible) instance $(\nu,\tau),$ let the set of optimal arms be denoted by $\mathcal{K}^{*}(\nu).$ The total number of pulls (or horizon) is denoted by $T.$ For an algorithm (a.k.a., policy) $\pi$, the number of pulls of an arm $k$ over the first $t$ pulls, for $t\in[T],$ is denoted by $N^{\pi}_{k}(t),$ though we often suppress the dependence on $\pi$ for simplicity.

In this section, we present an algorithm for constrained regret minimization, and provide performance guarantees for the same, assuming that we have estimators for each attribute that satisfy the concentration inequality (1).

The algorithm, which we refer to as constrained lower confidence bound (Con-LCB) algorithm, is based on the well-known principle of optimism under uncertainty. Con-LCB uses lower confidence bounds (LCBs) on attribute $g_{1}$ of each arm to maintain a set of plausibly feasible arms, and uses LCBs for attribute $g_{0}$ of the arms in this set to select the arm to be played. Note that LCBs are used for attribute $g_{0}$ because we are dealing with minimization of $g_{0}.$ If, at some instant, the set of plausibly feasible arms maintained by Con-LCB becomes empty, the algorithm turns conservative and plays the arm which violates the constraint least, i.e., the one with the smallest LCB on $g_{1}$. Finally, at the end of $T$ rounds, Con-LCB sets the feasiblity flag as true if the set of plausibly feasible arms is found to be non-empty, and false otherwise.

The remainder of this section is devoted to performance guatantees for Con-LCB. We consider feasible and infeasible instances separately.

The main takeaways from Theorem 3.1 are as follows.

$\bullet$ The upper bound on the expected number of pulls for feasible, suboptimal arms is logarithmic in the horizon $T,$ and inversely proportional to the square of the suboptimality gap. This is similar to bounds known for classical (unconstrained) MABs.

$\bullet$ For deceiver arms, the upper bound on the expected number of pulls, also logarithmic in $T,$ is inversely proportional to the square of the feasiblity gap. This is similar to the bound one would obtain in a pure $g_{1}$ minimization problem for a hypothetical instance consisting of all the originally infeasible arms, and a single hypothetical (optimal) arm having $g_{1}$ equal to $\tau.$

$\bullet$ The bound on the expected number of pulls of non-deceiver, infeasible arms involves a minimum of the dominant terms in the above two cases. Intuitively, this is because these arms can disambiguated in two ways: via the suboptimality gap, and the feasibility gap.

$\bullet$ The probability that the feasiblity flag incorrectly identifies the instance as infeasible is upper bounded by a power law in the horizon $T.$ Note that the specific form of the probability of mis-identification bound is not fundamental; a small modification in the algorithm would make this bound a faster decaying power law at the expense of a multiplicative bump in regret. However, that this probability is not much smaller (for example, exponentially decaying in the horizon $T$) is a consequence of an inherent tension between regret minimization and feasiblity identification. A similar tension is known to exist between regret minimization and best arm identification in the unconstrained setting; see Bubeck et al. [2009]). We provide lower bounds on the probability that any consistent algorithm makes a mistake in feasibility identification in Section 4.

Finally, we note that the suboptimality regret, as well as the infeasibility regret are logarithmic in the horizon $T.$ Indeed, the suboptimality regret for a feasible instance is bounded as

which is similar to regret bounds in the classical unconstrained setting. To express the infeasibility regret compactly, let us interpret $\Delta(k)$ as 0 (and $1/\Delta(k)$ as $\infty$) for deceiver arms. With this notation, the infeasibility regret of a feasible instance is bounded as

Next, we move on to characterizing the performance of Con-LCB over infeasible instances.

For an infeasible instance, the upper bound on the expected number of pulls of a non-optimal arm, logarithmic in the horizon, and inversely proportional to the the square of the constraint gaps $\Delta_{\rm{con}}(k),$ is structurally similar to the bound one would obtain in pure $g_{1}$ minimization problem on the same instance. However, note that when faced with an infeasible instance, Con-LCB would only start playing the optimal arm regularly after all $K$ arms appear to be infeasible; this explains the appearance of $K$ in our bounds.

Here, the constraint regret is bounded as

Finally, as before, the probability that the feasibility flag wrongly identifies the infeasible instance as feasible decays as a power law in the horizon for $T>T^{*}(\nu);$ the threshold $T^{*}$ accounts for the time it takes for the algorithm to ‘detect’ that the instance is infeasibile with high probability.

In this section, we establish fundamental limits on the performance of algorithms for constrained regret minimization. First, we show that the regret bounds obtained for Con-LCB are asymptotically tight upto universal multiplicative constants (on a class of Gaussian bandit instances). We then prove a lower bound on the probability that any consistent algorithm misidentifies a feasible instance as infeasible or vice-versa. This result illustrates an inherent tension between regret minimization and feasibility identification—consistent algorithms (recall that consistency means regret is $o(T^{a})$ for all $a>0$) cannot have a misidentification probability that decays exponentially in the horizon, and algorithms that enjoy a mid-identification probability that decays exponentially in the horizon cannot be consistent.

To state our information theoretic lower bound on regret, suppose that the class of arm distributions $\mathcal{G}$ is the class of 2-dimensional Gaussian distributions with covariance matrix $\Sigma=\mathrm{diag}(\nicefrac{{1}}{{2a_{0}}},\nicefrac{{1}}{{2a_{1}}}).$ Let attribute $g_{0}$ be the mean of the first dimension, and $g_{1}$ be the mean of the second dimension. For the assumed covariance matrix structure, the standard empirical mean estimators for $g_{0}$ and $g_{1}$ satisfy the concentration properties stated in (1).

The proof of Theorem 4.1 can be found in Appendix B. Comparing the lower bounds in Theorem 4.1 with the upper bounds for Con-LCB in Theorems 3.1 and 3.2, we conclude that Con-LCB is asymptotically optimal on regret, up to universal multiplicative constants.²²2Information theoretic lower bounds on the expected number of pulls of non-optimal arms can be derived (in terms of a certain optimization over KL divergences) for any general class of arm distributions; see Appendix B.1. We have specialized the statement of Theorem 4.1 to a class of Gaussian bandit instances to enable an easy comparison with our upper bounds for Con-LCB.

Next, we address the fundamental tradeoff between regret minimization and feasibility identification.

Theorem 4.2 states that for any consistent algorithm, the probability that $(\nu,\tau)$ get misidentified as infeasible, and the probability that $(\nu^{\prime},\tau)$ get misidentified as feasible, cannot both decay exponentially in the horizon. This is of course consistent with the power-law probability of misidentification under Con-LCB. In other words, slower-than-exponential decay of the probability of feasibility misidentification with respect to the horizon is an unavoidable consequence of the exploration-exploitation interplay in regret minimization. A similar tension between regret minimization and best arm identification was noted for the unconstrained MABs by Bubeck et al. [2009].

In this section, we provide a general formulation for constrained stochastic MABs with multiple criteria. We allow each arm to be associated with a $D$-dimensional probability distibution, the goal being to optimize one (dominant) attribute associated with this distribution, subject to constraints on $m$ others. The algorithm design and performance evaluation performed in Sections 2–4 for the special case of $m=1$ extend naturally to this general formulation, which can in turn be applied in a variety of application scenarios. We illustrate a few here.

$\bullet$ For clinical trials, with the arms corresponding to various treatment protocols, the dominant attribute might, for example, correspond to the success/recovery probability, whereas the constraints might capture recovery time, severity of side effects, etc.

$\bullet$ For product/service rating, where the arms correspond to various service providers, the dominant attribute might correspond to product quality, with constraints capturing reliability, pricing, customer service, etc.

$\bullet$ In wireless networks, the arms might correspond to various access networks or channels, with, for example, the dominant attribute corresponding to throughput, and constraints capturing delay, energy efficiency, etc.

If the instance is infeasible, the optimal arm is defined via a ranked list of the constraints, that orders them by ‘importance’. Without loss of generality assume that the order of importance increases from $(g_{1},\tau_{1})$ to $(g_{m},\tau_{m}).$ The idea is to relax the constraints one-by-one, starting with the least important, until a compliant arm is found. Formally, for a given infeasible instance $\nu$, for $2\leq i\leq m,$ let $\mathcal{K}_{i}(\nu)$ denote the set of arms that satisfy the constraints $\{g_{j}(\nu(k))\leq\tau_{j}\}_{j=i}^{m}.$ Let us also define $\mathcal{K}_{m+1}(\nu):=[K].$ Now, let $i^{*}(\nu)=\min\{k\in[m]:\ \mathcal{K}_{k+1}(\nu)\neq\phi\}.$ Here, $i^{*}(\nu)$ is the fewest number of constraints one must relax, in order of increasing importance, in order to have at least one compliant arm. An optimal arm is then defined to be $\operatorname*{arg\,min}\{g_{i^{*}(\nu)}(\nu(k)):\ k\in\mathcal{K}_{i^{*}(\nu)+1}(\nu)\}.$

Similar to the case where $m=1,$ we make the following assumption to simplify algorithm design. Suppose that for $0\leq i\leq m,$ and $\overline{\nu}\in\mathcal{C},$ there exist an estimator $\hat{g}_{i,n}(\overline{\nu})$ for $g_{i}(\overline{\nu})$ using $n$ i.i.d. samples from $\overline{\nu},$ satisfying the following concentration inequality: There exists $a_{i}>0$ such that for all $\Delta>0,$

Note that this is simply an extension of our assumption (1).

The remainder of this section is devoted to performance guarantees for Multi-Con-LCB. The suboptimality gap for an arm $k$ is given by $\Delta(k)=\max(g_{0}(\nu_{k})-g_{0}^{*},0).$ The infeasibility gap of an arm $k$ for constraint $i$ is given by $\Delta_{i,\tau_{i}}(k)=\max(g_{i}(\nu(k))-\tau_{i},0).$ We restrict our attention to feasible instances here; infeasible instances can be handled on similar lines as Theorem 3.2 for Con-LCB.

We omit the proof of Theorem 5.1 in the appendix because it is very similar to the proof of Theorem 3.1. However, we state the key takeaways from Theorem 5.1 below.

$\bullet$ Similar to Theorem 3.1, the upper bound on the expected number of pulls of suboptimal arms is logarithmic in $T$ and is inversely proportional to the suboptimality gap squared. Moreover, the upper bound for a deceiver arm is also logarithmic in $T$ but there is a minimum over $m$ terms because the deceiver arm can be identified by any of the constraints that the arm does not satisfy. Similarly, the upper bound for a non-deceiver, suboptimal arm is also logarithmic in $T$ and there is a minimum over $m+1$ terms because the arm can be identified as suboptimal, or infeasible for not satisfying one of the $m$ constraints.

$\bullet$ With an increase in the number of constraints to $m$, the upper bounds on the expected number of pulls of non-optimal arms increase linearly in $m$ and the probability of mis-identification also gets scaled by a factor of $m.$ The slight degradation in the guarantees is expected because with more constraints, the optimal arm has to satisfy more conditions, and it becomes harder to compare different arms.

In this section, we present a simple experiment to show the numerical performance of our algorithm Con-LCB.

We consider an instance with four arms and two criteria. Each arm is associated with a (two-dimensional) multivariate Gaussian distribution, different arms having the same covariance matrix $\Sigma,$ but different mean vectors. The means corresponding to the first dimension are 0.3, 0.4, 0.2, and 0.5 and the means corresponding to the second dimension are 0.4, 0.4, 1.0, and 1.0. The covariance matrix is given by $\Sigma=\begin{pmatrix}1&0.5\\ 0.5&1\end{pmatrix}.$

The goal is to maximize the pulls of the arm with the minimum mean of the first dimension, subject to a constraint on the mean of the second. Specifically, we require that the mean of the second dimension should be less than or equal to $\tau:=0.5.$ The instance is summarized in Table 1. We use empirical averages as the estimators for the means and standard confidence bounds based on sub-Gaussianity assumption. We run the algorithm 1000 times for values of $T$ in $[10000,100000].$ The suboptimality regret $R_{T}^{\text{sub}}$ and the infeasibility regret $R_{T}^{\text{inf}}$ are plotted in Figure 2. Clearly, the regrets grow sub-linearly with respect to the horizon. Also, in our experiments, the algorithm correctly detected the feasibility of the instance in all of the 1000 runs.

We consider another instance where arms are Beta distributed. The parameters are chosen in such a way that the mean of the arms are 0.3, 0.4, 0.2, and 0.5 and the variance of the arms are 0.08, 0.06, 0.15, 0.15. The goal here is to maximize the pulls of the arm with the minimum mean, subject to a constraint on the variance. Specifically, we require that the variance of the arms should be less than or equal to $\tau:=0.1.$ The instance is summarized in Table 2. We use the empirical average to estimate the mean of the arms and the standard unbiased estimator of variance to estimate the variance of the arms. The concentration bounds we use are based on fact that underlying arms are bounded between 0 and 1. We run the algorithm 100 times for values of $T$ in $[10000,100000].$ The suboptimality regret $R_{T}^{\text{sub}}$ and the infeasibility regret $R_{T}^{\text{inf}}$ are plotted in Figure 3. Similar to the previous case, the algorithm correctly detected feasibility in all of the 100 runs.

We also compare Con-LCB with an algorithm designed to optimize a single ’Lagrange relaxed’ objective of the form $g_{0}+\beta g_{1}.$ Since there is no systematic way of setting $\beta$ such that the original (constrained) optimal arm also minimizes this metric, this approach is very fragile, as we demonstrate in Figure 4. The instance we use is the Beta distributed instance in Table 2, and we are plotting the sum of the infeasible regret and suboptimal regret. When $\beta$ is small ($\beta<10/7$ here), a deceiver arm is optimal for the relaxed objective, and when $\beta$ is large ($\beta>5$ here), a suboptimal arm is optimal for the relaxed objective; the ‘correct’ range of $\beta$ being instance dependent (and a priori unknown).

In this paper, we have introduced a general formulation of multi-criterion constrained MABs, which is applicable when there are multiple attributes of interest associated with each arm. We propose algorithms that incur logarithmic regret, and also provide information theoretic lower bounds on the performance of any algorithm. An interesting departure from the classical MAB formulation is the aspect of instance feasibility; our algorithms predict, post facto, with a probability of error that decays as a power law in the horizon, whether the instance presented was feasible. Interestingly, this illustrates a fundamental tradeoff between regret minimization and feasibility detection. Finally, our algorithms ‘auto-tune’ to an infeasible instance, relaxing the constraints on arm attributes (in order of increasing importance), until a compliant arm is found.

The proposed framework and our algorithms can be applied in a wide range of application scenarios (see Bouneffouf and Rish [2019] for a survey). Further, this work motivates the study of multi-criterion variants of other bandit formulations, including contextual bandits, combinatorial bandits, and correlated bandits.

• •

  Funding: the authors did not receive support from any organization for the submitted work.

• •

  Conflicts of interest: the authors frequently collaborate with researchers from IIT Bombay (@iitb.ac.in), researchers from IIT Madras (@iitm.ac.in), and researchers from Stanford University (@stanford.edu). Jayakrishnan Nair received his PhD from Caltech (@caltech.edu) in 2012 and Krishna Jagannathan received his PhD from MIT (@mit.edu) in 2010.

• •

  Ethics approval: not applicable

• •

  Consent to participate: not applicable

• •

  Consent for publication: not applicable

• •

  Availability of data and material: not applicable

• •

  Code availability: the Python code for the numerical experiments can be found in this Github repository.

• •

  Author’s contributions: all authors contributed equally to the paper