Data-Driven Predictive Control With Adaptive Disturbance Attenuation For Constrained Systems

In addition to stability, disturbance rejection and constraint satisfaction are important topics in control systems engineering, especially as modern systems often operate in complex and dynamic environments while increasingly stringent safety and performance requirements are imposed on them. The $\mathcal{H}_{\infty}$ control is an effective approach to addressing the former – it aims to minimize the effect of disturbances on system outputs [1]. Since introduced in the early 1980s, $\mathcal{H}_{\infty}$ control has been successfully implemented in aerospace, automotive, and many other sectors. However, time-domain constraints on state and control variables are not handled in conventional $\mathcal{H}_{\infty}$ control. Meanwhile, Model Predictive Control (MPC) stands out from many other control approaches for its ability to explicitly and non-conservatively handle time-domain constraints [2]. Therefore, combing $\mathcal{H}_{\infty}$ control and MPC to achieve desired disturbance rejection properties while satisfying constraints is appealing and has been studied in, e.g., [3, 4, 5, 6, 7, 8, 9, 10, 11, 12]. A major strength of such a combination is the ability to dynamically adapt $\mathcal{H}_{\infty}$ disturbance attenuation performance (depending on measured/estimated system state and forecasted disturbance level) to satisfy constraints through solving an MPC optimization problem repeatedly in an online manner [3, 4].

Conventional $\mathcal{H}_{\infty}$ control and MPC methods rely on parametric models of the system to be controlled. As engineered systems are becoming increasingly complex and cyber-physical, first-principle models are more difficult to obtain. Meanwhile, with the rapid advances in sensing, computation, and communication technologies, data is more readily available. This has spurred the development of data-driven methods for system modeling, analysis, and control. In particular, practitioners may favor an end-to-end solution that bypasses the intermediate steps of modeling and analysis and produces a controller with desired properties directly from measured data of system behavior. Therefore, it is beneficial to extend model-based methods for $\mathcal{H}_{\infty}$ control, MPC, and the aforementioned combination to their data-driven counterparts.

Here we provide a brief review of existing data-driven control methods in the literature: Reinforcement Learning (RL) can be used to train optimal controllers from data [13, 14]. However, the majority of RL methods optimize the average behavior of the closed-loop system when it is subject to disturbances with certain statistics and do not provide a worst-case robustness guarantee. Furthermore, they typically handle constraints through penalties, hence making constraints soft. Integrating data-driven uncertainty estimation/system identification with(in) the MPC framework can lead to reliable usage of data for improving MPC performance while maintaining certain robustness guarantees especially for satisfying constraints [15, 16]. Along these lines, the Data-enabled Predictive Control (DeePC) is an emerging technique and has attracted increasing attention from researchers recently. The DeePC uses measured input-output data to create a non-parametric system model based on behavioral systems theory and uses the model to predict future trajectories [17]. It has demonstrated superior performance in various applications [18, 19, 20, 21]. However, DeePC entails higher computational cost than conventional MPC due to its high-dimensional, data-based non-parametric system model [22, 23]. Also, how to handle noisy data and provide certain robustness guarantees under noisy data remains to be an open question in DeePC [24, 25, 26]. Furthermore, to the best of our knowledge, there has been no previous work integrating $\mathcal{H}_{\infty}$ control and MPC (or, designing an MPC that has an $\mathcal{H}_{\infty}$-type disturbance attenuation property) in the general data-driven MPC literature. An approach to synthesizing an $\mathcal{H}_{\infty}$ controller using noisy data based on a matrix S-lemma was introduced in [27]. The approach reduces a data-driven $\mathcal{H}_{\infty}$ control synthesis problem to a low-dimensional Linear Matrix Inequality (LMI) optimization problem, which is computationally tractable with state-of-the-art interior-point LMI solvers. However, time-domain constraints are not handled by the approach of [27], and moving-horizon, MPC-type implementation of the approach was not considered in [27]. More classical data-driven control methods also include self-tuning regulators [28] and iterative learning control [29]. A comprehensive survey of data-driven control methods can be found in [30].

In this paper, we fill the gap in the literature by proposing a novel data-driven control approach that combines the strengths of $\mathcal{H}_{\infty}$ control for rejecting disturbances and MPC for handling constraints. Our approach can be viewed as a data-driven counterpart of the model-based moving-horizon $\mathcal{H}_{\infty}$ control approach of [3, 4] and enjoys similar properties including dynamic adaptation of $\mathcal{H}_{\infty}$ performance depending on measured system state and forecasted disturbance level for satisfying constraints. Specifically, the contributions include:

1. 1.

   Our approach is the first data-driven MPC method in the literature that focuses on $\mathcal{H}_{\infty}$-type disturbance attenuation for systems with time-domain constraints.

2. 2.

   We conduct a comprehensive analysis of the theoretical properties of our approach. The results include robust guarantees of closed-loop stability, disturbance attenuation, and constraint satisfaction under noisy data, as well as conditions for online problem recursive feasibility.

The paper is organized as follows: We describe the problem treated in this paper including key assumptions and preliminaries in Section 2. We develop an approach to synthesizing an $\mathcal{H}_{\infty}$ controller that also enforces time-domain constraints for an unknown system using noisy trajectory data in Section 3. The development in this section has merit in its own right because it extends the data-driven $\mathcal{H}_{\infty}$ control synthesis approach of [27] (which does not handle constraints) to the constrained case. It is also an essential building block of the data-driven MPC algorithm developed in Section 4. Then, Section 4 presents our proposed data-driven MPC algorithm and analyzes its theoretical properties. We illustrate the algorithm with a numerical example in Section 5. Finally, we conclude the paper in Section 6.

The notations used in this paper are mostly standard. We use $\mathbb{R}^{n}$ to denote the space of $n$-dimensional real vectors, $\mathbb{R}^{n\times m}$ the space of $n$-by-$m$ real matrices, and $\mathbb{N}$ the set of natural numbers including zero. Given a vector $x\in\mathbb{R}^{n}$, we use $\|x\|$ to denote its Euclidean norm, i.e., $\|x\|=\sqrt{x^{\top}x}$. Given a matrix $M\in\mathbb{R}^{n\times m}$, its kernel is the subspace of all $x\in\mathbb{R}^{m}$ such that $Mx=0$, i.e., $\text{ker}(M)=\{x\in\mathbb{R}^{m}:Mx=0\}$. Given two symmetric matrices $M,N\in\mathbb{R}^{n\times n}$, $M\succ N$ means that $M-N$ is positive definite, i.e., $x^{\top}(M-N)x>0$ for all non-zero $x\in\mathbb{R}^{n}$, and $M\succeq N$ means that $M-N$ is positive semidefinite, i.e., $x^{\top}(M-N)x\geq 0$ for all $x\in\mathbb{R}^{n}$. Similarly, $M\prec N$ and $M\preceq N$ means that $M-N$ is negative definite and negative semidefinite, respectively. For an optimization problem, by “solution” we mean a feasible solution, i.e., a set of values for the decision variables that satisfies all constraints, and by “optimal solution” we mean a feasible solution where the objective function (almost) reaches its maximum (or minimum) value. Because the optimization problems appearing in this paper are all convex problems, an “optimal solution” is globally optimal.

We consider the control of dynamic systems that can be represented by the following linear time-invariant model:

where $x(t)\in\mathbb{R}^{n}$ denotes the system state at the discrete time $t\in\mathbb{N}$, $u(t)\in\mathbb{R}^{m}$ denotes the control input, $w(t)\in\mathbb{R}^{n}$ denotes an unmeasured disturbance input, $y_{1}(t)\in\mathbb{R}^{p_{1}}$ and $y_{2}(t)\in\mathbb{R}^{p_{2}}$ are two outputs the roles of which are introduced below. The goal is to design a control algorithm that achieves the following three objectives:

1. 1.

   Guaranteeing closed-loop stability;

2. 2.

   Optimizing disturbance attenuation in terms of the $\mathcal{H}_{\infty}$ performance from $w$ to output $y_{1}$;

3. 3.

   Enforcing the following constraints on output $y_{2}$ at all times $t\in\mathbb{N}$:

   $$y_{2v}(t)\leq y_{2v,\max},\quad v=1,2,\dots,p_{2}$$

   (2)

   where $y_{2v}(t)$ denotes the $v$th entry of $y_{2}(t)$ and $y_{2v,\max}\geq 0$ for all $v=1,2,\dots,p_{2}$. Constraints in this form can represent state/output variable bounds, control input limits, etc.

In addition to the standard assumptions of $(A_{\text{o}},B_{\text{o}})$ being stabilizable and $(C_{1},A_{\text{o}})$ being detectable, we also assume that the system model $(A_{\text{o}},B_{\text{o}})$ is unknown and only trajectory data are available. This calls for a data-driven control approach.

Assume we have data $\{(x^{+}_{j},x_{j},u_{j})\}_{j=1}^{J}$, where $(x_{j},u_{j})$ denotes a pair of previous state and control input values, $x^{+}_{j}$ denotes the corresponding next state value, and the subscript $j$ indicates the $j$th data point. The data can be collected from a single or multiple trajectories. According to (1a), we have

where $w_{j}$ is the disturbance input value at the time of collecting the data point $(x^{+}_{j},x_{j},u_{j})$. Organizing data with the following matrices:

the relation (3) implies

Because the disturbance input $w_{j}$ is not measured, $W$ in (4d) and (5) is unknown. We make the following assumption about the data:

Assumption 1. The disturbance input values $w_{1},w_{2},$ $\dots,w_{J}$, collected in $W$, satisfy the following quadratic matrix inequality:

where $\Phi_{11}=\Phi_{11}^{\top}\in\mathbb{R}^{n\times n}$, $\Phi_{12}\in\mathbb{R}^{n\times J}$, and $\Phi_{22}=\Phi_{22}^{\top}\prec 0$ are known matrices.

When $\Phi_{12}=0$ and $\Phi_{22}=-I$, (6) reduces to $\sum_{j=1}^{J}w_{j}w_{j}^{\top}\preceq\Phi_{11}$, which has the interpretation that the total energy of the disturbance inputs in data is bounded by $\Phi_{11}$. A known norm bound on each individual disturbance $w_{j}$, $\|w_{j}\|\leq\varepsilon$, implies a bound in the form of (6) with $\Phi_{11}=(J\varepsilon^{2})I$, $\Phi_{12}=0$, and $\Phi_{22}=-I$.

Plugging $W=X^{+}-AX-BU$ into (6) and after some algebra, we obtain

where $\Theta=\Phi_{11}+X^{+}\Phi_{12}^{\top}+\Phi_{12}(X^{+})^{\top}+X^{+}\Phi_{22}(X^{+})^{\top}$, and $\star$ indicates the transpose of the related element below the diagonal. We let $\Sigma$ be the collection of $(A,B)$ satisfying (2):

According to (5), $(A_{\text{o}},B_{\text{o}})\in\Sigma$.

Under Assumption 1, we propose a control approach to achieving the three objectives below (1) for the unknown system (1). Our approach is based on the following matrix S-lemma developed in [27]:

Lemma 1 [27]. Let $M,N\in\mathbb{R}^{(2n+m)\times(2n+m)}$ be symmetric and partitioned as follows:

where $M_{11},N_{11}\in\mathbb{R}^{n\times n}$. Suppose $M_{22}\preceq 0$, $N_{22}\preceq 0$, $\text{ker}(N_{22})\subseteq\text{ker}(N_{12})$, and there exists $\bar{Z}\in\mathbb{R}^{(n+m)\times n}$ satisfying $\begin{bmatrix}I\\ \bar{Z}\end{bmatrix}^{\top}N\begin{bmatrix}I\\ \bar{Z}\end{bmatrix}\succ 0$. Then, we have that

if and only if there exist $\alpha\geq 0$ and $\beta>0$ such that

Proof. See Theorem 13 of [27]. $\blacksquare$

In this section, we consider a linear-feedback $u(t)=Kx(t)$ with a constant gain matrix $K\in\mathbb{R}^{m\times n}$. This controller yields the following closed-loop system:

The closed-loop transfer matrix from disturbance input $w$ to performance output $y_{1}$ is given by

For a performance level $\gamma>0$, the closed-loop matrix $A_{\text{c}}=A_{\text{o}}+B_{\text{o}}K$ is Schur stable and the $\mathcal{H}_{\infty}$ norm of $G_{1}(z)$ satisfies $\|G_{1}(z)\|_{\mathcal{H}_{\infty}}<\gamma$ if and only if there exists a matrix $P=P^{\top}\succ 0$ satisfying

See Theorem 2.2 of [31]. Now let $Q=P^{-1}$ (hence, $Q=Q^{\top}\succ 0$) and $Y=KQ$. With some algebra and Schur complement arguments, it can be shown that (14) is equivalent to

Note that (15b) can be written as

Now consider the quadratic Lyapunov function $V(x(t))=x(t)^{\top}Px(t)$. When (14) holds, we can derive the following dissipation inequality:

where $S=\|(\gamma^{2}I-P)^{-1/2}PA_{\text{c}}x(t)-(\gamma^{2}I-P)^{1/2}w(t)\|^{2}$ satisfies $S\geq 0$. We define, for $r>0$, the ellipsoidal set

and we arrive at the following result:

Lemma 2. Suppose (14) holds and the energy of the disturbance input is bounded as $\sum_{t=0}^{\infty}\|w(t)\|^{2}\leq\sigma_{0}$ for some $\sigma_{0}\geq 0$. Then, for any $r_{0}\geq x(0)^{\top}Px(0)+\gamma^{2}\sigma_{0}$, $\mathcal{E}(P,r_{0})$ is an invariant set of (12), i.e., $x(t)\in\mathcal{E}(P,r_{0})$ for all $t\in\mathbb{N}$.

Proof. Suppose (14) holds, $\sum_{t=0}^{\infty}\|w(t)\|^{2}\leq\sigma_{0}$, and $r_{0}\geq x(0)^{\top}Px(0)+\gamma^{2}\sigma_{0}$. Then, the dissipation inequality (3) implies

for all $t\in\mathbb{N}$. This shows $x(t)\in\mathcal{E}(P,r_{0})$ for all $t\in\mathbb{N}$. $\blacksquare$

We now consider the optimization problem (20) for designing the feedback gain $K=YQ^{-1}$. In (20), $e_{v}$ denotes the $v$th standard basis vector, $\sigma_{0}>0$ is a forecasted energy bound of the disturbance input, and $r_{0}>0$ is a design parameter to be tuned so that (20) is feasible. We note that (20) is an LMI (hence, convex) problem in the decision variables $(\eta,Q,Y,\alpha,\beta)$. A design of $K$ based on (20) has several desirable properties, stated in the following theorem:

Theorem 1. Suppose

1. (a)

   The offline data satisfy Assumption 1 and there exists $(A,B)=(\bar{A},\bar{B})$ such that (2) is strictly positive-definite;

2. (b)

   The online disturbance inputs satisfy the energy bound $\sum_{t=0}^{\infty}\|w(t)\|^{2}\leq\sigma_{0}$;

3. (c)

   The tuple $(\eta_{0},Q_{0},Y_{0},\alpha_{0},\beta_{0})$ is a solution to (20).

Then, if we control the unknown system (1) using the linear-feedback $u(t)=K_{0}x(t)$, with $K_{0}=Y_{0}Q_{0}^{-1}$, the closed-loop system has the following properties:

1. (i)

   The closed-loop matrix $A_{\text{c},0}=A_{\text{o}}+B_{\text{o}}K_{0}$ is Schur stable and the closed-loop $\mathcal{H}_{\infty}$ norm from $w$ to $y_{1}$ is less than $\gamma_{0}=\eta_{0}^{-1/2}$;

2. (ii)

   The constraints in (2) are satisfied at all times.

Proof. First, let

where $R=Q-(C_{1}Q+D_{1}Y)^{\top}(C_{1}Q+D_{1}Y)$. Using a Schur complement argument, it can be shown that (20b) is equivalent to

Meanwhile, for the $M$ and $N$ defined and partitioned as above, the following conditions can be verified: 1) $M_{22}\preceq 0$, using the fact that $R\succ 0$ due to the constraint (20c), 2) $N_{22}\preceq 0$, using $\Phi_{22}\prec 0$ according to Assumption 1, and 3) $\text{ker}(N_{22})\subseteq\text{ker}(N_{12})$. Now with $Z=[A,B]^{\top}$ and $\bar{Z}=[\bar{A},\bar{B}]^{\top}$ (with $(\bar{A},\bar{B})$ given in assumption (a)), it can be seen that the assumptions of Lemma 1 are all satisfied. According to Lemma 1, (34) holds for some $\alpha\geq 0$ and $\beta>0$ if and only if

where $\Sigma$ is defined in (8). Recall that $(A_{\text{o}},B_{\text{o}})\in\Sigma$. Therefore, we have shown that if $(\eta,Q=Q^{\top},Y,\alpha,\beta)$ satisfies (20b), (20c), and $\eta>0,\alpha\geq 0,\beta>0$, then (16) (hence, (15b)), with $\gamma=\eta^{-1/2}$, necessarily holds. Note that (20c) also implies (15a) and $Q\succ 0$. Now, if we let $P=Q^{-1}$ and $K=YQ^{-1}$, we arrive at (14) with $\gamma=\eta^{-1/2}$. This proves part (i).

Now suppose $(\eta,Q=Q^{\top},Y,\alpha,\beta)$ satisfies (20b)-(20e) and $\eta>0,\alpha\geq 0,\beta>0$. Above we have shown that, with $P=Q^{-1}$, $K=YQ^{-1}$, and $\gamma=\eta^{-1/2}$, (14) holds. Meanwhile, using a Schur complement argument, (20d) implies $r_{0}\geq x(0)^{\top}Q^{-1}x(0)+\eta^{-1}\sigma_{0}=x(0)^{\top}Px(0)+\gamma^{2}\sigma_{0}$. In this case, according to Lemma 2, $\sum_{t=0}^{\infty}\|w(t)\|^{2}\leq\sigma_{0}$ in assumption (b) implies that $\mathcal{E}(P,r_{0})$ is an invariant set of the closed-loop system, i.e., $x(t)\in\mathcal{E}(P,r_{0})$ for all $t\in\mathbb{N}$. Recall that $\mathcal{E}(P,r)$ is the ellipsoidal set defined in (18). The support function of $\mathcal{E}(P,r)$ is $h_{\mathcal{E}(P,r)}(\zeta)=\sqrt{r\,\zeta^{\top}P^{-1}\zeta}$. Hence, $x(t)\in\mathcal{E}(P,r_{0})$ implies

Meanwhile, using a Schur complement argument, (20e) is equivalent to

Combing (3) and (3), we obtain $y_{2v}(t)\leq y_{2v,\max}$. This completes the proof of part (ii). $\blacksquare$

Theorem 1 states the stability, disturbance attenuation, and constraint enforcement properties of the feedback gain $K$ for the unknown system (1) synthesized using its trajectory data according to (20). The existence of $(\bar{A},\bar{B})$ such that (2) is strictly positive-definite in assumption (a) of Theorem 1 can be checked offline. We make the following two remarks about Theorem 1:

Remark 1. The $\mathcal{H}_{\infty}$ norm from $w$ to $y_{1}$ represents a level of disturbance attenuation of the closed-loop system. In particular, using (3) recursively we can obtain the following dissipation inequality that holds for all $t\in\mathbb{N}$:

and this indicates that the $\l_{2}$-gain from disturbance $w$ to output $y_{1}$ is bounded by $\gamma$. Because (20) maximizes $\eta>0$, which is equivalent to minimizing $\gamma=\eta^{-1/2}$, the obtained controller seeks to maximize disturbance attenuation while satisfying the time-domain constraints in (2).

Remark 2. Differently from many other robust control formulations that assume set-bounded disturbances (i.e., $w(t)\in\mathbb{W}$ for some known set bound $\mathbb{W}$ and for all $t\in\mathbb{N}$), our formulation (20) enforces constraints for disturbance inputs that have bounded total energy where the energy can be distributed arbitrarily over time (i.e., $\sum_{t=0}^{\infty}\|w(t)\|^{2}\leq\sigma_{0}$). This total energy model is particularly suitable for modeling transient disturbances such as wind gusts in aircraft flight control or wind turbine control, power outages in power systems, temporary actuator or sensor failures, etc. Such disturbances occur infrequently and typically last a short period of time (as compared to persistent disturbances), but they can have a significant magnitude. Meanwhile, predicting exactly when they will occur can be difficult or impossible. In such a case, on the one hand, our formulation based on a total energy disturbance model may lead to a less conservative solution (hence, better performance) than those assuming a set bound at all times; on the other hand, estimating/predicting an energy bound for such transient disturbance events using historic data and/or real-time information (such as weather conditions) is possible in many applications. In what follows we assume a mechanism that is able to forecast an energy bound for future disturbances is available. Treating set-bounded persistent disturbances is left for future research.

We now consider a strategy for implementing the control developed in Section 3 in a moving-horizon manner: At each time $t\in\mathbb{N}$, one solves (20) with the current system state as the initial condition $x(0)$ in (20d) for a feedback gain $K_{t}$ and uses the control $u(t)=K_{t}x(t)$ over one time step. This way, the feedback gain becomes adaptive to system state and the control becomes nonlinear, possibly leading to improved performance while satisfying constraints. However, this simple implementation may fail to guarantee stability and disturbance attenuation, as discussed in [3, 4]. To recover a disturbance attenuation guarantee, following the strategy of [3, 4], we consider the following inequality:

where $P_{t}$ is associated with the feedback gain $K_{t}$ that is used at time $t$ and with an $\mathcal{H}_{\infty}$ performance level of $\gamma_{t}$ (i.e., the triple $(P_{t},K_{t},\gamma_{t})$ satisfies (14)), $P_{t-1}$ is associated with $K_{t-1}$ and $\gamma_{t-1}$, and $\Delta_{t}$ keeps track of a previous dissipation level and is defined recursively according to

for $t\geq 2$ and $\Delta_{1}=0$. Note that the definition of $\Delta_{t}$ in (40) only uses state and $P$-matrix information up to time $t-1$.

Lemma 3. Suppose (39) holds for all $t\geq 1$ where $\Delta_{t}$ is defined recursively according to (40). Then, the following dissipation inequality will be satisfied for all $t\in\mathbb{N}$:

where $\bar{\gamma}_{t}=\max\{\gamma_{0},\gamma_{1},\dots,\gamma_{t}\}$.

Proof. For each $i$, because $(P_{i},K_{i},\gamma_{i})$ satisfies (14), similar to (3), we can derive the following inequality:

Summing over $i=0,1,\dots,t$, we obtain

The definition (40) yields the following closed-form expression for $\Delta_{t}$:

Supposing (39) holds at $t$, we have

Combining (43), (45), and $\bar{\gamma}_{t}=\max_{i=0,\dots,t}\gamma_{i}$, we obtain

We now consider the following moving-horizon approach to constrained $\mathcal{H}_{\infty}$ control for the unknown system (1): At each time $t\in\mathbb{N}$, we solve the optimization problem (47) for designing the feedback gain $K_{t}=Y_{t}Q_{t}^{-1}$ and use the control $u(t)=K_{t}x(t)$ over one time step. In particular, the constraint (47f) is excluded at the initial time $t=0$ and included for $t\geq 1$. In (47), $x(t)$ denotes the measured current system state, $P_{t-1}=Q_{t-1}^{-1}$ is from the previous time, $\Delta_{t}$ is defined according to (40), $\sigma_{t}$ represents a forecasted bound on the total energy of present and future disturbances, i.e., $\sum_{i=t}^{\infty}\|w(i)\|^{2}\leq\sigma_{t}$, and $r_{t}>0$ is a design parameter, of which a design method is informed by Lemma 4 and Theorem 2. For a moving-horizon control algorithm, recursive feasibility (i.e., the online optimization problem being feasible at a given time implies the problem being feasible again at the next time) is a highly desirable property. Before we discuss the closed-loop properties of the proposed algorithm, the following lemma provides a recursive feasibility result: