Data-driven design of explicit predictive controllers
using model-based priors

The introduction of the regularizer in (20) allows us to derive the explicit DDPC law through the manipulation of the Karush-Kuhn-Tucker (KKT) conditions associated with the new DDPC problem. To ease the computations, let us consider the following further assumption.

Based on this assumption, we now follow the same steps used to derive the explicit model-based predictive control law in [3].

The KKT conditions for the regularized DDPC problem in (20) are:

Let us focus on the $i$-th set of active constraints only, distinguishing between the Lagrange multipliers associated with a given active and inactive inequality constraints. We respectively denote them as $\tilde{\lambda}_{i}$ and $\bar{\lambda}_{i}$. It is straightforward to notice that the combination of (21b) and (21c) leads to the following condition on $\bar{\lambda}_{i}$:

By merging (21b) and (21e) for the $i$-th set of active constraints, it is also straightforward to show that the optimal solution $G_{i}(t)$ satisfies

where

and $\tilde{\Xi}_{d,i}$, $\tilde{\Psi}_{i}$ and $\tilde{\mathcal{D}}_{i}$ are the rows of $\Xi_{d}$, $\Psi$ and $\mathcal{D}$ coupled with the considered set active constraints. By leveraging on (21a), we can now express our optimization variable $G_{i}(t)$ as a function of the Lagrange multipliers, i.e.,

where

We can now replace the latter into (22) to obtain an explicit expression of the Lagrange multipliers as functions of the matrices characterizing (20):

In turn, this allows us to explicitly retrieve $G_{i}(t)$ as:

and the associated optimal input sequence as

Thus, the input to be fed to the system when the $i$-th set of constraints is active is defined as:

Through (21c) and (21d), we can finally define the polyhedral region associated with the considered combination of active constraints, which is dictated by the following inequalities:

The complete data-driven expression for (11) is then straightforwardly obtained by following the above steps for all possible combinations of the active constraints. This operation ultimately yields an optimal input sequence $U(x(t))$ of the form:

where $M$ is given by the number of possible combinations of active constraints, $U_{i}$ corresponds to (26), for all $i\in\{1,\ldots,M\}$, while $\{\mathcal{F}_{d,i},\mathcal{E}_{d,i}\}_{i=1}^{M}$ can be easily obtained from (28). Consequently, the input to be fed to $\mathcal{S}$ starting from $x(t)$ can be retrieved by evaluating the PWA law

with $u_{i}(x(t))$ given by (27), for $i=1,\ldots,M$.

Based on the available batch of data and the features of the considered predictive control problem, the explicit DDPC law can be completely retrieved offline from the available measurements, as summarized in Algorithm 1.

Given the data, one has to initially construct the Hankel matrices needed to build the DDPC problem (see steps 0.-0.). Once all the possible combinations of active constraints have been detected at step 0. and degenerate scenarios have been handled (see step 0.), at step 0. the local controllers and the associated polyhedral regions are retrieved according to (26)-(28). Lastly, at step 0., the optimal control sequence is simplified, by merging polyhedral regions whenever possible. After this step, the explicit optimal input can simply be retrieved by extracting the first element of the input sequence $U(x(t))$ (see step 0.).

Once the explicit DDPC law has been retrieved, the computation of the optimal input at each time instant simply consists of a function evaluation. Specifically, one has to $(i)$ search for the polyhedral region the current state $x(t)$ belongs to, and $(ii)$ apply the corresponding parametric law. We stress that this computational advantage is retained for simple control problems only (i.e., for short prediction horizon and small systems). Indeed, the complexity of the PWA law is known to rapidly increase [7] with the one of the DDPC problem to be solved, analogously to the model-based case.

When designing a controller in a data-driven setting, it is crucial to check the stability of the resulting closed-loop system before the controller deployment. Towards this objective, we now show how the peculiar features of the explicit data-driven predictive control can be leveraged in combination with existing techniques to devise an off-line, data-driven stability test. To this end, let us assume that the $i$-th set of constraints is active and consider the following multi-step ahead closed-loop model:

obtained by combining (13) with the result of our explicit derivation in (25), where $\hat{X}$ stacks the state predicted by the learned closed-loop model, i.e.,

From (31), we can then isolate the learned one-step ahead closed-loop model, namely

When performed for each mode $i\in\{1,\ldots,M\}$, these manipulations allow us to retrieve the data-driven closed-loop transition matrix $A_{d,i}^{cl}$ for each $i\in\{1,\ldots,M\}$. Retrieving these matrices ultimately enables us to apply model-based techniques, e.g., the ones presented in [22], to shed a light on the features of the final closed-loop. As an example, one can search for a matrix $\mathcal{P}\in\mathbb{R}^{n\times n}$ satisfying the following sufficient conditions for asymptotic stability:

or, alternatively, look for the set of matrices $\{\mathcal{P}_{i}\in\mathbb{R}^{n\times n}\}_{i=1}^{M}$ verifying the following LMIs:

As highlighted in Remark 1, all the equivalences we rely on to derive the explicit predictive control law are verified when $\mathcal{D}_{T}$ is noiseless. However, in practice, $\Omega$ in Assumption 2 is generally a non-zero matrix.

To cope with noisy data, we follow the footsteps of [5, 13] and propose to leverage on the regularization term introduced in (19). Indeed, as in standard ridge regression [18], this additional element of the cost steers all the components of $G(t)$ towards small values. In turn, this potentially limits the impact of noise on the constraints in (20b)-(20c) and, thus, on the final explicit law. This shrinkage effect is modulated by the regularization parameter $\gamma$, with the reduction in the magnitude of $G(t)$ being stronger whenever large values of $\gamma$ are considered. At the same time, $\gamma$ implicitly changes the balance between the penalties in the original data-driven control problem in (15), with excessively high values of $\gamma$ potentially driving the explicit data-driven controller far away from its model-based counterpart. The choice of this hyper-parameters thus becomes an important tuning-knob of the approach, requiring one to trade-off between handling noise and keeping explicit DDPC as close as possible to the implicit DDPC problem.

Although the stability checks in (33)-(34) can be used to have a preliminary assessment on the effect of different choices of $\gamma$, at the moment this balance can only be attained through closed-loop trials for several values of $\gamma$. Such a procedure allows one to ultimately select the hyper-parameter that best fits one’s needs, at the price of requiring closed-loop experiments that can be rather safety-critical in practice, especially when a simulator of the system is not available.

Whenever multiple experiments can be performed by feeding the plant with the same input sequence $\mathcal{U}_{T}$, the burden associated to the choice of $\gamma$ can be alleviated by exploiting the features of Assumption 2 itself. In this scenario, one can indeed replace $\mathcal{D}_{T}$ with the averaged dataset $\bar{\mathcal{D}}_{T}=\{\mathcal{U}_{T},\bar{\mathcal{Y}}_{T}\}$, where $\bar{\mathcal{Y}}_{T}=\{\bar{y}_{t}\}_{t=0}^{T}$ and

with $y_{t}^{(i)}$ denoting the output of the $i$-th experiment. Since the noise is assumed to be zero mean, the law of large numbers asymptotically yields

As such, when the number $N$ of experiments increases, the role of $\gamma$ in handling noise is progressively less dominant. In this case, $\gamma$ should then be used only to make the DDPC problem well-defined. Any small $\gamma>0$ is acceptable for this purpose.

Let us consider the benchmark system described by:

Our goal is to regulate both components of the state to zero, while enforcing the following box-constraint on the input:

Towards this goal, we collect a set $\mathcal{D}_{T}$ of $T=100$ input/state pairs, by feeding $\mathcal{S}$ with an input sequence uniformly distributed within the interval $[-5,5]$. According to Assumption 2, the measured states are corrupted by an additive zero-mean white noise sequence, with variance yielding $\overline{\mbox{SNR}}=20$ dB. The parameters characterizing the DDPC problem to be solved are selected as in [3], namely $L=2$, $Q=I$, $R=0.01$ and $P$ is chosen as the solution of the data-driven Lyapunov in (35). By setting $\gamma=10$, the partition associated with the explicit data-driven predictive controller³³3The partition is plotted thanks to the Hybrid Toolbox [1]. is the one shown in Figure 1, which approximately correspond to that reported in [3]⁴⁴4The negligible differences with respect to the model-based partition are due to the noise on the batch data.. Prior to the controller deployment, we have performed the data-based closed-loop stability check in (33), resulting in⁵⁵5The LMIs in (33) are solved with CVX [17, 16].

This indicates that the explicit law preserves the stability of the open-loop system. Figure 2 report the trajectories of the state and the optimal input obtained over a noise-free closed-loop tests with the explicit data-driven law, which confirm its effectiveness and validate the result of the data-driven stability check.

We now assess the sensitivity of the explicit controller to the choice of $\gamma$ over $20$ Monte-Carlo realizations of the batch datasets $\mathcal{D}_{T}$, for different noise levels. This evaluation is performed by looking at the cost of the controller over the same noiseless closed-loop test of length $T_{v}=50$ considered previously, i.e.,

As shown in Figure 3, the value of $\gamma$ that leads to the minimum closed-loop cost tends to decrease when the noise level increases, supporting our considerations in Section 4.4. At the same time, by properly selecting $\gamma$ we attain a cost $\mathcal{J}=16.37\pm 0.58$, which is generally close to the oracle $\mathcal{J}^{\mathcal{O}}=15.77$, which is the one achieved by the oracle law, i.e., the model-based predictive controller obtained by exploiting the true model of $\mathcal{S}$. These results additionally show that, for increasing levels of noise, the choice of $\gamma$ becomes more challenging, since the range of values leading to the minimum $\mathcal{J}$ progressively shrinks. Note that, when $\gamma$ is excessively small the optimal input is always zero and $\mathcal{S}$ evolves freely⁶⁶6This behavior is also observed for $\gamma\!\leq\!10^{-6}$, $\gamma\!\leq\!10^{-4}$ and $\gamma\!\leq\!10^{-3}$ when $\overline{\mbox{SNR}}\!=\!40$ dB, $\overline{\mbox{SNR}}\!=\!20$ dB and $\overline{\mbox{SNR}}\!=\!10$ dB, respectively..

We additionally evaluate the effect of averaging, by looking at the performance index $\mathcal{J}$ in (41) over $30$ Monte-Carlo data-collections for an increasing number $N$ of repeated experiments of length $T=100$. The measurements are affected by noise, yielding $\overline{\mbox{SNR}}=20$ dB. Figure 4 shows that the use of the averaged dataset $\bar{\mathcal{D}}_{T}$ has a similar effect to a reduction of the noise level. Indeed, for increasing $N$ the optimal $\gamma$ slowly shifts towards smaller values, thus further implying the gradual reduction in the impact of $\gamma$ on noise handling.

As a final case study, we consider a nonlinear system, namely the problem of controlling the altitude of a quadcopter, to perform landing or take-off maneuvers. To this end, we exploit the same simulator used in [15] to collect the data and to carry out the closed-loop experiments with the learned explicit law. Let $z(t)$ [m] be the altitude of the quadcopter, $v_{z}(t)$ [m/s] be its vertical velocity and $(\theta(t),\phi(t),\psi(t))$ [deg] its roll, pitch and yaw angles at time $t$. Both the the altitude $z(t)$ [m] and the vertical velocity $v_{z}(t)$ are assumed to be measured, with the measurement being corrupted by a zero-mean white noise, resulting in $\overline{\mbox{SNR}}=$ 30 dB over these two outputs. As this system is open-loop unstable, the data collection phase is carried out in closed-loop for $20$ [s] at a sampling rate of $40$ [Hz], by using the four proportional derivative (PD) controllers introduced in [15]. The altitude set point used at this stage is generated uniformly at random in the interval $[0,4]$ [m]. The set points for all the attitude angles are instead selected as slowly variable random signals within the interval $[-0.2,0.2]$ [rad]. These choices yield a dataset $\mathcal{D}_{T}$ of length $T=800$, that satisfies Assumption 1 and allows us to retain information on possible non-zero angular configurations.