Deep unfolding-based output feedback control design for linear systems with input saturation

Let us consider the following linear time-invariant system with actuator saturation:

where $x_{p}\in\mathbb{R}^{n}$ is the state, $u\in\mathbb{R}^{m}$ is the control input, $y\in\mathbb{R}^{l}$ is the measured output, and $A_{p}$, $B_{p}$, and $C_{p}$ are real-valued matrices of appropriate dimensions. Furthermore, the function $\sigma\colon\mathbb{R}^{m}\to\mathbb{R}^{m}$ denotes the standard vector-valued saturation function and is defined by

for all $u\in\mathbb{R}^{m}$ and $i=1,\dotsc,m$.

In absence of the saturation in the system (1), it is standard to apply a linear and dynamic controller of the form

where $x_{c}\in\mathbb{R}^{n_{c}}$ is the controller state. However, it is recognized in the literature that the standard controller (3) does not necessarily mitigate the deterioration of the control performance introduced by the saturation. A common approach to address this problem is to introduce an anti-windup compensation term [17] to the dynamic controller (3) as

where $E_{c}\in\mathbb{R}^{n_{c}\times m}$ is the anti-windup gain. The compensator (4) behaves to hinder the integral operation of the dynamic controller when input saturation occurs.

When the anti-windup controller (4) is applied to the system (1), we obtain the following closed-loop system:

where

denotes the joint state of the closed-loop system and the matrices $A$, $B$, and $F$ are defined by

respectively. We can further rewrite the system (5) in a compact form as

In this subsection, we describe the stability analysis [11] of the system (6). We start with the following definition.

To state a criteria [11] for analyzing the contractive invariance of an ellipsoid, we introduce the following notations. For a vector $\nu\in\mathbb{R}^{m}$ and matrices $F,H\in\mathbb{R}^{m\times(n+n_{c})}$, define the matrix $M(\nu,F,H)$ by

where $I$ denotes the identity matrix of an appropriate dimension and $\operatorname{diag}\nu$ denotes the diagonal matrix having the elements $\nu_{1}$, …, $\nu_{m}$ as its diagonal elements. Also, define the set

which consists of $2^{m}$ vectors. The following lemma is given in [11].

Lemma 1 provides a condition for checking if a given ellipsoid is contractively invariant. As a scalar measure for evaluating the size of ellipsoids, we adopt the methodology developed in [11], where the authors measure the size of ellipsoids by using a shape reference set. Let $\mathcal{X}_{R}$ be a subset of $\mathbb{R}^{n+n_{c}}$ containing the origin. Then, for a set $S\subset\mathbb{R}^{n+n_{c}}$, we define the quantity

which gives us a measure for the size of the set $\mathcal{S}$. Although the authors in [11] allow the shape reference set $\mathcal{X}_{R}$ to be either an ellipsoid or a polyhedron, in this paper we focus on the case where the set is the polyhedron given as

by the set of pre-determined vectors $\tilde{x}_{1}$, $\tilde{x}_{2}$, …, $\tilde{x}_{l}$ in $\mathbb{R}^{n+n_{x}}$. Then, by using Lemma 1, one can formulate the problem of finding the maximum contractively invariant set measured by the shape reference set $\mathcal{X}_{R}$ as the following optimization problem [11]:

It is not impossible to use the optimization problem  (7) to optimally design the parameters $A_{c}$, $B_{c}$, $C_{c}$, $D_{c}$, and $E_{c}$ in the controller (4) for maximizing the contractively invariant ellipsoid. However, the optimization problem reduces to a set of bilinear matrix inequalities, which is not easy to solve in practice [18]. Due to this difficulty, the conventional method [7] first fixes the parameters $A_{c}$, $B_{c}$, $C_{c}$, and $D_{c}$ in such a way that the closed-loop system is stabilized in absence of the input saturation and, then, design the anti-windup gain $E_{c}$ separately. This implies that the parameters $A_{c}$, $B_{c}$, $C_{c}$, and $D_{c}$ are not necessarily designed for enlarging the contractively invariant ellipsoid. In order to fill in this gap, in the next section we propose a novel deep unfolding-based approach for designing all the parameters simultaneously for enlarging the contractively invariant ellipsoid.

As mentioned in the previous section, it is difficult to effectively design a dynamic anti-windup controller by using the optimization problem (7). In order to avoid this difficulty, in this paper we propose the following approach we first design a candidate controller by using a date-driven, deep learning-based approach described below. We then test the validity of the candidate controller by using the linear matrix inequalities (7).

In order to obtain a candidate controller, we propose that the following problem is solved:

In order to describe our motivation for introducing Problem 1, we below show how we can intuitively relate Problem 1 and the controller design problem. Suppose that the region $\mathcal{C}$ satisfies $\mathcal{C}\subset\Omega(P,\rho)$ for a set of parameters $(A_{c},B_{c},C_{c},D_{c},E_{c})$. Then, the state trajectory of the system (6) for $x(0)\in\mathcal{C}$ satisfies $\lim_{t\to\infty}x(t)=0$. Therefore, when $T$ is sufficiently large, we can expect that the set of the gains gives a good solution to Problem 1. Conversely, if the solution of Problem 1 for sufficiently large $T$ makes the value of the objective function $\mu$ sufficiently small, i.e., when $x(t)$ is sufficiently small at the $T$ for the initial state $x(0)\in\mathcal{C}$, then we can expect that the ellipsoid $\Omega(P,\rho)$ obtained in this case is larger than the region $\mathcal{C}$.

Although one can expect that the solution of Problem 1 could allow us to design a feedback controller for enlarged invariant set, it is not straightforward to directly minimize the quantity $\mu$ given in (8). In this paper, we show that we can efficiently perform the minimization by using the technique called deep unfolding.

In the framework of deep unfolding, adjust parameters using deep learning techniques, the signal-flow of the iterative algorithm is unfolded, and the network with the structure of this signal-flow graph is created. Therefore, in this paper, we unfold the signal-flow of the system (6) to obtain a signal-flow graph. For training this network, we propose that ODE-Net [16] is used. In ODE-Net, it is allowed that the structure of the network is represented by an ordinary differential equation $\dot{x}(t)=f(x(t),t,\theta)$, with $\theta$ being a trainable parameter. Therefore, we construct a network with the structure of the state equation (9) as

for using ODE-Net to learn the gain of a dynamic controller with an anti-windup compensator.

We employ the technique of incremental learning [14] to efficiently train the network. We specifically determine the number of incremental learning iterations $N$ and let $t_{k}=kT/N$ for $k=1$, …, $N$. We then calculate the candidate controller parameter sets $(A_{c,k},B_{c,k},C_{c,k},D_{c,k},E_{c,k})$ to minimize the loss function

at each step. We then solve an optimization problem (7) for $A_{c,k}$, $B_{c,k}$, $C_{c,k}$, $D_{c,k}$, and $E_{c,k}$ at each step of the incremental training, and use the controller with the largest ellipsoid.

To effectively enlarge the ellipsoid, we set the region $\mathcal{C}$ as

for some parameter $\beta>1$, and train the network by using the initial states outside the obtained region $\Omega(P_{0},\rho_{0})$. In order to implement the process described above, one may use DiffEqFlux.jl [19] together with DifferentialEquations.jl [20]. We remark that, because these tools are only applicable to dynamical systems with smooth vector fields, it is necessary to to approximate the non-smooth saturation function (2) by a smooth function. For example, one can use the approximation of the form (see, e.g., [21])

In Fig. 1, we present a comparison of the original saturation mapping and its approximation (10) for the value $\zeta=1.0\times 10^{-6}$, in which a small approximation error is achieved.

In summary, we below present the design algorithm that we propose in this paper:

1. Step 1)

   Let a reference set $\mathcal{X}_{R}$ and initial gains $A_{c,0}$, $B_{c,0}$, $C_{c,0}$, $D_{c,0}$ be given. Let $E_{c,0}=0$ and solve the optimization problem (7). Denote the solutions of the optimization problem as $\alpha_{0}$ and $P_{0}$. Set

   $$\mathcal{C}=(\beta\Omega(P_{0},\rho_{0}))\backslash\Omega(P_{0},\rho_{0})$$

   and $\alpha_{\max}=\alpha_{0}$.

2. Step 2)

   Set $k=1$.

3. Step 3)

   Generate $J$ initial states $\tilde{x}(0)=\tilde{x}_{k,1}$, …, $\tilde{x}_{k,J}$ randomly and uniformly from region $\mathcal{C}$.

4. Step 4)

   Use ODE-Net to train the network under the loss function $L(0,t_{k})$ and find a set of controller parameters.

5. Step 5)

   For $A_{c,k}$, $B_{c,k}$, $C_{c,k}$, $D_{c,k}$, and $E_{c,k}$, Solve the optimization problem (7). Denote the solution as $\alpha_{k}$. If $\alpha_{\max}<\alpha_{k}$, let $\alpha_{\max}=\alpha_{k}$, $A_{c}=A_{c,k}$, $B_{c}=B_{c,k}$, $C_{c}=C_{c,k}$, $D_{c}=D_{c,k}$, and $E_{c}=E_{c,k}$.

6. Step 6)

   If $k<N$, increment $k$ and go to step (3). Else, go to step (7).

7. Step 7)

   $A_{c}$, $B_{c}$, $C_{c}$, $D_{c}$ and $E_{c}$ are controller parameters and stop.