Instabilizability Conditions for Continuous-Time Stochastic Systems Under Control Input Constraints

Stabilization under control input constraints is an important research problem due to its wide applicability to systems with actuator saturation. The works [1, 2] describe the challenges of this problem and provide comprehensive discussions of the important results. A key result on this problem is an impossibility result: linear deterministic systems with strictly unstable system matrices cannot be globally stabilized if the norm of the control input is constrained to stay below a constant threshold [3, 4].

There is a rapidly growing interest in exploring control input constraints for stochastic systems. For instance, [5, 6, 7, 8] proposed stochastic model predictive controllers with control constraints; [9] and [10] developed reinforcement learning control frameworks with constraints, [11] investigated fuzzy controllers for stochastic systems with actuator saturation. Constrained control of nonlinear stochastic systems was investigated by [12] and [13], and moreover, [14] explored control constraints in stochastic networked control systems.

The work [15] presented an impossibility result for constrained control of discrete-time stochastic systems. It was shown there that if the control input of a strictly unstable discrete-time stochastic system is subject to hard norm-constraints, then the second moment of the state always diverges under nonvanishing and unbounded additive stochastic process noise. A common approach to overcome the difficulties in the stabilization of strictly unstable systems is to consider probabilistic constraints instead of hard deterministic constraints. However, it was shown in [16] that under certain conditions, stabilization of a discrete-time linear stochastic system is impossible even under probabilistic constraints.

The scope of the impossibility results provided in the abovementioned articles covers discrete-time stochastic systems with additive noise. In this paper, we are motivated to expand this scope by addressing two issues. First, we want to know if similar impossibility results can be obtained for continuous-time stochastic systems. Secondly, we want to investigate the effects of both additive and multiplicative noise terms. Handling multiplicative noise terms is important, since such terms can characterize parametric uncertainties in the system (see [17, 18]). As our main contribution, we identify the scenarios where stabilization of a continuous-time stochastic system (with both additive and multiplicative noise) is not possible under probabilistically-constrained control policies. Specifically, we consider control policies that have bounded time-averaged second moments. This class of control policies encapsulate many types of controllers with (probabilistic or deterministic) control constraints. We obtain conditions on the bounding value of the control constraint, under which the second moment of the state diverges regardless of the controller choice and regardless of the initial state value.

Our analysis for the continuous-time systems with additive and multiplicative noise has a few key differences from that for the discrete-time case with additive-only noise provided in [15, 16]. First, in our case, we handle Ito-type stochastic differential equations with state-dependent noise terms characterizing multiplicative Wiener noise. In addition, we develop a form of reverse Gronwall’s inequality to obtain lower bounds on functions with superlinear growth. Through our analysis, we observe that combination of additive and multiplicative noise can make systems harder to stabilize. Even systems that have Hurwitz-stable system matrices can be impossible to stabilize with constrained controllers under the combination of additive and multiplicative noise.

We organize the rest of the paper as follows. In Section 2, we describe the constrained control problem. Then in Sections 3 and 4, we provide our impossibility results for constrained control of continuous-time stochastic systems. Finally, we present numerical examples in Section 5 and conclude the paper in Section 6.

Notation: We denote the Euclidean norm by $\|\cdot\|$, the trace operator by $\mathrm{tr}(\cdot)$, and the maximum eigenvalue of a Hermitian matrix $H\in\mathbb{C}^{n\times n}$ by $\lambda_{\max}(H)$. We use $H^{\frac{1}{2}}$ to represent the unique nonnegative-definite Hermitian square root of a nonnegative-definite Hermitian matrix $H\in\mathbb{C}^{n\times n}$, satisfying $H^{\frac{1}{2}}H^{\frac{1}{2}}=H$ and $(H^{\frac{1}{2}})^{*}=H^{\frac{1}{2}}$. The identity matrix in $\mathbb{R}^{n\times n}$ is denoted by $I_{n}$. The notations $\mathrm{\mathbb{P}}[\cdot]$ and $\mathbb{E}[\cdot]$ respectively denote the probability and expectation on a probability space $(\Omega,\mathcal{F},\mathbb{P})$ with sample space $\Omega$ and $\sigma$-algebra $\mathcal{F}$. We consider a continuous-time filtration $\{\mathcal{F}_{t}\}_{t\geq 0}$ with $\mathcal{F}_{t_{1}}\subseteq\mathcal{F}_{t_{2}}\subseteq\mathcal{F}$ for $t_{1}\leq t_{2}$. Throughout the paper $\{W(t)=[W_{1}(t),\ldots,W_{\ell}(t)]^{\mathrm{T}}\in\mathbb{R}^{\ell}\}_{t\geq 0}$ denotes the Wiener process. Here, for every $i\in\{1,\ldots,\ell\}$, the process $\{W_{i}(t)\in\mathbb{R}\}_{t\geq 0}$ is $\mathcal{F}_{t}$-adapted; $\{W_{i}(t)\in\mathbb{R}\}_{t\geq 0}$, $i\in\{1,\ldots,\ell\}$, are independent processes. Moreover, $\overline{c}$ denotes the complex conjugate of a complex number $c\in\mathbb{C}$, and $\mathrm{Re}(c)$ denotes its real part. We use $C^{*}$ to denote the complex conjugate transpose of a complex matrix $C\in\mathbb{C}^{n\times m}$, that is, $(C^{*}){}_{i,j}=\overline{C_{j,i}}$, $i\in\{1,\ldots,m\}$, $j\in\{1,\ldots,n\}$. Given a vector $v\in\mathbb{R}^{n}$, and indices $i,j\in\{1,\ldots,n\}$, $i\leq j$, we define $v_{i:j}\in\mathbb{R}^{j-i+1}$ as $v_{i:j}\triangleq[v_{i},\ldots,v_{j}]^{\mathrm{T}}$.

Consider the continuous-time linear stochastic system described by the Ito-type stochastic differential equation

for $t\geq 0$, where $x(t)\in\mathbb{R}^{n}$ is the state with deterministic initial value $x(0)=x_{0}$, $u(t)\in\mathbb{R}^{m}$ is the control input, and moreover, $\{W(t)\in\mathbb{R}^{\ell}\}_{t\geq 0}$ is the Wiener process.

The matrices $A\in\mathbb{R}^{n\times n}$ and $B\in\mathbb{R}^{n\times m}$ are called system and input matrices, respectively. Moreover, $\Psi(x(t))\in\mathbb{R}^{n\times\ell_{1}}$ and $D\in\mathbb{R}^{n\times\ell_{2}}$ (with $\ell_{1}+\ell_{2}=\ell$) are noise matrices. The matrix-valued function $\Psi\colon\mathbb{R}^{n}\to\mathbb{R}^{n\times\ell_{1}}$ characterizes the effects of multiplicative noise and it is given by

where $C_{i}\in\mathbb{R}^{n\times n}$, $i\in\{1,\ldots,\ell_{1}\}$. The matrix $D\in\mathbb{R}^{n\times\ell_{2}}$ in (1) is used for characterizing the effects of additive noise.

Notice that $W_{1:\ell_{1}}(\cdot)$ enters in the dynamics as multiplicative noise and $W_{\ell_{1}+1:\ell}(\cdot)$ enters as additive noise, since $\left[\Psi(x(t)),\,D\right]\mathrm{d}W(t)=\Psi(x(t))\mathrm{d}W_{1:\ell_{1}}(t)+D\mathrm{d}W_{\ell_{1}+1:\ell}(t)$.

In this paper, we are interested in a stabilization problem. Since the Wiener process $W_{\ell_{1}+1:\ell}(\cdot)$ enters in the dynamics in an additive way, the state and its moments cannot converge to $0$ regardless of the control input, unless $D=0$. For this reason, asymptotic stabilization is not possible and a weaker notion of stabilization is needed. In this paper, we consider the bounded second-moment stabilization notion, where the control goal is to achieve $\sup_{t\geq 0}\mathbb{E}[\|x(t)\|^{2}]<\infty$.

We consider a stochastic constraint such that the time-averaged $2$nd moment of $u(t)$ is bounded by $\hat{u}\geq 0$, i.e.,

This constraint is a relaxation of other types of control constraints, i.e., the satisfaction of (3) does not necessarily imply satisfaction of other constraints. Note on the other hand that norm-constraints (e.g., $\|u(t)\|\leq\overline{u}$ or $\|u(t)\|_{\infty}\leq\overline{u}$), time-averaged norm constraints (e.g., $\frac{1}{t}\int_{0}^{t}\|u(\tau)\|\mathrm{d}\mathrm{\tau}\leq\overline{u}$), as well as first- and second-moment constraints (e.g., $\mathbb{E}[\|u(t)\|]\leq\overline{u}$ or $\mathbb{E}[\|u(t)\|^{2}]\leq\overline{u}$) all satisfy (3) for certain values of $\hat{u}$.

For given $A,B,\Psi,D$, our goal is to find a threshold for $\hat{u}$, below which stabilization of (1) becomes impossible and the second moment $\mathbb{E}[\|x(t)\|^{2}]$ diverges regardless of the controller design.

The following lemmas are used in the derivation of our main result in Section 3. The first lemma is related to the bounding value $\hat{u}$ of the control constraint (3). The second lemma is an extension of Gronwall’s lemma (see, e.g., [19]), where the key condition involves a linear term and the result provides a lower bound instead of an upper bound.

In this section, we present our main result, which provides conditions on the control constraint (3), under which the stochastic system (1) is impossible to be stabilized.

Theorem 3.1 provides sufficient conditions under which the system (1) is instabilizable and the second moment of the state diverges regardless of the controller design and the initial state value. Condition (7) in Theorem 3.1 quantifies the instability of the uncontrolled ($u(t)\equiv 0$) system, and the term $\mathrm{tr}\left(D^{\mathrm{T}}RD\right)$ in (8) represents the effect of additive noise characterized with the matrix $D$. If there is no multiplicative noise (i.e., $C_{i}=0$ for $i\in\{1,\ldots,\ell_{1}\}$), then (7) requires $A$ to be strictly unstable. On the other hand, when there is multiplicative noise, (7) may hold even if $A$ is Hurwitz-stable. Notice also that $R$ is a nonnegative-definite matrix and it may have $0$ as an eigenvalue. This property is essential in our analysis, since it allows us to deal with the cases where some of the states are diverging, while the others are stable.

Theorem 3.1 implies that if conditions (7), (8) hold, then it is not possible to stabilize the system by using control inputs with too small average second moments as in (3). The impossibility threshold on the average second moment of control input $u(t)$ is characterized in (9). If $\lambda_{\max}(B^{\mathrm{T}}RB)=0$, then this threshold value becomes infinity indicating that stabilization is impossible regardless of the input constraint. We note that the case $\lambda_{\max}(B^{\mathrm{T}}RB)=0$ represents the situation, where $u(t)$ does not have any effect on $x^{\mathrm{T}}(t)Rx(t)$.