Safety Embedded Control of Nonlinear Systems via Barrier States*

Control theory has been a central element in today’s fast growing, interdisciplinary technologies from simple decision making problems to terrifically complex autonomous systems. Undoubtedly, advancements in technologies are faced with unprecedented challenges. One vital challenge is safety. Nonetheless, conflicting safety restrictions and control objectives potentially appear in safety-critical control systems. The main goal of this letter is to develop a control design methodology that satisfies safety constraints and evades the possible conflict between safety constraints and control objectives. The objective is achieved by embedding the safety constraints in the system’s model through barrier states (BaS) which are provided to the control law used to attain performance objectives.

For a dynamical system, safety is classically verified by invariance of the set of permitted states. The set of permitted states is (forward) invariant if at some point in time it contains the system’s state then it contains the state for all (future) times [1]. To formally prove invariance and verify safety, barrier certificates were introduced in the control literature [2] (for a brief historical overview, see [3]). Extending the idea to control dynamical systems, a set is said to be controlled invariant, also called viable, if for any initial condition in the set, the associated trajectory is forced to be in the set for all future times using a proper control. Influenced by barrier certificates and control Lyapunov functions (CLFs), control barrier functions (CBFs) were introduced in [4], which were further developed in [5, 6, 7, 8]. CBFs can be looked at as state-dependent hard input constraints which are commonly used in quadratic programming (QP) fashion to produce a safe controller [6, 8, 8]. CBFs are becoming increasingly popular in multi-objective control due to their ability of rendering sets invariant and flexible unification with CLFs. Nonetheless, to avoid conflicts in the CLF-CBF QP, one of the conditions needs to be relaxed. For a complete review on CBFs, the reader may refer to [3].

In this letter, inspired by adopting barrier functions to establish inequality constraints forming CBFs, we utilize barrier functions to construct barrier states. Those barrier states (BaS) create barriers in the state space forcing the search of a stabilizing feedback control law to the set of safe controls. Specifically, the barrier states are appended to the model of the safety-critical dynamical system to generate a system that is safe if the equilibrium point of interest is asymptotically stable. Therefore, since safety and stability have been unified, designing a stabilizing controller for the new model means a stabilizing safe control for the original dynamical system. In other words, the safety property is embedded in the stability of the system as the feedback controller is a function of the system’s states and the barrier states, hence the name safety embedded control. The proposed method is general enough to be used with any valid barrier function. For standard barrier functions, such as Log or inverse, we show that it is possible to generate smooth or even analytic state equations for barrier states if desired. Unlike control barrier functions, no explicit knowledge of the relative degree of the system with respect to the output describing the safe region is required.

The letter is organized as follows. Section II presents the problem statement. In Section III, utilizing barrier functions, we develop barrier states used to embed safety in the stabilization problem. Subsequently, we implement the developed concept to design simple safe linear controls in Section IV. In Section V, we employ barrier states in the context of constrained optimal control to meet safety and performance objectives through minimizing an infinite horizon cost functional. In Section VI, we show the effectiveness of the proposed technique through safely stabilizing an unstable pendulum on a cart model where it is desired to avoid low velocity regions near the $90^{\text{o}}$ angle. Moreover, a multi-robot example is used where two simple mobile robots are to go to specific locations safely, i.e. without colliding while avoiding obstacles on the way. Lastly, conclusion remarks are driven in Section VII.

Consider the nonlinear control-affine dynamical system

$$\dot{x}=f(x)+g(x)u$$

(1)

where $x\in\mathcal{D}\subset\mathbb{R}^{n}$, $u\in\mathcal{U}\subset\mathbb{R}^{m}$, $f:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$ and $g:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n\times m}$ are continuously differentiable and $f(0)=0$, without loss of generality. We wish to formulate a continuous stabilizing feedback controller $u=K(x)$ that renders the nonempty open subset $\mathcal{C}$ of $\mathcal{D}$ defined as $\mathcal{C}=\{x\in\mathcal{D}:h(x)>0\}$ controlled invariant, where $h:\mathcal{D}\rightarrow\mathbb{R}$ is a continuously differentiable scalar valued function that represents the safe set. The set $\mathcal{D}$ represents the domain of operation which will be further prescribed later in the letter. The set $\mathcal{C}$, referred to as the safe set, is said to be forward invariant with respect to the closed-loop system $\dot{x}=f(x)+g(x)K(x)$ if $x(0)\in\mathcal{C}$ implies $x(t)\in\mathcal{C}$, $\forall t\geq 0$.

Enforcement of the safety constraint may be facilitated by means of a smooth scalar valued function $B:\mathcal{C}\rightarrow\mathbb{R}$ known as a barrier function (BF) in the optimization literature. Popular barrier functions include the inverse barrier function (a.k.a. Carroll barrier) $B(\eta)=1/\eta$ and the logarithmic barrier function $B(\eta)=\log(\frac{1+\eta}{\eta})$. The main properties of those types of barrier functions are that they blow up at the boundaries of the complement of $\mathcal{C}$, i.e. $\lim_{\eta\rightarrow 0}B(\eta)=\infty$, $\lim_{\eta\rightarrow\infty}B(\eta)=0$ and $\inf_{\eta\in\mathbb{R}^{+}}B(\eta)\geq 0$. Another choice that has the advantage of being analytic with a known power series expansion is $B(\eta)=\tanh^{-1}(e^{-\eta})$. The composite BF corresponding to $h$ that defines $\mathcal{C}$ is defined to be $\beta(x):=B\circ h(x)$. Note that $\beta(x)\rightarrow\infty$ if and only if $h(x)\rightarrow 0$. The following Proposition follows from Definition 1 and the choice of BFs:

As possibly conflicting safety constraints and control performance objectives need to be avoided in safety-critical control, current multi-objective frameworks, e.g. CBF-CLF safe stabilization framework, avoid possible conflicts by relaxing one of the constraints to ensure feasibility of the solution. This may result in an undesirable performance. To untangle such a problem, we propose a provable safe control technique that satisfies the safety constraints and the performance objectives simultaneously with no relaxation.

For a barrier function $\beta(x)=B(h(x))$ where $h(x)$ defines the safe set $\mathcal{C}$, the idea is to augment the open loop system with a new state variable $z$ that is related to the recentered barrier function [9], $\beta(x)-\beta(0)$ to ensure that $z=0$ is an equilibrium state. If we simply set $z$ to $\beta(x)-\beta(0)$, the resulting state equation would be

	$\displaystyle\dot{\beta}(x)$	$\displaystyle=B^{\prime}(h(x))(L_{f}h(x)+L_{g}h(x)u)$
		$\displaystyle=\phi_{0}(\beta(x))(L_{f}h(x)+L_{g}h(x)u)$

where $\phi_{0}=B^{\prime}\circ B^{-1}$ and $B^{-1}$ is the inverse of the BF. To safely stabilize the origin of (1), however, we need to ensure stabilizability of the origin of the augmented system. The main issue with augmenting $\dot{\beta}$ as the state equation is that $\beta(x)$ becomes a redundant state and the resulting augmented system may not be stabilizable in some cases. Fortunately, this issue can be resolved by perturbing the barrier state equation through an auxiliary function $\phi_{1}$ that ensures stabilizability of the origin of the augmented system without affecting the safety guarantees. More specifically, we modify the state equation for $z$ according to

$$\dot{z}=\phi_{0}(z+\beta_{0})\dot{h}(x)-\gamma\phi_{1}(z+\beta_{0},h(x))$$

(3)

where $\beta_{0}=\beta(0)$, $\dot{h}(x)=L_{f}h(x)+L_{g}h(x)u$, $\gamma\in\mathbb{R}^{+}$ and $\phi_{1}(\zeta,\eta)$ is an analytic function of two variables satisfying

$$\phi_{1}(\beta(x),h(x))=0\ \text{and}\ \frac{\partial\phi_{1}}{\partial\zeta}(\beta_{0},h(0))>0$$

It can be seen that for all three BFs mentioned earlier, the function $\phi_{0}:=B^{\prime}\circ B^{-1}$ is analytic with no singularities. Moreover, it is worth noting that both $\phi_{0}$ and $\phi_{1}$ are formulated independently of the system’s model based on the barrier function and $h(x)$. For instance, if $\zeta=B(\eta)=1/\eta$, then $\phi_{0}(\zeta)=B^{\prime}(B^{-1}(\zeta))=-\zeta^{2}$. Furthermore, letting $\phi_{1}(\zeta,\eta)=\zeta(\eta\zeta-1)$ satisfies the first condition $\phi_{1}(\zeta,\eta)=0$ along $\zeta=\beta(x)$ and $\eta=h(x)$ since $\beta(x)h(x)=1$ and the second condition $\frac{\partial\phi_{1}}{\partial\zeta}=2\zeta\eta-1|_{\beta_{0},h(0)}=2\beta_{0}h(0)-1=1>0$. Table I provides the explicit expressions for $\phi_{0}$ and possible $\phi_{1}$ for most commonly used barrier functions. It should be noted that the positive scalar $\gamma$ is a design parameter that adjusts the rate at which $z$ returns to $\beta(x)-\beta_{0}$ if it deviates from it at any instant of time. As a consequence, $\gamma$ has some influence on the design of the safe feedback control gains as we will demonstrate in the simulation examples.

The following proposition shows that if $z(0)$ is defined properly, then $z(t)=\beta(x)-\beta_{0}$ and $\phi_{1}(z+\beta_{0},h(x))=0$, $\forall t\geq 0$, implying that boundedness of $z$ guarantees satisfaction of the safety constraint.

If desired, multiple constraints can be combined to form one BF, as done in the optimization literature, to create a single BaS. Creating a single BaS is favorable in many applications and helps avoiding adding many nonlinear state equations to the safety embedded model which may increase the complexity of the controller. A single BaS that represents different constraints, however, may be highly nonlinear and may be more difficult to use to design a safety embedded control. Additionally, some flexibility on choosing design parameters for each constraint, such as penalization of the barrier states in the case of optimal control, will be lost since we will have only one barrier state. It is important to mention that in some applications, one could represent multiple constraints with one function $h(x)$ defining the safety set and then use a single BaS.

For $q$ constraints, let $\beta(x)=\sum_{i=1}^{q}B(h_{i}(x))$. Then,

$$\dot{\beta}(x)=\sum_{i=1}^{q}B^{\prime}\circ B^{-1}\Big{(}z+\beta_{0}-\sum_{j=1,j\neq i}^{q}B\big{(}h_{j}(x)\big{)}\Big{)}\dot{h}_{i}$$

and therefore the BaS can be found to be

$\displaystyle\begin{split}\dot{z}=\sum_{i=1}^{q}&\Bigg{[}\phi_{0}\Big{(}z+\beta_{0}-\sum_{j=1,j\neq i}^{q}B\big{(}h_{j}(x)\big{)}\Big{)}\dot{h}_{i}\\ &-\gamma_{i}\phi_{1}\Big{(}z+\beta_{0}-\sum_{j=1,j\neq i}^{q}B\big{(}h_{j}(x)\big{)},h_{i}\Big{)}\Bigg{]}\end{split}$

(5)

In the simulation examples, we use a single BaS to represent multiple constraints for one problem and we use multiple ones in the other to validate the proposed technique.

Now, we are in a position to create the safety embedded model. By defining $z=[z_{1},\dots,z_{q}]^{\rm{T}}$, if more than one BaS is used, and augmenting it to the system (1), we get

$$\begin{split}&\dot{x}=f(x)+g(x)u\\ &\dot{z}=f_{b}(x,z)+g_{b}(x,z)u\end{split}$$

(6)

where $f_{b}(x,z)$ and $g_{b}(x,z)$ are defined according to (3) or (5). By the definition of $\phi_{0}$ and the restrictions imposed on $\phi_{1}$, it follows that both $f_{b}$ and $g_{b}$ are analytic if $f$, $g$, and $h$ are analytic, and the subsystem $\dot{z}=f_{b}(x,z)+g_{b}(x,z)u$ is stabilizable at the origin. Hence, the combined system, which can be more compactly described as

$$\begin{split}&\dot{\bar{x}}=\bar{f}(\bar{x})+\bar{g}(\bar{x})u\\ \end{split}$$

(7)

where $\bar{x}=\begin{bmatrix}x\\ z\end{bmatrix},\bar{f}=\begin{bmatrix}f\\ f_{b}\end{bmatrix}$ with $\bar{f}(0)=0$ and $\bar{g}=\begin{bmatrix}g\\ g_{b}\end{bmatrix}$, preserves the continuous differentiability and stabilizability of the original control system (1). Therefore, the safety constraint is embedded in the closed-loop system’s dynamics and stabilizing the safety embedded system (LABEL:new_system,_safety_augmented) implies enforcing safety for the safety-critical system (1), i.e. forward invariance of the safe set $\mathcal{C}$ with respect to (1).

The remainder of the letter is devoted to synthesizing safe feedback controllers that simultaneously stabilize (1) and satisfy the required safety constraint (2) by stabilizing the dynamical system (LABEL:new_system,_safety_augmented) which unifies the two requirements. In the next sections, we apply the proposed methodology to constrained linear control systems to generate safe linear controls using pole placement and then to constrained optimal control problems to synthesize optimal safe controllers.

Consider the linear time-invariant system $\dot{x}=Ax+Bu$ subject to some safety constraint defined by a smooth function $h$. Defining the associated barrier state, augmenting it and linearizing the safety embedded model (LABEL:new_system,_safety_augmented) around the origin yields the linearized safety embedded system $\dot{\bar{x}}=\bar{A}\bar{x}+\bar{B}u$ where

	$$\displaystyle\bar{A}=\begin{bmatrix}A&0_{n\times 1}\\ -\gamma\phi_{1x}\big{(}\beta_{0},h_{x}(0)\big{)}+\phi_{0}(\beta_{0})h_{x}(0)A&-\gamma\phi_{1z}\big{(}\beta_{0},h(0)\big{)}\end{bmatrix}$$
	$$\displaystyle\bar{B}=\begin{bmatrix}B\\ \phi_{0}(\beta_{0})h_{x}(0)B\end{bmatrix}$$

It should be noted that this system may not be controllable. This should not pose any problem, however, as $\phi_{1}$ guarantees stabilizability which should be enough for us to design a safe stabilizing controller. Although we may not be able to change the location of the barrier state’s pole, it is sufficient to use the states and the barrier state to construct a safe stabilizing control. Next, we show an example where we form a barrier state to design a safe stabilizing control while the linearized system is stabilizable but not controllable.

Consider the infinite horizon optimal control problem of minimizing some cost functional subject to the dynamics (1) and the safety condition (2). This has been a sought-after problem recently [10, 11, 12] where CBFs are used to solve constrained optimal control problems. In these efforts, it can be seen how difficult the problem is and thus various complex techniques have been developed to solve the problem. Using the proposed BaS, this problem can be solved directly using well-known unconstrained optimal control methods. A systematic approach toward solving this problem is to seek an optimal feedback controller $u=K(\bar{x})$ that minimizes

$$V(x(0),u(t))=\frac{1}{2}\int_{0}^{\infty}Q(\bar{x})+u^{\rm{T}}Ru\;dt$$

(8)

where $Q:\mathbb{R}^{n+q}\rightarrow\mathbb{R}^{+}\ \forall x\neq 0$ is analytic and its Hessian is positive definite and $R\succ 0$, subject to (LABEL:new_system,_safety_augmented). By Theorem 3, if this optimal control problem is successfully solved, then both safety and stability requirements are met. For such an infinite horizon optimal control problem, a necessary and sufficient condition is that the well-known Hamilton-Jacobi-Belmman (HJB) equation is satisfied,

$$\text{HJB}:=\min_{u}\ V_{\bar{x}}^{*}\big{(}\bar{f}(\bar{x})+\bar{g}(\bar{x})u\big{)}+\frac{1}{2}u^{\rm{T}}Ru+\frac{1}{2}Q(\bar{x})=0$$

(9)

with a boundary condition $V^{*}(0)=0$ where $V^{*}$ is the optimal solution, a.k.a. the value function, and $V_{\bar{x}}^{*}=\frac{\partial V^{*}}{\partial\bar{x}}$.

Various techniques have been proposed in the literature to approximate the solution to the HJB equation or the associated optimal control [17, 15, 18, 19, 20, 13, 14]. In the next section, we utilize these efforts to produce a power series solution of the value function and its gradient to produce the optimal safe control (10) for the optimal control problem (8). Specifically, we mainly utilize the recursive analytic solution proposed in [14] and the nonlinear quadratic regulator (NLQR) in [13].

To demonstrate the efficacy of the presented technique to produce a safe and stabilizing control, we use a single BaS to enforce safety for a model of an inverted pendulum on a cart, and multiple barrier states for a multi-mobile robot navigation task where two point-robots are asked to go to intersecting targets while avoiding collision and avoiding some unsafe region.

A novel construction of safety embedded controls through the development of barrier states was presented. Through a proper conversion of barrier functions, barrier states were augmented to generate a nominal model which is safe if is asymptotically stable. Using barrier states, the constrained control problem was transformed to an unconstrained control problem, which makes the safety problem easier to be considered with various control techniques such as optimal control. Moreover, the BaS method is agnostic to the relative degree of the function describing the safe set with respect to the system unlike existing CBF based methods. Furthermore, there is no need to relax the stability requirement to guarantee safety as the two conditions are coupled and achieved simultaneously. The disadvantages of this approach include increasing the dimension of the model and adding nonlinearity to the model. The safety embedded model was used in constrained linear control and in the context of optimal control to generate safe stabilizing controllers. Simple linear controls and nonlinear quadratic controls were used to show the efficacy of the proposed method in various simulation examples.

Future work will include generalizations to nonlinear stochastic systems and applications to infinite and finite horizon stochastic optimal control as well as sampling-based model predictive control formulations. Another line of research includes generalizations of the proposed optimal control framework to min-max and $H_{\infty}$ optimal control problem formulations. Finally, incorporating uncertainty quantification methods into the augmented state space representation in (LABEL:new_system,_safety_augmented) is an active research.