Discrete-time Contraction-based Control of Nonlinear Systems with Parametric Uncertainties using Neural Networks

Contraction theory [4, 3] facilitates stability analysis and control of discrete-time nonlinear systems with respect to arbitrary, time-varying (feasible) references, without redesigning the control algorithm, through the study of corresponding displacement dynamics or differential dynamics. The analysis and controller synthesis to ensure stability/contraction of the nonlinear system is simultaneously completed via discrete-time control contraction metrics (DCCMs). To introduce the contraction-based methodology, we firstly consider the discrete-time nonlinear control affine system without uncertainty (extended in later sections)

where state and control are $x_{k}\in\mathcal{X}\subseteq\mathbb{R}^{n}$ and $u_{k}\in\mathcal{U}\subseteq\mathbb{R}^{m}$. The corresponding differential system of (1) is as follows

where Jacobian matrices of $f$ and $g$ in (1) are defined as $A_{k}=A(x_{k}):=\frac{\partial(f(x_{k})+g(x_{k})u_{k})}{\partial x_{k}}$ and $B_{k}=B(x_{k}):=\frac{\partial(f(x_{k})+g(x_{k})u_{k})}{\partial u_{k}}$ respectively, $\delta_{u_{k}}:=\frac{\partial u_{k}}{\partial s}$ and $\delta_{x_{k}}:=\frac{\partial x_{k}}{\partial s}$ are vectors in the tangent space $T_{x}\mathcal{U}$ at $u_{k}$ and $T_{x}\mathcal{X}$ at $x_{k}$ respectively, where $s$ parameterises a path, $c(s):[0,1]\rightarrow\mathcal{X}$ between two points such that $c(0)=x,c(1)=x^{*}\in\mathcal{X}$ (see Fig. 1). Considering a state-feedback control law for the differential dynamics (2),

where $K$ is a state dependent function, we then have from [4, 3] the following definition for a contracting system.

A region of state space is called a contraction region if condition (4) holds for all points in that region. In Definition 1, $M$ is a metric used in describing the geometry of Riemannian space, which we briefly present here. We define the Riemannian distance, $d(x,x^{*})$, as a measure for the tracking performance (convergence/stability of the state, $x$, to the reference, $x^{*}$), and is defined as (see, e.g., [16])

where $\delta_{c(s)}:=\frac{\partial c(s)}{\partial s}$. The shortest path in Riemannian space, or geodesic, between $x$ and $x^{*}$ is defined as

Leveraging Riemannian tools, one feasible feedback tracking controller for (1), can be obtained by integrating the differential feedback law (3) along the geodesic, $\gamma(s)$ (6), as

Note, that this particular formulation is reference-independent, since the target trajectory variations do not require structural redesign of the feedback controller and is naturally befitting to the flexible manufacturing paradigm. Moreover, the discrete-time control input, $u_{k}$ (7), is a function with arguments ($x_{k}$, $x^{*}_{k}$, $u^{*}_{k}$) and hence the control action is computed from the current state and target trajectory.

Then, under Definition 1 for a contracting system, e.g., (1) driven by (7), we have

which states that the Riemannian distance between the state trajectory, $x$, and a desired trajectory, $x^{*}$ is shrinking (i.e., convergence of the state to the reference), with respect to the DCCM, $M$, with at least constant rate.

To address parametric uncertainties, for the remainder of this article, we consider the following discrete-time control affine nonlinear system with parametric uncertainty

where vector $r$ represents the bounded uncertain parameters,

and functions $f$ and $g$ are smooth along the $x$ direction and Lipschitz continuous along $r$. The corresponding differential dynamics (and hence the contraction condition) can be determined for any specific value of the parameter $r$, i.e.,

where $A(r,x_{k}):=\frac{\partial(f(r,x_{k})+g(r,x_{k})u_{k})}{\partial x_{k}}$ and $B(r,x_{k}):=\frac{\partial(f(r,x_{k})+g(r,x_{k})u_{k})}{\partial u_{k}}$. Hence, from Section II-A, a function pair ($M,K$), satisfying the contraction condition (4) for any $r\in\mathcal{R}$, i.e., satisfying

ensures contraction of the uncertain system (9) for the full range of uncertainty.

The main objective is to ensure offset-free tracking of (a priori unknown) time-varying references for an uncertain nonlinear system (9). Time-varying references are generated using an estimate of the uncertain parameter $\hat{r}_{k}$, such that the reference sequence $(x^{*},u^{*})$ satisfies the following

where $r^{*}_{k}=\hat{r}_{k}$. Suppose that we have the desired state trajectory $(x^{*}_{k},x^{*}_{k+1})$. The corresponding target control input, $u_{k}$, can be obtained, given a system model and an estimated parameter value, $\hat{r}_{k}$, by solution to (13). These solutions, $(x^{*}_{k},x^{*}_{k+1},u^{*}_{k})$, are only feasible solutions for the actual system dynamics (9) when the estimated parameter, $\hat{r}$, is equal to the physical system value, $r$. Consequently, generating control references subject to parameter modelling error will result in incorrect control targets and hence state tracking offsets (see, e.g., [17]). Thus, our ensuing objective is to force the parameter estimate, $\hat{r}$, to approach the real value, $r$, online, whilst ensuring stability (to reference targets) for the full range of parameter variation.

To ensure stability/convergence to any (feasible) reference trajectories for the full range of parameter variation, a contraction theory-based structure is imposed during training of a neural network embedded controller offline. Instead of using the process model to generate process data for general neural network training, we propose to use the model to directly learn the crucial information for the contraction-based control design (satisfying (12)): the contraction metric (DCCM), which implies how the process nonlinearity affects the contraction behavior; and a differential feedback gain, from which the control action is computed. This trained DCCM neural network is then embedded in a contraction-based structure for (state-feedback) real-time control providing stability guarantees across the full range of parametric uncertainty (“exploitation”). This then facilitates online learning of uncertain system parameters within the control-loop (“safe exploration” see, e.g., [14, 15] for further discussion).

By integrating the power of well-studied, model-based modern control methods with the inherent ability of neural networks to handle system uncertainties, the proposed approach provides: (1) a systematic and efficient approach to embedding neural networks in the closed-loop control scheme; (2) certificates for stabilizability to arbitrary (feasible) time-varying references without structural redesign of the controller, by imposing a contraction-based controller structure and conditions; and (3) online model correction through iterative learning of uncertain system parameters.

The first step in the proposed methodology is to generate data, $\mathcal{D}$, from a family of system models (9). As discussed in the Introduction, to operate a chemical process using random operating conditions to generate process data to learn an accurate model is infeasible due to stability/safety concerns. The idea proposed in the following is to use a model with uncertain parameters (which characterizes the inherent uncertain nonlinear nature of modern processes) to generate data, which can be done safely offline for an explicit range of uncertainty in the system model. The contraction-based analysis is performed for the full range of system uncertainty to ensure the contraction-based controller to be robust. In this way, provided the actual system model behaves inside the family of models considered, efficient and stabilizing control combined with online parameter learning can be achieved.

In order to impose the contraction conditions (12) during training, which utilizes the generated data set, consideration as to which parameters must be included in $\mathcal{D}$ is required. The Jacobian matrices, $A_{k}(r,x_{k})$ and $B_{k}(r,x_{k})$, can be explicitly calculated from the system model (9) for specific $r$ values (see (11)). If the distribution for the uncertain parameter is known, then $r$ can be generated as a random variable with such a distribution to produce a more realistic data set for the uncertain model. Calculation of $M_{k+1}$ requires the possible next-step states (i.e., given a specific state, $x_{k}\in\mathcal{X}$, generate all possible next-step states, $x_{k+1}\in\mathcal{X}$, for all possible inputs, $u_{k}\in\mathcal{U}$, using (9)). Consequently, the Jacobian matrices $A_{k}(r,x_{k}),B_{k}(r,x_{k})$ and the two-step trajectories, $x_{k},x_{k+1}$, under specific $r$, are needed in the data set, $\mathcal{D}$. Algorithm 1 summarizes the data set generation procedure.

In this work, a DCCM neural network is employed to represent the function pair ($M$,$K$) satisfying (12). The structure of this DCCM neural network is shown in Fig. 2, whereby the inputs are the states of the system and the outputs are the numerical values of the matrices $M_{NN}$ and $K_{NN}$ for the corresponding states. Since the DCCM, $M_{NN}$, is a symmetric matrix, only the lower triangular components of $M_{NN}$ are of interest (i.e., only the lower triangular matrix is required to fully construct $M_{NN}$). Consequently, the first group of outputs are the components in the lower triangular matrix of $M_{NN}$ and the second group of outputs are the components of the controller gain $K_{NN}$ (see Fig. 2). Moreover, by exploiting the symmetric property of $M_{NN}$ the computational complexity is significantly reduced (only requiring $n(n+1)/2$ decision variables or network outputs).

This section details the implementation of a contraction-based controller, of the form in (7), that is obtained by embedding a neural network representation of the function pair $(M,K)$, i.e., $M_{NN}$ and $K_{NN}$ (calculated using Algorithms 1 and 2). Foremost, the proposed neural network embedded contraction-based controller is described by (cf. (7))

where $(x_{k}^{*},u_{k}^{*})$ are the state and control reference trajectories at time $k$. When the desired state value, $x_{k}^{*}$, changes, the feed-forward component, $u_{k}^{*}$, can be instantly updated, and the feedback component, $\int K_{NN}\delta_{\gamma}\,ds$, can be automatically updated through online geodesic calculation. Note that this approach results in setpoint-independent control synthesis. From (5) and (6), the geodesic, $\gamma$, is calculated as

where $M_{NN}$ and $K_{NN}$ are the function pair $(M,K)$ represented by the DCCM neural network (see Fig. 2), and recall from Section II-A that $c(s)$ is an $s$-parameterized smooth curve connecting $x$ ($s=0$) to $x^{*}$ ($s=1$). Implementing the contraction-based controller (7) requires integrating the feedback law along the geodesic, $\gamma$, in (6). Subsequently, one method to numerically approximate the geodesic is shown. Since (17) is an infinite dimensional problem over all smooth curves, without explicit analytical solution, the problem must be discretized to be numerically solved. Note that the integral can be approximated by discrete summation provided the discrete steps are sufficiently small. As a result, the geodesic (17) can be numerically calculated by solving the following optimization problem,

where $\bar{\gamma}(x,x^{*})\approx\gamma(x,x^{*})$ represents the numerically approximated geodesic, $x$ and $x^{*}$ are the endpoints of the geodesic, $\tilde{x}_{i}$ represents $i$-th point on a discrete path in the state space, $\Delta\tilde{x}_{s_{i}}:=\Delta{\tilde{x}_{i}}/\Delta{s_{i}}\approx{\partial c(s)}/{\partial s}$ can be interpreted as the displacement vector discretized with respect to the $s$ parameter, $\Delta\tilde{x}_{s}:=(\Delta\tilde{x}_{s_{1}},\dots,\Delta\tilde{x}_{s_{N}})$ is the discretized path joining $x$ to $x^{*}$ (i.e., discretization of c(s) in (17)), all $\Delta{s_{i}}$ are small positive scalar values chosen such that $\sum_{i=1}^{N}\Delta s_{i}=1$, $N$ is the chosen number of discretization steps (of s), $\tilde{x}_{i}=\sum_{j=1}^{i}\Delta\tilde{x}_{s_{j}}\Delta s_{j}+x$ represents the numerical state evaluation along the geodesic.

After the geodesic is numerically calculated using (18), the control law in (7) can be analogously calculated using an equivalent discretization as follows

The state reference, $x^{*}$, is chosen to follow some desired trajectory, for which the corresponding instantaneous input, $u^{*}$, can be computed via real-time optimization methods (see “Reference Generator” in Fig. 5), such that the triplet $(x_{k}^{*},u_{k}^{*},x_{k+1}^{*})$, obtained from (13), satisfies (9) for a specific value of the uncertain parameter, i.e., only when the modeled parameter matches the physical value, or $r^{*}=r$. The choice for this reference value, $r^{*}$, for the purpose of reference design, can be selected as the most likely or expected value for the uncertain parameter, i.e., $r^{*}=\operatorname{\mathbb{E}}[r]$. Hence, the corresponding desired control input at any time, $u^{*}_{k}$, can be calculated from (13) as the expected corresponding control effort, $u^{*}=\operatorname{\mathbb{E}}[u^{*}_{k}]$, given both the desired state values, $x^{*}_{k},x^{*}_{k+1}$, and expected value for $r$. Suppose then, that there was some error (e.g., due to modeling) between the chosen uncertain parameter, $r^{*}$, and the exact value for $r$. Consequently, there will be some error when computing the corresponding control effort for the desired state trajectory (via (9)), and moreover, for the resulting control effort in (16), denoted by $\tilde{u}_{k}=\bar{u}_{k}-u_{k}^{*}$, where $\bar{u}_{k}$ represents the control input reference generated using the correct parameter value $r$. The resulting disturbed system, can be modeled, using (9), as

where $u_{k}$ has the same form as (7). Inspired by the results in [4], we have then have the following contraction result.

The choice for the uncertain parameter, $r^{*}$, when designing the reference trajectory, $(x^{*},u^{*})$, directly affects the radius of the ball to which the system (1) contracts, and naturally, a finite upper limit on the radius, due to this design choice, exists and can be described by the maximum disturbance, i.e., $\max\sqrt{\alpha_{2}}\|g_{k}\tilde{u}_{k}\|$. For continuity, note that by designing the reference about the expected value of the uncertain parameter, the expected radius of the bounding ball in (21) is zero and hence (cf. (8)) recovers the undisturbed or exact contraction results of Section II-A (see specifically Definition 1) when the expected parameter value correctly matches the physical value. Following this idea, we will present a method in the following section to adjust the reference parameter, $r^{*}$, such that it converges to the physical value, $r$, hence facilitating offset-free tracking.

As Lemma 1 provides conditions for closed-loop system stability for a range of uncertain parameters, it serves the foundation for “safe exploration” [14, 15], through the addition of an online parameter estimation module, whilst maintaining stabilizing control. The additional parameter learning module (e.g., neural network training algorithm) is included in the closed-loop to learn the correct value of any uncertain parameters, such that correct reference generation and hence offset-free control can be obtained. Naturally befitting the existing approach, we present here a neural network based online parameter identification method. The estimation neural network is constructed as shown in Fig. 4, whereby the input is the current system state, $x_{k}=(x_{1},\cdots,x_{n})^{\top}$, and the output is the uncertain parameter estimate, $\hat{r}_{k}=(\hat{r}_{1},\cdots,\hat{r}_{\ell})^{\top}$. This chosen neural network structure is a generalized treatment for parameter identification/estimation. It allows for extensions to state-dependent uncertain system parameters (e.g., the rate of a chemical reaction can be a function of the temperature (a state variable) in a chemical reactor) or a more general case that uncertain parameters can represent unknown/unmodeled system dynamics.

To facilitate the learning process, as shown in Fig. 4, a loss function, $L_{ol}$, is constructed to represent the error of prediction, defined as

To clarify, $r^{*}$ is the value used for reference generation (as per Section III-C), which is updated by online parameter estimates, $\hat{r}$, for the physical system parameter, $r$, using the proposed Algorithm 3. The initial estimate (used for reference generation), is taken as the expected (albeit potentially incorrect) value for the uncertain parameter, $\bar{r}$ (as per Section III-C).

Algorithm 3 describes the procedure of parameter estimation in the time interval $[k,k+1)$. Suppose $H$ denotes the number of available recorded time steps (historical data elements), then the data set, $\mathcal{D}_{e}$, contains the collection of elements $\{x_{k-H},x_{k-H+1},\cdots,x_{k-1}\}$. Since the parameter estimation neural network is trained using backward propagation, if the number of elements is sufficiently large, this process can be batch executed (in parallel). The parameter learning process is finally terminated when the largest loss among elements in $\mathcal{D}_{e}$ is small enough (using the arbitrarily small threshold $\epsilon_{ol}$), or the max number of predefined iterations is reached, implying that the estimation error is sufficiently small for each historical element in the data set. Importantly, the estimated parameter, $\hat{r}_{k}$, is forced to lie inside the known bound $(r_{min},r_{max})$, satisfying (10), as required to ensure that the contraction-based controller maintains stability (as per Lemma 1). As the estimated parameter, $\hat{r}$, converges to the physical value, $r$, the reference model (13) converges to that of the physical system (9), leading to the following conclusion for the closed-loop.

Lemma 1 guarantees that a contraction-based controller (designed offline via Algorithms 1 and 2) will at least drive an uncertain nonlinear system to a small neighborhood about any reference generated using the model in (13). Since stability is guaranteed (in the sense of boundedness about the target trajectory), we can update online the reference model using the identified parameter in Algorithm 3 (provided the estimation satisfies (10)). As stated in Corollary 1, when the reference model matches precisely the physical system, i.e., the parametric uncertainty is removed through online identification (estimation), the neural network embedded contraction-based controller (16) also guarantees error free tracking. The proposed control synthesis and implementation (see Fig. 5) approach can be summarized as follows:

The proposed control approach is well suited for the dynamic control of modern industrial processes that are naturally highly nonlinear and modeled with uncertainty. Since the contraction metric, $M_{NN}$ (and thus the corresponding feedback control law (19)) is valid for the full range of parameter variation in $r$ (and hence $\hat{r})$, the parameter used by the reference generator, $r^{*}$, can be safely updated (explored) online simultaneously with control of the process. As a result, the proposed approach is capable of providing reference flexibility and efficient setpoint/trajectory tracking to ensure market competitiveness. Certificates for stability, to ensure safe and reliable operation, are explicitly detailed in the following section. The proposed contraction-based controller embeds neural networks, as shown in Fig. 5.