Secure distributed adaptive optimal coordination of nonlinear cyber-physical systems with attack diagnosis

Recently, Cyber-Physical Systems (CPSs), which closely connect cyber and physical worlds, have gained much research interest in many fields, such as computer field, control field, battle field. By utilizing the close interaction between cyber and physical parts, the “intelligence” of physical systems can be sufficiently enhanced in order to fulfil some complex, precise or dangerous tasks, such as remote diagnosis, deep sea exploration [1]. However, the networked connection between cyber and physical parts also often leads to large attack space, such that the CPSs are vulnerable to various types of adversarial attacks. Some famous examples such as the Maroochy water breach [2] and Stuxnet [3] indicate the CPS security as a fundamental issue to be studied.

With potential applications of distributed optimization in large-scale CPSs [21], many important results on discrete- or continuous-time DO algorithms have been reported [4, 5, 6, 7, 8]. In these algorithms, each individual (or said agent in multi-agent systems) only performs the designed optimization dynamics, ignoring its own dynamics. Note that the physical dynamic systems are usually indispensable parts for achieving DO task, such as the cooperative search of radio sources [9], the motion coordination [14] and the distributed optimal power flow [10, 11]. Hence, it is relevant to study the distributed optimization problems together with physical dynamics, termed distributed optimal coordination (DOC). In fact, the DOC can be completed based on the CPS architecture by effectively combining of cyber computation/communication and physical dynamics/control [12]. Recently, many important results for DOC have been reported for multi-agent system with various physical dynamics by designing integrated closed-loop control laws, such as integrator-type dynamics [13, 14, 15], continuous-time linear dynamics [16, 17], Euler-Lagrangian dynamics [12]. More references for DOC can be found in [18]. Motivated by these and considering the non-ignorable nonlinear uncertain dynamics in many physical agents [40, 41, 12], this paper investigates the DOC problem for a class of certain nonlinear large-scale systems on the CPS platforms.

Given the growing threat of malicious attacks in large-scale (and safety-critical) CPSs, the vulnerability of consensus-based DOC algorithms is also with respect to cyber attacks. Hence, the other main objective of this paper is to address the issue of security of consensus-based DOC dynamics by providing certain safety guarantees based on attack diagnosis. The recent works [20, 21, 22] also consider the problem of resilient DO under different adversarial models than the ones that we consider here, and the agent’s own physical dynamics is not considered there. Other related important works on distributed/decentralized sensor fault diagnosis and secure state estimation against sensor attacks for large-scale CPSs have been reported in [24, 25, 26, 27, 28, 29, 30]. In the existing fault diagnosis and fault-tolerant results, the fault detectors are in general designed for given failure types [42, 43], such as loss of effectiveness [27], bias faults [24, 25, 26]. As we will see later, in our problem formulation the attack model can be considered to contain infinite number of failure types and one cannot afford to construct a fault detector for each possible failure type. In [28, 29, 30], the attack-resilient mechanisms are designed in the presence of arbitrary adversarial behaviors under the framework of distributed estimation, outside of DOC framework.

In this paper, we propose a secure DOC architecture for a class of nonlinear large-scale CPSs in the presence of cyber attacks. The overall architecture consists of cyber and physical parts, and each physical subsystem is modeled as a nonlinear parametric strict-feedback system equipped with a dedicated decision-making agent in the cyber superstratum. The cyber core (multi-agent network) focuses on the design of DOC and attack diagnosis, and the physical part performs the corresponding optimization task following the cyber-core control command. The objective is to steer the physical systems to achieve the output consensus at the minimizer of a given team performance function in a distributed fashion and provide certain safety guarantees based on attack diagnosis. The contributions of this paper are threefold.

The first contribution of this paper is, differing from the existing integrated closed-loop control schemes proposed in [12, 13, 14, 15, 16, 17], to propose a two-layer DOC structure where the cyber-layer optimizer generates a control command which is transmitted to the physical-layer control for local regulation. To overcome the difficulty caused by the dynamic mismatch between the traditional DO algorithms [4, 5, 6, 7, 8] and adaptive backstepping control systems [36, 37, 38, 39], a novel command-driven control strategy is designed. It is proved that the proposed algorithm ensures all subsystems to achieve the optimal consensus under the healthy (attack-free) environment.

As the second contribution, we provide the design and analysis of an attack detection and isolation (ADI) methodology. The existing fault diagnosis schemes are usually designed for given fault types [42, 43] and cannot guarantee the detectability for arbitrary malicious behaviors in theory. Also, due to the coupling effects of multiple propagated attacks on the physical dynamics and cyber dynamics which are interacted, the attack isolation becomes more challenging. To this end, double coupling residuals and analytical redundancy relations (ARRs) are generated by a carefully-designed distributed filter. The adaptive thresholds with prescribed performance are designed to enhance the detectability and isolability. It is theoretically guaranteed that any attack signal cannot bypass the ADI methodology to destroy the convergence of the DOC algorithm, and the locally-occurring detectable attacks can be isolated from the propagating attacks from the neighboring subsystems.

The last contribution is to develop a secure version of the DOC protocol based on the ADI methodology, which can provide a safety guarantee in the sense that the healthy physical subsystems (satisfying ARRs) reach the output consensus at the optimal solution of the sum of their objective functions, while the attacked physical subsystems (not satisfying ARRs) converge to preset secure states.

Consider a CPS consisting of $N$ subsystems, which aims at achieving the DOC task. The $j$th subsystem, $j=1,\cdots,N$ is described by the pair $(\mathcal{P}^{(j)},\mathcal{C}^{(j)})$, where $\mathcal{C}^{(j)}$ denotes the cyber part which is responsible for task decision-making, while $\mathcal{P}^{(j)}$ denotes the physical part which is responsible for task execution. The physical part $\mathcal{P}^{(j)}$ is modeled as a nonlinear dynamical system

where $i=1,\cdots,n-1$, $x_{i}^{(j)}(t)\in\mathbb{R}^{m}$, $\bar{x}_{i}^{(j)}(t)=\mathrm{vec}(x_{1}^{(j)}(t),\cdots,x^{(j)}_{i}(t))$ $\in\mathbb{R}^{im}$, $x^{(j)}(t)=\mathrm{vec}(x^{(j)}_{1}(t),\cdots,$ $x^{(j)}_{n}(t))\in\mathbb{R}^{nm}$ is the measurable state; $u^{(j)}(t)\in\mathbb{R}^{m}$ is the control input; $\varphi_{i}^{(j)}(\bar{x}^{(j)}(t))\in\mathbb{R}^{m\times p}$ and $\beta_{j}\in\mathbb{R}^{m\times m}$ are known nonsingular matrix; $\theta_{j}\in\mathbb{R}^{p}$ is unknown constant; $y^{(j)}(t)\in\mathbb{R}^{m}$ is the output measurement transmitted to the cyber superstratum through a wireless network channel and $a^{(j)}(t)\in\mathbb{R}^{m}$ denotes the cyber attacks corrupting the sensor transmitting signal. In particular, in order to provide security guarantees against worst case adversarial behavior, we allow the adversarial attacker to know the overall system model, system state, control input and the possible fault detector $\mathfrak{D}$ (e.g., distributed adaptive observers [24, 25, 26]) equipped on the CPS. Thus, the attack signal can be modeled as

where $\kappa^{(j)}(t)$ is the time profile and $\phi^{(j)}(\cdot,\cdot,\cdot,\cdot)\in\mathbb{R}^{m}$ is an unknown function that occurs at the unknown time instant $T_{a}^{(j)}$. We make no assumption on $\phi^{(j)}(\cdot,\cdot,\cdot,\cdot)$, which may be any (such as unbounded, discontinuous) function vector. The time profile of the attack is modeled as $\kappa^{(j)}(r)=0$ if $r<0$ and $\kappa^{(j)}(r)=1$ otherwise. Multiple cyber attacks may occur simultaneously or sequentially, for example, $T_{a}^{(1)}\leq\cdots\leq T_{a}^{(j^{\prime})}$ with $j^{\prime}\leq N$.

Remark 1. System (1) can represent many practical systems such as mobile robots, chemical reactors, wind tunnels, and autonomous vehicles [36]. Some extensive researches for system (1) in presence of external disturbances, actuator failures, etc., have been studied well [37, 38, 39] (these are easily extended to the current framework and thus no longer considered here). Particularly, the works in [40, 41] investigated the adaptive leader-following consensus control of system (1). In this paper, the problem of DOC further minimizes a given team performance function on the basis of consensus.

Remark 2. In general, the fault detector $\mathfrak{D}$ is designed for given failure types and cannot guarantee the detectability for arbitrary malicious behaviors in theory. Due to adversary’s strategic design, here we can assume that the attack signal $a^{(j)}(t)$ in system (1) denotes a strategic attack model which can potentially bypass the fault detector $\mathfrak{D}$ to destroy the system convergence based on the knowledge of system and detector $\mathfrak{D}$ (see stealthy attack design methods against various fault detectors, e.g., [32, 33, 34, 35] and references therein).

The overall DOC architecture of the considered CPS is illustrated in Fig. 1. Similar CPS architectures can also be found in [31, 26]. The cyber part $\mathcal{C}^{(j)}$ consists of a decision-making multi-agent network. Each decision-making agent, denoted by $\mathcal{D}^{(j)}$, is responsible for sending the control command $\Im^{(j)}$ to $\Sigma^{(j)}$. The agent $\mathcal{D}^{(j)}$ contains an optimization module and a monitoring module, denoted by $\mathcal{O}^{(j)}$ and $\mathcal{M}^{(j)}$, respectively. The module $\mathcal{O}^{(j)}$ is used to optimize its local objective function, while exchanging its information with its neighbors under a network topology $\mathcal{G}$. Since the adversarial attackers can perform the attack $a^{(j)}(t)$ to corrupt the communication $y^{(j)}(t)$ from physical stratum to cyber stratum, a cyber attack in the subsystem $(\mathcal{P}^{(j)},\mathcal{C}^{(j)})$ can also be propagated to neighbors via the information exchange between the agent $\mathcal{D}^{(j)}$ and its neighboring agents. This complicates the identification of attacked subsystems. To address it, each module $\mathcal{M}^{(j)}$ is used to detect and isolate the cyber attack $a^{(j)}$. In the physical part $\mathcal{P}^{(j)}$, the control module $\mathcal{K}^{(j)}$ consisting of an inner-loop stabilizing module $\mathcal{K}^{(I,j)}$ and an outer-loop tracking module $\mathcal{K}^{(O,j)}$ drives the physical dynamics in accordance with the control command $\Im^{(j)}$ coming from the decision-making agent $\mathcal{D}^{(j)}$. Fig. 1 illustrates that sufficient interaction between cyber and physical parts in the CPS architecture.

Objective of this paper: Design the decision-making agent $\mathcal{D}^{(j)}$ (including the optimization module $\mathcal{O}^{(j)}$ and the monitoring module $\mathcal{M}^{(j)}$) and the control agent $\mathcal{K}^{(j)}$, such that

1) Optimality: Under the healthy (attack-free) condition, all the physical subsystems cooperatively reach the optimal output that minimizes the following team performance function:

where $g^{(j)}:\mathbb{R}^{m}\to\mathbb{R}$ is a local performance function privately known to the agent $\mathcal{D}^{(j)}$; and all the closed-loop signals of the physical system are uniformly ultimately bounded (UUB).

2) Security: Detect and isolate multiple cyber attacks occurring at the network communication between physical stratum and cyber superstratum, and guarantee that the attacked subsystem $\mathcal{P}^{(j)}$, $j\in\mathbf{N}_{a}$ converges to a given secure state $y_{s}^{(j)}$, while the healthy subsystems achieve the output consensus at the optimal solution of

where $\mathbf{N}_{h}$ represents the index set of the healthy subsystems, i.e., $\mathbf{N}_{h}=\{j\in\{1,\cdots,N\}:a^{(j)}(t)=0,\forall t\geq 0\}$, and $\mathbf{N}_{a}=\{1,\cdots,N\}\backslash\mathbf{N}_{h}$.

Here we consider the scenario that the communication from physical stratum to cyber superstratum can be attacked, while the communication from the cyber superstratum to the physical stratum is secure [35]. For example, for the GPS spoofing attacks on multiple Unmanned Aerial Vehicles (UAVs) [45], the GPS attacker in fact can tamper with the location information transmitted from the UAV to the ground station, but cannot tamper with the control command from the ground station to the UAV or other communications among the ground stations. Indeed, in many practical situations, the adversarial attackers may also tamper with the control command $\Im^{(j)}$ transmitted from cyber superstratum to physical stratum or communication among decision-making agents in the cyber superstratum; however, the design of the attack diagnosis and attack-tolerant strategy is beyond the scope of this paper.

Assumption 1. The function $g^{(j)}$ is differentiable and convex for all $j=1,\cdots,N$.

Assumption 2. The graph $\mathcal{G}$ of the network is undirected and connected.

Remark 3. Assumptions 1 and 2 are common in the DO or DOC literature [8, 12, 13, 14, 15, 16, 17, 21]. In fact, many practical optimization problems can be formalized by the current convex DOC problem or approximated by it using convex relaxation, such as the motion coordination [14], target aggregation [12], search of radio sources [12], optimal power flow [10], and so on. The DOC problems for pure-integrator dynamics [13, 14, 15], Euler-Lagrangian systems [12] and linear time-invariant systems [16, 17] have been studied well. This paper further considers more complex nonlinear case (system (1)). Note that a simple combination of traditional DO algorithms [4, 5, 6, 7, 8] and adaptive backstepping controls [36, 37, 38, 39] will cause the mismatch between (cyber) optimization dynamics and physical dynamics and resultantly cannot guarantee the overall convergence.

Let $y_{r}^{(j)}$ denote the estimate of agent $\mathcal{D}^{(j)}$ about the value of the solution to (3) and denote $y_{r}=\mathrm{vec}(y_{r}^{1},\cdots,y_{r}^{N})$ and $L=\mathcal{L}\otimes I_{m}$. Then problem (3) is equivalent to

Since $g(y_{r})$ is convex and the constraint in (4) is linear, the constrained optimization problem is feasible. The following lemma gives the analysis on the optimal solution of (4).

Lemma 1 [8]. Under Assumptions 1 and 2, define by

Then $G$ is differentiable and convex in its first argument and linear in its second, and:

(i) if $(y^{*},v^{*})$ is a saddle point of $G$, then $y^{*}$ is a solution of (4);

(ii) if $y^{*}$ is a solution of (4), then there exists $v^{*}$ with $Lv^{*}=-\nabla g(y^{*})$ such that $(y^{*},v^{*})$ is a saddle point of $G$.

In what follows, we present the resilient DOC scheme by three steps. First, under the healthy conditions, we provide a basic version of DOC protocol (Section 4); Second, under the adversarial conditions, an ADI methodology is proposed to identify the attacked subsystems (Section 5); Third, the final secure DOC scheme is derived by formulating appropriate ADI-based attack countermeasure strategy for the basic version (Section 6).

Remark 4. In the cyber superstratum, all the decision-making agents $\mathcal{D}^{(j)}$ themselves are assumed to be healthy in the sense that they will follow any algorithm that we prescribe. However, due to the occurrence of cyber attack $a^{(j)}$, the agent $\mathcal{D}^{(j)}$ and its neighbors will receive false measurements $y^{(j)}$ and thus be induced as “malicious” agents in network topology. Moreover, these agents also send false data to other healthy agents and cause cascading failures [19]. These tamped measurements allow the corrupted agents to update their states to arbitrary values such that the corresponding physical subsystems follow false decision commands. In this paper, a resilient DOC algorithm will be developed, where the corresponding secure countermeasure based on an ADI method can effectively avoid the occurrences of “malicious agents” and cascading failures.

This section deals with the designs of the optimization module $\mathcal{O}^{(j)}$ and control agent $\mathcal{K}^{(j)}$ that form a basic DOC scheme under healthy conditions, i.e., $a^{(j)}(t)=0$ for any $t\geq 0$ and $j\in\{1,\cdots,N\}$. In the sequel, we drop the time argument of the signals for notational brevity.

This section deals with the design of the monitoring module $\mathcal{M}^{(j)}$, $j\in\{1,\cdots,N\}$. The ADI structure follows the standard one of fault detection and isolation (FDI), formulated by the ARRs of residuals and detection thresholds, e.g., [26, 25]. However, in this section we will focus on achieving the detection and isolation for arbitrary malicious behaviors by constructing new residuals and thresholds. Also, due to the coupling effects of multiple propagated attacks on the physical dynamics and optimization dynamics, the design of attack diagnosis becomes more challenging.

Before giving the main result of this section, we make the following assumption.

Assumption 3. The unknown parameter vector $Z:=\mathrm{vec}(y^{*},v^{*},\theta,\|L\|)$ lies in a known bounded convex set

where $\sigma(Z)$ is a convex function.

Assumption 3 is common in the existing results for fault diagnosis [25, 26, 37], and this is also necessary to detect the attack in transient response phase. It implies that the upper bounds of $y^{*},v^{*},\theta$ and $\|L\|$, say, $y_{M},v_{M},\theta_{M},L_{M}$, can be obtained, respectively. Noting that $V(0)$ depends on the unknown vector $Z$, then we define function $\Omega(Z):=V(0)$ and $\bar{\Omega}:=\sup_{Z\in\Upsilon_{Z}}\Omega(Z)$, where $V$ is the Lyapunov function defined in proof of Theorem 1.