Bayesian Meta-Learning on Control Barrier Functions with Data from On-Board Sensors

ENSURING safety while achieving certain tasks is a fundamental yet challenging problem that has drawn significant attention from researchers in the control and robotics communities over the past few decades. In the control community, such safety requirements are often handled by imposing corresponding state constraints on an optimal control problem. Model Predictive Control (MPC) [1] and control synthesis methods based on the certificate functions such as Control Lyapunov Function (CLF) and Control Barrier Function (CBF) [2, 3, 4] are notable examples of such a method. Desirable theoretical results for these methods such as feasibility, stability, and safety are offered for known system dynamics [1, 2, 3, 4] or noisy dynamics with known noise distribution [5, 6, 7].

The CBF is a prominent tool to impose forward invariance of a safe set in state space. If a dynamical system is control-affine, quadratic programming formulation called CBF-QP [4] is available, which can produce safe control inputs much faster than many other control methods including MPC, and enables real-time implementations for challenging applications such as autonomous vehicles [8] and bipedal locomotion [9]. However, the majority of the existing works presuppose the availability of a valid CBF that represents safe and unsafe regions in state space. This assumption, however, cannot be maintained if a robot is expected to operate autonomously in unknown or uncertain environments. In such situations, it is essential for a robotic system to automatically identify unsafe regions on the fly, employing data from sensory devices. Thus, in this paper, we consider a learning-based method for constructing a CBF based on online measurements obtained from onboard sensors such as LiDAR scanners.

In the previous studies regarding this line of the topic (e.g., [31, 32, 33]), the robotic system iteratively improves the prediction of the CBF by collecting online sensor measurements without any prior knowledge about the environment or learning task. However, in such cases, even a slight change in the environment necessitates a complete retraining process, which is inefficient and requires a substantial amount of data. To efficiently and effectively learn CBF, adapting it to a new environment with minimal or small amount of training data, we suggest implementing meta-learning [10, 11], a technique that allows an agent to learn how to learn, as opposed to creating it from scratch. During the meta-training process, the model parameters are trained with data collected from various different environments, aiming to learn common underlying knowledge or structure of the learning task. The parameters obtained through meta-learning are then utilized as initial parameters for the online phase in a new environment and updated with online measurements in the environment. In this paper, we specifically employ a learning scheme combining the recently developed Bayesian meta-learning method [12] which is equipped with probabilistic bounds on the prediction [13], and the Bayesian surface reconstruction method [14, 15, 16] that can learn the unsafe regions from noisy sensor measurements. With this method, we can learn CBF and synthesize a safe controller with few amount of data and provide a probabilistic guarantee on the resulting controller.

Related works on learning and CBF : Learning-based approaches for certificate functions such as CBF and CLF are one of the active research topics in the area of intersections between control and learning [17]. In the following, we summarize the recent progress in the topic of learning and CBF. The authors of the works [18, 19, 20, 21, 22, 23] used CBF for active safe learning of uncertain dynamics. The works [18, 19] consider machine learning approaches to safely reduce the model uncertainty and show the methods yield empirically good performances, while the Gaussian Process (GP) is also used in [20, 21, 22, 23] and rigorous theoretical results for safety requirements are offered under certain assumptions. CBF is also used in Imitation Learning (IL) and Reinforcement Learning (RL) [25, 26, 24] to account for safety concerns.

While the above studies assume a valid CBF is given and consider leveraging it to impose safety conditions, several previous studies also consider parameterizing CBF itself by Neural Network (NN) and learning it to be a valid barrier function [27, 28, 29, 30, 31, 32, 33]. The work [27] considers jointly learning NNs representing CBF and CLF as well as a control policy based on safe and unsafe samples. The method proposed in [28] considers recovering CBF from expert demonstrations and theoretically guarantees that the learned NN meets the CBF conditions under the Lipsitz continuity assumptions. In [31], GP is also used to synthesize CBF. The works most related to this paper are [32, 33] which propose ways to learn CBF with sensor measurements. In [32], Support Vector Machine (SVM) classifier representing CBF is trained with safe and unsafe samples constructed by using the LiDAR measurements. Similarly, the authors of the work [33] present a synthesis method using Signed Distance Function (SDF).

Related works on meta-learning and control : Meta-learning, also known as “learning to learn,” is a technique capable of learning from previous learning experiences or tasks in order to improve the efficiency of future learning [10]. Meta-learning has recently become increasingly popular in the control literature. In several previous studies such as [13, 34], meta-learning is used to learn system dynamics to enable quick adaptation to the changes in surrounding situations. In these works, the trained models are incorporated into MPC which requires relatively heavy online computation. Meta-learning is also used in policy learning in the context of adaptive control [35] and reinforcement learning [10, 36].

Contributions: Our contributions compared to the existing methods are threefold: First, compared to the methods that synthesize CBF with sensor measurements [31, 32, 33] which rely on traditional supervised learning, the proposed meta-learning scheme can effectively use past data collected from different environments and produce a prediction of CBF with few amount of data. Consequently, the resulting control scheme realizes less conservative control performance with few amount of data (small number of online CBF updates) as we see in the case study. Second, different from the previous NN-based CBF synthesis methods [27, 29, 30, 32, 33], our method can readily take into account uncertainty in data and calculate formal probabilistic bounds on CBF. Based on this, we provide a probabilistic safety guarantee on the proposed control scheme. Although the work [33] takes into account the prediction error in SDF, such an error bound is assumed to be given and actual computation is not provided. Third, the learned CBF can readily be incorporated into ordinary QP formulation which can be efficiently solved. Thus, our method can produce control input much faster than the previous methods regarding meta-learning and MPC such as [13, 34].

Notations: A continuous functions $\alpha_{1}:\mathbb{R}_{\geq 0}\rightarrow\mathbb{R}_{\geq 0}$ and $\alpha_{2}:\ \mathbb{R}\rightarrow\mathbb{R}$ are class $\mathcal{K}$ and extended class $\mathcal{K}_{\infty}$ function, respectively, if they are strictly increasing with $\alpha_{1}(0)=0$ and $\alpha_{2}(0)=0$. $\mathcal{X}_{d}^{2}(p)$ is the $p$-th quantile of the $\mathcal{X}^{2}$ distribution with $d$ degrees of freedom. The maximum and minimum eigenvalues for any positive definite matrix $A$ are defined as $\bar{\lambda}(A)$ and $\underline{\lambda}(A)$, respectively. For vector fields $f$ and a function $F$, $L_{f}F(x):=\frac{\partial F(x)}{\partial x}f(x)$ is the Lie derivatives of the function $F$ in the direction of $f$.

Throughout this paper, we consider the following control affine system:

where $x\in\mathcal{D}\subset\mathbb{R}^{n}$ and $u\in\mathcal{U}:=\{u\in\mathbb{R}^{m}\mid Au\leq b\}$ with $A\in\mathbb{R}^{m\times m}$ and $b\in\mathbb{R}^{m}$ are the admissible system states and control inputs respectively. The functions $f:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$ and $g:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n\times m}$ are continuously differentiable functions representing dynamics of a robotic system (e.g., ground vehicles or drones) equipped with sensors such as LiDAR. The state space is assumed to contain initially unknown $N_{\mathrm{obs}}$ obstacles and the outside of each obstacle is defined by the following:

where $h(\cdot;\xi):\mathbb{R}^{n}\rightarrow\mathbb{R}$ is a continuously differentiable function with unobserved latent variable $\xi$ that encodes the geometric information about an obstacle, e.g., the position and shape of an obstacle. We assume realizations of the variable, $\xi_{i}$, $i\in\mathbb{N}_{1:N_{\mathrm{obs}}}$ are determined by samples from a probabilistic distribution $p(\xi)$ and they are fixed during the control execution. For simplicity, the obstacles are chosen not to overlap with each other.

Our goal is to synthesize a controller that achieves a given task (e.g., goal-reaching task) without deviating from the safe region $\mathcal{C}=\bigcap_{i=1}^{N_{\mathrm{obs}}}\mathcal{C}_{\xi_{i}}$ given initial state $x(0)\in\mathcal{C}$. Since the safe/unsafe regions are initially unknown, the robotic system needs to identify them based on the sensor measurements. A naive approach toward this end is to use supervised learning based on measurement data obtained under a pre-determined environment. However, in this case, the trained model cannot be generalized to different environments, and even slight changes in the environment necessitate a complete re-fitting of the model, which is not only inefficient but also demands a substantial amount of data. Thus, this paper considers a way to effectively use past data regarding the different environments sampled from the distribution $p(\xi)$ to quickly learn safe/unsafe regions in a new environment and synthesize the safe controller with few amount of online data.

In the following, we first introduce the control notions suited for our purpose called Control Barrier Function (CBF) and Control Lyapunov Function (CLF) in Section III. Then, the meta-learning procedures of CBF and a way to synthesize QP with the learned CBF are discussed in Section IV and V.

In this section, we first introduce the Zeroing Control Barrier Function (ZCBF) as a tool to enforce safety on the system (1). The following discussion is a summary of the works [2, 4]. In ZCBF, the safety is defined as the forward-invariance of a safe set $\mathcal{C}$ which is defined by the super zero level set of a function $h:\mathbb{R}^{n}\rightarrow\mathbb{R}$, i.e., the system (1) is considered to be safe if $x(t)\in\mathcal{C}:=\{x\mid h(x)\geq 0\}$ holds for all $t\geq 0$ when $x(0)\in\mathcal{C}$. Given an extended class $\mathcal{K}_{\infty}$ function $\alpha_{C}:\mathbb{R}\rightarrow\mathbb{R}$, the ZCBF is defined as follows.

If the function $h$ is a valid CBF for (1) on $\mathcal{C}$, the safety of the system (1) is guaranteed as the following theorem.

In this paper, the control objectives, except for the safety requirements, are encoded through the control Lyapunov function (CLF) which is given in advance. With a class $\mathcal{K}$ function $\alpha_{V}$, the definition of CLF is formally given as follows [2].

Given a CLF $V$, we consider the set of all control inputs that satisfy (4) for a state $x\in\mathcal{D}$ as

If a function $V$ is CLF, the stability of the equilibrium point $x_{e}$ is guaranteed as the following theorem.

Having CBF and CLF defined above, we can synthesize a controller satisfying CBF and CLF conditions (3) and (4) with quadratic programming (QP) as follows:

where $H\in\mathbb{R}^{m\times m}$ is a positive definite matrix and $\lambda\ (>0)$ is a tunable parameter for relaxing the CLF condition (5b). The relaxation of the CLF condition ensures the solvability of the QP. If the safety is defined by multiple CBFs, we can handle them by adding the corresponding CBF constraints to the optimization problem.

In this section, we consider learning the safe region (2) based on the measurement data from sensor devices. To this end, we employ a technique based on the Gaussian process implicit surfaces (GPIS) [14, 15, 16] which is developed for non-parametric probabilistic reconstruction of object surfaces. Note that we consider the case of $N_{\mathrm{obs}}=1$ for a while, and the case of multiple obstacles is discussed later in the next section. In this paper, the implicit surfaces (IS) are defined by the distance from the surface of objects to a certain point $z$ in 2D or 3D space, and the inside and outside of the object surfaces are interpreted through the sign of the function as follows.

where $z\in\mathbb{R}^{b}$, $b\in\{2,3\}$ is a position in a 2D or 3D space and $d_{\xi}:\mathbb{R}^{b}\rightarrow\mathbb{R}_{\geq 0}$ is the function that returns the minimum Euclidean distance between $z$ and surface of the obstacle corresponding to $\xi\sim p(\xi)$. Though this paper focuses on a 2D environment, our method can easily be extended to a 3D case. In GPIS, the function $h_{\mathrm{IS}}$ is estimated with the Gaussian process (GP), which enables us to deal with uncertainty arising from measurement noise and scarcity of data. The GP is estimated with a dataset $\{(z^{i},h_{\mathrm{IS}}(z^{i},\xi)+\varsigma_{i})\}_{i=1}^{n_{\mathrm{data}}}$, where $\varsigma_{i}$ is bounded $\sigma$-subgaussian noise and $n_{\mathrm{data}}$ is the number of data.

In the following, we explain how to construct a dataset and apply Bayesian meta-learning to the training of $h_{\mathrm{IS}}$.

Given system (1), meta-learned parameters $(w,\bar{\theta}_{0},\Lambda_{0})$, and CLF $V$ that encodes a control objective, the procedures in the online control execution is as summarized in Algorithm 1. Before the execution, an environment is determined by sampling the obstacles $\xi_{i},\ \forall i\in\mathbb{N}_{1:N_{\mathrm{obs}}}$ from the distribution $p(\xi)$ and the data buffers $\mathcal{B}_{i}$ for $\xi_{i},\ \forall i\in\mathbb{N}_{1:N_{\mathrm{obs}}}$ are initialized by empty sets (Line 1). Then, the control is executed by repeatedly solving the QP (5) with the CBFs (17) defined using the current posteriors $\bar{\theta}_{\xi_{i}}$ and $\Lambda_{\xi_{i}}$ calculated by the data in the current buffers (Line 13,14). The buffers $\mathcal{B}_{i}$ and the posterior parameters $\bar{\theta}_{\xi_{i}}$ and $\Lambda_{\xi_{i}}$ $\forall i\in\mathbb{N}_{1:N_{\mathrm{obs}}}$ are updated at every $\Delta_{\mathrm{lidar}}\in\mathbb{R}_{>0}$ [s] through the procedures in the following subsection (Line 3-11). Since each buffer $\mathcal{B}_{i}$ is empty at the beginning of the execution, we ignore the CBF constraint for $\xi_{i}$ until surface points of the corresponding obstacle are detected. To justify this, we make the following assumption.

In the following subsections, we discuss the update scheme of each buffer $\mathcal{B}_{i}$ and the theoretical results of the proposed control scheme.

We test the proposed approach through a simulation of vehicle navigation in a 2D space. All the experiments are conducted on Python running on a Windows 10 with a 2.80 GHz Core i7 CPU and 32 GB of RAM. TensorFlow and PyBullet [38] are used for the meta-learning of the IS function and implementation of the vehicle system with LiDAR scanners, respectively. We define the controlled system by the unmanned vehicle that has the following dynamics:

where $q_{x}$, $q_{y}$, and $\vartheta$ are $x-y$ coordinates of the robot position and heading angle of the robot, $v$ and $\omega$ are the velocity and angular velocity of the vehicle, respectively. The control inputs and system states are defined as $u=[v,\omega]^{\top}\in\mathbb{R}^{2}$ and $x=[q_{x},q_{y},\vartheta]\in\mathbb{R}^{3}$, respectively. Note that in (18), we consider the dynamics of a point off the axis of the robot by a distance $\ell\ (>0)$ to make the system relative degree one [40]. Moreover, the LiDAR scanner which has a 360-degree field of view and radiates 150 rays with a 3-meter length is assumed to be mounted on the vehicle.

In this paper, we proposed an efficient and probabilistically ensured online learning method of CBF with sensor measurements. In the proposed method, we utilized a technique based on the GPIS to train unsafe regions from the measurements. Specifically, a Bayesian meta-learning scheme was employed to learn IS function, which enables us to effectively use past data from different settings. To avoid the volume of data buffers becoming unnecessarily large in the online control execution, we also considered the data selection scheme which effectively uses the uncertainty information provided by the current model. The prediction made by the proposed scheme was readily incorporated in the CBF-CLF-QP formulation and the probabilistic safety guarantee for the control scheme was also provided. In the case study, we have shown the efficacy of our method in terms of the conservativeness of the prediction, the time required for the online training, and the conservativeness of the control performance.