Neural Operators for Adaptive Control of Freeway Traffic

The control strategy for freeway traffic congestion is usually based on static road infrastructure to regulate traffic flow, such as ramp metering and varying speed limits. Various traffic boundary control designs have been proposed to smooth traffic in the works of Bekiaris-Liberis and Delis [7], Zhang [8] as well as Karafyllis, Bekiaris-Liberis, and Papageorgiou [9]. While Bekiaris-Liberis and Delis utilize Adaptive Cruise Control vehicles for in-domain actuation as control inputs [7], Karafyllis et al. design a boundary feedback law to manage inlet demand [9]. The boundary control strategy using PDE backstepping is first proposed in [10] to stabilize the linearized ARZ system, including full state feedback and output feedback. Recent efforts [11, 12, 13, 14, 15, 16] have further developed backstepping controllers for various traffic scenarios including multi-lane, multi-class and mixed-autonomy traffic. This paper primarily focuses on adaptive control of traffic PDE systems with uncertain parameters.

In traffic flow modeling, relaxation time is a critical parameter representing drivers’ reaction delays to evolving traffic conditions. However, heterogeneity and unpredictability of individual driver behavior makes it impossible to obtain the relaxation time in practice. This uncertainty in relaxation time can significantly impact the stability and performance of traffic systems. Traditional control methods struggle to handle such uncertainties, making it difficult to ensure system stability and optimal performance under varying traffic conditions. To address these challenges, we adopt adaptive control strategies that allow for real-time adjustment of the controller gains to accommodate unknown or time-varying system characteristics, ensuring the desired system performance.

Adaptive control for PDEs first appears in the late 2000s, primarily focused on parabolic PDEs [17, 18]. Adaptive control methods [19, 20, 21] can be categorized into Lyapunov-based design, identifier-based design and swapping-based design. After a decade of research, advancements in adaptive control have begun to be applied to coupled hyperbolic PDEs [22].

Although adaptive control for PDE systems with unknown parameters has been extensively studied [23, 24, 25, 26, 27, 28] and was first applied for the ARZ PDE model in [12]. The practical implementation of the adaptive controller for the traffic systems still faces challenge. This is because the adaptive control process simultaneously requires the estimation of unknown system parameters and PDE states. After each time step, it is necessary to recalculate the solution to the PDE corresponding to the gain kernel function in order to update the estimated system parameter functions. This places extremely high demands on real-time computation. The computational resources required for calculation of the gain function increase significantly with spatial sampling precision when applying traditional finite difference and finite element methods. In this paper, we adopt neural operators to accelerate computation of adaptive PDE backstepping controllers.

With rapid advances in machine learning, data-driven methods for solving, modeling and control of PDEs have received widespread attention including physics-informed learning, reinforcement learning and operator learning. Physics-Informed Neural Networks (PINNs) directly incorporates physical constraints into neural networks training by embedding the physical laws of PDEs into the loss function. This enables PINNs to solve PDEs without large amounts of training data. Mowlavi and Nabi extend PINNs method to PDE optimal control problems in [29]. Zhao proposed a novel hybrid Traffic state estimation (TSE) approach called Observer-Informed Deep Learning (OIDL), which integrates a PDE observer and deep learning paradigm to estimate spatial-temporal traffic states from boundary sensing data in [30]. However, PINNs need to be retrained for each new set of boundary and initial conditions, which poses limitations in adaptive control applications.

Reinforcement learning (RL) has also been increasingly applied for PDE control problems, particularly in boundary and feedback control. RL continuously optimizes strategies to achieve real-time control of complex PDE systems. In the field of traffic management, researchers have been applying RL to various traffic issues. Wu et al. used the city mobility traffic micro-simulator SUMO to design a deep RL framework for hybrid autonomous traffic in various experimental scenarios [31]. Under the same framework, [32] proposed a reinforcement learning-based car-following model for electric, connected, and automated vehicles to reduce traffic oscillations and improve energy efficiency. [33] presented the exploration using RL for traffic PDE boundary control. However, RL has limited generalization ability in practical applications. RL may perform well under the specific initial conditions. However, for initial conditions outside the training range, there may be performance degradation or even failure. RL may be sensitive to hyperparameters and exhibit unpredictable behavior, making it difficult to ensure consistent and stable performance in different scenarios.

Traditional neural networks typically learn mappings between finite dimensional Euclidean spaces, but with the advancement of research, this method has been extended to the field of NO [34]. NO-based learning methods focus on mapping between function spaces and are specifically designed for solving PDEs and dynamical systems. Compared with traditional machine learning methods, NO have two unique advantages. Firstly, theoretically speaking, NO can learn the mapping of the entire system parameter set, rather than being limited to a single system parameter like standard neural networks. Secondly, from an empirical perspective, research work [34][35] has shown that NO have significantly better accuracy than traditional deep learning methods when simulating complex functions. Therefore, NO not only solves individual equation instances, but can also handle the problems of the entire PDE family.

Recent research has effectively utilized DeepONet for one-dimensional transport PDEs[36] and reaction-diffusion equations and observer designs[37]. These studies establish the stability of PDEs under approximate kernels by employing the general operator approximation theorem. Subsequent developments focus on hyperbolic PDEs[38], parabolic PDEs with delays[39], and the ARZ PDE system for traffic flow control[40]. NO are applied for gain scheduling to enable real-time control of nonlinear PDEs[41]. This framework integrates PDE backstepping and DeepONet methods for offline learning, ensuring closed-loop stability and enhancing the accuracy in managing more complex PDEs. The application of NO-approximated gain kernels becomes even more valuable for adaptive control, where the kernel must be recomputed online at each time step to accommodate updated estimates of the plant parameters. This was first explored for first-order hyperbolic PDE in [42] and extended to the reaction-diffusion equation in [43]. Different from [42][43], where the kernel equation involves a single kernel, in this work, we extend the results of [42] to the ARZ traffic models which involved the coupled heterogeneous hyperbolic PDEs. The technical challenges arise from both the more complex kernel computations and the proof analysis of the higher-order PDE systems with the approximated controllers.

Contributions: The main contributions are summarized as follows:

• $\bullet$

  We present an NO-based adaptive control method to stabilize the ARZ traffic PDE model with unknown relaxation time. Additionally, we extend stability schemes for more general 2$\times$2 hyperbolic systems with uncertain spatially varying in-domain coefficients and boundary parameter. Compared to the relevant works [42][43], which approximate single kernel, a key technical challenge is dealing with the approximation of coupled 2$\times$2 Goursat-form PDE kernels in the stabilization of coupled 2$\times$2 hyperbolic PDEs.

• $\bullet$

  To address the computational challenges associated with solving gain kernel equations, we integrate DeepONet into the adaptive control framework. It is shown that the NO is almost two orders of magnitude faster than the PDE solver in solving kernel functions, and the loss remains on the order of $10^{-3}$. To the best of our knowledge, this is the first study to integrate DeepONet with adaptive control in traffic flow systems, demonstrating its potential to improve the computational efficiency of control schemes in congested traffic scenarios.

• $\bullet$

  Through comparative experiments with RL, it has been proven that our method does not rely on initial values compared to RL and provides a model-based solution with guaranteed stability. In addition, we theoretically prove the system’s stability through Lyapunov analysis when replacing with the DeepONet approximation kernels in the adaptive controller.

Organization of paper:The paper is organized as follows. Section 2 introduces ARZ traffic PDE model and a nominal adaptive backstepping control scheme designed for 2$\times$2 hyperbolic PDEs. Section 3 gives a series of properties for the gain kernel and its time derivative and introduces the approximation of feedback kernel operators. Section 4 presents the stabilization achieved through the application of approximate controller gain functions via DeepONet. Numerical simulations are presented in Section 5. Section 6 presents the conclusion.

The ARZ PDE model is used to describe the formation and dynamics of the traffic oscillations which refer to variations of traffic density and speed around equlibrium values. It consists of a set of 2$\times$2 hyperbolic PDEs for traffic density and velocity. The ARZ model of $(\rho_{1}(x,t),v_{1}(x,t))$-system is given by

where $(x,t)\in[0,L]\times\mathbb{R}_{+}$, $\rho_{1}(x,t)$ represents the traffic density, $v_{1}(x,t)$ represents the traffic speed, and $\tau$ denotes the relaxation time, which refers to the time required for driver behavior to adapt to equilibrium. This parameter is used to describe the process by which vehicle speed adjusts to match the traffic density. The variable $p_{1}(\rho_{1})$, defined as the traffic system pressure, is related to the density by the equation

and $c_{0},\gamma\in\mathbb{R}_{+}$. The equilibrium velocity-density relationship $V(\rho_{1})$ is given in Greenshield model:

where $v_{f}$ is the free flow velocity, $\rho_{m}$ is the maximum density, and $(\rho_{1}^{\star},v_{1}^{\star})$ are the equilibrium points of the system with $v_{1}^{\star}=V(\rho_{1}^{\star})$. We consider a constant traffic flux $q_{1}^{\star}=\rho_{1}^{\star}V(\rho_{1}^{\star})$ entering the domain from $x=0$. We can apply the change of coordinates introduced in [14] to rewrite it in the Riemann coordinates and then map it to a decoupled first-order 2$\times$2 hyperbolic system.

where

The relaxation time $\tau$ describes how fast drivers adapt their speed to equilibrium speed-density relations. Its value is usually difficult to measure in practice and is easily affected by various external factors. Therefore, we propose adaptive control law. Motivated by the second-order ARZ model, we first propose NO-based adaptive design for a more general framework of 2$\times$2 hyperbolic PDEs with spatially varying coefficients, as the linearized ARZ model is a special case of such systems.

We consider the first-order coupled 2$\times$2 hyperbolic PDE system with four spatially variable coefficients,

where $t\in\mathbb{R}_{+}$ is the time, $x\in[0,1]$ is the space, the states are given by $u,v$ and the initial conditions are $u(x,0)=u_{0}(x)$, $v(x,0)=v_{0}(x)$ where

The transport speeds

are known, while the spatially variable coefficients and boundary coefficient

are unknown.

Note that system (16)-(19) is 2$\times$2 hyperbolic system with spatially variable coefficients in domain, which is different from the system in [25] with the constant coefficients. System (16)-(19) is a direct extension of the system in [25], where the difference lies in the designed adaptive update law.

To ensure the well-posedness of the kernel PDEs, the adaptive control estimation requires bounded assumptions. Our basic assumption is as follows.

We first propose an adaptive control design using passive identifier design method, which includes the exact estimated backstepping kernels $\breve{K}^{u},\breve{K}^{v}$.

We consider the identifier

where

are errors between $u$ and $v$ and their estimates $\hat{u}$ and $\hat{v}$ , $\hat{c}_{i}$ and $\hat{r}$ are estimates $c_{i}$ and $r$. We define

for some initial conditions

The error signals (29) can straightforwardly be shown to have dynamics

where $\tilde{c}_{i}=c_{i}-\hat{c}_{i},i=1,\cdots,4$, $\tilde{r}=r-\hat{r}$. We choose the following update laws

where $\rho,\gamma,\gamma_{1},\gamma_{2},\gamma_{3},\gamma_{4},\gamma_{5}>0$ are scalar design gains. The adaptive law (35)-(39) have the following properties for all $t>0$ [44]

where

Considering the plant (16)-(19) with unknown parameters $c_{i},i=1,\cdots,4$ and $r$, we will design a nominal adaptive control law to achieve global stability.

We consider the following adaptive backstepping transformation

where the kernels $\breve{K}^{u}$ and $\breve{K}^{v}$ satisfy the following kernel functions

The coupled $2\times 2$ Goursat-form PDEs, governed by two gain kernels, are defined over the triangular domain $\mathcal{T}_{1}$, given by:

Using the transformation (48) and (49), we get the following target system

where the coefficient $\omega$ and $\kappa$ are chosen to satisfy

From the boundary conditions (19), (49) and (58), the nominal stabilizing controller is straightforwardly derived as follows [25]

Next, we present the stability of exact adaptive backstepping control, which serves as a guide for what we aim to achieve under the NO-based approximate adaptive backstepping design.

In summary, the exact adaptive backstepping feedback law (61) can achieve global stability at the equilibrium point, with the system states $u(x,t),v(x,t)$ converging pointwise to zero. However, this method is computationally intensive because it requires solving Volterra equations (2.2)-(53) at each time step t. To simplify the computation, we propose using the NO $\hat{\mathcal{K}}:(\hat{c}_{1},\hat{c}_{2},\hat{c}_{3},\hat{c}_{4},\hat{r})\mapsto(\hat{K}^{u},\hat{K}^{v})$ to approximate the exact adaptive backstepping gain operator $\mathcal{K}:(\hat{c}_{1},\hat{c}_{2},\hat{c}_{3},\hat{c}_{4},\hat{r})\mapsto(\breve{K}^{u},\breve{K}^{v})$. This approach allows for neural network evaluation at each time step instead of solving the complex equations. In the following section, we will introduce how to approximate the operator $\mathcal{K}$ using DeepONet and use the resulting approximate control gain functions for boundary stabilization of plant (16)-(19). And we will use the universal approximation theorem of DeepONet [45] to derive the stability theorem from control gain kernel approximations.

In the following discussion, we will first introduce the universal approximation theorem of DeepONet. This theorem demonstrates DeepONet’s ability to approximate operators, enabling us to use it to learn the backstepping gain kernel mapping of PDEs and theoretically guarantee the stability of the adaptive control system.

Figure 1 presents a schematic diagram of the control circuit, illustrating the utilization of neural operators to accelerate the generation process of gain kernel functions in PDE adaptive control.

We use the deepxde package to train DeepONet. As shown in Figure 2, we constructed a branch network consisting of two layers of convolutional neural networks and two layers of fully connected networks, and a trunk network consisting of two layers of fully connected networks. This framework contains a total of 32,094,465 parameters. $n_{x}$ represents the number of discrete lattices in the $x$ direction. This framework consists of a branch network and a trunk network, encoding the input function space and output function domain respectively. Firstly, based on the kernel functions (2.2)-(53), multiple input-output data pairs are calculated offline using a traditional PDE numerical solver. Subsequently, the computed kernel functions are used as training data for training the neural network. Through the combination of multi-layer neural networks, DeepONet learns the hidden nonlinear relationships in the system, namely the mapping relationship between system parameters and kernel functions. During the training process, DeepONet continuously adjusts the network parameters through the backpropagation algorithm to minimize the error between the predicted output and the true kernels.

Next, we define the operator $\mathcal{K}:(C^{0}([0,1]))^{4}\times\mathbb{R}\mapsto(C^{1}(\mathcal{T}))^{2}$