Input-to-State Stable Neural Ordinary Differential Equations with Applications to Transient Modeling of Circuits

We consider the problem of learning input-to-state stable (ISS) dynamics from observed input and output trajectories. Stability constraints provide an inductive bias that can help a dynamics learning algorithm choose one model over another and may improve the generalization accuracy of the dynamical system model when it is simulated in novel interconnection configurations. In some cases, imposing a stability constraint can guide the learning algorithm and improve performance.

This work was motivated by a need for fast-to-simulate behavioral models of electronic circuits. A system-on-a-chip (SoC), or larger microelectronic system, is composed of multiple functional blocks, often referred to as IP blocks, where IP denotes intellectual property. Prior to manufacturing, simulation is used to verify system performance. Each IP block may be represented by its transistor-level netlist, which specifies the interconnection of transistors that comprise its nonlinear dynamics. Transient simulation of the complete model of the non-linear system can be prohibitively slow, so it is preferable to replace the transistor-level description of each IP block with a cheaper-to-simulate behavioral model.

Transient circuit simulation presents a challenge for dynamics learning. At test time, an adaptive timestep solver is used to simulate the learned dynamics model, which is usually interconnected (in feedback) with external circuits and other IP blocks (Hajj2016). We refer to these external elements collectively as the load. In order to be useful, the model must be accurate when interconnected with a variety of loads, and those loads are generally not known a priori. Due to uncertainty over the load, a model that performs well in training and validation may fail when embedded in a circuit simulation at test time.

In this work, we assume that the circuit of interest is well-modeled by a system of ODEs. A natural approach is to directly learn a parametric system of controlled ODEs

which has state $x\in\mathbb{R}^{n}$, input $u\in\mathbb{R}^{m}$, and output $y\in\mathbb{R}^{p}$. $u$ and $y$ consist of node voltages and currents, and possibly their time derivatives.

In this work, we directly learn a neural ODE model of the form \eqrefeq:node1 – \eqrefeq:node2, which may be trained by either directly backpropagating through an ODE solver or implicitly differentiating through the solution to the ODEs using an adjoint method (ChenRBD2018). Our models are trained using interpolated trajectories of $u(t)$ and $y(t)$ obtained from a circuit simulator; this approach is similar to prior works on neural ODE models of physical systems with continuous inputs (KidgerMFL2020; ZhongDC2020a).

We focus on the case where $f$ takes the form of a continuous-time recurrent neural network (CTRNN) and $h$ is an affine function of the state. Models of this form are universal approximators on finite time intervals (FunahashiN1993) and, in some cases, infinite time intervals (HansonR2020). In addition, a CTRNN may be implemented as a generic circuit block using the Verilog-A behavioral modeling language and subsequently simulated by commercial circuit simulators (ChenRR2017).

In Section 2, we propose a CTRNN parametrization that is guaranteed to be ISS with respect to a Lur’e-Postnikov type (quadratic plus integral) ISS-Lyapunov function $V(x)$, which has parameters that are learned jointly with the CTRNN model parameters. ISS is a natural a priori assumption for many circuits; the state is guaranteed to be bounded given bounded inputs, and the state converges to a unique equilibrium if the input is set to zero.

There has been recent interest in learning neural ODEs jointly with a Lyapunov-like function $V(x)$. For autonomous systems, a sufficient condition for global asymptotic stability (GAS) is to ensure that $V$ is strictly decreasing along any system trajectory, i.e.,

RichardsBK2018 encourage \eqrefeq:dissipation_gas via regularization, but do not guarantee that the dissipation inequality holds everywhere. ManekK2019 and MassaroliPBPYA2020 define the model dynamics as a function of $V(x)$ such that \eqrefeq:dissipation_gas holds for all $x$. CranmerGHBSH2020 and ZhongDC2020a considered the related problem of learning dynamics with Lagrangian and Hamiltonian structure, respectively.

Our approach is similar to that of ManekK2019 in the sense that we guarantee that a dissipation inequality on $V$ holds everywhere, although we consider ISS, which can be seen as a generalization of GAS to systems with inputs. We use a stability condition that generalizes the ISS condition derived by Ahn2011, which is based on a quadratic ISS-Lyapunov function. In another related work, cao2006state first learn an unconstrained circuit model and subsequently stabilize the model using nonlinear constrained optimization. In contrast, we build the stability constraint directly into the model parametrization.

Besides providing stability guarantees, we observed that our proposed model parametrization can accelerate training convergence. In this sense, it is related to prior works on regularization methods for accelerating neural ODE training. For example, FinlayJNO2020 penalized the complexity of the model dynamics while KellyBJD2020 penalized the forward ODE solution time. Unlike those methods, our stability constraint does not introduce additional penalty terms, which can be difficult to tune.

We also show that our ISS parametrization is directly compatible with aging-aware circuit modeling. The dynamics of a circuit drift over time due to semiconductor degradation. Aged dynamics, estimated using physics-based approaches (tu1993berkeley), can be used to verify lifetime specifications and identify aging-induced failures. rosenbaum2020machine directly learn an aging-aware circuit model by choosing the dynamics $f$ in \eqrefeq:node1 and output map $h$ in \eqrefeq:node2 to themselves be learned functions of a periodic stress waveform $u_{\text{stress}}$, which is assumed to have been applied to the circuit continuously for an operating time $T_{\text{op}}$ on the order of years. Aging analysis can greatly benefit from fast-to-simulate surrogate models since separate aging simulations are needed to characterize different possible use condition profiles, each of which is specified by a pair ($u_{\text{stress}}$, $T_{\text{op}}$).

Section 2 presents our ISS-constrained model and describes how it can be used for transient circuit simulation, with and without aging effects. Section LABEL:sec:experiments evaluates the proposed methods on a variety of circuit modeling tasks.