An adaptive model hierarchy for data-augmented training of kernel models for reactive flow

We are interested in constructing efficient and accurate models to approximate time-dependent quantities of interest (QoI) $f\in L^{2}\big{(}\mathcal{P};L^{2}([0,T])\big{)}$ in the context of reactive flow, with $T>0$ and where $\mathcal{P}\subset\mathbb{R}^{p}$ for $p\geq 1$ denotes the set of possible input parameters. As a class of QoI functions, we consider those obtained by applying linear functionals $s_{\mu}\in V^{\prime}$ to solution trajectories $c_{\mu}\in L^{2}(0,T;V)$ of, e.g., parametric parabolic partial differential equations. Thus, $f(\mu;t):=s_{\mu}(c_{\mu}(t))$, where for each parameter $\mu\in\mathcal{P}$, the concentration $c_{\mu}$ with $\partial_{t}c_{\mu}\in L^{2}(0,T;V^{\prime})$ and initial condition $c_{0}\in V$ is the unique weak solution of

Here, $V\subset H^{1}(\Omega)\subset L^{2}(\Omega)\subset V^{\prime}$ denotes a Gelfand triple of Hilbert-spaces associated with a spatial Lipschitz-domain $\Omega$ and, for $\mu\in\mathcal{P}$, $l_{\mu}\in V^{\prime}$ denotes a continuous linear functional and $a_{\mu}:V\times V\to\mathbb{R}$ a continuous coercive bilinear form.

As a basic model for reactive flow in catalytic filters, (1) could stem from a single-phase one-dimensional linear advection-diffusion-reaction problem with Dammköhler- and Péclet-numbers as input (thus $p=2$), where $c$ models the dimensionless molar concentration of a species and the break-through curve $s$ measures the concentration at the outflow, as detailed in Gavrilenko et al. (2022).

Since direct evaluations of $f$ are not available, we resort to a full order model (FOM) as reference model, yielding

which we assume to be a sufficiently accurate approximation of the QoI. For simplicity, we consider a $P^{1}$-conforming Finite Element space $V_{h}\subset V$ and obtain the FOM solution trajectory $c_{h,\mu}\in L^{2}(0,T;V_{h})$ by Galerkin projection of (1) onto $V_{h}$ and an implicit Euler approximation of the temporal derivative.

The evaluation of (2) may be arbitrarily costly, in particular in multi- or large-scale scenarios where $\dim V_{h}\gg 1$, but also if $N_{T}\gg 1$ due to long-time integration or when a high resolution of $f_{h}$ is required. We thus seek to build a machine learning (ML) based surrogate model

to predict all values $\{f_{\text{ml}}(\mu;t_{n})\}_{n=1}^{N_{T}}$ at once, without time-integration. Such models based on Neural Networks or Kernels typically rely on a large amount of training data

rendering their training prohibitively expensive in the aforementioned scenarios; we refer to Gavrilenko et al. (2022) and the references therein and in particular to Santin and Haasdonk (2021). In Gavrilenko et al. (2022) we thus seek to employ an intermediate surrogate to generate sufficient training data.

Keil et al. (2021) introduced an approach beyond the classical offline/online splitting where a RB ROM is adaptively enriched based on rigorous a posteriori error estimates, following the path of an optimization procedure through the parameter space. Similarly, we propose an adaptive enrichment yielding a hierarchy of FOM, RB ROM and ML models, based on the standard residual-based a posteriori estimate on the RB output error, $\|f_{h}(\mu)-f_{\text{rb}}(\mu)\|_{L^{2}([0,T]))}\leq\Delta_{\text{rb}}(\mu)$, for which we refer to the references in Milk et al. (2016).

As a means to judge if a ML model is trustworthy, we propose a manual validation using the following a posteriori error estimate on the ML QoI error. While not as cheaply computable as $f_{\text{ml}}$, it still allows to validate the ML model without computing $f_{h}$.

Applying Algorithm 1 to the example of one-dimensional single-phase reactive flow from the last row of Table 1 in Gavrilenko et al. (2022), with $\dim\mathcal{P}=2$, $N_{T}=24576$ time steps, $\dim V_{h}=65537$ gives the behaviour shown in Figure 1, where we set $\varepsilon=10^{-2}$, retrain the ML model every 10 collected samples and unconditionally trust the ML model as soon as $|X_{\text{train}}|\geq 50$.¹¹1The experiments were performed using pyMOR from Milk et al. (2016) and dune-gdt from https://docs.dune-gdt.org/. For the considered diffusion dominated regime, we only require a single evaluation of $f_{h}$ (yielding a $\dim V_{\text{rb}}=15$-dimensional RB ROM), which results in even further computational savings, compared to the results obtained in Gavrilenko et al. (2022).