Data-driven modelling of nonlinear spatio-temporal fluid flows using a deep convolutional generative adversarial network

Numerical simulations of fluid dynamics have been indispensable in many applications relevant to physics and engineering. For improving predictive capability, numerical algorithms have become increasingly sophisticated and complicated by using more spatial and temporal resolution of datasets. This often leads to very large-scale models and unavoidably raises computational challenges. In this work, an Artificial Intelligence (AI) fluid flow model is presented for rapidly providing spatio-temporal prediction of nonlinear fluid flows while avoiding the excessive computational cost.

Recently, deep learning has gained popularity in rapidly predicting nonlinear fluid flows (Kalteh, 2013; Kasiviswanathan et al., 2016; Kisi et al., 2012; Lohani et al., 2014; Yang et al., 2017; Wang et al., 2018; Brunton et al., 2019). Deep learning techniques are capable to handle the large-scale data from the dynamic and nonlinear systems, and provide accurate nonlinear fluid flow prediction without prior knowledge of underlying physical relationships. Various deep learning techniques have been applied to nonlinear fluid flow prediction, for example, modifying the model parameters by Bayesian deep neural networks to enhance the flow prediction in the turbulence model (Geneva and Zabaras, 2019), obtaining the time-series streamflow prediction by Long Short-Term Memory (LSTM) networks (Hu et al., 2019a), and simulating unsteady wake flows by convolutional neural network (CNN) -based deep learning (Miyanawala and Jaiman, 2017).

More recently, Generative Adversarial Networks (GANs) have been introduced to reconstruction of fluid dynamic features (Farimani et al., 2017; Xie et al., 2018). GANs represent the most recent progress in deep learning techniques (Goodfellow, 2016; Laloy et al., 2017; Ströfer et al., 2018) and have been attractive for producing high-resolution samples (e.g. images) (Ledig et al., 2017; Xie et al., 2018; Creswell et al., 2018). The main specificity of GANs is the competition game between the generator and discriminator modules. The generator consists of deep neural networks, which can produce high-dimensional distributions of data to “fool” the discriminator. The discriminator is responsible for distinguishing the real data from the generated data. Nowadays applications of GANs focus mostly on images (Reed et al., 2016; Li and Wand, 2016; Isola et al., 2017), rarely on fluid flow problems (Farimani et al., 2017; Xie et al., 2018). Farimani et al. (2017) first introduced the conditional generative adversarial networks (cGAN) to simulate steady state heat conduction and incompressible fluid flow. This study proves that GANs can be effective in exhibiting complex nonlinear fluid structures. Xie et al. (2018) introduced a temporal discriminator to GANs for solving the super-resolution problem for smoke flows.

In this work, we have developed a deep convolutional generative adversarial network (DCGAN) (Radford et al., 2015) for nonlinear unsteady fluid flow problems. This is the first attempt to use the DCGAN for predictive analysis in fluid flow problems. As is well known, both spatially and temporally flow dynamic properties are important in prediction. Classic machine learning approaches exploit dependencies in time or space, rarely both spatio-temporal dependencies (Reichstein et al., 2019). Here, the DCGAN is first used to extract the spatio-temporal coherence of the original high-dimensional fluid fields. The high-dimensional nonlinear fluid flow features can be converted into low-dimensional “latent” representations. With discriminating the real fluid fields and their realizations in the low-dimensional latent space, the training process herein is fast. This alleviates the problem of large data-driven modelling. The way of dimensionality reduction in the deep convolutional networks is similar to that of Galerkin reduced order model (ROM) (Fang et al., 2013, 2014; Xiao et al., 2019), Variational Autoencoder (VAE) (Gonzalez and Balajewicz, 2018) and cluster-based reduced order model (CROM) (Kaiser et al., 2014; Nair et al., 2019; Östh et al., 2015). These ROMs are able to compress spatial-temporal flow snapshots into a small number of low-dimensional states for dynamic modelling, which is also a strategy to reduce computational requirements of ROMs (Kaiser et al., 2014). Nevertheless, the prominent difference between the DCGAN and existing ROMs lies in the fact that the CROM or POD based ROMs result in linear combinations of the original snapshots (Kaiser et al., 2014; Xiao et al., 2016), while the deep convolutional neural networks are able to capture the nonlinear flow transients since the effective basis functions are non-linear (Miyanawala and Jaiman, 2017). In addition, GAN also has the key advantage that, in principle, any point in the latent space of inputs will be associated with a realistic looking solution.

The DCGAN is a general and efficient numerical tool for rapidly predicting nonlinear fluid flows for given inputs. In this work, we have successfully applied the DCGAN to a tsunami case in the southwestern Hokkaido Island, Japan. The performance of the DCGAN has been evaluated by comparing the results obtained with those from an unstructured mesh finite element model (Fluidity) (Pain et al., 2001).

The structure of this article is organized as follows. Section 2 presents the governing equation for nonlinear fluid flows in dynamic systems. The architecture of the DCGAN is briefly introduced in section 3. The detailed process for the large data-driven modelling is described in section 4. Section 5 illustrates the capability of the DCGAN in the tsunami case, which entails the snapshot-based datasets generation with the different wave characteristics and the DCGAN training. Results of cases and the model efficiency are then presented in section 6. Conclusions are drawn in section 7.

Nonlinear fluid flows are usually simulated by solving the complex non-linear equations (for example, Navier-Stokes (NS) equations, Lorenz equations and shallow water equations) in the computational fluid dynamic models. In a general way, the governing equation of dynamic flows can be expressed as:

where ${\phi}$ is the state variable to be predicted (for example, water depth and velocity), ${\mu}$ represents the inputs (for example, initial and boundary conditions, modelling parameters) , ${\bf x}=(x,y,z)$ denotes the spatial coordinate system, $s$ is the source term, and $t$ is the time.

This work focuses on the spatio-temporal fluid simulations that are driven by the inputs (for example, initial and boundary conditions, modelling parameters). The nonlinear relationship between the fluid flow solutions $\phi$ and the inputs $\mu$ in Eq.(1) can be re-written in a general way:

where ${\mathcal{P}}$ represents a dynamic model. In this paper, the dynamic model ${\mathcal{P}}$ will be represented by the DCGAN introduced in section 4. Based on a series of inputs ${\mu}$, the main objective of this work is to develop a new DCGAN tool for simulating the spatio-temporal distribution of nonlinear fluid flows ${\phi}$.

A generative adversarial network comprises a generator module (G) and a discriminator module (D), as shown in Fig. 1. The role of the generator is to map a random dataset ${\mu}$ (taking as inputs) into a desired output dataset ${\phi}$. In traditional GANs, the random dataset $\mu$ is typically a Gaussian or uniform noise ranging from zero to one. The discriminator is used for distinguishing the real datasets ${\phi_{d}}$ (the targeted outputs) from the generated outputs ${\phi}$. In the discriminator, ${D(\phi_{d})=1}$ if the targeted outputs are accepted while ${D(G(\mu))=0}$ if they are rejected.

In general, the training process in GAN can be considered as a minimization-maximization problem based on a cross entropy loss function ${\bf J}(D,G))$:

where ${p_{\mu}(\mu)}$ is a prior distribution for the random dataset ${\mu}$, and ${p_{data}(\phi_{d})}$ is the corresponding probability data distribution for the targeted outputs ${\phi_{d}}$.

In particular, the min-max process consists of two steps:

• 1.

  Given a fixed generator, updating the discriminator by maximizing the discriminator function $\mathop{\max}\limits_{\theta^{(D)}}{{\bf J}^{(D)}}\left({{\theta^{\left(D\right)}},{\theta^{\left(G\right)}}}\right)$ (where ${\theta^{\left(D\right)}}$ represents the parameters in the cost functions of the discriminator, and the function ${\bf J}^{(D)}$ is differentiable.).

  $${\bf{J}}^{(D)}\left({{\theta^{\left(D\right)}},{\theta^{\left(G\right)}}}\right)={E_{{\phi_{d}}\sim{p_{data}(\phi_{d})}}}[\log D({\phi_{d}},\theta^{(D)})]+{E_{{\mu}\sim{p_{\mu}(\mu)}}}[\log(1-D(G({\mu}),\theta^{(G)},\theta^{(D)})];$$

  (4)

• 2.

  Updating the generator by minimizing the generator function, i.e. $\mathop{\min}\limits_{\theta^{(G)}}{\bf J}^{(G)}\left({{\theta^{\left(D\right)}},{\theta^{\left(G\right)}}}\right)$ (where ${\theta^{\left(G\right)}}$ represents the parameter in the cost functions of the generator, and the function ${\bf J}^{(G)}$ is differentiable.)

  $${\textbf{J}^{(G)}}\left({{\theta^{\left(D\right)}},{\theta^{\left(G\right)}}}\right)={E_{\mu\sim{p_{\mu}(\mu)}}}[\log(1-D(G(\mu)))].$$

  (5)

Following Goodfellow et al. (2014), the min-max game has a global optimum for ${{p_{g}}(\phi)={p_{data}(\phi_{d})}}$ (where ${{p_{g}}}$ is a probability distribution of the generated output ${\phi}$ when ${\mu\sim{p_{\mu}(\mu)}}$), if $D$ and $G$ have enough capacity. The optimal discriminator for the ${\textbf{J}^{(G)}(G,D)}$ can be characterized by:

The minimax game in (3) can be rewritten as

When the ${{D^{*}}(\phi_{d})=\frac{1}{2}}$, the generator can use any random dataset ${\mu}$ to synthesis the generated outputs ${\phi}$, and the discriminator loses the ability to distinguish between the real dataset ${\phi_{d}}$ and generated ones ${\phi}$.

Gradient-based algorithms can be used for optimization of parameters $\theta^{(G)},\theta^{(D)}$ in the generator and discriminator. For details see sections below.

Details about model development for spatio-temporal fluid flow modelling, please see full paper.

In this section, two flooding examples are presented to illustrate the capabilities of the DCGAN in resolving nonlinear fluid flow problem governed by the NS equations. In these examples, the inputs are a series of incoming waves along the coastal line while the corresponding outputs are the flooding solutions (e.g. water depth).

To demonstrate the predictive capability of the DCGAN for dynamic fluid flows, it has been applied to a case of flooding in Hokkaido, Japan. The flooding happened in 1993, caused by the Hokkaido Nansei-Oki earthquake offshore of southwestern Hokkaido Island, Japan. Incoming waves along the coastal line have been identified as an important driver for flood hazards in coastal areas. The increased demand to improve flood prediction requires better understanding of uncertainties associated with flood hazards due to the variability of waves (Hu et al., 2018, 2019b). In our flooding study cases, the characteristics of waves including the wave phase and wave height were considered. The phase variation represents the lags in timing of simulated flows (Prakash et al., 2014; Chang and Tsai, 2016), and the wave height variation indicates the magnitude of the flow peak (Kasiviswanathan et al., 2016; Chang and Tsai, 2016).

To investigate the danger caused by the wave run-up and flooding, two series of incoming waves $(H^{(e)},e=(1,2))$ are given and included in the inputs ${\mu}$ in two study cases, where each series of waves contains 13 types of waves, as shown in Fig. 2.

In this work, a DCGAN has been proposed for nonlinear spatio-temporal fluid flow prediction. It is the first time that the DCGAN is applied to prediction of spatially-temporally-coherent and high-dimensional nonlinear fluid flows. This novel DCGAN enables rapid and reliable prediction and can be used for providing risk reduction strategies in emergencies.

The performance of the DCGAN has been illustrated in the case of Hokkaido tsunami in this paper. Two study cases are set up with two series of waves with varying wave phase and wave height. To estimate the accuracy of the DCGAN, a comparison of nonlinear flow results obtained by the DCGAN and the original high fidelity model has been carried out. It is shown that the DCGAN is able to capture the complex flow features and exhibits an overall good agreement with the original high fidelity model. Error analysis has also been performed for further evaluation of the DCGAN through RMSE and correlation coefficient. In comparison to the original high fidelity model, the DCGAN achieves a reasonable accuracy in the whole simulation, while the online CPU cost is reduced by five orders of magnitude.

Overall, a general DCGAN for spatio-temporal fluid flow modelling is presented in this paper. It will be highly promising to apply the above DCGAN to various modelling problems, for example, turbulence modelling, ocean modelling, air pollution, and urban flows. Future work will focus on optimization of uncertainties in models and explore a hybrid machine learning approach (auto-encoder and GAN) for improving the accuracy and efficiency in prediction for complex problems. In this work we focus on the simulation of spatio-temporal distribution during a fixed simulation period for given varied boundary conditions, where the training and validation simulation periods are the same. In future, we will also focus on the real-time forecast– using the DCGAN for the lead-time prediction of spatio-temporal fluid flows. In addition, for realistic applications, a large number of snapshots (historical datasets), over a long time period, are needed for model training. The Classical Multidimensional Scaling (CMDS) proposed by (Kaiser et al., 2014, 2017) can be an efficient method to address this compression issue, which is able to convert a large set of observations into a set of low-dimensional states for visualization. Our future work will further explore the use of other compression methods (e.g. CMDS) along with the GAN to handle high dimensional data for real-time prediction in realistic applications.

The Authors acknowledge the support of: China Scholarship Council (No. 201806270238), funding from the EPSRC (MAGIC (EP/N010221/1), INHALE (EP/T003189/1), MUFFINS (EP/P033148/1) and PREMIERE (EP/T000414/1)), and the Royal Society (IEC/ NSFC/170563) in the UK.