Discovering Phase Field Models from Image Data with the Pseudo-spectral Physics Informed Neural Networks

The phase field models have been widely appreciated for investigating moving interface or multiphase problems. Among them, the well-known models are the Allen-Cahn equation (AC) and the Cahn-Hilliard (CH) equation [3]. In general, most existing phase field models are thermodynamically consistent, i.e., they satisfy the first and second thermodynamic laws. In particular, for most scenarios, the temperature is assumed constant, such that the Helmholtz free energy is non-increasing in time. Therefore, Denoting the state variable as $\phi$ and the free energy as $E$, the phase field models can usually be written in a generic gradient flow form

with proper initial values and boundary conditions. Here $\mathcal{G}$ is a semi negative-definite operator known as the mobility operator. In general, the free energy $E$ contains a conformation entropy and a bulk potential part. In this paper, we consider the free energy

with $\varepsilon$ a model parameter. We use $f(\phi)=F^{\prime}(\phi)$ to denote the bulk chemical potential. For the Allen-Cahn equation, the mobility is usually chosen as $\mathcal{G}=-M$, with $M>0$ a constant. Then, the Allen-Cahn equation reads

Similarly, for the Cahn-Hilliard equation, the mobility is usually chosen as $\mathcal{G}=M\Delta$, with $M>0$ a constant. Then the Cahn-Hilliard equation reads as

Here the choice of the bulk potential $F(\phi)$ will depend on the material properties, and the double-well potential and Flory-Huggins free energy are the common choices.

Though there is a considerable amount of work on solving the phase field models [19, 18, 8, 6, 22, 7], there is less achievement on solving the inverse problem, i.e., discovering the phase field models from data. In the classical approach, the PDE models are usually derived based on empirical observation. The PDE models with free parameters are introduced first, and the free parameters will be fitted with data afterward. In this paper, we introduce a novel method using the physics informed neural network approach, where the model would be directly learned from data. In particular, we assume the bulk potential $F(\phi)$ is unknown, and we will discover it via learning from the existing data using the deep neural network.

The deep neural network has been widely used to investigate problems in various fields. Mathematically, the feed-forward neural network could be defined as compositions of nonlinear functions. Given an input $x\in\mathbb{R}^{n_{1}}$, and denote the output of the $l$-th layer as $a^{[l]}\in\mathbb{R}^{n_{l}}$, which is the input for $l+1$-th layer. In general, we can define the neural network as [10]

where $W^{[k]}\in\mathbb{R}^{n_{k}\times n_{k-1}}$ and $b^{[k]}\in\mathbb{R}^{n_{k}}$ denote the weights and biases at layer $k$ respectively, $\sigma$ denotes the activation function. Many interesting work have been published in terms of solving or discovery of partial differential equations [16, 13, 15, 14, 2, 20, 9, 5, 1, 11, 23]. Here we briefly discuss some relevant work of PDE discovery with deep neural network or machine learning method. In [16], the authors assume the PDE model is a linear combination of a few known terms, and try to fit the coefficient from data. Based on the format of known data, they propose two approaches for calculating the loss function. In [17], the authors assume the PDE model is a linear combination of terms in a predefined dictionary (with many terms), and use the sparse regularization to coefficients are zero. Thus, after fitting with the data, only a few non-zero terms will be left, and the model is discovered. In [12], the authors introduce a feed-forward deep neural network to predict the PDE dynamics and uncover the underlying PDE model (in a black box form) simultaneously.

However, in most of the existing work, there is strong assumption on the availability of data. For instance, most existing PDE discovery strategies request a well sampled data across time and spatial domain, and they usually assume the data collected at different time slots are from a single time sequence. In reality, especially from lab experiments, the data are usually sampled as snapshots (images) from various time slots, and the images might have limited spatial resolution, i.e. the solution values are only known at certain locations. The major goal of this paper is to address the problem of PDE discovery with image snapshots. Therefore, we add to the efforts by introducing the pseudo-spectral PINN, which provides a handy approach to deal with data that are provided as image snapshots. And in particular, we focus on the phase field models alone.

The rest of of this paper is organized as follows. In Section 2, we introduce some notations, and set up the PDE discovery problem. In Section 3, we introduce the pseudo-spectral physics informed neural networks by introducing the network stricture and the various definitions of the loss function. Afterward, several examples are shown to demonstrate the approximation power of the pseudo-spectral PINN in Section 4. A brief conclusion is drawn in the last section.

Recall from (1.1), we use $\phi(\mathbf{x},t)$ to denote the state variables, where $\mathbf{x}\in\Omega\subset\mathbb{R}^{d}$ are the spatial coordinates, and $t\in(0,T]$ denotes the time. The baseline PINN [16] would require a rather amount of data pairs $\{(\mathbf{x}_{i},t_{i},\phi(\mathbf{x}_{i},t_{i}))\}_{i=1}^{N}$ properly sampled from the domain $\Omega\times[0,T]$. Also, it requires all the data are from a single time sequence, given a deep neural network $\mathcal{N}(\mathbf{x},t)$ is needed to approximate $\phi(\mathbf{x},t)$ as an aiding neural network for the discovery the PDE. However, this is usually not the case. In practice, the data are usually sampled as image snapshots at various time slots. For instance, for certain experiments, a picture is taken at $t_{0}$, and after $\delta t$, another picture is taken at $t_{1}=t_{0}+\delta t$.

To clearly describe the problem, we need to introduce some notations. Let $N_{x},N_{y}$ be two positive even integers. The spatial domain $\Omega=[0,L_{x}]\times[0,L_{y}]$ is uniformly partitioned with mesh size $h_{x}=L_{x}/N_{x},h_{y}=L_{y}/N_{y}$ and

In order to derive the algorithm conveniently, we denote the discrete gradient operator and the discrete Laplace operator

following our previous work [4].

Let $V_{h}=\{u|u={u_{j,k}(x_{j},y_{k})\in\Omega_{h}}\}$ be the space of grid functions on $\Omega_{h}$. For such cases, the data are collected in the form as

where $\Phi_{i}^{(1)}\in V_{h}$ and $\Phi_{i}^{(2)}\in V_{h}$ are two snapshots with a time lag $\delta_{i}>0$ between the two states, and $N$ is the total number of snapshot pairs. And the goal is to discover the bulk function $f$ in the phase field models, with the data collected in the form of (2.1).

We emphasis that $\delta_{i}$ is not required to be constant, removing the assumption in [21]. Also, different data pairs $(\Phi_{i}^{(1)},\Phi_{i}^{(2)})$ and $(\Phi_{j}^{(1)},\Phi_{j}^{(2)})$ $i\neq j$ are not necessarily from the same single time sequence, removing the requirement of baseline PINN [16].

Next, we investigate the proposed SPINN with several examples. To identify the phase field models from data snapshots.

Recall that we assume $f(\phi)$ is unknown, and the goal is to identify it via learning the existing data in the form of (2.1). As explained, we define a neural network. $\mathcal{N}_{f}:(\phi)\rightarrow\mathcal{N}_{f}(\phi;\theta)$ to approximate $f$ and use the loss functions as defined in the previous section. In the rest of this paper, we assume $\mathcal{N}_{f}$ a feed-forward neural network, with $2$-hidden layers, with each hidden layer having $20$ neurons. The $\tanh$ activation function is applied in both hidden layers. During the training process, the $Adam$ method with default learning rate is used for 10,000 training iterations, followed by an L-BFGS-B optimization training process. The algorithms are implemented with Tensorflow.

For simplicity of discussion, in the all the examples below, $rand()$ generates random numbers between $[-1,1]$ we chosen the domain $\Omega=[-1,1]^{2}$, and choose $N_{x}=N_{y}=128$ in $\Omega_{h}$, i.e. the collected data $\Phi_{i}^{(1)},\Phi_{i}^{(2)}$ are matrices in $\mathbb{R}^{128,128}$. And we solve the PDE first with high-order-accurate scheme with uniform time steps and uniform spatial discretization. The numerical solutions are randomly sampled at different time slots as training data to inversely discover the bulk function $f$. Given the free energy in (1.2), we get $g(\phi)=-\varepsilon^{2}\Delta\phi$, and $L_{g}(\Phi)=-\varepsilon^{2}\Delta_{h}\Phi$. And we will chose $C=2$ for AC equation, and $C=-2\Delta_{h}$ for CH equation.

Example 1. In the first example, we generate data by solving the Allen-Cahn equation in (1.3) with $F(\phi)=\frac{1}{4}(1-\phi^{2})^{2}$, which means $f(\phi)=\phi^{3}-\phi$. And the parameters used are $\varepsilon=0.02$, $M=10$, with the initial condition $\phi(\mathbf{x},t=0)=0.25rand(\mathbf{x})$. Some snapshots of $\phi$ at various time slots are shown in Figure 4.1.

First of all, we test out the Runge-Kutta SPINN approach. We choose $N=1$, i.e. use only a single data pair $(\Phi^{(1)},\Phi^{(2)},\delta)$ to train the neural network. We randomly choose $\Phi^{(1)}$, and study how the size of $\delta$ would affect the learned result. One experiment result with $\Phi^{(1)}$ chosen at $t=0.325$ and $\delta=0.05,0.1,0.25,0.5$ are summarized in Figure 4.2. We observe that when the two snapshots are close, i.e. $\delta$ is small, the Runge-Kutta SPINN approach can accurately learn the bulk function $f$. However, when the time step $\delta$ is large, its accuracy drops.

To overcome the inaccuracy when $\delta$ is large, we utilize the recursive Runge-Kutta SPINN approach. Here we fix the time step $\delta=0.25$, and test the effect by using different recursive stage $K=1,20,50,100$. The results are summarized in Figure 4.3. We observe that, by increasing the recursive stages, the accuracy improves, and the bulk function $f$ can be learned accurately when sufficient stage is used.

Example 2. However, when the time step is large enough, the recursive Runge-Kutta SPINN approach will not provide accurate approximate to $f$. For instance, with the time step $\delta=0.5$, the recursive Runge-Kutta SPINN approach fails. Meanwhile, the linear SPINN approach shows superior accuracy. As an example, we use a single data pair$(\Phi^{(1)},\Phi^{(2)},\delta)$ with a fixed time step $\delta=0.5$. We vary the recursive stage $K$ for the recursive linear SPINN, and the results of the learned function $f$ are summarized in Figure 4.4. We observe that, even when the time step $\delta$ is large, the recursive linear SPINN approach still provides accurate approximate to $f$, so long as the recursive stage $K$ is large enough.

We remark that the training strategy and the quality of training data might also be factors for the approximation accuracy, which we will not pursue in detail. Interested readers are strongly encouraged to explore.

Example 3. In the next example, we increase the problem complexity to identify a highly nonlinear bulk function. In details, we get the data by solving an Allen-Cahn equation with the Flory-Huggins free energy $F(\phi)=\phi\ln(\phi)+0.5(1-\phi)\ln(1-\phi)+2\phi(1-\phi)$, which means the bulk function $f=\ln\phi-0.5\ln(1-\phi)+1.5-2\phi$. The parameters used are $M=10$, $\varepsilon=0.1$, along with the initial condition $\phi(\mathbf{x},t=0)=\frac{1}{2}(1+\tanh\frac{0.8-\sqrt{x^{2}+y^{2}}}{\sqrt{2}\varepsilon})$. We randomly sample two snapshots with $\delta t=0.05$, and train the neural network. A example of using two snapshots at $t=0.325,0.375$ are shown in Figure 4.5, where the two snapshots are shown in Figure 4.5(a), and the predicted function $f$ is shown in Figure 4.5(b). We observe the linear SPINN approach can learn the bulk function $f$ from only two images accurately.

Example 4. In the last example, we use the linear SPINN approach to discover the bulk function $f$ from the solution snapshots of the Cahn-Hilliard equation in (1.4). Here the data is obtained by solving the Cahn-Hilliard equation with $F(\phi)=\frac{1}{4}(\phi^{2}-1)^{2}$, i.e. $f(\phi)=\phi^{3}-\phi$, $M=1$ $\varepsilon=0.05$, and initial condition $\phi(t=0)=0.2+0.001rand(\mathbf{x})$. For the loss function, we add an extra term $10^{3}\|\mathcal{N}_{f}(0)\|^{2}$, to enforce the uniqueness of $\mathcal{N}_{f}$. Some temporal snapshots for $\Phi$ are summarized in Figure 4.6. We randomly choose data $\{(\Phi_{i}^{(1)},\Phi_{i}^{(2)},\delta_{i})\}_{i=1}^{N}$, with $N=1,5,10$ data points with fixed time step $\delta_{i}=0.05$. The learned result $f$ is summarized in Figure 4.7. We obverse the predicted bulk function $f$ has improved accuracy with more data used to train the neural network.

In this paper, we introduce a pseudo-spectral physics informed neural network to discover the bulk function in the phase field models. This newly proposed method well fits the common data collection strategy, i.e., taking snapshots/images at various time slots. The effectiveness of the proposed pseudo-spectral PINN, or SPINN, has been verified through identifying the bulk function $f$ of several phase field models. The idea of pseudo-spectral PINN is rather general, and it can be readily applied to discover other PDE models from image data, which will be investigated in our later research projects.