Stability and regularization for ill-posed Cauchy problem of a stochastic parabolic differential equation

To beginning with, we introduce some notations concerning stochastic analysis. More details for these can be found in [24].

Let $(\Omega,{\cal F},{\mathbb{F}},{\mathbb{P}})$ with ${\mathbb{F}}=\{{\cal F}_{t}\}_{t\geq 0}$ be a complete filtered probability space on which a one-dimensional standard Brownian motion $\{W(t)\}_{t\geq 0}$ is defined. Let $H$ be a Fréchet space. We denote by $L_{{\mathbb{F}}}^{2}(0,T;H)$ the Fréchet space consisting of all $H$-valued ${\mathbb{F}}$-adapted processes $X(\cdot)$ such that $\mathbb{E}(|X(\cdot)|_{L^{2}(0,T;H)}^{2})<\infty$; by $L_{{\mathbb{F}}}^{\infty}(0,T;H)$ the Fréchet space consisting of all $H$-valued ${\mathbb{F}}$-adapted bounded processes; and by $L_{{\mathbb{F}}}^{2}(\Omega;C([0,T];H))$ the Fréchet space consisting of all $H$-valued ${\mathbb{F}}$-adapted continuous processes $X$ such that $\mathbb{E}(|X|_{C([0,T];H)}^{2})<\infty$. All of the above spaces are equipped with the canonical quasi-norms. Furthermore, all of the above spaces are Banach spaces equipped with the canonical norms, if $H$ is a Banach space.

For simplicity, we use the notation $y_{i}\equiv y_{i}(x)=\frac{\partial y(x)}{\partial x_{i}},$ where $x_{i}$ is the $i$-th coordinate of a generic point $x=(x_{1},\cdots,x_{n})$ in ${\mathbb{R}}^{n}$. In a similar manner, we use the notation $z_{i}$, $v_{i}$, etc. for the partial derivatives of $z$ and $v$ with respect to $x_{i}$. Also, we denote the scalar product in ${\mathbb{R}}^{n}$ by $\big{\langle}\cdot,\cdot\big{\rangle}$, and use $C$ to denote a generic positive constant independent of the solution $y$, which may change from line to line.

Let $T>0$, $G\subset{\mathbb{R}}^{n}$ ($n\in{\mathbb{N}}$) be a given bounded domain with the $C^{4}$ boundary $\partial G$, and $\Gamma$ be a given nonempty open subset of $\partial G$.

Let $a_{1}\in L^{\infty}_{\mathbb{F}}(0,T;L^{\infty}(G;{\mathbb{R}}^{n}))$, $a_{2}\in L^{\infty}_{\mathbb{F}}(0,T;L^{\infty}(G))$ and $a_{3}\in L^{\infty}_{\mathbb{F}}(0,T;W^{1,\infty}(G))$. We assume $f\in L^{2}_{\mathbb{F}}(0,T;L^{2}(G))$, $g_{1}\in H^{1}(\Gamma)$, $g_{2}\in L^{2}_{\mathbb{F}}(0,T;L^{2}(\Gamma))$, and $a^{ij}:\;\Omega\times[0,T]\times\overline{G}\to{\mathbb{R}}^{n\times n}\;$ ($i,j=1,2,\cdots,n$) satisfies

(H1) $a^{ij}\in L_{{\mathbb{F}}}^{2}(\Omega;C^{1}([0,T];W^{2,\infty}(G)))$ and $a^{ij}=a^{ji}$;

(H2) $\sum\limits^{n}_{i,j=1}a^{ij}(\omega,t,x)\xi^{i}\xi^{j}\geq s_{0}|\xi|^{2},\ (\omega,t,x,\xi)\equiv(\omega,t,x,\xi^{1},\cdots,\xi^{n})\in\Omega\times(0,T)\times G\times{\mathbb{R}}^{n}$ for some $s_{0}>0$.

Now, the Cauchy problem of the forward stochastic parabolic differential equation can be described as follows.

$$\begin{cases}\displaystyle dy-\sum^{n}_{i,j=1}(a^{ij}y_{i})_{j}dt=[\big{\langle}a_{1},\nabla y\big{\rangle}+a_{2}y+f]dt+a_{3}ydW(t)&\mbox{ in }(0,T)\times G,\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr\displaystyle y=g_{1}&\mbox{ on }(0,T)\times\Gamma,\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr\displaystyle\frac{\partial y}{\partial\nu}=g_{2}&\mbox{ on }(0,T)\times\Gamma.\end{cases}$$

(1.1)

Let $\mathbf{G}_{\Gamma}\buildrel\triangle\over{=}\{G^{\prime}\subset G|\partial G^{\prime}\cap\partial G\subset\Gamma\}$ and

$$H_{\Gamma}^{2}(G)\mathop{\buildrel\Delta\over{=}}\{\eta\in H^{2}_{loc}(G):\eta|_{G^{\prime}}\in H^{2}(G^{\prime}),\quad\forall G^{\prime}\in\mathbf{G}_{\Gamma},\quad\eta|_{\Gamma}=g_{1},\;\partial_{\nu}\eta|_{\Gamma}=g_{2}\}.$$

The aim of this paper is to study the Cauchy Problem with the Lateral Data for the stochastic parabolic differential equation:

Problem (CP) Find a function $y\in L_{{\mathbb{F}}}^{2}(\Omega;C([0,T];L^{2}_{loc}(G)))\cap L_{{\mathbb{F}}}^{2}(0,T;H_{\Gamma}^{2}(G))$ that satisfies system (1.1).

In certain applications involving diffusive, thermal, and heat transfer problems, measuring some boundary data can be challenging. For instance, in nuclear reactors and steel furnaces in the steel industry, the interior boundary can be difficult to measure. Similarly, the use of liquid crystal thermography in visualization can pose the same problem. To address this issue, engineers attempt to reconstruct the status of the boundary using measurements from the accessible boundary. This, in turn, creates the Cauchy problem for parabolic equations. Due to the requirements in real applications, numerous researchers have focused on solving the Cauchy problem of parabolic differential equations, particularly on the inverse problems for deterministic parabolic equations, as seen in [14, 19, 17, 16, 15, 22, 31, 33], and their respective references. Among these works, some have studied the identification of coefficients in parabolic equations through lateral Cauchy observations, such as uniqueness and stability [14, 33, 19, 31], and numerical reconstruction [15], assuming that the initial value is known. Meanwhile, the determination of the initial value has been considered in [17, 16, 22] under the assumption that all coefficients in the governing equation are known.

Stochastic parabolic equations have a wide range of applications for simulating various behaviors of stochastic models which are utilized in numerous fields such as random growth of bacteria populations, propagation of electric potential in neuron models, and physical systems that are subject to thermal fluctuations (e.g.,[18, 24]). In addition, they can be considered as simplified models for complex phenomena such as turbulence, intermittence, and large-scale structure (e.g.,[8]). Given the significant applications of these models, stochastic parabolic equations have been extensively studied in both physics and mathematics. Therefore, it is natural to study the Cauchy problem of stochastic parabolic equations in such situations. However, due to the complexity of stochastic parabolic equations, some tools become ineffective for solving these problems. Thus, the research on inverse problems for stochastic parabolic differential equations is relatively scarce. In [1], the author proved backward uniqueness of solutions to stochastic semilinear parabolic equations and tamed Navier-Stokes equations driven by linearly multiplicative Gaussian noises, via the logarithmic convexity property known to hold for solutions to linear evolution equations in Hilbert spaces with self-adjoint principal parts. In [11, 27], authors studied inverse random source problems for the stochastic time fractional diffusion equation driven by a spatial Gaussian random field, proving the uniqueness and representation for the inverse problems, as well as proposing a numerical method using Fourier methods and Tikhonov regularization. Carleman estimates play an important role in the study of inverse problems for stochastic parabolic differential equations, such as inverse source problems [23, 35], determination of the history for stochastic diffusion processes [4, 23, 30, 34], and unique continuation properties [6, 32]. We refer the reader to [25] for a survey on some recent advances in Carleman estimates and their applications to inverse problems for stochastic partial differential equations.

In this paper, our objective is to solve Problem (CP), i.e., we aim to retrieve the solution of equation (1.1) with observed data from the lateral boundary. To this end, we first prove the conditional stability based on a new Carleman estimate for the stochastic parabolic equation (1.1). Then we construct a Tikhonov functional for the Cauchy problem based on the Tikhonov regularization strategy and prove the uniqueness of the minimizer of the Tikhonov functional, as well as the convergence rate for the optimization problem using variational principles, Riesz theorems, and Carleman estimates established previously.

Generally, the optimization problem for the Tikhonov functional is difficult to solve in the study of inverse problems in stochastic partial differential equations (SPDEs). This is because it involves solving the adjoint problem of the original problem, which is challenging to handle. In fact, one of the primary differences between stochastic parabolic equations and deterministic parabolic equations is that at least one partial derivative of the solution does not exist, making it impossible to express the solution of that equation explicitly. Fortunately, we can express the mild solution of the stochastic parabolic equations using the fundamental solution of the corresponding deterministic equation[7, 9]. This idea suggests that we can use kernel-based theory to numerically solve the minimization problem of the Tikhonov functional without computing the adjoint problem. Furthermore, we can solve the problem in one step without iteration, thus reducing the computational cost to some extent. This technique has gained attention in the study of numerical computation for ordinary and partial differential equations, and the use of fundamental solutions as kernels has proven effective for solving inverse problems in deterministic evolution partial differential equations. As far as we know, our work is the first attempt to apply the regularization method combining with kernel-based theory to solve inverse problems for stochastic parabolic differential equations.

The strong solution of equation (1.1) is useful in the proof of convergence rate of regularization method. Thus, we recall it here.

It should be noted that the assumption regarding the solution, namely that $y\in L_{{\mathbb{F}}}^{2}(\Omega;C([0,T];$ $L^{2}_{loc}(G)))\cap L_{{\mathbb{F}}}^{2}(0,T;H_{\Gamma}^{2}(G))$, implies a higher degree of smoothness than is strictly necessary for establishing Hölder stability of the Cauchy problem. However, this additional smoothness is required in order to facilitate the regularization process.

The remainder of this paper is organized as follows. Section 2 presents a proof of a Hölder-type conditional stability result, along with the establishment of the Carleman estimate as a particularly useful tool in this proof. The regularization method, using the Tikhonov regularization strategy, is introduced in Section 3 and we showcase both the uniqueness and convergence rate of the regularization solution. Section 4 provides numerically simulated reconstructions aided by kernel-based learning theory.

In this section, we prove a stability estimate for the Cauchy problem.

We first recall the following exponentially weighted energy identity, which will play a key role in the sequel.

In the sequel, for a positive integer $p$, denote by $O(\mu^{p})$ a function of order $\mu^{p}$ for large $\mu$ (which is independent of $\lambda$); by $O_{\mu}(\lambda^{p})$ a function of order $\lambda^{p}$ for fixed $\mu$ and for large $\lambda$.

Take a bounded domain $J\subset{\mathbb{R}}^{n}$ such that $\partial J\cap\overline{G}=\Gamma$ and that $\widetilde{G}=J\cup G\cup\Gamma$ enjoys a $C^{4}$ boundary $\partial\widetilde{G}$. Then we have

$$G\subset\widetilde{G},\quad\overline{\partial G\cap\widetilde{G}}\subset\Gamma,\quad\partial G\setminus\Gamma\subset\partial\widetilde{G}\quad\mbox{and }\widetilde{G}\setminus G\mbox{ contains some nonempty open subset. }$$

(2.6)

Let $G_{0}\subset\subset\widetilde{G}\setminus G$ be an open subdomain. We know that there is a $\psi\in C^{4}(\widetilde{G})$ satisfying (see [29, Lemma 5.1] for example)

$$\left\{\begin{array}[]{ll}\displaystyle\psi>0&\mbox{ in }\widetilde{G},\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr\displaystyle\psi=0&\mbox{ on }\partial\widetilde{G},\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr\displaystyle|\nabla\psi|>0&\mbox{ in }G\subset\widetilde{G}\setminus G_{0}.\end{array}\right.$$

(2.7)

Since $G^{\prime}\subset\subset G$, we can choose a sufficiently large $N>0$ such that

$$G^{\prime}\subset\Big{\{}x:\,x\in\widetilde{G},\;\psi(x)>\frac{4}{N}|\psi|_{L^{\infty}(\widetilde{G})}\Big{\}}\cap G.$$

(2.8)

Further, let $\rho=\frac{1}{\sqrt{2}}\Big{(}\frac{1}{2}-\kappa\Big{)}T>0$, there exists a positive number $c$ such that

$$c\rho^{2}<|\psi|_{L^{\infty}(\widetilde{G})}<2c\rho^{2}.$$

(2.9)

Then, define

$$\phi(t,x)=\psi(x)-c(t-t_{0})^{2},\quad\alpha(t,x)=e^{\mu\phi(t,x)}$$

(2.10)

for a fix $t_{0}\in[\sqrt{2}\rho,T-\sqrt{2}\rho]$, and denote $\beta_{k}=\beta_{k}(\mu)=e^{\mu\big{[}\frac{k}{N}|\psi|_{L^{\infty}(\widetilde{\Omega})}-\frac{c\rho^{2}}{N}\big{]}},k=1,2,3,4$. Set

$$Q_{k}=\{(t,x):\,x\in\overline{G},\;\alpha(x,t)>\beta_{k}\},\quad k=1,2,3,4.$$

(2.11)

Clearly, $Q_{k}$ is independent of $\mu$. Moreover, $\psi(x)>\frac{4}{N}|\psi|_{L^{\infty}(\widetilde{G})}$ for any $(t,x)\in\Big{(}t_{0}-\frac{\rho}{\sqrt{N}},t_{0}+\frac{\rho}{\sqrt{N}}\Big{)}\times G^{\prime}$, and thus

$$\psi(x)-c(t-t_{0})^{2}>\frac{4}{N}|\psi|_{L^{\infty}(\widetilde{G})}-\frac{c\rho^{2}}{N}.$$

Hence, we see $\alpha(t,x)>\beta_{4}$ in $(t,x)\in\Big{(}t_{0}-\frac{\rho}{\sqrt{N}},t_{0}+\frac{\rho}{\sqrt{N}}\Big{)}\times G^{\prime}$, and thus $Q_{4}\supset\Big{(}t_{0}-\frac{\rho}{\sqrt{N}},t_{0}+\frac{\rho}{\sqrt{N}}\Big{)}\times G^{\prime}$. On the other hand, for any $(t,x)\in Q_{1}$,

$$\psi(x)-c(t-t_{0})^{2}>\frac{1}{N}|\psi|_{L^{\infty}(\widetilde{G})}-\frac{c\rho^{2}}{N}.$$

This yields

$$|\psi|_{L^{\infty}(\widetilde{G})}-\frac{1}{N}|\psi|_{L^{\infty}(\widetilde{G})}+\frac{c\rho^{2}}{N}>c(t-t_{0})^{2}.$$

Together with (2.9) we have

$$2\Big{(}1-\frac{1}{N}\Big{)}c\rho^{2}+\frac{c\rho^{2}}{N}>c(t-t_{0})^{2}.$$

Therefore, we conclude

$$\Big{(}t_{0}-\frac{\rho}{\sqrt{N}},t_{0}+\frac{\rho}{\sqrt{N}}\Big{)}\times G^{\prime}\subset Q_{1}\subset(t_{0}-\sqrt{2}\rho,t_{0}+\sqrt{2}\rho)\times\overline{G}.$$

(2.12)

Next, for any $(t,x)\in\partial Q_{1}$, we know $x\in\overline{G}$ and $\alpha(t,x)\geq\beta_{1}$. In above set, if $x\in G$, $\alpha(t,x)=\beta_{1}$, and if $x\in\partial G$, it must hold $x\in\Gamma$. Indeed, if $x\in\partial G\setminus\Gamma$, from $\partial G\setminus\Gamma\subset\partial\widetilde{G}$ we get $\psi(x)=0$. On the other hand, since $\alpha(t,x)\geq\beta_{1}$, we know

$$\psi(x)-c(t-t_{0})^{2}=-c(t-t_{0})^{2}\geq\frac{1}{N}|\psi|_{L^{\infty}(\widetilde{G})}-\frac{c\rho^{2}}{N}.$$

And thus

$$0\leq c(t-t_{0})^{2}\leq\frac{1}{N}(c\rho^{2}-|\psi|_{L^{\infty}(\widetilde{G})}),$$

which contradicts (2.9). Therefore, we have

$$\partial Q_{1}=\Sigma_{1}\cup\Sigma_{2},$$

(2.13)

where

$$\begin{array}[]{ll}\displaystyle\Sigma_{1}\subset[0,T]\times\Gamma,\qquad\Sigma_{2}=\{(t,x)\in[0,T]\times G:\,x\in G,\,\alpha(t,x)=\beta_{1}\}.\end{array}$$

(2.14)

Let $\eta\in C_{0}^{\infty}(Q_{2})$ such that $0\leq\eta\leq 1$ and $\eta=1$ in $Q_{3}$. For any $y$ solving (1.1), let $z=\eta y$, then $z$ solves

$$\left\{\begin{array}[]{ll}\displaystyle dz-\sum^{n}_{i,j=1}(a^{ij}z_{i})_{j}dt=\big{(}\big{\langle}a_{1},\nabla z\big{\rangle}+a_{2}z+\tilde{f}+\eta f\big{)}dt+a_{3}zdB(t)&\mbox{ in }Q_{1},\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr\displaystyle z=\frac{\partial z}{\partial\nu}=0&\mbox{ on }\Sigma_{2}.\end{array}\right.$$

(2.15)

Here $\displaystyle\tilde{f}=-\sum^{n}_{i,j=1}(a^{ij}_{j}\eta_{i}y+a^{ij}\eta_{ij}y+2a^{ij}\eta_{i}y_{j})-\big{\langle}a_{1},\nabla\eta\big{\rangle}y+\eta_{t}y\in L^{2}_{\mathbb{F}}(0,T;L^{2}(G))$ and $\tilde{f}$ is supported in $Q_{2}\setminus Q_{3}$.

Applying Lemma 2.1 to (2.15) with $u=z$, $b^{ij}=a^{ij}$, $\ell=\lambda\alpha$ and $\displaystyle\Psi=2\sum^{n}_{i,j=1}a^{ij}\ell_{ij}$, integrating it in $G$ and taking mean value, noting that $z$ is supported in $Q_{1}$, we see

$$\begin{array}[]{ll}\displaystyle 2{\mathbb{E}}\int_{Q_{1}}\theta\Big{[}-\sum^{n}_{i,j=1}(a^{ij}v_{i})_{j}+{\cal A}v\Big{]}\Big{[}dz-\sum^{n}_{i,j=1}(a^{ij}z_{i})_{j}dt\Big{]}dx+2\int_{Q_{1}}\sum^{n}_{i,j=1}(a^{ij}v_{i}dv)_{j}dx\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr\displaystyle\quad+2{\mathbb{E}}\int_{Q_{1}}\sum^{n}_{i,j=1}\Big{[}\sum_{i^{\prime},j^{\prime}}\Big{(}2a^{ij}a^{i^{\prime}j^{\prime}}\ell_{i^{\prime}}v_{i}v_{j^{\prime}}-a^{ij}a^{i^{\prime}j^{\prime}}\ell_{i}v_{i^{\prime}}v_{j^{\prime}}\Big{)}+\Psi a^{ij}v_{i}v-a^{ij}\Big{(}{\cal A}\ell_{i}+\frac{\Psi_{i}}{2}\Big{)}v^{2}\Big{]}_{j}dxdt\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr\displaystyle\geq 2\sum^{n}_{i,j=1}{\mathbb{E}}\int_{Q_{1}}c^{ij}v_{i}v_{j}dxdt+\int_{0}^{T}{\cal B}v^{2}dxdt+{\mathbb{E}}\int_{Q_{1}}\Big{|}-\sum^{n}_{i,j=1}(a^{ij}v_{i})_{j}+{\cal A}v\Big{|}^{2}dxdt\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle\quad-{\mathbb{E}}\int_{Q_{1}}\theta^{2}\sum^{n}_{i,j=1}a^{ij}(dz_{i}+\ell_{i}dz)(dz_{j}+\ell_{j}dz)dx-{\mathbb{E}}\int_{Q_{1}}\theta^{2}{\cal A}(dz)^{2}dx,\end{array}$$

(2.16)

where

$$\left\{\begin{array}[]{ll}\displaystyle{\cal A}=-\sum^{n}_{i,j=1}\big{(}a^{ij}\ell_{i}\ell_{j}-a^{ij}_{j}\ell_{i}-a^{ij}\ell_{ij}\big{)}-\Psi,\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr\displaystyle{\cal B}=2\Big{[}A\Psi-\sum^{n}_{i,j=1}\big{(}Aa^{ij}\ell_{i}\big{)}_{j}\Big{]}-A_{t}-\sum^{n}_{i,j=1}\big{(}a^{ij}\Psi_{j}\big{)}_{i}-\ell_{t}^{2},\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr\displaystyle c^{ij}=\sum_{i^{\prime},j^{\prime}}\Big{[}2a^{ij^{\prime}}\big{(}a^{i^{\prime}j}\ell_{i^{\prime}}\big{)}_{j^{\prime}}-\big{(}a^{ij}a^{i^{\prime}j^{\prime}}\ell_{i^{\prime}}\big{)}_{j^{\prime}}\Big{]}-\frac{a_{t}^{ij}}{2}+\Psi a^{ij}.\end{array}\right.$$

(2.17)

Now, we estimate ${\cal A}$, ${\cal B}$ and $c^{ij}$.

$$\begin{array}[]{ll}\displaystyle{\cal A}\!\!\!&\displaystyle=-\sum^{n}_{i,j=1}\big{(}a^{ij}\ell_{i}\ell_{j}-a^{ij}_{j}\ell_{i}-a^{ij}\ell_{ij}\big{)}-\Psi\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr&\displaystyle=-\lambda^{2}\mu^{2}\alpha^{2}\sum^{n}_{i,j=1}a^{ij}\psi_{i}\psi_{j}+\lambda\mu\alpha\sum^{n}_{i,j=1}a^{ij}_{j}\psi_{i}-\lambda\mu^{2}\alpha\sum^{n}_{i,j=1}a^{ij}\psi_{i}\psi_{j}-\lambda\mu\alpha\sum^{n}_{i,j=1}a^{ij}\psi_{ij}\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr&\displaystyle=-\lambda^{2}\mu^{2}\alpha^{2}\sum^{n}_{i,j=1}a^{ij}\psi_{i}\psi_{j}+\lambda\alpha O(\mu^{2}),\end{array}$$

(2.18)

In order to estimate ${\cal B}$, we first do some computations. For $\Psi$, we have

$$\Psi=2\sum^{n}_{i,j=1}a^{ij}\big{(}\lambda\mu^{2}\alpha\psi_{i}\psi_{j}+\lambda\mu\alpha\psi_{ij}\big{)}=2\lambda\mu^{2}\alpha\sum^{n}_{i,j=1}a^{ij}\psi_{i}\psi_{j}+\lambda\alpha O(\mu).$$

(2.19)

Next, we have

$$\ell_{i^{\prime}j^{\prime}j}=\lambda\mu^{3}\alpha\psi_{i^{\prime}}\psi_{j^{\prime}}\psi_{j}+\lambda\alpha O(\mu^{2}),\quad\ell_{i^{\prime}j^{\prime}ij}=\lambda\mu^{4}\alpha\psi_{i^{\prime}}\psi_{j^{\prime}}\psi_{i}\psi_{j}+\lambda\alpha O(\mu^{3}).$$

Therefore, we get

$$\Psi_{j}=2\sum_{i^{\prime},j^{\prime}=1}^{n}\big{(}a^{i^{\prime}j^{\prime}}\ell_{i^{\prime}j^{\prime}}\big{)}_{j}=2\sum_{i^{\prime},j^{\prime}}\big{(}a^{i^{\prime}j^{\prime}}_{j}\ell_{i^{\prime}j^{\prime}}+a^{i^{\prime}j^{\prime}}\ell_{i^{\prime}j^{\prime}j}\big{)}=2\lambda\mu^{3}\alpha\sum_{i^{\prime},j^{\prime}=1}^{n}a^{i^{\prime}j^{\prime}}\psi_{i^{\prime}}\psi_{j^{\prime}}\psi_{j}+\lambda\alpha O(\mu^{2}),$$

and

$$\Psi_{ij}=2\sum_{i^{\prime},j^{\prime}=1}^{n}\big{(}a^{i^{\prime}j^{\prime}}_{ij}\ell_{i^{\prime}j^{\prime}}+a^{i^{\prime}j^{\prime}}\ell_{i^{\prime}j^{\prime}ij}+2a^{i^{\prime}j^{\prime}}_{j}\ell_{i^{\prime}j^{\prime}i}\big{)}=2\lambda\mu^{4}\alpha\sum_{i^{\prime},j^{\prime}=1}^{n}a^{i^{\prime}j^{\prime}}\psi_{i^{\prime}}\psi_{j^{\prime}}\psi_{i}\psi_{j}+\lambda\alpha O(\mu^{3}).$$

Hence, we find

$$-\sum^{n}_{i,j=1}\big{(}a^{ij}\Psi_{j}\big{)}_{i}=-\sum^{n}_{i,j=1}\big{(}a^{ij}_{i}\Psi_{j}+a^{ij}\Psi_{ji}\big{)}=-2\lambda\mu^{4}\alpha\Big{(}\sum^{n}_{i,j=1}a^{ij}\psi_{i}\psi_{j}\Big{)}^{2}+\lambda\alpha O(\mu^{3}).$$

(2.20)

Further, from (2.18) and (2.19), we have

$${\cal A}\Psi=-2\lambda^{3}\mu^{4}\alpha^{3}\Big{(}\sum^{n}_{i,j=1}a^{ij}\psi_{i}\psi_{j}\Big{)}^{2}+\lambda^{3}\alpha^{3}O(\mu^{3})+\lambda^{2}\alpha^{2}O(\mu^{4}).$$

(2.21)

From the definition of $A$, we find

$$\begin{array}[]{ll}\displaystyle{\cal A}_{j}\!\!\!&\displaystyle=-\sum_{i^{\prime},j^{\prime}=1}^{n}\big{(}a_{j}^{i^{\prime}j^{\prime}}\ell_{i^{\prime}}\ell_{j^{\prime}}+2a^{i^{\prime}j^{\prime}}\ell_{i^{\prime}}\ell_{j^{\prime}j}-a_{j^{\prime}j}^{i^{\prime}j^{\prime}}\ell_{i^{\prime}}-a_{j^{\prime}}^{i^{\prime}j^{\prime}}\ell_{i^{\prime}j}+a_{j}^{i^{\prime}j^{\prime}}\ell_{i^{\prime}j^{\prime}}+a^{i^{\prime}j^{\prime}}\ell_{i^{\prime}j^{\prime}j}\big{)}\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr&\displaystyle=-\sum_{i^{\prime},j^{\prime}=1}^{n}\big{(}2a^{i^{\prime}j^{\prime}}\ell_{i^{\prime}}\ell_{j^{\prime}j}+a^{i^{\prime}j^{\prime}}\ell_{i^{\prime}j^{\prime}j}\big{)}+\big{(}\lambda\alpha+\lambda^{2}\alpha^{2}\big{)}O(\mu^{2})\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr&\displaystyle=-2\lambda^{2}\mu^{3}\alpha^{2}\sum_{i^{\prime},j^{\prime}=1}^{n}a^{i^{\prime}j^{\prime}}\psi_{i^{\prime}}\psi_{j^{\prime}}\psi_{j}+O_{\mu}(\lambda)+\lambda^{2}\alpha^{2}O(\mu^{2}).\end{array}$$

Hence, we see

$$\sum^{n}_{i,j=1}{\cal A}_{j}a^{ij}\ell_{i}=-2\lambda^{3}\mu^{4}\alpha^{3}\Big{(}\sum^{n}_{i,j=1}a^{ij}\psi_{i}\psi_{j}\Big{)}^{2}+O_{\mu}(\lambda^{2})+\lambda^{3}\alpha^{3}O(\mu^{3}),$$

which leads to

$$\begin{array}[]{ll}\displaystyle\sum^{n}_{i,j=1}\big{(}{\cal A}a^{ij}\ell_{i}\big{)}_{j}\negthinspace\negthinspace\negthinspace&\displaystyle=\sum^{n}_{i,j=1}{\cal A}_{j}a^{ij}\ell_{i}+{\cal A}\sum^{n}_{i,j=1}\big{(}a_{j}^{ij}\ell_{i}+a^{ij}\ell_{ij}\big{)}\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr&\displaystyle=-3\lambda^{3}\mu^{4}\alpha^{3}\Big{(}\sum^{n}_{i,j=1}a^{ij}\psi_{i}\psi_{j}\Big{)}^{2}+O_{\mu}(\lambda^{2})+\lambda^{3}\alpha^{3}O(\mu^{3}).\end{array}$$

(2.22)

Further, we have

$$\begin{array}[]{ll}\displaystyle{\cal A}_{t}\negthinspace\negthinspace\negthinspace&\displaystyle=-\sum^{n}_{i,j=1}\Big{(}a^{ij}\ell_{i}\ell_{j}-a^{ij}_{j}\ell_{i}+a^{ij}\ell_{ij}\Big{)}_{t}\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr&\displaystyle=-\sum^{n}_{i,j=1}\Big{[}a^{ij}(\ell_{i}\ell_{j})_{t}-a^{ij}_{j}\ell_{it}+a^{ij}\ell_{ijt}\Big{]}+\lambda^{2}\alpha^{2}O(\mu^{2})+\lambda\alpha O(\mu^{2})+\lambda\alpha TO(\mu^{3})\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr&\displaystyle=\lambda^{2}\alpha^{2}TO(\mu^{3})+\lambda^{2}\alpha^{2}O(\mu^{2})+\lambda\alpha O(\mu^{2})+\lambda\alpha TO(\mu^{3}).\end{array}$$

(2.23)

Finally, it holds

$$\ell_{t}^{2}=\lambda^{2}\mu^{2}\alpha^{2}\phi_{t}^{2}=O_{\mu}(\lambda^{2}).$$

(2.24)