Duality for Nonlinear Filtering II: Optimal Control

The linear Gaussian filtering model is as follows:

	$\displaystyle\,\mathrm{d}X_{t}$	$\displaystyle=A^{\hbox{\rm\tiny T}}X_{t}\,\mathrm{d}t+\sigma\,\mathrm{d}B_{t},\quad X_{0}\sim N(m_{0},\Sigma_{0})$		(1a)
	$\displaystyle\,\mathrm{d}Z_{t}$	$\displaystyle=H^{\hbox{\rm\tiny T}}X_{t}\,\mathrm{d}t+\,\mathrm{d}W_{t}$		(1b)

where $X:=\{X_{t}\in\mathbb{R}^{d}:0\leq t\leq T\}$ is the state process, the prior $N(m_{0},\Sigma_{0})$ is a Gaussian density with mean $m_{0}\in\mathbb{R}^{d}$ and variance $\Sigma_{0}\succeq 0$, $Z:=\{Z_{t}:0\leq t\leq T\}$ is the observation process, and both $B:=\{B_{t}:0\leq t\leq T\}$ and $W:=\{W_{t}:0\leq t\leq T\}$ are Brownian motion (B.M.). It is assumed that $X_{0},B,W$ are mutually independent. The model parameters $A\in\mathbb{R}^{d\times d}$, $H\in\mathbb{R}^{d\times m}$, and $\sigma\in\mathbb{R}^{d\times p}$.

For this problem, the dual optimal control formulations are well-understood. These are of following two types:

• •

  Minimum variance optimal control problem:

	$\displaystyle\mathop{\text{Minimize}}_{\stackrel{{\scriptstyle\{u_{t}\in\mathbb{R}^{m}:0\leq t\leq T\}}}{{=:u}}}\!:\quad{\sf J}(u)$	$\displaystyle=|y_{0}|^{2}_{\Sigma_{0}}+\int_{0}^{T}y_{t}^{\hbox{\rm\tiny T}}(\sigma\sigma^{\hbox{\rm\tiny T}})y_{t}+|u_{t}|^{2}\,\mathrm{d}t$		(2a)
	$\displaystyle\text{Subject to}\;\;:\;-\frac{\,\mathrm{d}y_{t}}{\,\mathrm{d}t}$	$\displaystyle=Ay_{t}+Hu_{t},\quad y_{T}=f\;\;\text{(given)}$		(2b)

• •

  Minimum energy optimal control problem:

	$\displaystyle\mathop{\text{Minimize}}_{\stackrel{{\scriptstyle\{u_{t}\in\mathbb{R}^{p}:0\leq t\leq T\}=:u}}{{\tilde{m}_{0}\in\mathbb{R}^{d}}}}\!:\quad$	$\displaystyle{\sf J}(u,\tilde{m}_{0};z)=|m_{0}-\tilde{m}_{0}|^{2}_{\Sigma_{0}^{-1}}$
		$\displaystyle\qquad+\int_{0}^{T}|u_{t}|^{2}+|\dot{z}_{t}-H^{\hbox{\rm\tiny T}}\tilde{m}_{t}|^{2}\,\mathrm{d}t$		(3a)
	$\displaystyle\text{Subject to}\;\;:\;$	$\displaystyle\frac{\,\mathrm{d}\tilde{m}_{t}}{\,\mathrm{d}t}=A^{\top}\tilde{m}_{t}+\sigma u_{t}$		(3b)

where $z=\{z_{t}\in\mathbb{R}^{m}:0\leq t\leq T\}$ is a given sample path of observations.

These two types of linear quadratic (LQ) optimal control problems are known since 1960s and described in [3, Sec. 7.3.1 and 7.3.2]. Because it is discussed in the seminal paper [2] of Kalman and Bucy, the minimum variance duality (2) is also referred to as the Kalman-Bucy duality [4]. The relationship of the two problems to the model (1) is as follows:

• •

  Minimum variance duality is related to the filtering problem for the model (1). The optimal control cost (2a) comes from specifying a minimum variance objective for estimating the random variable $f^{\hbox{\rm\tiny T}}X_{T}$ for $f\in\mathbb{R}^{d}$.

• •

  Minimum energy duality is related to a smoothing problem for the model (1). The optimal cost (3a) is obtained from specifying a maximum likelihood (ML) objective for estimating a trajectory $\{\tilde{m}_{t}:0\leq t\leq T\}$ given a sample path $\{z_{t}:0\leq t\leq T\}$ of observations.

Their respective solutions are related to (1) as follows:

• •

  The solution of the minimum variance duality (2) is useful to derive the Kalman filter for (1) [5, Ch. 7.6]. The derivation helps explain why the covariance equation of the Kalman filter is the same as the differential Ricatti equation (DRE) of the LQ optimal control. Note however that the time arrow is reversed: the DRE is solved in forward time for the Kalman filter. This is because the constraint (2b) is a backward (in time) ordinary differential equation (ODE).

• •

  The solution of the minimum energy duality (3) is a favorite technique to derive the forward-backward equations of smoothing for the model (1). The Hamilton’s equation for (3) is referred to as the Bryson-Frazier formula [6, Eq. (13.3.4)]. By introducing a DRE, other forms of solution, e.g., the Fraser-Potter smoother [7, Eq. (16)-(17)], are possible and useful in practice.

Given this background for the linear Gaussian model (1), there has been extensive work spanning decades on extending duality to the problems of nonlinear filtering and smoothing. The prominent duality type solution approaches in literature include the following:

• •

  Mortensen’s maximum likelihood estimator (MLE) [8].

• •

  Minimum energy estimator (MEE) in the model predictive control (MPC) literature [9, Ch. 4].

• •

  Log transformation relationship between the Zakai equation of nonlinear filtering and the Hamilton-Jacobi-Bellman (HJB) equation of optimal control [10].

• •

  Mitter and Newton’s variational formulation of the nonlinear smoothing problem [11].

In an early work [8], Mortensen considered a slightly more general version of the linear Gaussian model (1) where the drift terms in both (1a) and (1b) are nonlinear. Both the optimal control problem and its forward-backward solution are straightforward extensions of (3). Since 1960s, closely related extensions have appeared by different names in different communities, e.g., maximum likelihood estimation (MLE), maximum a posteriori (MAP) estimation, and the minimum energy estimation (MEE) which is discussed next.

Based on the use of duality, the theory and algorithms developed in the MPC literature are readily adapted to solve state estimation problems. The resulting class of estimators is referred to as the minimum energy estimator (MEE) [9, Ch. 4]. The MEE algorithms are broadly of two types: (i) Full information estimator (FIE) where the entire history of observation is used; and (ii) Moving horizon estimator (MHE) where only a most recent fixed window of observation is used. An important motivation is to also incorporate additional constraints in estimator design. Early papers include [12, 13, 14] and more recent extensions have appeared in [15, 16, 17, 18]. A historical survey is given in [9, Sec. 4.7] where Rawlings et. al. write “establishing duality [of optimal estimator] with the optimal regulator is a favorite technique for establishing estimator stability”. Although the specific comment is made for the Kalman filter, the remainder of the chapter amply demonstrates the utility of dual constructions for both algorithm design and convergence analysis (as the time-horizon $T\to\infty$). Convergence analysis typically requires additional assumptions on the model which in turn has motivated the work on nonlinear observability and detectability definitions. A literature survey of these definitions, including the connections to duality theory, appears in the introduction of the companion paper [19].

While the focus of MEE is on deterministic models, duality is also an important theme in the study of nonlinear stochastic systems (hidden Markov models). A key concept is the log transformation [20]. In [10], the log transformation was used to transform the Zakai equation into a Hamilton-Jacobi-Bellman (HJB) equation. Because of this, the negative log of a posterior density is a value function for some stochastic optimal control problem (this is how duality is understood in stochastic settings [21, Sec. 4.8]). While the problem itself was not clarified in [10] (see however [22]), Mitter and Newton introduced a dual optimal control problem in [11] based on a variational interpretation of the Bayes’ formula. This work continues to impact algorithm design which remains an important area of research [23, 24, 25, 26, 27]. A notable ensuing contribution appeared in the PhD thesis-work [28] where Mitter-Newton duality is used to obtain results on nonlinear filter stability.

Given the importance of duality for the purposes of stability analysis in both deterministic and stochastic settings of the problem, it is useful to return to the linear Gaussian model (1) and compare the two types of duality (2) and (3). An important point, that has perhaps not been stressed in literature, is that minimum variance duality (2) is more compatible with the classical duality between controllability and observability in linear systems theory. This is because of the following reasons:

• •

  Inputs and outputs. In (2), the control input $u$ has the same dimension $m$ as the output process while in (3), the control input $u$ is the dimension $n$ of the process noise. Evidently, it is natural to view the inputs and outputs as dual processes that have the same dimension.

• •

  Constraint. If we ignore the noise terms in (1) then the resulting deterministic state-output system ($\dot{x}_{t}=A^{\top}x_{t}$ and $z_{t}=H^{\top}x_{t}$) shares a dual relationship with the deterministic state-input system (2b). (It is shown in part I [19, Sec. III-F] that (2b) is also the dual for the stochastic system (1)). In contrast, the ODE (3b) is a modified copy of the model (1a).

• •

  Stability condition. The condition for asymptotic analysis of (2) is stabilizability of (2b) and by duality detectability of $(A^{\hbox{\rm\tiny T}},H^{\hbox{\rm\tiny T}})$. The latter is known to be also the appropriate condition for stability of the Kalman filter. In contrast, for (3), asymptotic convergence of the optimal $\tilde{m}_{T}$ is possible even with $\sigma=0$. The important condition again is detectability of $(A^{\hbox{\rm\tiny T}},H^{\hbox{\rm\tiny T}})$ but it is not at all easy to see from (3).

• •

  Arrow of time. Because the respective DREs are solved forward (resp. backward) in time for optimal filtering (resp. control), the arrow of time flips between optimal control and optimal filtering. Evidently, this is the case for minimum variance duality (2) but not so for the minimum energy duality (3): The constraint (2b) is a backward in time ODE while the constraint (3b) is a modified copy of the signal model which proceeds forward in time.

All of this suggests that a fruitful approach – for both defining observability and for using the definition for asymptotic stability analysis – is to consider the minimum variance duality, which naturally begets the following questions:

• •

  What are the appropriate extensions of (2) and (3) for nonlinear deterministic and stochastic systems?

• •

  What type of duality is implicit in Mitter-Newton’s work? It is already evident that MEE is an extension of (3).

Both of these questions are answered in the present paper (for the white noise observation model). Before discussing the original contributions, it is noted that the past work on minimum variance duality has been on refinement and extensions of the linear model with additional constraints. In [29], it is used to obtain the solution to a class of singular regulator problems, and in [30], the Lagrangian dual for an MEE problem with truncated measurement noise is considered. Numerical algorithms for (2) and its extensions appear in [31, 32, 33, 34]. Prior to our work, it was widely believed that the nonlinear extension of minimum variance duality is not possible [4].

The main contribution of this paper is to present a minimum variance dual to the nonlinear filtering problem. As in the companion paper (part I), the nonlinear filtering problem is for the HMM with the white noise observation model. The mathematical statement of the dual relationship between optimal filtering and optimal control is given in the form of a duality principle (Thm. 1). The principle relates the value of the control problem to the variance of the filtering problem. The classical Kalman-Bucy duality (2) is recovered as a special case for the linear-Gaussian model (1).

Two approaches are described to solve the optimal control problem: (i) Based on the use of the stochastic maximum principle to derive the Hamilton’s equation (Thm. 4.9); and (ii) Based on a martingale characterization (Thm. 5.14). A formula for the optimal control as a feedback control law is obtained and used to derive the equation of the optimal nonlinear filter. Our duality is also related to Mitter-Newton duality with a side-by-side comparison in Table 1.

This paper is drawn from the PhD thesis of the first author [35]. A prior conference version appeared in [36]. While the duality principle was already stated in the conference paper, it relied on a certain assumption [36, Assumption A1] which has now been proved. Various formulae are stated more simply, e.g., the use of carré du champ operator to specify the running cost. Issues related to function spaces have been clarified to a large extent. While the conference version relied on the innovation process, the present version directly works with the observation process. Such a choice is more natural for the problem at hand. As a result, most of the results and certainly their proofs are novel. Comparison with Mitter-Newton duality is also novel.

The outline of the remainder of this paper is as follows: The mathematical model and necessary background appears in Sec. 2. The dual optimal control problem together with the duality principle and its relation to the linear-Gaussian case is described in Sec. 3. Its solution using the maximum principle and the martingale characterization appears in Sec. 4 and Sec. 5, respectively. Duality-based derivation of the equation of the nonlinear filter appears in Sec. 6. A comparison with Mitter-Newton duality is contained in Sec. 7. The paper closes with some conclusions and directions for future work in Sec. 8. All the proof are contained in the Appendix.

The canonical filtration ${\cal F}_{t}=\sigma\big{(}\{(X_{s},W_{s}):0\leq s\leq t\}\big{)}$. The filtration generated by the observation is denoted by ${\cal Z}:=\{{\cal Z}_{t}:0\leq t\leq T\}$ where ${\cal Z}_{t}=\sigma\big{(}\{Z_{s}:0\leq s\leq t\}\big{)}$. A standard approach is based upon the Girsanov change of measure. Suppose the model satisfies the Novikov’s condition: ${\sf E}\left(\exp\big{(}{\mathchoice{\genfrac{}{}{}{1}{1}{2}}{\genfrac{}{}{}{2}{1}{2}}{\genfrac{}{}{}{3}{1}{2}}{\genfrac{}{}{}{4}{1}{2}}}\int_{0}^{T}|h(X_{t})|^{2}\,\mathrm{d}t\big{)}\right)<\infty$. Define a new measure ${\tilde{\sf P}}$ on $(\Omega,{\cal F}_{T})$ as follows:

$$\frac{\,\mathrm{d}{\tilde{\sf P}}}{\,\mathrm{d}{\sf P}}=\exp\Big{(}-\int_{0}^{T}h^{\hbox{\rm\tiny T}}(X_{t})\,\mathrm{d}W_{t}-{\mathchoice{\genfrac{}{}{}{1}{1}{2}}{\genfrac{}{}{}{2}{1}{2}}{\genfrac{}{}{}{3}{1}{2}}{\genfrac{}{}{}{4}{1}{2}}}\int_{0}^{T}|h(X_{t})|^{2}\,\mathrm{d}t\Big{)}=:D_{T}^{-1}$$

Then it is shown that the probability law for $X$ is unchanged but $Z$ is a ${\tilde{\sf P}}$-B.M. that is independent of $X$ [28, Lem. 1.1.5]. The expectation with respect to ${\tilde{\sf P}}$ is denoted by ${\tilde{\sf E}}(\cdot)$.

The two probability measures are used to define the un-normalized and the normalized (or nonlinear) filter are as follows: For $0\leq t\leq T$ and $f\in C_{b}(\mathbb{S})$,

	$\displaystyle\text{(un-normalized filter)}\quad\sigma_{t}(f)$	$\displaystyle:={\tilde{\sf E}}\big{(}D_{t}f(X_{t})|{\cal Z}_{t}\big{)}$
	$\displaystyle\text{(nonlinear filter)}\quad\pi_{t}(f)$	$\displaystyle:={\sf E}\big{(}f(X_{t})|{\cal Z}_{t}\big{)}$

As the name suggests, $\pi_{t}(f)=\frac{\sigma_{t}(f)}{\sigma_{t}({\sf 1})}$ which is referred to as the Kallianpur-Striebel formula [39, Thm. 5.3] (here ${\sf 1}$ is the constant function ${\sf 1}(x)=1$ for all $x\in\mathbb{S}$). Combining the tower property of conditional expectation with the change of measure gives

$${\sf E}(f(X_{t}))={\sf E}(\pi_{t}(f))={\tilde{\sf E}}(\sigma_{t}(f))$$

(8)

The notation $L^{2}_{{\cal Z}_{T}}(\Omega;\mathbb{R}^{m})$ and $L^{2}_{{\cal Z}}\big{(}[0,T];\mathbb{R}^{m}\big{)}$ is used to denote the Hilbert space of ${\cal Z}_{T}$-measurable random vector and ${\cal Z}$-adapted stochastic process, respectively. These Hilbert spaces suffice if the state-space is finite. In general settings, let ${\cal Y}$ denote a suitable Banach space of real-valued functions on $\mathbb{S}$, equipped with the norm $\|\cdot\|_{\cal Y}$. Then

• •

  For a random function, the Banach space $L^{2}_{{\cal Z}_{T}}(\Omega;{\cal Y}):=\big{\{}F:\Omega\to{\cal Y}:F\text{ is }{\cal Z}_{T}\text{-measurable},\;{\tilde{\sf E}}\big{(}\|F\|_{\cal Y}^{2}\big{)}<\infty\big{\}}$.

• •

  For a function-valued stochastic process, the Banach space is $L^{2}_{{\cal Z}}([0,T];{\cal Y}):=\Big{\{}Y:\Omega\times[0,T]\to{\cal Y}\;:\;Y\text{ is }{\cal Z}\text{-adapted},\;{\tilde{\sf E}}\Big{(}\int_{0}^{T}\|Y_{t}\|_{\cal Y}^{2}\,\mathrm{d}t\Big{)}<\infty\Big{\}}$.

In the remainder of this paper, we set ${\cal Y}:=C_{b}(\mathbb{S})$ (the space of continuous and bounded functions) equipped with the sup-norm. The dual space ${\cal M}(\mathbb{S})$ (the space of rba measures) is denoted by ${\cal Y}^{\dagger}$ where the duality pairing $\langle f,\rho\rangle=\rho(f)$ for $f\in{\cal Y}$ and $\rho\in{\cal Y}^{\dagger}$.

For a function $F\in L^{2}_{{\cal Z}_{T}}\big{(}\Omega;{\cal Y}\big{)}$, the nonlinear filter $\pi_{T}(F)$ is the minimum variance estimate of $F(X_{T})$ [3, Sec. 6.1.2]:

$$\pi_{T}(F)=\mathop{\operatorname{argmin}}_{S_{T}\in L^{2}_{{\cal Z}_{T}}(\Omega;\mathbb{R})}{\sf E}\big{(}|F(X_{T})-S_{T}|^{2}\big{)}$$

(9)

Our goal is to express the above minimum variance optimization problem as a dual optimal control problem.

The conditional variance is denoted by

$${\cal V}_{T}(F):={\sf E}\big{(}|F(X_{T})-\pi_{T}(F)|^{2}|{\cal Z}_{T}\big{)}=\pi_{T}(F^{2})-\big{(}\pi_{T}(F)\big{)}^{2}$$

For notational ease, the expected value of the conditional variance is denoted by

$$\operatorname{var}_{T}(F):={\sf E}\big{(}{\cal V}_{T}(F)\big{)}$$

Strictly speaking, the above is variance only at time $T=0$. However, the verbiage is consistent with the “minimum variance” interpretation of the nonlinear filter.

The function space of admissible control inputs is denoted by ${\cal U}:=L^{2}_{{\cal Z}}\big{(}[0,T];\mathbb{R}^{m}\big{)}$. An element of ${\cal U}$ is denoted $U=\{U_{t}:0\leq t\leq T\}$. It is referred to as the control input. The main contribution of this paper is the following problem.

• •

  Minimum variance optimal control problem:

	$\displaystyle\mathop{\text{Minimize:}}_{U\in\;{\cal U}}\;{\sf J}_{T}(U)=\operatorname{var}_{0}(Y_{0})+{\sf E}\Big{(}\int_{0}^{T}l(Y_{t},V_{t},U_{t}\,;X_{t})\,\mathrm{d}t\Big{)}$		(10a)
	Subject to (BSDE constraint):
	$\displaystyle-\!\,\mathrm{d}Y_{t}(x)=\big{(}({\cal A}Y_{t})(x)+h(x)(U_{t}+V_{t}(x))\big{)}\,\mathrm{d}t-V_{t}^{\hbox{\rm\tiny T}}(x)\,\mathrm{d}Z_{t}$
	$\displaystyle\quad\;Y_{T}(x)=F(x),\;\;x\in\mathbb{S}$		(10b)

where the running cost

$$l(y,v,u;x):=(\Gamma y)(x)+|u+v(x)|^{2}$$

and $\operatorname{var}_{0}(Y_{0})={\sf E}(|Y_{0}(X_{0})-\mu(Y_{0})|^{2})=\mu(Y_{0}^{2})-\mu(Y_{0})^{2}$.

The relationship of (10) to the minimum variance objective (9) is given the following theorem.

The problem (10) is a stochastic linear quadratic optimal control problem for which there is a well established existence-uniqueness theory for the optimal control solution. Application of this theory is the subject of the following section. For now, we assume that the optimal control is well-defined and denote it as $U^{\text{\rm(opt)}}=\{U_{t}^{\text{\rm(opt)}}:0\leq t\leq T\}$. Because the right-hand side of the identity (12) is bounded below by $\text{var}_{T}(F)$, the duality gap

$${\sf J}_{T}(U^{\text{\rm(opt)}})-\operatorname{var}_{T}(F)\geq 0$$

In order to conclude that the duality gap is zero, it is both necessary and sufficient to show that there exists a $U\in{\cal U}$ such that the estimator $S_{T}$, as given by (11), equals $\pi_{T}(F)$. Since $Z$ is a ${\tilde{\sf P}}$-B.M., the following lemma is a consequence of the Itô representation theorem for Brownian motion [38, Thm. 4.3.3].

Because the duality gap is zero, the following implications are to be had:

• •

  The optimal control $U^{\text{\rm(opt)}}$ gives the conditional mean

  $$\pi_{T}(F)=\mu(Y_{0})-\int_{0}^{T}\big{(}U_{t}^{\text{\rm(opt)}}\big{)}^{\hbox{\rm\tiny T}}\,\mathrm{d}Z_{t},\quad{\sf P}\text{-a.s.}$$

• •

  The optimal value is the expected value of the conditional variance

  $$\operatorname{var}_{T}(F)=\operatorname{var}_{0}(Y_{0})+{\sf E}\Big{(}\int_{0}^{T}l(Y_{t},V_{t},U_{t}^{\text{\rm(opt)}};X_{t})\,\mathrm{d}t\Big{)}$$

  where $(Y,V)$ is the optimally controlled stochastic process obtained with $U=U^{\text{\rm(opt)}}$ in (10b).

In fact, these two implications carry over to the entire optimal trajectory.

Consequently, the expected value of the conditional variance is the optimal cost-to-go (for a.e. $0\leq t\leq T$). We do not yet have a formula for the optimal control $U^{\text{\rm(opt)}}$. The difficulty arises because there is no HJB equation for BSDE-constrained optimal control problem. Instead, the literature on such problem utilizes the stochastic maximum principle for BSDE which is the subject of the next section. Before that, we discuss the linear Gaussian case.

The goal is to show that the classical Kalman-Bucy duality (2) described in Sec. 1 for the linear Gaussian model (1) is a special case. Consider a linear function $F(x)=f^{\hbox{\rm\tiny T}}x$ where $f\in\mathbb{R}^{d}$ is a given deterministic vector. The problem is to compute a minimum variance estimate of the scalar random variable $f^{\hbox{\rm\tiny T}}X_{T}$. It is given by ${\sf E}(f^{\hbox{\rm\tiny T}}X_{T}|{\cal Z}_{T})$. Now, it is a standard result in the theory of Gaussian processes that conditional expectation can be evaluated in the form of a linear predictor [42, Cor. 1.10]. For this reason, it suffices to consider an estimator of the form

$$S_{T}:=b-\int_{0}^{T}u_{t}^{\hbox{\rm\tiny T}}\,\mathrm{d}Z_{t}$$

where $b\in\mathbb{R}$ and $u=\{u_{t}\in\mathbb{R}^{m}:0\leq t\leq T\}$ are both deterministic (the lower case notation is used to stress this). Consequently, for linear Gaussian estimation, we can restrict the admissible space of control inputs to $L^{2}\big{(}[0,T];\mathbb{R}^{m}\big{)}$ which is a much smaller subspace of $L_{{\cal Z}}^{2}\big{(}[0,T];\mathbb{R}^{m}\big{)}$. Using a deterministic control $u$, and the terminal condition $F(x)=f^{\hbox{\rm\tiny T}}x$, the solution of the BSDE is given by

$$Y_{t}(x)=y_{t}^{\hbox{\rm\tiny T}}x,\quad V_{t}(x)=0,\quad x\in\mathbb{R}^{d},\;\;0\leq t\leq T$$

where $y=\{y_{t}\in\mathbb{R}^{d}:0\leq t\leq T\}$ is a solution of the backward ODE:

$$-\frac{\,\mathrm{d}y_{t}}{\,\mathrm{d}t}=Ay_{t}+Hu_{t},\quad y_{T}=f$$

Using the formula (7) for the carré du champ, the running cost

	$\displaystyle l(Y_{t},V_{t},U_{t};X_{t})$	$\displaystyle=(\Gamma Y_{t})(X_{t})+|U_{t}+V_{t}(X_{t})|^{2}$
		$\displaystyle=y_{t}^{\hbox{\rm\tiny T}}(\sigma\sigma^{\hbox{\rm\tiny T}})y_{t}+|u_{t}|^{2}$

With the Gaussian prior, the initial cost $\text{var}_{0}(y_{0})=y_{0}^{\hbox{\rm\tiny T}}\Sigma_{0}y_{0}$. Combining all of the above, the optimal control problem (10) reduces to (2) for the linear Gaussian model (1).

Because $y\in{\cal Y}$ is a function, the co-state $p\in{\cal Y}^{\dagger}$ is a measure. The Hamiltonian ${\cal H}:{\cal Y}\times{\cal Y}^{m}\times\mathbb{R}^{m}\times{\cal Y}^{\dagger}\times{\cal Y}^{\dagger}\to\mathbb{R}$ is defined as follows:

$${\cal H}(y,v,u,p;\rho)=-p\big{(}{\cal A}y+h^{\hbox{\rm\tiny T}}(u+v)\big{)}-\ell(y,v,u;\rho)$$

In the following, the Hamilton’s equations for the optimal trajectory are derived by an application of the maximum principle for BSDE constrained optimal control problems [44, Thm. 4.4].

The Hamilton’s equations are expressed in terms of the derivatives of the Hamiltonian. In order to take derivatives with respect to functions and measures, we adopt the notion of Gâteaux differentiability. Given a nonlinear functional $F:{\cal Y}\to\mathbb{R}$, the Gâteaux derivative $F_{y}(y)\in{\cal Y}^{\dagger}$ is obtained from the defining relation [3, Sec. 10.1.3]:

$$\frac{\,\mathrm{d}}{\,\mathrm{d}\varepsilon}F(y+\varepsilon\tilde{y})\Big{|}_{\varepsilon=0}=\big{\langle}\tilde{y},F_{y}(y)\big{\rangle},\quad\forall\,\tilde{y}\in{\cal Y}$$

For the problem at hand, the derivatives of the Hamiltonian are as follows:

	$\displaystyle{\cal H}_{y}(y,v,u,p;\rho)$	$\displaystyle=-{\cal A}^{\dagger}p-\big{(}\rho(\Gamma y)\big{)}_{y}$
	$\displaystyle{\cal H}_{v}(y,v,u,p;\rho)$	$\displaystyle=-ph-2(u+v)\rho$
	$\displaystyle{\cal H}_{u}(y,v,u,p;\rho)$	$\displaystyle=-p(h)-2\rho({\sf 1})u-2\rho(v)$
	$\displaystyle{\cal H}_{p}(y,v,u,p;\rho)$	$\displaystyle=-{\cal A}y-h^{\hbox{\rm\tiny T}}(u+v)$

where ${\cal A}^{\dagger}$ is the adjoint of ${\cal A}$ (whereby $({\cal A}^{\dagger}\rho)(f)=\rho({\cal A}f)$ for all $f\in{\cal Y},\rho\in{\cal Y}^{\dagger}$). Using this notation, the Hamilton’s equations are as follows: