Maximum principle for discrete-time control systems driven by fractional noises and related backward stochastic difference equations

From last century, stochastic maximum principle (SMP) and backward stochastic differential equations (BSDEs) are studied popularly to deal with optimal control problems of stochastic systems. For SMP, some original works refer to [1, 2, 3]. Right up to 1990, Peng [4] obtain the stochastic maximum principle for a general control system, where the control domain could be non-convex and the diffusion term contains control process. Then SMP for many different control problems are investigated, such as near-optimal control [5], doubly stochastic control systems [6, 7], mean-field optimal control [8, 9] and delayed control problems [10, 11]. For BSDEs, as the adjoint equation in SMP, it is originated from Bismut [12]. In 1990, Pardoux and Peng [13] prove the existence and uniqueness of the solution of nonlinear BSDEs. Then some related early works refer to [14, 15].

For SMP for discrete-time systems, the original work is by Lin and Zhang [16], a type of backward stochastic difference equations (BS$\Delta$Es) is introduced as adjoint equation. Based on this work, much progress has been made by researchers. Discrete-time stochastic games are studied by Wu and Zhang [17]. Dong et al. obtain the SMP for discrete-time systems with mean-field [18] and delay [19]. Ji and Zhang investigate the infinite horizon recursive optimal control and infinite horizon BS$\Delta$E. But as we known, there are poorly works consider the discrete-time control problems with fractional noises or other “colored noises”.

In this paper, motivated by stochastic optimal control of continuous systems driven by fractional Brownian motion (FBM), we concern the discrete-time optimal control for systems driven by fractional noises. Let $H\in(0,\,1)$ be a fixed constant, which is called Hurst parameter. The $m$-dimensional FBM $B_{t}^{H}=\left(B_{1}^{H}(t),\cdots,B_{m}^{H}(t)\right),t\in[0,T]$ is a continuous, mean 0 Gaussian process with the covariance

$i,\,j=1,\dots,m$. Moreover, the FBM could be generated by a standard Brownian motion through

for some explicit functions $Z_{H}(\cdot,\cdot)$. For control problem of continuous systems driven by FBM, Han et al. [20] firstly obtain the maximum principle for general systems. Some other related works refer to [21, 22, 23].

For our problem, the state equation is

to minimize the cost function

Here $\xi^{H}$ is called a fractional noise describe by the increment of a FBM.

To obtain the stochastic maximum principle, the following BS$\Delta$E is introduced as adjoint equation:

where $\eta$ is a Gaussian white noises constructed by $\xi^{H}$.

A difficulty is that it is hard to estimate the $L^{2}_{\mathcal{F}}$-norm for $X,p,q$ and the variation equation appears in section 3, because of the dependence of $\xi^{H}$ and its coefficient. We deal with this with more complex calculations, then we prove the uniqueness of the state S$\Delta$E and adjoint BS$\Delta$E, and obtain the maximum principle.

The rest of this paper is organized as follows. In section 2, we introduce the BS$\Delta$E driven by both fractional noise and white noise, and prove the existence and the uniqueness of the solution of this type of BS$\Delta$E. In section 3, we obtain the stochastic maximum principle by proving the existence and uniqueness of the state S$\Delta$E and showing the convergence of the variation equation. In section 4, the linear quadratic case is investigated to illustrate the main results.

Let $(\Omega,\mathcal{F},\{\mathcal{F}_{n}\}_{n\in\mathbb{Z}^{+}},P)$ be a filtered probability space, $\mathcal{F}_{0}\subset\mathcal{F}$ be a sub $\sigma$-algebra. $\{\xi_{n}^{H}\}_{n\in\mathbb{Z}^{+}}$ is a sequence of fractional noises described by the increment of a $m$-dimensional fractional Brownian motion, namely, $\xi_{n}^{H}=B^{H}(n+1)-B^{H}(n)$. Define the filtration $\mathbb{F}=(\mathcal{F}_{n})_{0\leq n\leq N}$ by $\mathcal{F}_{n}=\mathcal{F}_{0}\lor\sigma(\xi^{H}_{0},\xi_{1}^{H},...,\xi_{n-1}^{H})$.

Denote by $L^{\beta}(\mathcal{F}_{n};\mathbb{R}^{n})$, or $L^{\beta}(\mathcal{F}_{n})$ for simplify, the set of all $\mathcal{F}_{n}$-measurable random variables $X$ taking values in $\mathbb{R}^{n}$, such that $\mathbb{E}\|X\|^{\beta}<+\infty$. Denote by $L^{\beta}_{\mathcal{F}}(0,T;\mathbb{R}^{n})$, or $L^{\beta}_{\mathcal{F}}(0,T)$ for simplify, the set of all $\mathcal{F}_{n}$-adapted process $X=(X_{n})_{n\in\mathbb{Z}^{+}}$ such that

Then we consider the following BS$\Delta$E ,

where $y\in L^{2a}(\mathcal{F}_{N})$ for some $a>1$, and $\eta_{n}\in\mathcal{F}_{n+1}$, which will be defined in the following lemma, is a sequence of independent Gaussian random variables generated by $\{\xi_{n}^{H}\}$. We also assume the following conditions for $f,g$:

1. (H2.1)

   $f$ and $g$ are adapted processes: $f(\cdot,y,z),g(\cdot,y,z)\in L^{2a}_{\mathcal{F}}(1,N)$ for all $y\in\mathbb{R}^{n},z\in\mathbb{R}^{n\times m}$.

2. (H2.2)

   There are some constants $L>0$, such that

   	$\displaystyle|f(n,y_{1},z_{1})-f(n,y_{2},z_{2})|$	$\displaystyle+|g(n,y_{1},z_{1})-g(n,y_{2},z_{2})|\leq L\left(|y_{1}-y_{2}|+|z_{1}-z_{2}|\right),$
   		$\displaystyle\forall n\in[1,N],\quad y_{1},y_{2}\in\mathbb{R}^{n},\quad z_{1},z_{2}\in\mathbb{R}^{n\times m}.$

3. (H2.3)

   There exists a $\mathcal{F}_{N}$-measurable functions $f_{1},g_{1}$, such that $f(N,y,z)=f_{1}(y)$ and $g(N,y,z)=g_{1}(y)$ for all $y,z$.

Then we show the construction and properties of $\{\eta_{n}\}$.

Then we show the solvability of BS$\Delta$E (2.3).

In this section, we study the optimal control problem for discrete-time systems driven by fractional noises. Let $(\Omega,\mathcal{F},\{\mathcal{F}_{n}\}_{n\in\mathbb{Z}^{+}},P)$ be a filtered probability space, $\mathcal{F}_{0}\subset\mathcal{F}$ be a sub $\sigma$-algebra and $\mathbb{F}=(\mathcal{F}_{n})_{0\leq n\leq N}$ be the filtration defined by $\mathcal{F}_{n}=\mathcal{F}_{0}\lor\sigma(\xi^{H}_{0},\xi_{1}^{H},...,\xi_{n-1}^{H})$. The state equation is

with the cost function

Here $x\in L^{2a}(\mathcal{F}_{0})$ is independent with $\{\xi_{n}\}$, $b(n,x,u)$ and $\sigma(n,x,u)$ are measurable functions on $[0,N-1]\times\mathbf{R}^{d}\times\mathbf{R}^{k}$ with values in $\mathbf{R}^{d}$ and $\mathbf{R}^{d\times m}$, respectively. $l(n,x,u)$ and $\Phi(x)$ be measurable functions on $[0,N-1]\times\mathbf{R}^{d}\times\mathbf{R}^{k}$ and $\mathbf{R}^{d}$, respectively, with values in $\mathbf{R}$.

Denote by $\mathbb{U}$ the set of progressively measurable process u$=(u_{n})_{0\leq n\leq N-1}$ taking values in a given closed-convex set $\textbf{U}\subset\mathbb{R}^{k}$ and satisfying $\mathbb{E}\sum_{n=0}^{N-1}|u_{n}|^{2a}<+\infty$. The problem is to find an optimal control $u^{*}\in\mathbb{U}$ to minimized the cost function, i.e.,

To simplify the notation without losing the generality, we assume that $d=k=m=1$. We give the following assumptions:

1. (H3.1)

   $b$ and $\sigma$ are adapted processes: $b(\cdot,x,u),\sigma(\cdot,y,z)\in L^{2a}_{\mathcal{F}}(0,N-1)$ for all $x,u\in\mathbb{R}$.

2. (H3.2)

   $b(n,x,u),\sigma(n,x,u)$ are differential w.r.t $(x,u)$ and there exists some constants $L>0$, such that

   	$\displaystyle|\phi(n,x_{1},u_{1})-\$	$\displaystyle\phi(n,x_{2},u_{2})|\leq L\left(|x_{1}-x_{2}|+|u_{1}-u_{2}|\right),$
   		$\displaystyle\forall n\in[0,N-1],\quad\forall x_{1},x_{2},u_{1},u_{2}\in\mathbb{R},$

   for $\phi=b,\sigma,b_{x},\sigma_{x},b_{u},\sigma_{u}.$

3. (H3.3)

   $l(n,x,u),\Phi(x)$ are differential w.r.t $(x,u)$ and

   $\displaystyle|l_{x}(n,x,u)|+|\Phi_{x}(x)|\leq L(1+|x|+|u|),\quad\forall n\in[0,N-1],\,x,u\in\mathbb{R}.$

4. (H3.4)

   $b(N,x,u)=\sigma(N,x,u)=l(N,x,u)=0,$ for all $(x,u)$.

Then we show the solvability of the state equation.

For any $u\in\mathbb{U}$ and $\varepsilon\in(0,1)$, let

Denote

for $\phi=b,\sigma,l,b_{x},\sigma_{x},l_{x},b_{u},\sigma_{u},l_{u}$.

Define the variation equation by

Then we have the following convergence result.