Harris recurrent Markov chains and nonlinear monotone cointegrated models

In this section, we present the notation and main results related to Markov chains that are needed to present our main results. For further details, we refer the reader to the first section of the Supplementary Material [4] and the books [29, 27, 12].

Consider a time-homogeneous irreducible Markov Chain, denoted as $\textbf{X}=X_{0},X_{1},X_{2},\ldots$, defined on a probability space $\left(E,\mathcal{E},\mathbb{P}\right)$, where $\mathcal{E}$ is countably generated. The irreducibility measure of the chain is represented by $\psi$. The transition kernel of the chain is denoted as $P\left(x,A\right)$ and its initial distribution is represented by $\lambda$. If the initial measure of the chain is specified, we use $\mathbb{P}_{\lambda}$ (and $\mathbb{E}_{\lambda}$) to denote the probability (and the expectation) conditioned on the law of the initial state $\mathcal{L}\left(X_{0}\right)=\lambda$.

For any set $C\in\mathcal{E}$, we will denote by $\sigma_{C}$ and $\tau_{C}$, respectively, the times of first visit and first return of the chain to the set $C$, i.e. $\tau_{C}=\operatorname{inf}\left\{n\geqslant 1:X_{n}\in C\right\}$ and $\sigma_{C}=\operatorname{inf}\left\{n\geqslant 0:X_{n}\in C\right\}$. The subsequent visit and return times $\sigma_{C},\tau_{C}\left(k\right)$, $k\geqslant 1$ are defined inductively.

Given that our methods will only deal with the values of X in a fixed set $C$, if $A$ is a measurable set, we will write $\mathbb{I}_{C}\{X_{t}\in A\}$ instead of $\mathbb{I}\{X_{t}\in A\cap C\}$ and if $A=E$, then we will simply write $\mathbb{I}_{C}\left(X_{t}\right)$. We will use $T_{n}\left(C\right)$ to denote the random variable that counts the number of times the chain has visited the set $C$ up to time $n$, that is $T_{n}\left(C\right)=\sum_{t=0}^{n}{\mathbb{I}_{C}\left(X_{t}\right)}$. Similarly, we will write $T\left(C\right)$ for the total of numbers of visits the chain X to $C$. The set $C$ is called recurrent if $\mathbb{E}_{x}T\left(C\right)=+\infty$ for all $x\in C$ and the chain X is recurrent if every set $A\in\mathcal{E}$ such that $\psi\left(A\right)>0$ is recurrent. A recurrent chain is called Harris recurrent if for all $x\in E$ and all $A\in\mathcal{E}$ with $\psi(A)>0$ we have $\mathbb{P}\left({{X_{n}}\in A\;\text{infinitely often}\left|{{X_{0}}=x}\right.}\right)=1$.

Denote by $\mathcal{E}^{+}$ the class of nonnegative measurable functions with positive $\psi$ support. A function $s\in\mathcal{E}^{+}$ is called small if there exists an integer $m_{0}\geqslant 1$ and a measure $\nu\in\mathscr{M}{\left({\mathcal{E}}\right)_{+}}$ such that

When a chain possesses a small function $s$, we say that it satisfies the minorization inequality $M\left(m_{0},s,\nu\right)$. A set $A\in\mathcal{E}$ is said to be small if the function $\mathbb{I}_{A}$ is small. Similarly, a measure $\nu$ is small if there exist $m_{0}$, and $s$ that satisfy (4). By Theorem 2.1 in [29], every irreducible Markov chain possesses a small function and Proposition 2.6 of the same book shows that every measurable set $A$ with $\psi\left(A\right)>0$ contains a small set. In practice, finding such a set consists in most cases in exhibiting an accessible set, for which the probability that the chain returns to it in $m$ steps is uniformly bounded below. Moreover, under quite wide conditions a compact set will be small, see [15].

An irreducible chain possesses an accessible atom, if there is a set $\boldsymbol{\alpha}\in\mathcal{E}$ such that for all $x,y$ in $\boldsymbol{\alpha}$: $P(x,.)=P(y,.)$ and $\psi(\boldsymbol{\alpha})>0$. When an accessible atom exists, the stochastic stability properties of X amount to properties concerning the speed of return time to the atom only. Moreover, it follows from the strong Markov property that the sample paths may be divided into independent blocks of random length corresponding to consecutive visits to $\boldsymbol{\alpha}$. The sequence $\left\{\tau_{\boldsymbol{\alpha}}(j)\right\}_{j\geqslant 1}$ defines successive times at which the chain forgets its past, called regeneration times. Similarly, the sequence of i.i.d. blocks $\{\mathcal{B}_{j}\}_{j\geqslant 1}$ are named regeneration blocks. The random variable $T\left({n}\right)=T_{n}\left(\boldsymbol{\alpha}\right)-1$ counts the number of i.i.d. blocks up to time $n$. This term is called number of regenerations up to time $n$.

If X does not possess an atom but is Harris recurrent (and therefore satisfies a minorization inequality $M\left(m_{0},s,\nu\right)$), a splitting technique, introduced in [28, 29], allows us to extend in some sense the probabilistic structure of X in order to artificially construct an atomic chain (named the split chain and denoted by ${\check{\textbf{X}}}$) that inherits the communication and stochastic stability properties from X. One of the main results derived from this construction is the fact that every Harris recurrent Markov chain admits a unique (up to multiplicative constant) invariant measure (see Proposition 10.4.2 in [27]), that is, a measure $\pi$ such that

The invariant measure is finite if and only if $\mathbb{E}_{\check{\boldsymbol{\alpha}}}{\tau_{\check{\boldsymbol{\alpha}}}}<+\infty$, in this case we say the chain is positive recurrent, otherwise, we say the chain is null recurrent. A null recurrent chain is called $\beta$-null recurrent (c.f. Definition 3.2 in [24]) if there exists a small nonnegative function $h$, a probability measure $\lambda$, a constant $\beta\in\left({0,1}\right)$ and a slowly varying function $L_{h}$ such that

As argued in [24], is not a too severe restriction to assume $m_{0}=1$. Therefore, throughout this paper we assume that X satisfies the minorization inequality $M(1,s,\nu)$, i.e, there exist a measurable function $s$ and a probability measure $\nu$ such that $0\leqslant s\left(x\right)\leqslant 1$, $\int_{E}{s(x)d\nu(x)}>0$ and

The following theorem is a compendium of the main properties of Harris’s recurrent Markov chains that will be used throughout the paper. Among other things, it shows that the asymptotic behavior of $T\left({n}\right)$ is similar to the function $u\left(n\right)$ defined as

The aim of the section is to show that for an arbitrary $x_{0}$ in the support of $f_{0}$, the LSE $\widehat{f}_{n}(x_{0})$ is consistent. We make the following assumptions on the processes $\textbf{X}=\{X_{t}\}$ and $\textbf{W}=\{W_{t}\}$.

• (A1)

  X is a Harris recurrent Markov chain whose kernel $P(x,A)$ satisfies the minorization condition (5).

Let $\mathcal{F}_{n}=\sigma\left(\left\{X_{0},\ldots,X_{n}\right\}\right)$ be sigma algebra generated by the chain X up to time $n$.

• (A2)

  For each $n$, the random variables $W_{1},\ldots,W_{n}$ are conditionally independent given ${{\mathcal{F}_{n}}}$, $\mathbb{E}{\left(W_{t}|\mathcal{F}_{n}\right)}=0$ and $\operatorname{Var}\left(W_{t}|\mathcal{F}_{n}\right)\leqslant\sigma^{2}$ for some $\sigma>0$.

It follows from Assumption (A1) that the Markov Chain X admits a unique (up to a multiplicative constant) $\sigma$-finite invariant measure $\pi$. Let $C$ be a set such that $0<\pi\left(C\right)<\infty$ and $x_{0}\in C$. We denote by $F_{n}$ the process defined by

for all $y\in\mathbb{R}$, which is a localized version of the empirical distribution function of the $X_{t}$’s. It is proved in Lemma 5.1 that $F_{n}$ converges almost surely to the distribution function $F$ supported on $C$ and defined by

Our next two assumptions guarantee that there is a compact $C$, that is a small set and contains $x_{0}$ as an interior point. Sets like this can be found under very wide conditions (cf [15]).

• (A3)

  There is $\delta=\delta(x_{0})$ such that the set $C=\left[{{x_{0}}-\delta,{x_{0}}+\delta}\right]$ is small.

• (A4)

  $x_{0}$ belongs to the interior of the support of $X_{t}$.

Notice that by part 1 of Theorem 2.1, (A3)  guarantees that $\pi(C)$ is finite and positive, and hence, $F$ is properly defined.

In addition to the assumptions on the processes $\{X_{t}\}$ and $\{W_{t}\}$, we need smoothness assumptions on $F$ and on $f_{0}$. In particular, we will assume that $F$ and $f_{0}$ are continuous and strictly monotone in $C$. This implies that $f_{0}$ and $F$ are invertible in $C$, so we can find neighborhoods of $f_{0}(x_{0})$ and $F(x_{0})$ respectively, over which the inverse functions are uniquely defined. We denote by $f_{0}^{-1}$ and $F^{-1}$ respectively the inverses of $f_{0}$ and $F$ over such a neighborhood of $f_{0}(x_{0})$ and $F(x_{0})$ respectively. The function $f_{0}$ is assumed to be monotone on its whole support.

• (A5)

  $F$ is locally continuous and strictly increasing in the sense that for all $x^{\prime}$ in $C$, for all $\varepsilon>0$, there exists $\gamma>0$ such that $|F^{-1}(u)-x^{\prime}|>\gamma$ for all $u$ such that $|u-F(x^{\prime})|\geqslant\varepsilon$.

• (A6)

  $f_{0}$ is non-increasing, and $f_{0}$ is locally strictly decreasing in the sense that for all $x^{\prime}$ in $C$, for all $\varepsilon>0$, there exists $\gamma>0$ such that $|f_{0}(x^{\prime})-f_{0}(y)|>\gamma$ for all $y$ such that $|y-x^{\prime}|\geqslant\varepsilon$.

• (A7)

  $f_{0}$ continuous in $C$.

Assumptions (A1), (A3) and (A5) ensure that $X_{t}$ visits infinitely many times any small enough neighborhood of $x_{0}$ with probability 1, and guarantee that $x_{0}$ is not at the boundary of the recurrent states. Assumptions (A1) and (A3) and Lemma 3.2 in [24] imply that $T_{n}(C)\to\infty$ almost surely.

To compute rates of convergence, we need stronger assumptions than for consistency. We replace assumption (A1) for the following stronger version

• (B1)

  $\{X_{t}\}$ is a positive or $\beta$-null recurrent, aperiodic and irreducible Markov Chain whose kernel $P(x,A)$ satisfies the minorization condition (5).

We replace, (A5), (A6) and (A7), for the following slightly more restrictive assumption

• (B2)

  The function $f_{0}$ is non-increasing, the functions $f_{0}$ and $F_{C}$ are differentiable in $C$, and the derivatives $F_{C}^{\prime}$ and $f_{0}^{\prime}$ are bounded, in absolute value, above and away from zero in $C$.

Let $\lambda$ be the initial measure of X. Our next hypothesis imposes some control on the behavior of the chain in the first regenerative block.

• (B3)

  There exists a constant $K$ and a neighborhood $V$ of 0, such that

  $$\mathbb{E}_{\lambda}\left(\sum\limits_{t=0}^{{\tau_{\check{\boldsymbol{\alpha}}}}}{\left({{\mathbb{I}_{C}}\left\{{{X_{t}}\leqslant{x_{0}}+\gamma}\right\}-{\mathbb{I}_{C}}\left\{{{X_{t}}\leqslant{x_{0}}-\gamma}\right\}}\right)}\right)\leqslant K\gamma\quad\forall\gamma\in V.$$

Assumption (B3)is satisfied if we assume that the initial measure of the chain is the small measure used for the construction of the split chain (see equation 4.16c in [29]). In the positive recurrent case, taking $\lambda$ equal to the unique invariant probability measure of the chain also satisfies (B3).

And finally, we need to control the number of times the chain visits $C$ in a regeneration block.

• (B4)

  $\ell_{C}(\mathcal{B}_{1})=\sum_{t\in\mathcal{B}_{1}}\mathbb{I}_{C}{\left\{X_{t}\right\}}$ has finite second moment.

The rate $u\left(n\right)$ comes from Lemmas 5.3 and 6.7, and as it can be seen from Theorem 2.1, it is a deterministic approximation of $T\left({n}\right)$. Note that in the positive recurrent case, $u\left(n\right)=n$, hence we obtain the same rate $n^{-1/3}$ as in the i.i.d. case [18, Chapter 2]. In the $\beta$-null recurrent case, however, the rate of convergence is $n^{\beta/3}L^{1/3}\left(n\right)$ which is slower than the usual rate. This is due to the null-recurrence of the chain because it takes longer for the process to return to a neighborhood of the point $x_{0}$ and it is these points in the neighborhood of $x_{0}$ which are used in nonparametric estimation.

Given the localized nature of our approach, in this section, we present some results that are particularly useful in this scenario. These results are well known for positive recurrent chains but are new in the null recurrent case. The detailed proofs of these results can be found in Section 2 of the Supplementary material [4].

The first result can be viewed as an extension of the Glivenko-Cantelli theorem to the localized scenario.

Our next result (Lemma 5.2), which is an extension of Lemma 2 in [5] to the localized $\beta$-null recurrent case, deals with the properties of classes of functions defined over the regeneration blocks. Before presenting the result, we need some machinery.

Recall that $E\subseteq\mathbb{R}$ denotes the state space of $X$. Define $\widehat{E}=\cup_{k=1}^{\infty}E^{k}$ (i.e. the set of finite subsets of $E$) and let the localized occupation measure $M_{C}$ be given by

The function that gives the size of the localized blocks is $\ell_{C}:\widehat{E}\to\mathbb{N}$

Let $\widehat{\mathcal{E}}$ denote the smallest $\sigma$-algebra formed by the elements of the $\sigma$-algebras $\mathcal{E}^{k}$, $k\geqslant 1$, where $\mathcal{E}^{k}$ stands for the classical product $\sigma$-algebra. Let $\widehat{Q}$ denote a probability measure on $(\widehat{E},\widehat{\mathcal{E}})$. If $B(\omega)$ is a random variable with distribution $Q^{\prime}$, then $M_{C}(B(\omega),\mathrm{d}y)$ is a random measure, i.e., $M_{C}(B(\omega),\mathrm{d}y)$ is a (counting) measure on $(E,\mathcal{E})$, almost surely, and for every $A\in\mathcal{E}$, $M_{C}(B(\omega),A)=\int_{A}\,M_{C}(B(\omega),\mathrm{d}y)$ is a measurable random variable (valued in $\mathbb{N}$). Henceforth $\ell(B(\omega))\times\int f(y)\,M_{C}(B(\omega),\mathrm{d}y)$ is a random variable and, provided that $\widehat{Q}(\ell^{2})<\infty$, the map $Q_{C}$, defined by

is a probability measure on $(E,\mathcal{E})$. The notation $E_{Q_{C}}$ stands for the expectation with respect to the underlying measure $Q_{C}$. Introduce the following notations: for any function $g:E\to\mathbb{R}$, let $\widehat{g}_{C}:\widehat{E}\to\mathbb{R}$ be given by

and for any class $\mathcal{G}$ of real-valued functions defined on $E$, denote the localized version of the sums on the blocks by $\widehat{G}_{C}=\{\widehat{g}_{C}\,:\,g\in\mathcal{G}\}$.

Notice that, for any function $g$,

To put into perspective Lemma 5.2, consider a class of bounded functions $\mathcal{G}$ that is VC with finite envelope. Lemma 5.2 tells us that the class of unbounded functions $\widehat{\mathcal{G}}_{C}$ is also VC. If we also have that (B4) holds, then Theorem 2.5 in [25] tells us that $\widehat{\mathcal{G}}_{C}$ is a Donsker class. A reasoning like this is used in the proof of the following result, which is a stronger version of Lemma 5.1 under assumptions (B1) and (B2) and has some interest on its own.

In this section, we provide the proof of Theorems 3.1 and 4.1. These proofs make use of several intermediate lemmas, whose proofs can be found in Sections 7 and 8.