Empirical process theory for locally stationary processes

Empirical process theory is a powerful tool to prove uniform convergence rates and weak convergence of composite functionals. The theory for independent variables is well-studied (cf. [19], [23], [44] or [43] for an overview) based on the original ideas of [13], [17], [18], [37] and [34] among others. For random variables with dependence structure, various approaches have been discussed. There exists a well-developed empirical process theory and large deviation results for Harris-recurrent Markov chains based on regenerative schemes (cf. [28], [41], [21] and [1] among others) or geometric ergodicity (cf. [27]). To quantify the speed of convergence in maximal inequalities, additional assumptions like $\beta$-recurrence (cf. [26]) have to be imposed. The theory covers a rich class of Markov chains, but for instance, can not discuss linear processes.

An empirical process theory for stationary processes under high-level assumptions on the moments of means was derived in [12] and further discussion papers. In the paradigm of weak dependence (which measures the size of covariances of Lipschitz functions of the random variables), [16] derived Bernstein-type inequalities. Focusing on the analysis of the empirical distribution function (EDF), much more techniques were discussed: For instance, [20, Theorem 4] provide uniform convergence of the EDF by using bounds for covariances of Hölder functions of the random variables. Another abstract concept was introduced by [4] via S-mixing (for stationary mixing), which imposes the existence of $m$-dependent approximations of the original observations. They then derive strong approximations and uniform central limit theorems for the EDF.

A different idea to measure dependence of random variables is given by mixing coefficients. Here, several concepts were introduced, the most common (with increasing strength) being $\alpha$-, $\beta$- and $\phi$-mixing (for an overview about mixing coefficients, cf. [15]). Large deviation results and uniform central limit theorems for general classes of functions (not only EDF) were derived by using coupling techniques, cf. [38], [30] for $\alpha$-mixing, [3], [50], and successively refined by [14], [39], [11], [9] (the last two developed for EDFs only) and [40] for $\beta$-mixing, and [10], [6] for $\beta$- and $\phi$-mixing. See also [2], [10] and [40] for comprehensive overviews.

In [10] it is argued that $\beta$-mixing is the weakest mixing assumption that allows for a “complete” empirical process theory which incorporates maximal inequalities and uniform central limit theorems. There exist explicit upper bounds for $\beta$-mixing coefficients for Markov chains (cf. [25]) and for so-called V-geometric mixing coefficients (cf. [32]). For several stationary time series models like linear processes (cf. [35] for $\alpha$-mixing), ARMA (cf. [33]), nonlinear AR (cf. [26]) and GARCH processes (cf. [22]) there also exist upper bounds on mixing coefficients. A common assumption in these results is that the observed process or, more often, the innovations of the corresponding process, have a continuous distribution. This is a crucial assumption to handle the relatively complicated mixing coefficients defined over a supremum over two different sigma-algebras. A relaxation of $\beta$-mixing coefficients was investigated by [11, Theorem 1] and is specifically designed for the analysis of the EDF. In opposite to sigma-algebras, these smaller coefficients are defined with conditional expectations of certain classes of functions and are easier to upper bound for a wide range of time series models.

During the last years, another measure for dependence, the so-called functional dependence measure, became popular (cf. [46]), which uses a Bernoulli shift representation (see (1.1) below) and decomposition into martingales and $m$-dependent sequences. It has been shown in various applications that the functional dependence measure allows, when combined with the rich theory of martingales, for sharp large deviation inequalities (cf. [49] or [51]). In [47] and [31], uniform central limit theorems for the EDF were derived for stationary and piecewise locally stationary processes.

Up to now, no general empirical process theory (allowing for general classes of functions) using the functional dependence measure is available. In this paper we will fill this gap and prove maximal inequalities and functional central limit theorems under functional dependence. Furthermore, we will draw connections and compare our results to already existing empirical process concepts for dependent data we mentioned above. While the empirical process theory for Markov chains and mixing cited above was developed for stationary processes, we will work in the framework of locally stationary processes and therefore automatically provide the first general empirical process theory in this setting ([7] investigated spectral empirical processes for linear processes, [31] proved a functional central limit theorem for a localized empirical distribution function). Locally stationary processes allow for a smooth change of the distribution over time but can locally approximated by stationary processes. Therefore, they provide more flexible time series models (cf. [8] for an introduction).

The functional dependence measure uses a representation of the given process as a Bernoulli shift process and quantifies dependence with a $L^{\nu}$-norm. More precisely, we assume that $X_{i}=(X_{ij})_{j=1,...,d}$, $i=1,...,n$, is a $d$-dimensional process of the form

where $\mathcal{G}_{i}=\sigma(\varepsilon_{i},\varepsilon_{i-1},...)$ is the sigma-algebra generated by $\varepsilon_{i}$, $i\in\mathbb{Z}$, a sequence of i.i.d. random variables in $\mathbb{R}^{\tilde{d}}$ ($d,\tilde{d}\in\mathbb{N}$), and some measurable function $J_{i,n}:(\mathbb{R}^{\tilde{d}})^{\mathbb{N}_{0}}\to\mathbb{R}$, $i=1,...,n$, $n\in\mathbb{N}$. For a real-valued random variable $W$ and some $\nu>0$, we define $\|W\|_{\nu}:=\mathbb{E}[|W|^{\nu}]^{1/\nu}$. If $\varepsilon_{k}^{*}$ is an independent copy of $\varepsilon_{k}$, independent of $\varepsilon_{i},i\in\mathbb{Z}$, we define $\mathcal{G}_{i}^{*(i-k)}:=(\varepsilon_{i},...,\varepsilon_{i-k+1},\varepsilon_{i-k}^{*},\varepsilon_{i-k-1},...)$ and $X_{i}^{*(i-k)}:=J_{i,n}(\mathcal{G}_{i}^{*(i-k)})$. The uniform functional dependence measure is given by

Graphically, $\delta_{\nu}^{X}$ measures the impact of $\varepsilon_{0}$ in $X_{k}$. Although representation (1.1) appears to be rather restrictive, it does cover a large variety of processes. In [5] it was motivated that the set of all processes of the form $X_{i}=J(\varepsilon_{i},\varepsilon_{i-1},...)$ should be equal to the set of all stationary and ergodic processes. We additionally allow $J$ to vary with $i$ and $n$ to cover processes which change their stochastic behavior over time. This is exactly the form of the so-called locally stationary processes discussed in [8].

Since we are working in the time series context, many applications ask for functions $f$ that not only depend on the actual observation of the process but on the whole (infinite) past $Z_{i}:=(X_{i},X_{i-1},X_{i-2},...)$. In the course of this paper, we aim to derive asymptotic properties of the empirical process

where

Let $\mathbb{H}(\varepsilon,\mathcal{F},\|\cdot\|)$ denote the bracketing entropy, that is, the logarithm of the number of $\varepsilon$-brackets with respect to some distance $\|\cdot\|$ that is necessary to cover $\mathcal{F}$ (this is made precise at the end of this section). We will define a distance $V_{n}$ which guarantees weak convergence of (1.3) if the corresponding bracketing entropy integral $\int_{0}^{1}\sqrt{\mathbb{H}(\varepsilon,\mathcal{F},V_{n})}d\varepsilon$ is finite.

The definition of the functional dependence measure for locally stationary processes is similar to its stationary version and is easy to calculate for many time series models. It does not rely on the stationarity assumption but on the representation of the process as a Bernoulli shift. Therefore, many well-known upper bounds for stationary time series given in [48], including recursively defined models and linear models, directly carry over to the locally stationary case (1.2). It seems reasonable to use it as a starting point to generalize empirical process theory for stationary processes to the more general setting of local stationarity. While the other two paradigms mentioned above should also allow such a generalization in principle, there are open questions:

• •

  The theory for Harris-recurrent Markov chain relies on stationarity and intrinsically needs some knowledge about the whole time series due to the assumption of null-recurrence. There exist generalizations of local stationary Markov chains (cf. for instance [42]), but the corresponding recurrence properties and examples of locally stationary time series are not worked out yet. Furthermore, it is not directly clear how to deal with processes $Z_{i}$ which incorporate the infinite past of $X_{i}$, and linear processes can not be discussed easily.

• •

  Absolutely regular $\beta$-mixing is shown in most examples by assuming some continuity in the distribution or in the corresponding innovations. Especially for linear processes, the bounds are quite hard to obtain and seem not to be optimal. Moreover, there exist no “invariance rules” which would directly allow to transfer the mixing properties of $X_{i}$ to $f(Z_{i},\frac{i}{n})$ which incorporates infinitely many lags of $X_{i}$.

• •

  Many of the more elaborated dependence concepts developed in e.g. [11], [6], [20], [4] are restricted (at least in their original formulation) to the discussion of the EDF or connected one-dimensional indexed function classes.

Contrary to $\beta$-mixing, the functional dependence measure can easily deal with $f(Z_{i},\frac{i}{n})$ by using Hölder-type assumptions on $f$. Furthermore, it easily can be calculated in many situations and is not restricted to noncontinuous distributions of $X_{i}$. Also, linear processes $X_{i}$ are covered.

However, there are also some peculiarities using the functional dependence measure (1.2). While for Harris-recurrent Markov chains and $\beta$-mixing, the empirical process theory is independent of the function class considered, the situation for the functional dependence measure is more complicated. In order to quantify the dependence of $f(Z_{i},\frac{i}{n})$ by $\delta_{\nu}^{X}$, we have to impose smoothness conditions on $f$ in direction of its first argument. The distance $V_{n}$ therefore will not only change with the dependence structure of $X$, but also has to be “compatible” with the function class $\mathcal{F}$. The smoothness condition on $f$ also poses a challenging issue when considering chaining procedures where rare events are excluded by (non-smooth) indicator functions.

Our main contributions in this paper are the following:

• •

  We derive maximal inequalities for $\mathbb{G}_{n}(f)$ for classes of functions $\mathcal{F}$,

• •

  a chaining device which preserves smoothness during the chaining procedure and

• •

  conditions to ensure asymptotic tightness and functional convergence of $\mathbb{G}_{n}(f)$, $f\in\mathcal{F}$.

The paper is organized as follows. In Section 2, we present our main result Theorem 2.3, the functional central limit theorem under minimal moment conditions. As a special case, we derive a version for stationary processes. We give a discussion on the distance $V_{n}$ and compare our result with the empirical process theory for $\beta$-mixing. Some Assumptions are postponed to Section 3, where a new multivariate central limit theorem for locally stationary processes is presented. In Section 4, we provide new maximal inequalities for $\mathbb{G}_{n}(f)$ for both finite and infinite $\mathcal{F}$. In Section 5, we apply our theory to prove uniform convergence rates and weak convergences of several estimators. The aim of the last section is to highlight the wide range of applicability of our theory and to provide the typical conditions which have to be imposed as well as some discussion. In Section 6, a conclusion is drawn. We illustrate the main steps of the proofs in the Appendix of the article but postpone all detailed proofs to the Supplementary Material.

We now introduce some basic notation. For $a,b\in\mathbb{R}$, let $a\wedge b:=\min\{a,b\}$, $a\vee b:=\max\{a,b\}$. For $k\in\mathbb{N}$,

which naturally appears in large deviation inequalities. For a given finite class $\mathcal{F}$, let $|\mathcal{F}|$ denote its cardinality. We use the abbreviation

if no confusion arises. For some distance $\|\cdot\|$, let $\mathbb{N}(\varepsilon,\mathcal{F},\|\cdot\|)$ denote the bracketing numbers, that is, the smallest number of $\varepsilon$-brackets $[l_{j},u_{j}]:=\{f\in\mathcal{F}:l_{j}\leq f\leq u_{j}\}$ (i.e. measurable functions $l_{j},u_{j}:(\mathbb{R}^{d})^{\mathbb{N}_{0}}\times[0,1]\to\mathbb{R}$ with $\|u_{j}-l_{j}\|\leq\varepsilon$ for all $j$) to cover $\mathcal{F}$. Let $\mathbb{H}(\varepsilon,\mathcal{F},\|\cdot\|):=\log\mathbb{N}(\varepsilon,\mathcal{F},\|\cdot\|)$ denote the bracketing entropy. For $\nu\geq 1$, let

Roughly speaking, a process $X_{i}$, $i=1,...,n$ is called locally stationary if for each $u\in[0,1]$, there exists a stationary process $\tilde{X}_{i}(u)$, $i=1,...,n$ such that $X_{i}\approx\tilde{X}_{i}(u)$ if $|u-\frac{i}{n}|$ is small (cf. [8]). Typical estimators are of the form

where $K$ is a kernel function and $h=h_{n}>0$ is a bandwidth. Clearly, such a localization changes the convergence rate. To cover these cases, we suppose that any $f\in\mathcal{F}$ has a representation

where $\bar{f}$ is independent of $n$ and $D_{f,n}(u)$ is independent of $z$. We put

The function class $\bar{\mathcal{F}}$ is considered to consist of Hölder-continuous functions in direction of $z$. For $s\in(0,1]$, a sequence $z=(z_{i})_{i\in\mathbb{N}_{0}}$ of elements of $\mathbb{R}^{d}$ (equipped with the maximum norm $|\cdot|_{\infty}$) and an absolutely summable sequence $\chi=(\chi_{i})_{i\in\mathbb{N}_{0}}$ of nonnegative real numbers, we set

and $|z|_{\chi}:=|z|_{\chi,1}$.

The basic assumption for our main result is the following compatibility condition on $\mathcal{F}$.

While (2.3) summarizes moment assumptions on $X_{ij}$ which are balanced by $p$, the sequence $\Delta(k)$ reflects the intrinsic dependence of $f(Z_{i},\frac{i}{n})$ and $\mathbb{D}_{n}$ measures the influence of the factor $D_{f,n}(u)$ to the convergence rate of $\mathbb{G}_{n}(f)$.

Based on Assumption 2.2, we define for $f\in\mathcal{F}$

Clearly, $V_{n}$ satisfies a triangle inequality. Therefore, $V_{n}(f-g)$ is a distance between $f,g\in\mathcal{F}$. We are now able to state our main result. The weak convergence takes place in the normed space

cf. [43] for a detailed discussion of this space.

The proof of Theorem 2.3 consists of two ingredients, convergence of the finite-dimensional distributions (cf. Theorem 3.4) and asymptotic tightness (cf. Corollary 4.5). The more challenging part is asymptotic tightness; its proof only relies on Assumption 2.2 and consists of a new maximal inequality presented in Theorem 4.1 which may be of independent interest. To ensure convergence of the finite-dimensional distributions, we have to formalize local stationarity (Assumption 3.1) and pose conditions in time direction on $\bar{f}(z,\cdot)$ (cf. Assumption 3.2) and $D_{f,n}(\cdot)$ (cf. Assumption 3.3) which is done in Section 3. In particular, it is needed that $D_{f,n}(u)$ is properly normalized.

Let us note that in the case that $X_{i}$ is stationary, $\bar{f}(z,u)=\bar{f}(z)$ and $D_{f,n}(u)=1$, Assumptions 3.1, 3.2 and 3.3 are directly fulfilled. That is, in the stationary case, Assumption 2.2 is sufficient for Theorem 2.3. We formulate this finding as a simple corollary. Let

where $X_{i}=J(\mathcal{G}_{i})$, $i=1,...,n$, is a stationary process and $h$ are functions from

with the property that for all $x,y\in\mathbb{R}^{d}$, $|h(x)-h(y)|\leq L_{\mathcal{H}}|x-y|_{\infty}^{s}$.

In this section, we introduce the remaining assumptions needed in Theorem 2.3 which pose regularity conditions on the process $X_{i}$ and the function class $\mathcal{F}$ in time direction. They are used to derive a multivariate central limit theorem for $(\mathbb{G}_{n}(f_{1}),...,\mathbb{G}_{n}(f_{k}))$ under minimal moment conditions in Theorem 3.4. Comparable results in different and more specific contexts were shown in [8] or [42].

We first formalize the property of $X_{i}$ to be locally stationary (cf. [8]). We ask that for each $u\in[0,1]$ there exists a stationary process $\tilde{X}_{i}(u)$, $i=1,...,n$, such that $X_{i}\approx\tilde{X}_{i}(u)$ if $|u-\frac{i}{n}|$ is small. Recall $R(\cdot),s,p$ from Assumption 2.2.

The behavior of the functions $f(z,u)=D_{f,n}(u)\cdot\bar{f}(z,u)$ of the class $\mathcal{F}$ in the direction of time $u\in[0,1]$ is controlled by the following two continuity assumptions which state conditions on $\bar{f}(z,\cdot)$ and $D_{f,n}(\cdot)$ separately.

For $f\in\mathcal{F}$, let $D^{\infty}_{f,n}:=\sup_{i=1,...,n}D_{f,n}(\frac{i}{n})$.

Assumption 3.3 looks rather technical. The first part including (3.1) guarantees the right normalization of $D_{f,n}(\cdot)$. The second part ensures the convergence of the asymptotic variances $\mathrm{Var}(\mathbb{G}_{n}(f))$ and covariances $\mathrm{Cov}(\mathbb{G}_{n}(f),\mathbb{G}_{n}(g))$.

We obtain the following central limit theorem.