Gaussian Approximation for Lag-Window Estimators and the Construction of Confidence Bands for the Spectral Density

We consider the problem of constructing (simultaneous) confidence bands for the spectral density of a stationary time series. Toward this goal we develop Gaussian approximation results for the maximum deviation over all (positive) Fourier frequencies of a lag-window spectral density estimator and for its bootstrap counterpart. Based on observations $X_{1},X_{2},\ldots,X_{T}$ stemming from a strictly stationary and centered stochastic process $\{X_{t},t\in\mbox{$\mathbb{Z}$}\}$, a lag-window estimator of the spectral density $f(\lambda)$, $\lambda\in[0,\pi]$, is given by

In (1) and for $j=0,1,2,\ldots,M_{T}<T$,

are estimators of the autocovariances $\gamma(j)=\mathrm{Cov}(X_{0},X_{j})$ of the process $\{X_{t},t\in\mbox{$\mathbb{Z}$}\}$ and $\widehat{\gamma}(j)=\widehat{\gamma}(-j)$ for $j<0$. The function $w:[-1,1]\rightarrow{\mathbb{R}}$ is a so-called lag-window, which assigns weights to the $M_{T}$ sample autocovariances effectively used in the calculation of the estimator for $f(\lambda)$ and which satisfies some assumptions to be specified later.

It is well-known that for a fixed frequency $\lambda$, under suitable assumptions on the dependence structure of the underlying time series and for $M_{T}$ converging not too fast to infinity, asymptotic normality for lag-window estimators can be shown. To elaborate and assuming sufficient smoothness of $f$, which is equivalent to assuming a sufficiently fast decay of the underlying autocovariances $\gamma(h)$ as $|h|\to\infty$, one typically can show under certain conditions on the lag-window $w$ and for $M_{T}\to\infty$ such that $T/M_{T}^{5}\to C^{2}\geq 0$ as $n\to\infty$, that

for $0<\lambda<\pi$, where $f^{\prime\prime}$ denotes the second derivative of $f$ and $W=\lim_{u\rightarrow 0}(1-w(u))/u^{2}$, where the latter is assumed to exist and to be positive.

Moreover, for the estimator $\widehat{f}_{T}(\lambda)$ with sophisticated selected so-called flat-top lag-windows $w$ and under the additional assumption that $\sum_{j\in\mathbb{Z}}|j|^{r}|\gamma(j)|<\infty$ for some $r\geq 1$ convergence rates for the mean squared error $\mbox{MSE}(\widehat{f}_{T}(\lambda))=O(T^{-2r/(2r+1)})$ can be achieved, which corresponds to $M_{T}\sim T^{1/(2r+1)}$. See \citeasnounPolitisRomano87 and \citeasnounBergPolitis2009 for details.

Instead of point-wise inference we aim in this paper for simultaneous inference using the estimator $\widehat{f}_{T}$. More precisely, we aim for confidence bands covering the spectral density uniformly at all positive Fourier frequencies $\lambda_{k,T}=2\pi k/T,k=1,\ldots,\lfloor T/2\rfloor=:N_{T}$ with a desired (high) probability. Hence, we keep all information contained in $X_{1},\dots,X_{T}$ in the sense that the discrete Fourier transform of any vector $x\in\mbox{$\mathbb{R}$}^{T}$ is a linear combination of trigonometric polynomials at exactly these frequencies.

Note that so far simultaneous confidence bands for the spectral density had only been derived for special cases. In particular, for autoregressive processes relying on parametric spectral density estimates (see e.g. \citeasnounNP84 and \citeasnounT87) while \citeasnounNP08 proposed a bootstrap-aided approach for Gaussian time series using the integrated periodogram to construct simultaneous confidence bands for (a smoothed version of) the spectral density. Generalizing results from \citeasnounLiuWu2010, \citeasnounYZ22 derived simultaneous confidence bands for the spectral density on a grid with mesh size wider than $2\pi/T$ and for locally stationary Bernoulli shifts satisfying a geometric moment contraction condition. To this end, they proved that the maximum deviation of a suitably standardized lag-window spectral density estimator over such a grid asymptotically possesses a Gumbel distribution. Motivated by the slow rate of convergence of the maximum deviation to a Gumbel variable, they propose an asymptotically valid bootstrap method to improve the finite sample behavior of their confidence regions.

As mentioned above, we aim to construct simultaneous confidence bands on the finer grid of all (positive) Fourier frequencies. To achieve this goal, we establish a Gaussian approximation result for the distribution of a properly standardized statistic based on the random quantity

In this context and in order to stabilize the asymptotic variance and to construct confidence bands that automatically adapt to the local variability of $f(\lambda)$, we are particularly interested in deriving a Gaussian approximation for the normalized statistic

Various Gaussian approximations of max-type statistics in the time domain have been derived during the last decades. For means of high dimensional time series data we refer the reader to \citeasnounZW17 as well as \citeasnounZC18 and references therein. In the context of i.i.d. functional data, \citeasnounCCH22 show that Gaussian approximation results for means of high dimensional vectors can be adapted to establish a Gaussian approximation for the maximum of the periodogram. To elaborate, they use that the periodogram at frequency $\lambda_{k,T}$ is the squared norm of $\frac{1}{\sqrt{T2\pi}}\sum_{t=1}^{T}X_{t}\,e^{-it\lambda_{k,T}}$. However, their arguments do not directly carry over to lag-window estimators as the latter have a more complex structure. It is shown in (3) below, that they still have a mean-type representation, but the degree of dependence of the summands increases with the sample size which prevents the application of the afore-mentioned results. A Gaussian approximation result for high dimensional spectral density matrices of $\alpha$-mixing time series has been derived in \citeasnounChangetal23. However, they require the dimension of the process to increase with sample size making their results not applicable for our purposes.

The paper is organized as follows. Section 2 summarizes key assumptions for our results. The core material on Gaussian approximation for spectral density estimators is contained in Section 3. In Section 4 we discuss application of the Gaussian approximation results to the construction of simultaneous confidence bands over the positive Fourier frequencies. Section 5 reports the results of a small simulation study. Auxiliary lemmas and proofs are deferred to Section 6.

We begin by imposing the following assumption on the dependence structure of the stochastic process generating the observed time series $X_{1},X_{2},\ldots,X_{T}$.

Assumption 1 implies that the autocovariance function $\gamma(h)=\mathrm{Cov}(X_{0},X_{h}),$ $h\in\mbox{$\mathbb{Z}$},$ is absolute summable, that is that the process $\{X_{t},t\in\mbox{$\mathbb{Z}$}\}$ possesses a continuous and bounded spectral density $f(\lambda)=(2\pi)^{-1}\sum_{h\in\mbox{$\mathbb{Z}$}}\gamma(h)e^{-ih\lambda}$, $\lambda\in(-\pi,\pi]$.
It can be even shown that (4) implies $\sum_{j\in\mathbb{Z}}|j|^{r}|\gamma(j)|<\infty$ for $r<\alpha-1$, see Lemma 1 in Section 6. As has been mentioned in the Introduction this allows for the option of convergence rates for $\widehat{f}_{T}(\lambda)$ of order $O(T^{-r/(2r+1)})$, $r<\alpha-1$. Note that our assumptions on the decay of the dependence coefficients are less restrictive than those in \citeasnounYZ22, where exponentially decaying coefficients are presumed. We assume for simplicity that $EX_{t}=0$. Our results can be generalized to non-centered time series using the modified autocovariance estimator $\widetilde{\gamma}(j)=\frac{1}{T}\sum_{t=j+1}^{T}(X_{t}-\overline{X})(X_{t-j}-\overline{X})\ \ \mbox{with}\ \ \overline{X}=\sum_{t=1}^{T}X_{t}/T$, instead of the estimator $\widehat{\gamma}(j)$ defined in (2).

Additional to Assumption 1 we also require the following boundedness condition for the spectral density $f$.

For the following considerations and in order to separate any bias related problems, we consider the centered sequence

with an obvious abbreviation for $Z_{t,k}$ and where for $\ k=1,\ldots,N_{T}$,

Due to the differentiability of the lag-window $w$ (cf. Assumption 3), we have

The following theorem is our first result and establishes a valid Gaussian approximation for the maximum over all positive Fourier frequencies of the centered estimator given in (1).

To construct confidence bands that appropriately take into account the local variability of the lag-window estimator, we make use of the following Gaussian approximation result for the properly standardized lag-window estimators.

Step 1.

For $k_{1},k_{2}\in\{1,2,\ldots,N_{T}\}$, let $\widehat{C}_{T}(k_{1},k_{2})$ be an estimator of the covariance $C_{T}(k_{1},k_{2})$ given in (2) which ensures that the $N_{T}\times N_{T}$ matrix $\widehat{\Sigma}_{N_{T}}$ with the $(i,j)$-th element equal to $\widehat{C}_{T}(i,j)$ is non-negative definite.

Step 2.

Generate random variables $\xi^{\ast}_{1},\xi^{\ast}_{2},\ldots,\xi^{\ast}_{N_{T}}$, where

$$\big{(}\xi^{\ast}_{1},\xi^{\ast}_{2},\ldots,\xi^{\ast}_{N_{T}}\big{)}^{\top}\sim{\mathcal{N}}\Big{(}0_{N_{T}},\widehat{\Sigma}_{N_{T}}\Big{)},$$

with $0_{N_{T}}$ a $N_{T}$-dimensional vector or zeros and covariance matrix $\widehat{\Sigma}_{N_{T}}$. Let $\xi^{\ast}_{\max}=\max\big{\{}|\xi^{\ast}_{1}|,|\xi^{\ast}_{2}|,\ldots,|\xi^{\ast}_{N_{T}}|\big{\}}$ and for $\alpha\in(0,1)$ given, denote by $q^{\ast}_{1-\alpha}$ the upper $(1-\alpha)$ percentage point of the distribution of $\xi^{\ast}_{\max}$.

Step 3.

A simultaneous $(1-\alpha)$-confidence band for $\widetilde{f}(\lambda_{j,T})$, $j=1,2,\ldots,N_{T}$, is then given by

$$\Big{\{}\Big{[}\widehat{f}(\lambda_{j,T})\Big{(}1-q^{\ast}_{1-\alpha}\sqrt{\frac{M_{T}}{T}}\Big{)},\ \ \widehat{f}(\lambda_{j,T})\Big{(}1+q^{\ast}_{1-\alpha}\sqrt{\frac{M_{T}}{T}}\Big{)}\Big{]},\ j=1,2,\ldots,N_{T}.\Big{\}}$$

(13)

Notice that the distribution of $\xi^{\ast}_{\max}$ in Step 2 can be estimated via Monte-Carlo simulation. While $f(\lambda_{k_{i},T})$, $i=1,2$, appearing in the expression for $C_{T}(k_{1},k_{2})$ can be replaced by the estimator $\widehat{f}_{T}(\lambda_{k_{i},T})$, the important step in the bootstrap algorithm proposed, is the estimation of the covariances

appearing in $C_{T}(k_{1},k_{2})$. To obtain an estimator for this quantity which has some desired consistency properties, we follow \citeasnounZhang_etal2022. In particular, let $K$ be a kernel function satisfying the following conditions.

In this section we investigate by means of simulations, the finite sample performance of the Gaussian approximation and of the corresponding multiplier bootstrap procedure proposed for the construction of simultaneous confidence bands. For this purpose, time series of length $T=256,\;512$ and $1024$ have been generated from the following three time series models:

1. Model I:  $X_{t}=0.8X_{t-1}+\varepsilon_{t}$,

2. Model II:  $X_{t}=1.3X_{t-1}-0.75X_{t-2}+u_{t}$ with $u_{t}=\varepsilon_{t}\sqrt{1+0.25u^{2}_{t-1}}$,

3. Model III:   $X_{t}=\big{(}0.4+0.1\varepsilon_{t-1}\big{)}X_{t-1}+\varepsilon_{t}$.

In all models the innovations $\varepsilon_{t}$ are chosen to be i.i.d. standard Gaussian. The empirical coverage over $R=500$ repetitions of each model has been calculated for two different nominal coverages, $90\%$ and $95\%$. The lag-window estimator of the spectral density used has been obtained using the Parzen lag-window and truncation lag $M_{T}$. The Gauss kernel with different values of the parameter $b_{T}$ has been used for obtaining the covariance estimators $\widehat{\sigma}_{T}(j_{1},j_{2})$ given in (14) and which are used for the calculation of the covariance matrix $\widehat{C}_{T}(k_{1},k_{2})$; see equation (15). All bootstrap approximations are based on $B=1,000$ replications. Table 1 presents empirical coverages as well as mean lengths of the confidence bands obtained, where the mean lengths are calculated as

with $R$ the number of repetitions and $q^{\ast,(\ell)}_{1-\alpha}$ the upper $(1-\alpha)$ percentage point of the distribution of $\xi^{\ast}_{\max}$ obtained in the $\ell$-th repetition; also see expression (13). Figure 1 shows averaged 90% and 95% confidence bands obtained using the method proposed in this paper for sample sizes of $T=512$ and $T=1024$ and for Model II.

Table 1. Empirical coverages (Cov) and mean lengths (ML) of simultaneous confidence bands (13) for different sample sizes and different values of the parameters $M_{T}$ and $b_{T}$.

We also compare the performance of the approach proposed in this paper with that based on a Gumbel-type approximation of the maximum deviation of the centered lag-window spectral density estimator evaluated over a much coarser grid of frequencies in the interval $(0,\pi]$. To elaborate and based on Theorems 3-5 of \citeasnounLiuWu2010 derived under different conditions, an asymptotically $(1-\alpha)$ simultaneous confidence band for $\mathrm{E}(\widehat{f}_{T}(\lambda_{s}))$ over the set of frequencies $\lambda_{s}=s\pi/M_{T}$ for $s=1,2,\ldots,M_{T}$, is given by

where