Inference in parametric models with many L-moments

As in the previous section, we consider a setting where we have a sample with $T$ identically distributed observations, $Y_{1},Y_{2}\ldots Y_{T}$, $Y_{t}\sim F$ for $t=1,2\ldots T$, where $F$ belongs to a parametric family $\{F_{\theta}:\theta\in\Theta\}$, $\Theta\subseteq\mathbb{R}^{d}$; and $F=F_{\theta_{0}}$ for some $\theta_{0}\in\Theta$. We will analyse the behaviour of the estimator:

where $\hat{Q}_{Y}(\cdot)$ is the empirical quantile process given by (4); $Q_{Y}(\cdot|\theta)$ is the quantile function associated with $F_{\theta}$; $\{w_{k,l}^{R}\}_{1\leq k,l\leq R}$ are a set of (possibly estimated) weights; $\{P_{k}\}_{1\leq k\leq R}$ are a set of quantile “weighting” functions with $\int_{0}^{1}P_{k}(u)^{2}du=1$; and $0\leq\underline{p}<\bar{p}\leq 1$. This setting encompasses the generalised-method-of-L-moment estimatior discussed in the previous section. Indeed, by choosing $P_{k}(u)=\sqrt{2(2k-1)}\cdot P^{*}_{k-1}(u)$, where $P^{*}_{k}(u)$ are the shifted Legendre polynomials on $[0,1]$, and $0=\underline{p}<\bar{p}=1$, we have the generalised L-moment-based estimator in (6) using (3) as an estimator for the L-moments.⁷⁷7The rescaling by $\sqrt{2(2k-1)}$ is adopted so the polynomials have unit $L^{2}[0,1]$-norm. We leave $\underline{p}<\bar{p}$ fixed throughout.⁸⁸8In Remark 4 later on, we briefly discuss an extension to sample-size-dependent trimming. All limits are taken jointly with respect to $T$ and $R$.

To facilitate analysis, we let $\mathbf{P}^{R}(u)\coloneqq(P_{1}(u),P_{2}(u)\ldots P_{R}(u))^{\prime}$; and write $W^{R}$ for the $R\times R$ matrix with entry $W_{i,j}^{R}=w_{i,j}^{R}$. We may then rewrite our estimator in matrix form as:

In this section, we present conditions under which our estimator is consistent.

We impose the following assumptions on our environment. In what follows, we write $Q_{Y}(\cdot)=Q_{Y}(\cdot|\theta_{0})$.

1 is satisfied in a variety of settings. For example, if $Y_{1}$, $Y_{2}$ …$Y_{T}$ are iid and the family $\{F_{\theta}:\theta\in\Theta\}$ is continuous with a (common) compact support; then (8) follows with $\underline{p}=0$ and $\overline{p}=1$ (Ahidar-Coutrix and Berthet, 2016, Proposition 2.1.). Yoshihara (1995) and Portnoy (1991) provide sufficient conditions for uniform consistency (8) to hold when observations are dependent. We also note that, for the result in this section, it would have been sufficient to assume weak convergence in the $L^{2}(\underline{p},\bar{p})$ norm (see Mason (1984), Barrio et al. (2005) and references therein). We only state results in the $\sup$-norm because convergence statements regarding the empirical quantile process available in the literature are usually proved in $L^{\infty}(\underline{p},\bar{p})$.

2 is satisfied by (rescaled) shifted Legendre polynomials, shifted Jacobi polynomials and other weighting functions.

Next, we impose restrictions on the estimated weights. In what follows, we write, for a $c\times d$ matrix $A$, $\lVert A\rVert_{2}=\sqrt{\lambda_{\max}(A^{\prime}A)}$.

3 restricts the range of admissible weight matrices. Notice that $W^{R}=\Omega^{R}=\mathbb{I}_{R}$ trivially satisfies these assumptions. By the triangle inequality, 3 implies that $\lVert W^{R}\rVert_{2}=O_{P^{*}}(1)$.

Finally, we introduce our identifiability assumption. For some $X\in L^{2}[0,1]$, let $\lVert X\rVert_{L^{2}[0,1]}=\left(\int_{0}^{1}X(u)^{2}du\right)^{\frac{1}{2}}$:

The first part of this assumption is closely related to the usual notion of identifiability in parametric distribution models. Indeed, if $\Theta$ is compact, $\theta\mapsto\lVert Q(\cdot|\theta)\rVert_{L^{2}[0,1]}$ is continuous, $\underline{p}=0$, $\bar{p}=1$, the $\{P_{l}\}_{l}$ constitute an orthonormal basis in $L^{2}[0,1]$ (this is the case for shifted Legendre polynomials), and $W^{R}=\mathbb{I}_{R}$, then part 1 is equivalent to identifiability of the parametric family $\{F_{\theta}\}_{\theta}$.

As for the second part of the assumption, we note that boundedness of the $L^{2}$ norm of parametric quantiles uniformly in $\theta$ is satisfied in several settings. If the parametric family $\{F_{\theta}:\theta\in\Theta\}$ has common compact support, then the assumption is trivially satisfied. More generally, if we assume $\Theta$ is compact and $Q_{Y}(u|\theta)$ is jointly continuous and bounded on $[\underline{p},\bar{p}]\times\Theta$, then the condition follows immediately from Weierstrass’ theorem, as in this case: $\sup_{\theta\in\Theta}\lVert Q_{Y}(\cdot|\theta)\mathbbm{1}_{[\underline{p},\bar{p}]}\rVert_{L^{2}[0,1]}\leq\sqrt{\bar{p}-\underline{p}}\cdot\sup_{(\theta,u)\in\Theta\times[\underline{p},\bar{p}]}|Q_{Y}(u|\theta)|<\infty$.

Under the previous assumptions, the estimator is consistent.

In this section, we provide conditions under which the estimator admits an asymptotic linear representation. In what, follows, define $h^{R}(\theta):=\int_{\underline{p}}^{\bar{p}}\left(\hat{Q}_{Y}(u)-Q_{Y}(u|\theta)\right)\mathbf{P}^{R}(u)du$; and write $\nabla_{\theta^{\prime}}h^{R}(\tilde{\theta})$ for the Jacobian of $h^{R}$ with respect to $\theta$, evaluated at $\tilde{\theta}$. We assume that:

Assumption 5 requires $\theta_{0}$ to be an interior point of $\Theta$. It also implies the objective function is countinuously differentiable on a neighborhood of $\theta_{0}$, which enables us to linearise the first order condition satisfied with high probability by $\hat{\theta}$.

Weak convergence of the empirical quantile process (6) has been derived in a variety of settings, ranging from iid data (Vaart, 1998, Corollary 21.5) to nonstationary and weakly dependent observations (Portnoy, 1991). In the iid setting, if the family $\{F_{\theta}:\theta\in\Theta\}$ is continuously differentiable with strictly positive density $f_{\theta}$ over a (common) compact support; then weak-convergence holds with $\underline{p}=0$ and $\overline{p}=1$. In this case, the covariance kernel is $\Gamma(i,j)=\frac{(i\land j-ij)}{f_{Y}(Q_{Y}(i))f_{Y}(Q_{Y}(j))}$. Similarly to the discussion of Assumption 1, Assumption 6 is stronger than necessary: it would have been sufficient to assume $\lVert\sqrt{T}(Q_{Y}(\cdot)-\hat{Q}_{Y}(\cdot))\mathbbm{1}_{[\underline{p},\bar{p}]}\rVert_{L^{2}[0,1]}^{2}=O_{P^{*}}(1)$, which is implied by weak convergence in $L^{2}$.

7 is a technical condition which enables us to provide an upper bound to the linearisation error of the first order condition satisfied by $\hat{\theta}$.

8 is similar to the rank condition used in the proof of asymptotic normality of M-estimators (Newey and McFadden, 1994), which is known to be equivalent to a local identification condition under rank-regularity assumptions (Rothenberg, 1971). In our setting, where $R$ varies with sample size, we show in Supplemental LABEL:app_relation that a stronger version of 4 implies 8.

Under Assumptions 1-8, we have that:

In the next subsection, we work out an asymptotic approximation to the distribution of the leading term in (9).

Finally, to work out the asymptotic distribution of the proposed estimator, we rely on a strong approximation concept. The idea is to construct, in the same underlying probability space, a sequence of Brownian bridges that approximates, in the supremum norm, the empirical quantile process. This can then be used to conduct inference based on a Gaussian distribution. In Supplemental Appendix LABEL:app_inferece_bahadur, we alternatively show how a Bahadur-Kiefer representation of the quantile process can be used to conduct inference in the iid case. In this alternative, one approximates the distribution of the leading term of (9) by a transformation of independent Bernoulli random variables.

We first consider a strong approximation to a Gaussian process in the iid setting. We state below a classical result, due to Csorgo and Revesz (1978):

The above theorem is much stronger than the weak convergence of 6. Indeed, Theorem 1 requires variables to be defined in the same probability space and yields explicit bounds in the sup norm; whereas weak convergence is solely a statement on the convergence of integrals (Vaart and Wellner, 1996). Suppose the approximation (10)/(11) holds in our context. Let $B_{T}$ be as in the statement of the theorem, and assume in addition that $\int_{\underline{p}}^{\overline{p}}\frac{1}{f_{Y}(Q_{Y}(u))^{2}}du<\infty$. A simple application of Bessel’s inequality then shows that:

Note that the distribution of the leading term in the right-hand side is known (by Riemann integration, it is Gaussian) up to $\theta_{0}$. This representation could thus be used as a basis for inference. The validity of such approach can be justified by verifying that the Kolmogorov distance between the distribution of $\sqrt{T}(\hat{\theta}-\theta_{0})$ and that of the leading term of the representation goes to zero as $T$ and $R$ increase. We show that this indeed is true later on, where convergence in the Kolmogorov distance is obtained as a byproduct of weak convergence.

Next, we reproduce a strong approximation result in the context of dependent observations. The result is due to Fotopoulos and Ahn (1994) and Yu (1996).

A similar argument as the previous one then shows that, under the conditions of the theorem above:

Differently from the iid case, the distribution of the leading term on the right-hand side is now known up to $\theta_{0}$ and the covariance kernel $\Gamma$. The latter could be estimated with a Newey and West (1987) style estimator.

To conclude the discussion, we note that the strong representation (12) (resp. (14)) allows us to establish asymptotic normality of our estimator. Indeed, let $L_{T}$ be the leading term of the representation on the right-hand side of (12) (resp. (14)), and $V_{T,R}$ be its variance. Observe that ${V}_{T,R}^{-1/2}R_{T}$ is distributed according to a multivariate standard normal. It then follows by Slutsky’s theorem that $V_{T,R}^{-1/2}\sqrt{T}(\hat{\theta}-\theta_{0})\overset{d}{\to}N(0,\mathbb{I}_{d})$. Since pointwise convergence of cdfs to a continuous cdf implies uniform convergence (Parzen, 1960, page 438), and given that $V_{T,R}^{-1/2}$ is positive definite, we obtain that:

which justifies our approach to inference based on the distribution of the leading term on the right-hand side of (12) (resp. (14)).

We collect the main results in this subsection under the corollary below.

Supplemental Appendix LABEL:app_efficiency discusses efficiency of our proposed L-moment estimator. Specifically, we show that, when no trimming is adopted ($0=\underline{p}<\overline{p}=1$), the optimal weighting scheme (16) is used, the $\{P_{l}\}_{l}$ constitue an orthonormal basis in $L^{2}[0,1]$ (recall this is satisfied by shifted Legendre polynomials), and the data is iid, the generalised method of L-moments estimator is assymptotically efficient, in the sense that its asymptotic variance coincides with the inverse of the Fisher information matrix of the parametric model.¹⁰¹⁰10In the dependent case, “efficiency” should be defined as achieving the efficiency bound of the semiparametric model that parametrises the marginal distribution of the $Y_{t}$, but leaves the time series dependence unrestricted up to regularity conditions (Newey, 1990; Komunjer and Vuong, 2010). Indeed, in general, our L-moment estimator will be inefficient with respect to the MLE estimator that models the dependency structure between observations. See Carrasco and Florens (2014) for further discussion. We leave details to the Supplemental Appendix, though we briefly outline the argument here. The idea is to consider the alternative estimator:

for a grid of $G_{T}$ points $\mathcal{G}_{T}=\{g_{1},g_{2},\ldots,g_{G_{T}}\}\subseteq(0,1)$ and weights $\kappa_{i,j}$, $i,j\in\mathcal{G}_{T}$. This is a weighted version of a “percentile-based estimator”, which is used in contexts where it is difficult to maximise the likelihood (Gupta and Kundu, 2001). It amounts to choosing $\theta$ so as to match a weighted combination of the order statistics in the sample. In the Supplemental Appendix, we show that, under a suitable sequence of gridpoints and optimal weights, this estimator is asympotically efficient. We then show, by using the fact that the $\{P_{l}\}_{l}$ form an orthonormal basis, that estimator (17) can be seen as a generalised L-moment estimator that uses infinitely many L-moments. The final step of the argument then consists in showing that a generalised L-moment estimator that uses a finite but increasing number of L-moments is asymptotically equivalent to estimator (17), which implies that a generalised L-moment estimator under optimal weights is no less efficient than the (efficient) percentile estimator.

Following Hosking et al. (1985) and Hosking (1990), we consider the family of distributions

and $\theta_{0}=(0,1,-0.2)^{\prime}$.

Table 1 reports the RMSE of each procedure, divided by the RMSE of the MLE, under the choice of $R$ that achieves the smallest RMSE. Values above 1 indicate the MLE outperforms the estimator in consideration; and values below 1 indicate the estimator outperforms MLE. The value of $R$ that minimises the RMSE is presented under parentheses. Some patterns are worth highlighting. Firstly, the L-moment estimator, under a proper choice of $R$ and (estimated) optimal weights (two-step estimators) is able to outperform MLE in most settings, especially at the tail of the distribution function. Reductions in these settings can be as large as $34\%$. At the median, two-step L-moment estimators behave similarly to the MLE. The performance of two-step càglàd and unbiased estimators is also quite similar. Secondly, the power of overidentifying restrictions is evident: except in four out of twenty-four cases, two-step L-moment estimators never achieve a minimum RMSE at $R=3$, the number of parameters. These exceptions are found at the smallest sample size ($T=50$), where the benefit of overidentifying restrictions may be outweighed by noisy estimation of the weighting matrix.¹³¹³13Indeed, as we discuss in Supplemental Appendix LABEL:app_selection, a higher-order expansion of our proposed estimator shows that correlation of the estimator of the optimal weighting matrix with sample L-moments plays a key role in the higher-order bias and variance of the two-step estimator. Thirdly, the relation between $T$ and $R$ in the two-step Càglàd estimator is monotonic, as expected from our theoretical discussion. Finally, the role of optimal weights is clear: first-step estimators tend to underperform the MLE as the sample size increases. In larger samples, and when optimal weights are not used, the best choice tends to be setting $R$ close to or equal to $3$, which reinforces the importance of weighting when overidentifying restrictions are included.

To better understand the patterns in the table, we report in Figure 1, the relative RMSE curve for different sample sizes and choices of $R$. The role of optimal weights is especially striking: first-step estimators usually exhibit an increasing RMSE, as a function of $R$. In contrast, two-step estimators are able to better control the RMSE across $R$. It is also interesting to note that the two-step unbiased L-moment estimator behaves poorly when $R$ is close to $T$. This suggests that, in settings where one may wish to make $R$ large, the càglád estimator is preferrable.¹⁴¹⁴14This is also in accordance with our theoretical results for the càglád-based estimator, which essentially place no restriction on the growth rate of $R$.

Following Hosking and Wallis (1987), we consider the family of distributions:

and $\theta_{0}=(1,-0.2)^{\prime}$.

Table 2 and Figure 2 summarise the results of our simulation. Overall patterns are similar to the ones obtained in the GEV simulations. Importantly, though, estimation of the optimal weighting matrix impacts two-step estimators quite negatively in this setup. As a consequence, we verify that the choice of $R=2$ (i.e. a just-identified estimator that effectively does not rely on the weights) is optimal for TS estimators at eight out of the sixteen cases in sample sizes smaller than $500$. This behaviour also leads to FS estimators, which do not use estimated weights, outperforming TS estimators at the $0.99$ and $0.999$ quantiles when $T=50$, and only slightly underperforming TS estimators at some quantiles in larger sample sizes. The latter phenomenon is also partly explained by the just-identified choice $R=2$ improving upon the MLE even in the largest sample size. In all settings, L-moment estimators compare favourably to the MLE.