Inference of Random Effects for Linear Mixed-Effects Models with a Fixed Number of Clusters

Chih-Hao Changlabel=e1 [ mark][email protected] Hsin-Cheng Huanglabel=e2][email protected] [ Ching-Kang Inglabel=e3][email protected] [ Institute of Statistics, National University of Kaohsiung, Kaohsiung, Taiwan. Institute of Statistical Science, Academia Sinica, Taipei, Taiwan. Institute of Statistics, National Tsing Hua University, HsinChu, Taiwan.

Abstract

We consider a linear mixed-effects model with a clustered structure, where the parameters are estimated using maximum likelihood (ML) based on possibly unbalanced data. Inference with this model is typically done based on asymptotic theory, assuming that the number of clusters tends to infinity with the sample size. However, when the number of clusters is fixed, classical asymptotic theory developed under a divergent number of clusters is no longer valid and can lead to erroneous conclusions. In this paper, we establish the asymptotic properties of the ML estimators of random-effects parameters under a general setting, which can be applied to conduct valid statistical inference with fixed numbers of clusters. Our asymptotic theorems allow both fixed effects and random effects to be misspecified, and the dimensions of both effects to go to infinity with the sample size.

confidence interval,

consistency,

maximum likelihood,

keywords:

\startlocaldefs\endlocaldefs

, and

1 Introduction

Over the past several decades, linear mixed-effects models have been broadly applied to clustered data [13], longitudinal data [12, 23], spatial data [15], and data in scientific fields [10, 11], particularly due to their usefulness in modeling data with clustered structures. Model parameters are traditionally estimated, for example, via minimum norm quadratic, maximum likelihood (ML), and restricted ML (REML) methods. ML and REML estimators are compared in Gumedze and Dunne [6].

Estimating random-effects variances in mixed-effects models is usually more challenging than estimating fixed-effects parameters. Although desired asymptotic properties have been developed for ML and REML estimators of random-effects variances [7, 8, 18], these are mainly obtained under the mathematical device of requiring the number of clusters (denoted as $m$ ) to grow to infinity with the sample size (denoted as $N$ ) and the numbers of fixed effects and random effects (denoted as $p$ and $q$ ) to be fixed. In fact, most asymptotic results for likelihood ratio tests and model selection in linear mixed-effects models are established under a similar mathematical device; see Self and Liang [21], Stram and Lee [22], Crainiceanu and Ruppert [4], Pu and Niu [20], Fan and Li [5], and Peng and Lu [19]. However, in many practical situations, we are faced with a small $m$ , which does not grow to infinity with $N$ . As pointed out by McNeish and Stapleton [16] and Huang [9], data collected in the fields of education or developmental psychology typically have a small number of clusters, corresponding, for example, to classrooms or schools. Unfortunately, to the best of our knowledge, no theoretical justification has been provided for random-effects estimators when $m$ is fixed.

As shown by Maas and Hox [14], Bell et al. [1], and McNeish and Stapleton [17], for a linear mixed-effects model with few clusters, random-effects variances are not well estimated by either ML or REML. This is because when $m$ is fixed, the Fisher information for random-effects variances fails to grow with $N$ , and hence the corresponding ML estimators do not achieve consistency. A similar difficulty arises in a spatial-regression model of Chang et al. [2] under the fixed domain asymptotics, in which the spatial covariance parameters cannot be consistently estimated. A direct impact of this difficulty is that the classical central limit theorem established under $m\rightarrow\infty$ for the ML (or REML) estimators [7, 8, 18] is no longer valid. Consequently, statistical inference based on the asymptotic results for $m\rightarrow\infty$ can be misleading.

In this article, we focus on the ML estimators in linear mixed-effects models with possibly unbalanced data. We first develop the asymptotic properties of the ML estimators, without assuming that fixed- and random-effects models are correctly specified, $p$ and $q$ are fixed, or $m\rightarrow\infty$ . Based on the asymptotic properties of the ML estimators, we provide, for the first time in the mixed-effects models literature, the asymptotic valid confidence intervals for random-effects variances when $m$ is fixed. In addition, we present an example illustrating that empirical best linear unbiased predictors (BLUPs) of random effects (which are the BLUPs with the unknown parameters replaced by their ML estimators) compare favorably to least squares (LS) predictors even when the ML estimators are not consistent; see Section 3.1 for details. Also note that our asymptotic theorems allow both fixed- and random-effects models to be misspecified. Consequently, our results are crucial to facilitate further studies on model selection for linear mixed-effects models with fixed $m$ , in which investigating the impact of model misspecification is indispensable.

This article is organized as follows. Section 2 introduces the linear mixed-effects model and the regularity conditions. The asymptotic results for the ML estimators are given in Section 3. Section 4 describes simulation studies that confirm our asymptotic theory, including a comparison between the conventional confidence intervals and the proposed ones for random-effects variances. A brief discussion is given in Section 5. The proofs of all the theoretical results are deferred to the online supplementary material.

2 Linear Mixed-Effects Models

Consider a set of observations with $m$ clusters, $\{(\bm{y}_{i},\bm{X}_{i},\bm{Z}_{i})\}_{i=1}^{m}$ , where $\bm{y}_{i}=(y_{i,1},\dots,y_{i,n_{i}})^{\prime}$ is the response vector, $\bm{X}_{i}$ and $\bm{Z}_{i}$ are $n_{i}\times p$ and $n_{i}\times q$ design matrices of $p$ and $q$ covariates with the $(j,k)$ -th entries $x_{i,j,k}$ and $z_{i,j,k}$ , respectively, and $n_{i}$ is the number of observations in cluster $i$ . A general linear mixed-effects model can be written as

\displaystyle\bm{y}_{i}=\bm{X}_{i}\bm{\beta}+\bm{Z}_{i}\bm{b}_{i}+\bm{\epsilon}_{i};\quad i=1,\dots,m,

(1)

where $\bm{\beta}=(\beta_{1},\dots,\beta_{p})^{\prime}$ is the $p$ -vector of fixed effects, $\bm{b}_{i}=(b_{i,1},\dots,b_{i,q})^{\prime}\sim N(\bm{0},\mathrm{diag}(\sigma^{2}_{1},\dots,\sigma^{2}_{q}))$ is the $q$ -vector of random effects, $\bm{\epsilon}_{i}\sim N(\bm{0},v^{2}\bm{I}_{n_{i}})$ , and $\bm{I}_{n_{i}}$ is the $n_{i}$ -dimensional identity matrix. Here $\{\bm{b}_{i}\}$ and $\{\bm{\epsilon}_{i}\}$ are mutually independent. Let $\bm{y}$ , $\bm{X}$ , $\bm{b}$ , and $\bm{\epsilon}$ be obtained by stacking $\{\bm{y}_{i}\}$ , $\{\bm{X}_{i}\}$ , $\{\bm{b}_{i}\}$ , and $\{\bm{\epsilon}_{i}\}$ . Also let $\bm{Z}=\mathrm{diag}(\bm{Z}_{1},\dots,\bm{Z}_{m})$ be the block diagonal matrix with diagonal blocks $\{\bm{Z}_{i}\}$ and dimension $N\times(mq)$ , where $N=n_{1}+\cdots+n_{m}$ is the total sample size. Let $\theta_{k}=\sigma^{2}_{k}/v^{2}$ ; $k=1,\dots,q$ and $\bm{D}=\mathrm{diag}(\theta_{1},\dots,\theta_{q})$ . Then we can rewrite (1) as

\displaystyle\bm{y}=\bm{X}\bm{\beta}+\bm{Z}\bm{b}+\bm{\epsilon}\sim N(\bm{X}\bm{\beta},v^{2}\bm{H}),

(2)

where $\bm{H}=\bm{R}+\bm{I}_{N}$ , $\bm{R}=\mathrm{diag}(\bm{R}_{1},\dots,\bm{R}_{m})$ , and $\bm{R}_{i}=\bm{Z}_{i}\bm{D}\bm{Z}^{\prime}_{i}$ ; $i=1,\dots,m$ .

Let $\mathcal{A}\times\mathcal{G}\subset 2^{\{1,\dots,p\}}\times 2^{\{1,\dots,q\}}$ be the set of candidate models with $\alpha\in\mathcal{A}$ and $\gamma\in\mathcal{G}$ corresponding to the fixed-effects and random-effects covariates indexed by $\alpha$ and $\gamma$ , respectively. Then a linear mixed-effects model corresponding to $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ can be written as

\bm{y}=\bm{X}(\alpha)\bm{\beta}(\alpha)+\bm{Z}(\gamma)\bm{b}(\gamma)+\bm{\epsilon}.

(3)

For $i=1,\dots,m$ , let $\bm{Z}_{i}(\gamma)$ be the sub-matrix of $\bm{Z}_{i}$ and $\bm{b}_{i}(\gamma)$ be the sub-vector of $\bm{b}_{i}$ corresponding to $\gamma$ . Then for $\gamma\in\mathcal{G}$ ,

\displaystyle\bm{R}_{i}(\gamma,\bm{\theta}(\gamma))\equiv

\displaystyle~{}\mathrm{var}(\bm{Z}_{i}(\gamma)\bm{b}_{i}(\gamma))=\sum_{k\in\gamma}\theta_{k}\bm{z}_{i,k}\bm{z}_{i,k}^{\prime},

where $\bm{z}_{i,k}$ is the $k$ -th column of $\bm{Z}_{i}$ and $\bm{\theta}(\gamma)$ is the parameter vector of $\theta_{k}$ ; $k\in\gamma$ . In other words, under $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ ,

\displaystyle\bm{y}\sim N(\bm{X}(\alpha)\bm{\beta}(\alpha),v^{2}\bm{H}(\gamma,\bm{\theta})),

(4)

where

	$\displaystyle\bm{H}(\gamma,\bm{\theta})=$	$\displaystyle~{}\bm{R}(\gamma,\bm{\theta})+\bm{I}_{N},$		(5)
	$\displaystyle\bm{R}(\gamma,\bm{\theta})=$	$\displaystyle~{}\mathrm{diag}(\bm{R}_{1}(\gamma,\bm{\theta}),\dots,\bm{R}_{m}(\gamma,\bm{\theta}))=\sum_{i=1}^{m}\sum_{k\in\gamma}\theta_{k}\bm{h}_{i,k}\bm{h}_{i,k}^{\prime},$

$\bm{h}_{i,k}=(\bm{0}_{n_{1}}^{\prime},\dots,\bm{0}_{n_{k-1}}^{\prime},\bm{z}_{i,k}^{\prime},\bm{0}_{n_{k+1}}^{\prime},\dots,\bm{0}_{n_{m}}^{\prime})^{\prime}$ , and $\bm{0}_{n_{i}}$ is the $n_{i}$ -vector of zeros. Here, for notational simplicity, we suppress the dependence of $\bm{\theta}$ on $\gamma$ .

For $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ , let $p(\alpha)$ be the dimension of $\alpha$ and let $q(\gamma)$ be the dimension of $\gamma$ . Assume that the true model of $\bm{y}$ is

\displaystyle\bm{y}\sim N(\bm{\mu}_{0},v_{0}^{2}\bm{H}_{0}),

(6)

where $\bm{\mu}_{0}$ is the underlying mean trend, $v_{0}^{2}>0$ is the true value of $v^{2}$ , $\bm{H}_{0}=\bm{R}_{0}+\bm{I}_{N}$ , $\bm{R}_{0}=\mathrm{diag}(\bm{Z}_{1}\bm{D}_{0}\bm{Z}^{\prime}_{1},\dots,\bm{Z}_{m}\bm{D}_{0}\bm{Z}_{m}^{\prime})$ , and $\bm{D}_{0}=\mathrm{diag}(\theta_{1,0},\dots,\theta_{q,0})$ for some $\theta_{k,0}\geq 0$ ; $k=1,\dots,q$ . Similarly, let $v_{0}^{2}\bm{D}_{0}=\mathrm{diag}(\sigma_{1,0}^{2},\dots,\sigma_{q,0}^{2})$ with $\sigma_{k,0}^{2}\geq 0$ being the true values of $\sigma_{k}^{2}$ , for $k=1,\dots,q$ . We say that a fixed-effects model $\alpha$ is correct if there exists $\bm{\beta}(\alpha)\in\mathbb{R}^{p(\alpha)}$ such that $\bm{\mu}_{0}=\bm{X}(\alpha)\bm{\beta}(\alpha)$ . Similarly, a random-effects model $\gamma$ is correct if $\{k:\theta_{k,0}>0,\,k=1,\dots,q\}\subset\gamma$ . Let $\mathcal{A}_{0}$ and $\mathcal{G}_{0}$ denote the sets of all correct fixed-effects and random-effects models, respectively. A linear mixed-effects model $(\alpha,\gamma)$ is said to be correct if $(\alpha,\gamma)\in\mathcal{A}_{0}\times\mathcal{G}_{0}$ . We denote the smallest correct model by $(\alpha_{0},\gamma_{0})$ , which satisfies

	$\displaystyle p_{0}\equiv p(\alpha_{0})=$	$\displaystyle~{}\inf_{\alpha\in\mathcal{A}_{0}}p(\alpha),$
	$\displaystyle q_{0}\equiv q(\gamma_{0})=$	$\displaystyle~{}\inf_{\gamma\in\mathcal{G}_{0}}q(\gamma),$

where $p_{0}>0$ and $q_{0}>0$ are assumed fixed.

Given a model $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ , the covariance parameters consist of $\bm{\theta}$ and $v^{2}$ . We estimate these by ML. We assume that $\bm{X}$ and $\bm{Z}$ are of full column rank. The ML estimators $\hat{\bm{\theta}}(\alpha,\gamma)$ and $\hat{v}^{2}(\alpha,\gamma)$ of $\bm{\theta}$ and $v^{2}$ based on model $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ can be obtained by minimizing the negative twice profile log-likelihood function:

\displaystyle\begin{split}-2\log L(\bm{\theta},v^{2};\alpha,\gamma)=&~{}N\log(2\pi)+N\log(v^{2})+\log\det(\bm{H}(\gamma,\bm{\theta}))\\ &~{}+\frac{\bm{y}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{A}(\alpha,\gamma;\bm{\theta})\bm{y}}{v^{2}},\\ \end{split}

(7)

where

	$\displaystyle\bm{A}(\alpha,\gamma;\bm{\theta})\equiv$	$\displaystyle~{}\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}),$		(8)
	$\displaystyle\bm{M}(\alpha,\gamma;\bm{\theta})\equiv$	$\displaystyle~{}\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}).$		(9)

Note that $\bm{M}^{2}(\alpha,\gamma;\bm{\theta})=\bm{M}(\alpha,\gamma;\bm{\theta})$ , $\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha)=\bm{X}(\alpha)$ and

\displaystyle\bm{M}(\alpha,\gamma;\bm{\theta})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})=

\displaystyle~{}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta}).

For model $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ , the ML estimator of $\bm{\beta}(\alpha)$ is given by

\displaystyle\hat{\bm{\beta}}(\alpha,\gamma;\hat{\bm{\theta}})=(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\hat{\bm{\theta}})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\hat{\bm{\theta}})\bm{y},

(10)

where $\hat{\bm{\theta}}=\hat{\bm{\theta}}(\alpha,\gamma)$ satisfies

\displaystyle(\hat{\bm{\theta}}(\alpha,\gamma),\hat{v}^{2}(\alpha,\gamma))=

\displaystyle~{}\operatorname*{arg\,min}_{\bm{\theta}\in[0,\infty)^{q(\gamma)},v^{2}\in(0,\infty)}\{-2\log L(\bm{\theta},v^{2};\alpha,\gamma)\}.

Then the ML estimator of $\sigma_{k}^{2}$ is

\hat{\sigma}_{k}^{2}(\alpha,\gamma)=\hat{\theta}_{k}(\alpha,\gamma)\hat{v}^{2}(\alpha,\gamma);\quad k\in\gamma,

where $\hat{\theta}_{k}(\alpha,\gamma)$ is the ML estimator of $\theta_{k}$ based on model $(\alpha,\gamma)$ .

To establish the asymptotic theory for the ML estimators of the parameters in linear mixed-effects models, we impose regularity conditions on covariates of fixed effects and random effects.

(A0)

Let $n_{\min}=\displaystyle\min_{i=1,\dots,m}n_{i}$ . Assume that $p=c_{p}+o(n_{\min}^{\tau})$ and $q=c_{q}+o(n_{\min}^{\tau})$ , for some constant $\tau\in[0,1/2)$ , where $c_{p}>0$ and $c_{q}>0$ .

(A1)

With $\tau$ given in (A0), there exist constants $\xi\in(2\tau,1]$ and $d_{i,j}>0$ ; $i=1,\dots,m$ , $j=1,\dots,p$ , with $0<\inf\{d_{i,j}\}\leq\sup\{d_{i,j}\}<\infty$ such that for $i=1,\dots,m$ and $1\leq j,j^{*}\leq p$ ,

\displaystyle\bm{x}_{i,j}^{\prime}\bm{x}_{i,j^{*}}=

\displaystyle~{}\left\{\begin{array}[]{ll}d_{i,j}n_{i}^{\xi}+o(n_{i}^{\xi});&\mbox{if }j=j^{*},\\ o(n_{i}^{\xi-\tau});&\mbox{if }j\neq j^{*}\>,\end{array}\right.

where $\bm{x}_{i,j}$ is the $j$ -th column of $\bm{X}_{i}$ , for $i=1,\dots,m$ and $j=1,\dots,p$ .

(A2)

With $\tau$ given in (A0), there exist constants $\ell\in(2\tau,1]$ and $c_{i,k}>0$ ; $i=1,\dots,m$ , $k=1,\dots,q$ , with $0<\inf\{c_{i,k}\}\leq\sup\{c_{i,k}\}<\infty$ such that for $i=1,\dots,m$ and $1\leq k,k^{*}\leq q$ ,

\displaystyle\bm{z}_{i,k}^{\prime}\bm{z}_{i,k^{*}}=

\displaystyle~{}\left\{\begin{array}[]{ll}c_{i,k}n_{i}^{\ell}+o(n_{i}^{\ell});&\mbox{if }k=k^{*},\\ o(n_{i}^{\ell-\tau});&\mbox{if }k\neq k^{*}\>.\end{array}\right.

(A3)

For $i=1,\dots,m$ , $j=1,\dots,p$ , and $k=1,\dots,q$ ,

$\displaystyle\bm{x}_{i,j}^{\prime}\bm{z}_{i,k}=$ $\displaystyle~{}o(n_{i}^{(\xi+\ell)/2-\tau}),$

where $\tau$ , $\xi$ , and $\ell$ are given in (A0), (A1), and (A2), respectively.

Condition (A0) allows the numbers of fixed effects and random effects (i.e., $p$ and $q$ ) to go to infinity with $n_{\min}$ at a certain rate. Conditions (A1)–(A3) impose correlation constraints on $\{\bm{x}_{i,j}\}$ and $\{\bm{z}_{i,k}\}$ . For example, Condition (A2) implies that the maximum eigenvalue satisfies $\lambda_{\max}(\bm{Z}_{i}\bm{D}\bm{Z}_{i}^{\prime})=O(n_{i}^{\ell})$ , which is similar to an assumption given in Condition 3 of Fan and Li [5].

3 Asymptotic Properties

In this section, we investigate the asymptotic properties of the ML estimators of $v^{2}$ and $\{\sigma_{k}^{2}:k\in\gamma\}$ for any $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ . We allow $p$ and $q$ to go to infinity with the sample size $N$ . In addition, as we allow $m$ to be fixed, we must account for the fact that $\{\sigma_{k}^{2}:k\in\gamma\}$ may not be estimated consistently.

3.1 Asymptotics under correct specification

In this subsection, we consider a correct (but possibly overfitted) model $(\alpha,\gamma)\in\mathcal{A}_{0}\times\mathcal{G}_{0}$ . We derive not only the convergence rates for the ML estimators of $v^{2}$ and $\{\sigma_{k}^{2}:k\in\gamma\}$ , but also their asymptotic distributions.

Theorem 1.

Consider the data generated from (2) with the true parameters given by (6). Let $(\alpha,\gamma)\in\mathcal{A}_{0}\times\mathcal{G}_{0}$ be a correct model defined in (4). Denote $\hat{\sigma}_{k}^{2}(\alpha,\gamma)$ and $\hat{v}^{2}(\alpha,\gamma)$ to be the ML estimators of $\sigma_{k}^{2}$ and $v^{2}$ , respectively. Assume that (A0)–(A3) hold. Then

	$\displaystyle\hat{v}^{2}(\alpha,\gamma)=$	$\displaystyle~{}v_{0}^{2}+O_{p}\Big{(}\frac{p+mq}{N}\Big{)}+O_{p}(N^{-1/2}),$		(1)
	$\displaystyle\hat{\sigma}_{k}^{2}(\alpha,\gamma)=$	$\displaystyle~{}\left\{\begin{array}[]{ll}\displaystyle\frac{1}{m}\sum_{i=1}^{m}b_{i,k}^{2}+O_{p}\bigg{(}\frac{1}{m}\sum_{i=1}^{m}n_{i}^{-\ell/2}\bigg{)};&\mbox{if }k\in\gamma\cap\gamma_{0},\\ O_{p}\big{(}n_{\max}^{-\ell}\big{)};&\mbox{if }k\in\gamma\setminus\gamma_{0},\end{array}\right.$		(4)

where $\displaystyle n_{\max}=\max_{i=1,\ldots,m}n_{i}$ . In addition, if $p+mq=o\big{(}N^{1/2}\big{)}$ , then

\displaystyle N^{1/2}\big{(}\hat{v}^{2}(\alpha,\gamma)-v_{0}^{2}\big{)}\xrightarrow{d}N\big{(}0,2v_{0}^{4}\big{)},\quad\mbox{as }N\rightarrow\infty.

When $m$ is fixed and $(\alpha,\gamma)\in\mathcal{A}_{0}\times\mathcal{G}_{0}$ , it follows from (4) that $\hat{\sigma}_{k}^{2}(\alpha,\gamma)$ does not converge to $\sigma_{k,0}^{2}$ , for $k\in\gamma\cap\gamma_{0}$ . This is because the data do not contain enough information for $\{\sigma_{k}^{2}:k\in\gamma\cap\gamma_{0}\}$ . Nevertheless, $\hat{\sigma}^{2}_{k}(\alpha,\gamma)$ converges to $\sigma_{k,0}^{2}=0$ , for $k\in\gamma\setminus\gamma_{0}$ , at a rate $n_{\max}^{-\ell}$ , which can be faster than $N^{-1/2}$ . On the other hand, when $m\rightarrow\infty$ , by applying the law of large numbers and the central limit theorem to $b_{i,k}$ ; $i=1,\dots,m,\,k\in\gamma_{0}$ , we immediately have the following corollary.

Corollary 1.

Under the assumptions of Theorem 1, $\hat{\sigma}_{k}^{2}(\alpha,\gamma)\xrightarrow{p}\sigma_{k,0}^{2}$ as $m\rightarrow\infty$ , for $k\in\gamma$ . If, in addition, $m=o(n_{\min}^{\ell})$ , then

\displaystyle m^{1/2}(\hat{\sigma}_{k}^{2}(\alpha,\gamma)-\sigma_{k,0}^{2})\xrightarrow{d}N(0,2\sigma_{k,0}^{4});\quad k\in\gamma\cap\gamma_{0},\quad\mbox{as }N\rightarrow\infty.

From Corollary 1, for $k\in\gamma_{0}$ , we obtain a $100(1-\alpha)\%$ confidence interval of $\sigma^{2}_{k,0}$ :

\displaystyle\bigg{(}\hat{\sigma}_{k}^{2}(\alpha,\gamma)-\bigg{(}\frac{2\hat{\sigma}^{4}_{k}(\alpha,\gamma)}{m}\bigg{)}^{1/2}\zeta_{1-\alpha/2},\,\hat{\sigma}_{k}^{2}(\alpha,\gamma)-\bigg{(}\frac{2\hat{\sigma}^{4}_{k}(\alpha,\gamma)}{m}\bigg{)}^{1/2}\zeta_{\alpha/2}\bigg{)},

(5)

where $\zeta_{a}$ is the $(100a)$ -th percentile of the standard normal distribution. Although this confidence interval is commonly applied in practice (e.g., Maas and Hox [14]; McNeish and Stapleton [17]), it is only valid when $m$ is large, as detailed in a simulation experiment of Section 4.2. Thanks to Theorem 1, we can derive a $100(1-\alpha)\%$ confidence interval of $\sigma_{k,0}^{2}$ valid for a fixed $m$ .

Theorem 2.

Under the assumptions of Theorem 1, suppose that $m$ is fixed. Then for $k\in\gamma\cap\gamma_{0}$ , a $100(1-\alpha)\%$ confidence interval of $\sigma^{2}_{k}$ is

\displaystyle\bigg{(}\frac{m\hat{\sigma}_{k}^{2}(\alpha,\gamma)}{\chi^{2}_{m,1-\alpha/2}},\frac{m\hat{\sigma}_{k}^{2}(\alpha,\gamma)}{\chi^{2}_{m,\alpha/2}}\bigg{)},

(6)

where $\chi^{2}_{m,a}$ denotes the $(100a)$ -th percentile of the chi-square distribution on $m$ degrees of freedom.

Note that the length of the confidence interval of $\sigma_{k,0}^{2}$ in (6) does not shrink to $0$ as $N\rightarrow\infty$ , which is not surprising due to the fact that $\hat{\sigma}_{k}^{2}(\alpha,\gamma)$ is not a consistent estimator of $\sigma_{k}^{2}$ when $m$ is fixed, for $k\in\gamma\cap\gamma_{0}$ .

We close this section by mentioning that although a fixed $m$ hinders us from consistently estimating $\sigma_{k}^{2}$ , the empirical BLUPs of random effects, based on the ML estimator of $\sigma_{k}^{2}$ , are still asymptotically more efficient than the LS predictors, as illustrated in the following example.

Example 1.

Consider model (2) with $p=0$ , $q=1$ , $n_{1}=\cdots=n_{m}=n$ and $m>1$ fixed. Assume that (A2) holds with $c_{1,1}=\cdots=c_{m,1}=1$ and $\ell=1$ . Let $\tilde{\bm{b}}_{i}$ be the LS predictor of $\bm{b}_{i}$ and $\hat{\bm{b}}_{i}(\sigma_{1}^{2},v^{2})$ be the BLUP of $\bm{b}_{i}$ given $(\sigma_{1}^{2},v^{2})$ . Define

\displaystyle D(\sigma_{1}^{2},v^{2})\equiv

\displaystyle~{}\sum_{i=1}^{m}\big{\|}\bm{Z}_{i}\big{(}\tilde{\bm{b}}_{i}-\bm{b}_{i})\big{\|}^{2}-\sum_{i=1}^{m}\big{\|}\bm{Z}_{i}\big{(}\hat{\bm{b}}_{i}(\sigma_{1}^{2},v^{2})-\bm{b}_{i}\big{)}\big{\|}^{2}.

Then, we show in Appendix B of the supplementary material that

\displaystyle nD(\hat{\sigma}_{1}^{2},\hat{v}^{2})=

\displaystyle~{}G_{n,m}+o_{p}(1),

where $\hat{\sigma}_{1}^{2}$ and $\hat{v}^{2}$ are the ML estimators of $\sigma_{1}^{2}$ and $v^{2}$ , and $G_{n,m}$ is some random variable depending on $n,m$ . Moreover, it is shown in the same appendix that the moments of $G_{n,m}$ do not exist for $m\leq 4$ and

\displaystyle\mathrm{E}(G_{n,m})=

\displaystyle~{}\frac{m(m-4)v_{0}^{4}}{(m-2)\sigma_{1,0}^{2}}

(7)

for $m>4$ . Equation (7) reveals that for any fixed $m>4$ , the empirical BLUP, $\bm{Z}_{i}\hat{\bm{b}}_{i}(\hat{\sigma}_{1}^{2},\hat{v}^{2})$ of $\bm{Z}_{i}\bm{b}_{i}$ , is asymptotically more efficient than its LS counterpart, $\bm{Z}_{i}\tilde{\bm{b}}_{i}$ , even when $\hat{\sigma}_{1}^{2}$ is not a consistent estimator of $\sigma_{1}^{2}$ . In addition, the advantage of the former over the latter rapidly increases with $m$ .

3.2 Asymptotics under misspecification

In this subsection, we consider a misspecified model $(\alpha,\gamma)\in(\mathcal{A}\times\mathcal{G})\setminus(\mathcal{A}_{0}\times\mathcal{G}_{0})$ . We derive not only the convergence rates for $\hat{v}^{2}(\alpha,\gamma)$ and $\{\hat{\sigma}_{k}^{2}(\alpha,\gamma):k\in\gamma\}$ , but also their asymptotic distributions. These results are crucial for developing model selection consistency and efficiency in linear mixed-effects models under fixed $m$ ; see Chang et al. [3].

We start by investigating the asymptotic properties for the ML estimators of $v^{2}$ and $\{\sigma_{k}^{2}:k\in\gamma\}$ for $(\alpha,\gamma)\in\mathcal{A}_{0}\times(\mathcal{G}\setminus\mathcal{G}_{0})$ under a misspecified random-effects model.

Theorem 3.

Under the assumptions of Theorem 1, except that $(\alpha,\gamma)\in\mathcal{A}_{0}\times(\mathcal{G}\setminus\mathcal{G}_{0})$ ,

\displaystyle\begin{split}\hat{v}^{2}(\alpha,\gamma)=&~{}v_{0}^{2}+\frac{1}{N}\sum_{i=1}^{m}\bigg{(}n_{i}^{\ell}\sum_{k\in\gamma_{0}\setminus\gamma}c_{i,k}b_{i,k}^{2}\bigg{)}+o_{p}\bigg{(}\frac{1}{N}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}\\ &~{}+O_{p}\Big{(}\frac{p+mq}{N}\Big{)}+O_{p}(N^{-1/2})\end{split}

(8)

and

\displaystyle\hat{\sigma}_{k}^{2}(\alpha,\gamma)=

\displaystyle~{}\left\{\begin{array}[]{ll}\displaystyle\frac{1}{m}\sum_{i=1}^{m}b_{i,k}^{2}+\displaystyle o_{p}(a_{N}(\xi,\ell))+o_{p}(1);&\mbox{if }k\in\gamma\cap\gamma_{0},\\ \displaystyle o_{p}(a_{N}(\xi,\ell))+o_{p}(1);&\mbox{if }k\in\gamma\setminus\gamma_{0},\end{array}\right.

(11)

where $a_{N}(\xi,\ell)=\displaystyle\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\xi-\ell}}{m}\bigg{)}$ . In addition, if $\ell<1$ , then

\hat{v}^{2}(\alpha,\gamma)\xrightarrow{p}v_{0}^{2},\quad\mbox{as }N\rightarrow\infty.

Furthermore, if $\ell\in(0,1/2)$ and $p+mq=o\big{(}N^{1/2}\big{)}$ , then

\displaystyle N^{1/2}(\hat{v}^{2}(\alpha,\gamma)-v_{0}^{2})\xrightarrow{d}N(0,2v_{0}^{4}),\quad\mbox{as }N\rightarrow\infty.

Note that $\displaystyle\frac{1}{N}\sum_{i=1}^{m}\bigg{(}n_{i}^{\ell}\sum_{k\in\gamma_{0}\setminus\gamma}c_{i,k}b_{i,k}^{2}\bigg{)}$ in (8) is the dominant bias term for $\hat{v}^{2}(\alpha,\gamma)$ , which is contributed by the non-negligible random effects missed by model $\gamma$ . It is asymptotically positive with probability one when $\ell=1$ . Hence $\hat{v}^{2}(\alpha,\gamma)$ has a non-negligible positive bias when $\ell=1$ . On the other hand, for $\xi=\ell$ or nearly balanced data, the following corollary shows that $\hat{\sigma}_{k}^{2}(\alpha,\gamma)\xrightarrow{p}\sigma_{k}^{2}$ ; $k\in\gamma$ , as $m\rightarrow\infty$ , even though $(\alpha,\gamma)\in\mathcal{A}_{0}\times(\mathcal{G}\setminus\mathcal{G}_{0})$ is misspecified.

Corollary 2.

Under the assumptions of Theorem 3, with $\xi=\ell$ or $n_{\max}=O(n_{\min})$ ,

\displaystyle\hat{\sigma}_{k}^{2}(\alpha,\gamma)=

\displaystyle~{}\left\{\begin{array}[]{ll}\displaystyle\frac{1}{m}\sum_{i=1}^{m}b_{i,k}^{2}+o_{p}(1);&\mbox{if }k\in\gamma\cap\gamma_{0},\\ o_{p}(1);&\mbox{if }k\in\gamma\setminus\gamma_{0}.\end{array}\right.

If $m\rightarrow\infty$ , then

\hat{\sigma}_{k}^{2}(\alpha,\gamma)\xrightarrow{p}\sigma_{k,0}^{2};\quad k\in\gamma,\quad\mbox{as }N\rightarrow\infty.

The following theorem presents the asymptotic properties of $\hat{v}^{2}(\alpha,\gamma)$ and $\{\hat{\sigma}_{k}^{2}(\alpha,\gamma):k\in\gamma\}$ for $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times\mathcal{G}$ under a misspecified fixed-effects model.

Theorem 4.

Under the assumptions of Theorem 1 except that $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times\mathcal{G}_{0}$ ,

\displaystyle\begin{split}\hat{v}^{2}(\alpha,\gamma)=&~{}v_{0}^{2}+\frac{1}{N}\sum_{i=1}^{m}\bigg{(}n_{i}^{\xi}\sum_{j\in\alpha_{0}\setminus\alpha}d_{i,j}\beta_{j,0}^{2}\bigg{)}+o_{p}\bigg{(}\frac{1}{N}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}\\ &~{}+O_{p}\Big{(}\frac{p+mq}{N}\Big{)}+O_{p}(N^{-1/2})\end{split}

(12)

and

\displaystyle\hat{\sigma}_{k}^{2}(\alpha,\gamma)=

\displaystyle~{}\left\{\begin{array}[]{ll}\displaystyle\frac{1}{m}\sum_{i=1}^{m}b_{i,k}^{2}+o_{p}\bigg{(}\frac{1}{m}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{)}+o_{p}(1);&\mbox{if }k\in\gamma\cap\gamma_{0},\\ \displaystyle o_{p}\bigg{(}\frac{1}{m}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{)}+o_{p}(1);&\mbox{if }k\in\gamma\setminus\gamma_{0}.\end{array}\right.

(15)

In addition, if $\xi<1$ , then

\hat{v}^{2}(\alpha,\gamma)\xrightarrow{p}v_{0}^{2},\quad\mbox{as }N\rightarrow\infty.

Furthermore, if $\xi\in(0,1/2)$ and $p+mq=o\big{(}N^{1/2}\big{)}$ , then

\displaystyle N^{1/2}(\hat{v}^{2}(\alpha,\gamma)-v_{0}^{2})\xrightarrow{d}N(0,2v_{0}^{4}),\quad\mbox{as }N\rightarrow\infty.

Note that $\displaystyle\frac{1}{N}\sum_{i=1}^{m}\bigg{(}n_{i}^{\xi}\sum_{j\in\alpha_{0}\setminus\alpha}d_{i,j}\beta_{j,0}^{2}\bigg{)}$ in (12) is asymptotically positive with probability one when $\xi=1$ . Therefore, under the assumptions of Theorem 4, $\hat{v}^{2}(\alpha,\gamma)$ has a non-negligible positive bias when $\xi=1$ . Nevertheless, $\hat{\sigma}_{k}^{2}(\alpha,\gamma)$ is consistent for $\gamma\in\mathcal{G}_{0}$ when $\xi\leq\ell$ , as $m\rightarrow\infty$ .

The following theorem establishes the asymptotic properties of $\hat{v}^{2}(\alpha,\gamma)$ and $\{\hat{\sigma}_{k}^{2}(\alpha,\gamma):k\in\gamma\}$ for $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times(\mathcal{G}\setminus\mathcal{G}_{0})$ when both the fixed-effects model and the random-effects model are misspecified.

Theorem 5.

Under the assumptions of Theorem 1 except that $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times(\mathcal{G}\setminus\mathcal{G}_{0})$ ,

\displaystyle\begin{split}\hat{v}^{2}(\alpha,\gamma)=&~{}v_{0}^{2}+\frac{1}{N}\sum_{i=1}^{m}\Bigg{(}n_{i}^{\xi}\sum_{j\in\alpha_{0}\setminus\alpha}d_{i,j}\beta_{j,0}^{2}+n_{i}^{\ell}\sum_{k\in\gamma_{0}\setminus\gamma}c_{i,k}b_{i,k}^{2}\Bigg{)}\\ &~{}+o_{p}\bigg{(}\frac{1}{N}\sum_{i=1}^{m}(n_{i}^{\xi}+n_{i}^{\ell})\bigg{)}+O_{p}\Big{(}\frac{p+mq}{N}\Big{)}+O_{p}(N^{-1/2})\end{split}

(16)

and

\displaystyle\hat{\sigma}_{k}^{2}(\alpha,\gamma)=

\displaystyle~{}\left\{\begin{array}[]{ll}\displaystyle\frac{1}{m}\sum_{i=1}^{m}b_{i,k}^{2}+\displaystyle o_{p}(a_{N}^{*}(\xi,\ell))+o_{p}(1);&\mbox{if }k\in\gamma\cap\gamma_{0},\\ \displaystyle o_{p}(a_{N}^{*}(\xi,\ell))+o_{p}(1);&\mbox{if }k\in\gamma\setminus\gamma_{0},\end{array}\right.

(19)

where $a_{N}^{*}(\xi,\ell)=\displaystyle\bigg{(}1+\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\xi-\ell}}{m}\bigg{)}$ . In addition, if $\max\{\xi,\ell\}<1$ , then

\hat{v}^{2}(\alpha,\gamma)\xrightarrow{p}v_{0}^{2},\quad\mbox{as }N\rightarrow\infty.

Furthermore, if $(\xi,\ell)\in(0,1/2)\times(0,1/2)$ and $p+mq=o\big{(}N^{1/2}\big{)}$ , then

\displaystyle N^{1/2}(\hat{v}^{2}(\alpha,\gamma)-v_{0}^{2})\xrightarrow{d}N(0,2v_{0}^{4}),\quad\mbox{as }N\rightarrow\infty.

Note that $\displaystyle\frac{1}{N}\sum_{i=1}^{m}\Bigg{(}n_{i}^{\xi}\sum_{j\in\alpha_{0}\setminus\alpha}d_{i,j}\beta_{j,0}^{2}+n_{i}^{\ell}\sum_{k\in\gamma_{0}\setminus\gamma}c_{i,k}b_{i,k}^{2}\Bigg{)}$ in (16) is asymptotically positive with probability one when either $\xi=1$ or $\ell=1$ . Therefore, under the assumptions of Theorem 5, $\hat{v}^{2}(\alpha,\gamma)$ has a non-negligible positive bias when either $\xi=1$ or $\ell=1$ . Also, we have the following corollary.

Corollary 3.

Under the assumptions of Theorem 5, with either (i) $\xi=\ell$ or (ii) $\xi<\ell$ and $n_{\max}=O(n_{\min})$ ,

\displaystyle\hat{\sigma}_{k}^{2}(\alpha,\gamma)=

\displaystyle~{}\left\{\begin{array}[]{ll}\displaystyle\frac{1}{m}\sum_{i=1}^{m}b_{i,k}^{2}+o_{p}(1);&\mbox{if }k\in\gamma\cap\gamma_{0},\\ o_{p}(1);&\mbox{if }k\in\gamma\setminus\gamma_{0}.\end{array}\right.

If $m\rightarrow\infty$ , then

\hat{\sigma}_{k}^{2}(\alpha,\gamma)\xrightarrow{p}\sigma_{k,0}^{2};\quad k\in\gamma,\quad\mbox{as }N\rightarrow\infty.

4 Simulations

We conduct two simulation experiments for linear mixed-effects models. The first one examines estimation of mixed-effects models, and the second concerns confidence intervals.

4.1 Experiment 1

We generate data according to (1) with $p=q=5$ , $(\sigma_{1,0}^{2},\sigma_{2,0}^{2},\sigma_{3,0}^{2},\sigma_{4,0}^{2},\sigma_{5,0}^{2})^{\prime}=(0,0.5,1,1.5,0)^{\prime}$ , $\bm{\beta}_{0}=(1.2,-0.7,0.8,0,0)^{\prime}$ , and $v^{2}=1$ , where $\bm{x}_{i,j}\sim N(\bm{0},\bm{I}_{n_{i}})$ and $\bm{z}_{i,k}\sim N(\bm{0},\bm{I}_{n_{i}})$ are independent, for $i=1,\dots,m$ and $j,k=1,\dots,5$ . This setup satisfies (A1)–(A3) with $\xi=\ell=1$ and $d_{i,j}=c_{i,k}=1$ , for $i=1,\dots,m$ and $j,k=1,\dots,5$ . We consider parameter estimation under two scenarios corresponding to balanced data and unbalanced data. We also consider model selection under balanced data.

For parameter estimation, we first consider balanced data with $m\in\{10,20,30\}$ , $n_{1}=\cdots=n_{m}=m$ , and hence $N=m^{2}$ . The ML estimators of $\sigma_{1}^{2},\dots,\sigma_{5}^{2}$ and $v^{2}$ under the full model $(\{1,\dots,5\},\{1,\dots,5\})\in\mathcal{A}_{0}\times\mathcal{G}_{0}$ based on 100 simulated replicates are summarized in Table 1. The ML estimators of $\sigma_{1}^{2},\dots,\sigma_{3}^{2}$ and $v^{2}$ under model $(\{1,2,3\},\{1,2,3\})\in\mathcal{A}_{0}\times(\mathcal{G}\setminus\mathcal{G}_{0})$ with correct fixed effects but misspecified random effects based on 100 simulated replicates are summarized in Table 2. The ML estimators of $\sigma_{4}^{2},\sigma_{5}^{2}$ , and $v^{2}$ under model $(\{2,3,4,5\},\{4,5\})\in(\mathcal{A}\setminus\mathcal{A}_{0})\times(\mathcal{G}\setminus\mathcal{G}_{0})$ with both misspecified fixed and random effects based on 100 simulated replicates are summarized in Table 3.

Table 1: Sample means and sample standard deviations (in parentheses) of ML estimators of

\sigma_{1}^{2},\dots,\sigma_{5}^{2}

and

v^{2}

for different values of

m

obtained from full model in Section 4.1 with balanced data based on 100 simulated replicates. Values in row for

m=\infty

are probability limits of ML estimators.

$m$	$\hat{\sigma}_{1}^{2}$	$\hat{\sigma}_{2}^{2}$	$\hat{\sigma}_{3}^{2}$	$\hat{\sigma}_{4}^{2}$	$\hat{\sigma}_{5}^{2}$	$\hat{v}^{2}$
10	0.033	0.467	0.994	1.453	0.041	0.850
	(0.062)	(0.281)	(0.564)	(0.615)	(0.073)	(0.165)
20	0.008	0.512	1.028	1.470	0.008	0.983
	(0.017)	(0.194)	(0.362)	(0.490)	(0.014)	(0.087)
30	0.003	0.490	0.994	1.534	0.004	0.989
	(0.006)	(0.116)	(0.260)	(0.396)	(0.007)	(0.049)
$\infty$	0.000	0.500	1.000	1.500	0.000	1.000
True	0.000	0.500	1.000	1.500	0.000	1.000

Table 2: Sample means and sample standard deviations (in parentheses) of ML estimators of

\sigma_{1}^{2},\sigma_{2}^{2},\sigma_{3}^{2}

and

v^{2}

for different values of

m

obtained from model

(\alpha,\gamma)=(\{1,2,3\},\{1,2,3\})

in Section 4.1 with balanced data based on 100 simulated replicates. Values in row for

m=\infty

are probability limits of ML estimators.

$m$	$\hat{\sigma}_{1}^{2}$	$\hat{\sigma}^{2}_{2}$	$\hat{\sigma}^{2}_{3}$	$\hat{v}^{2}$
10	0.151 (0.541)	0.583 (0.546)	0.944 (0.572)	2.363 (1.054)
20	0.047 (0.091)	0.555 (0.285)	1.050 (0.392)	2.415 (0.650)
30	0.030 (0.070)	0.521 (0.198)	0.961 (0.268)	2.442 (0.666)
$\infty$	0.000	0.500	1.000	2.500
True	0.000	0.500	1.000	1.000

Table 3: Sample means and sample standard deviations (in parentheses) of ML estimators of

\sigma_{4}^{2},\sigma_{5}^{2}

and

v^{2}

for different values of

m

obtained from model

(\alpha,\gamma)=(\{2,3,4,5\},\{4,5\})

in Section 4.1 with balanced data based on 100 simulated replicates. Values in row for

m=\infty

are probability limits of ML estimators.

$m$	$\hat{\sigma}_{4}^{2}$	$\hat{\sigma}_{5}^{2}$	$\hat{v}^{2}$
10	1.604 (1.073)	0.176 (0.465)	3.494 (0.788)
20	1.353 (0.567)	0.043 (0.077)	3.915 (0.540)
30	1.525 (0.427)	0.030 (0.065)	3.880 (0.436)
$\infty$	1.500	0.000	3.690
True	1.500	0.000	1.000

As seen in Table 1, the ML estimators, $\hat{\sigma}_{1}^{2},\dots,\hat{\sigma}_{5}^{2}$ and $\hat{v}^{2}$ , based on the full model, have small biases except for $\hat{v}^{2}$ with $m=10$ . We note that their standard deviations tend to be smaller when $m$ is larger. In particular, the standard deviations of $\hat{\sigma}_{1}^{2}$ and $\hat{\sigma}_{5}^{2}$ are much smaller than the others, which echoes Theorem 1, that which shows that $\hat{v}^{2}_{j}$ has a faster convergence rate when it converges to zero. For model $(\alpha,\gamma)=(\{1,2,3\},\{1,2,3\})$ with misspecified random effects, Table 2 shows that the ML estimator $\hat{v}^{2}$ overestimates $v_{0}^{2}=1$ by about $\sigma_{4,0}^{2}=1.5$ on average, particularly for larger values of $m$ , which also complies with Theorem 3. Finally, for model $(\alpha,\gamma)=(\{2,3,4,5\},\{4,5\})$ with both fixed and random effects misspecified, Table 3 confirms that $\hat{v}^{2}$ is far from its true value and reasonably close to its probability limit, $v_{0}^{2}+\sigma_{2,0}^{2}+\sigma_{3,0}^{2}+\beta_{1,0}^{2}=3.69$ , derived in Theorem 5. In addition, $\hat{\sigma}_{4}^{2}$ tends to be closer to $\sigma_{4,0}^{2}$ when $m$ is larger, as expected from Theorem 5.

Next, we consider unbalanced data with $m\in\{10,20,30\}$ and $N=m^{2}$ . We set $n_{1}=[N^{1/4}]$ , $n_{2}=[N^{3/4}]$ , $n_{3}=\cdots=n_{m-1}=[(N-n_{1}-n_{2})/(m-2)]$ , and hence $n_{m}=N-\sum_{i=1}^{m-1}n_{i}$ . The ML estimators of $\sigma_{1}^{2},\dots,\sigma_{5}^{2}$ and $v^{2}$ under the full model $(\{1,\dots,5\},\{1,\dots,5\})\in\mathcal{A}_{0}\times\mathcal{G}_{0}$ based on 100 simulated replicates are summarized in Table 4. The ML estimators of $\sigma_{1}^{2},\dots,\sigma_{3}^{2}$ and $v^{2}$ under model $(\{1,2,3\},\{1,2,3\})\in\mathcal{A}_{0}\times(\mathcal{G}\setminus\mathcal{G}_{0})$ with correct fixed effects but misspecified random effects based on 100 simulated replicates are summarized in Table 5. The ML estimators of $\sigma_{4}^{2}$ , $\sigma_{5}^{2}$ , and $v^{2}$ under model $(\{2,3,4,5\},\{4,5\})\in(\mathcal{A}\setminus\mathcal{A}_{0})\times(\mathcal{G}\setminus\mathcal{G}_{0})$ with both misspecified fixed and random effects based on 100 simulated replicates are summarized in Table 6. The ML estimators of $\sigma_{1}^{2},\dots,\sigma_{5}^{2}$ and $v^{2}$ based on unbalanced data can be seen to perform similarly to those based on balanced data.

Table 4: Sample means and sample standard deviations (in parentheses) of ML estimators of

\sigma_{1}^{2},\dots,\sigma_{5}^{2}

and

v^{2}

for different values of

m

obtained from full model in Section 4.1 with unbalanced data based on 100 simulated replicates. Values in row for

m=\infty

are probability limits of ML estimators.

$m$	$n_{\min}$	$n_{\max}$	$\hat{\sigma}_{1}^{2}$	$\hat{\sigma}_{2}^{2}$	$\hat{\sigma}_{3}^{2}$	$\hat{\sigma}_{4}^{2}$	$\hat{\sigma}_{5}^{2}$	$\hat{v}^{2}$
10	3	32	0.017	0.500	1.011	1.414	0.021	0.877
			(0.041)	(0.330)	(0.602)	(0.790)	(0.040)	(0.154)
20	4	89	0.009	0.516	1.029	1.490	0.008	0.974
			(0.020)	(0.175)	(0.399)	(0.493)	(0.014)	(0.082)
30	5	164	0.002	0.497	1.007	1.539	0.004	0.991
			(0.005)	(0.121)	(0.263)	(0.374)	(0.008)	(0.049)
$\infty$			0.000	0.500	1.000	1.500	0.000	1.000
True			0.000	0.500	1.000	1.500	0.000	1.000

Table 5: Sample means and sample standard deviations (in parentheses) of ML estimators of

\sigma_{1}^{2},\sigma_{2}^{2},\sigma_{3}^{2}

, and

v^{2}

for different values of

m

obtained from model

(\alpha,\gamma)=(\{1,2,3\},\{1,2,3\})

in Section 4.1 with unbalanced data based on 100 simulated replicates. Values in row for

m=\infty

are probability limits of ML estimators.

$m$	$n_{\min}$	$n_{\max}$	$\hat{\sigma}_{1}^{2}$	$\hat{\sigma}_{2}^{2}$	$\hat{\sigma}_{3}^{2}$	$\hat{v}^{2}$
10	3	32	0.213 (0.927)	0.576 (0.696)	1.175 (1.220)	2.283 (1.127)
20	4	89	0.044 (0.096)	0.536 (0.260)	1.091 (0.461)	2.456 (0.792)
30	5	165	0.028 (0.079)	0.500 (0.208)	0.959 (0.290)	2.426 (0.645)
$\infty$			0.000	0.500	1.000	2.500
True			0.000	0.500	1.000	1.000

Table 6: Sample means and sample standard deviations (in parentheses) of ML estimators of

\sigma_{4}^{2},\sigma_{5}^{2}

, and

v^{2}

for different values of

m

obtained from model

(\alpha,\gamma)=(\{2,3,4,5\},\{4,5\})

in Section 4.1 with unbalanced data based on 100 simulated replicates. Values in row for

m=\infty

are probability limits of ML estimators.

$m$	$n_{\min}$	$n_{\max}$	$\hat{\sigma}_{4}^{2}$	$\hat{\sigma}_{5}^{2}$	$\hat{v}^{2}$
10	3	32	1.522 (0.902)	0.065 (0.180)	3.535 (0.944)
20	4	89	1.362 (0.540)	0.057 (0.142)	3.960 (0.716)
30	5	164	1.494 (0.458)	0.030 (0.068)	3.892 (0.539)
$\infty$			1.500	0.000	3.690
True			1.500	0.000	1.000

4.2 Experiment 2

In the second experiment, we compare the conventional confidence interval given by (5) with the proposed confidence interval given by (6). Similar to Experiment 1, we generate data according to (1) with $p=q=5$ , $\bm{\beta}=(1.2,-0.7,0.8,0,0)^{\prime}$ , $v^{2}=1$ , and $(\sigma_{1,0}^{2},\sigma_{2,0}^{2},\sigma_{3,0}^{2},\sigma_{4,0}^{2},\sigma_{5,0}^{2})^{\prime}=(0,0.5,1,1.5,0)^{\prime}$ , where $\bm{x}_{i,j}\sim N(\bm{0},\bm{\Sigma}_{x})$ and $\bm{z}_{i,k}\sim N(\bm{0},\bm{\Sigma}_{z})$ are independent, for $i=1,\dots,m$ and $j,k=1,\dots,5$ . Here we consider a more challenging situation of dependent covariates. Specifically, we assume that $\bm{\Sigma}_{x}$ is a $5\times 5$ matrix with the $(i,j)$ -th entry $0.4^{|i-j|}$ , and $\bm{\Sigma}_{z}$ is a $5\times 5$ matrix with the $(i,j)$ -th entry $0.6^{|i-j|}$ . We consider balanced data with $n=n_{1}=\cdots=n_{m}\in\{10,50,100\}$ and three numbers of clusters, $m\in\{2,5,10\}$ , resulting in a total of nine different combinations.

We compare the 95 $\%$ confidence intervals of (5) and (6) for $\sigma_{2}^{2}$ and $\sigma_{4}^{2}$ based on model $(\alpha,\gamma)=(\{1,2,3\},\{2,3,4\})$ . The coverage probabilities of both confidence intervals obtained from the two methods for various cases based on 1,000 simulated replicates are shown in Table 7. The proposed method has better coverage probabilities than the conventional ones in almost all cases. The coverage probabilities of our confidence interval tend to the nominal level (i.e., $0.95$ ) as $n$ increases for all cases even when $m$ is very small. In contrast, the conventional method tends to be too optimistic for both $\sigma_{2}^{2}$ and $\sigma_{4}^{2}$ . For example, the coverage probabilities are less than $0.73$ when $m=2$ regardless of $n$ . Although the coverage probabilities are a bit closer to the nominal level when $m$ is larger, they are still in the range of $(0.82,0.87)$ when $m=10$ , showing that the conventional confidence interval is not valid for small $m$ .

Table 7: Coverage probabilities (denoted by

\hat{P}

) for

95\%

confidence intervals of

\sigma_{2}^{2}

and

\sigma_{4}^{2}

obtained from two methods in Section 4.2 based on 1,000 simulated replicates. Values given in parentheses are standard errors of coverage probabilities (evaluted by

\sqrt{\hat{P}(1-\hat{P})/1000}

$m$	$n$	Classical		Proposed
		$\sigma_{2}^{2}$	$\sigma_{4}^{2}$	$\sigma_{2}^{2}$	$\sigma_{4}^{2}$
2	10	0.651 (0.015)	0.649 (0.015)	0.814 (0.012)	0.763 (0.013)
	50	0.724 (0.014)	0.703 (0.014)	0.932 (0.008)	0.935 (0.008)
	100	0.725 (0.014)	0.722 (0.014)	0.942 (0.007)	0.929 (0.008)
5	10	0.778 (0.013)	0.738 (0.014)	0.895 (0.010)	0.871 (0.011)
	50	0.809 (0.012)	0.818 (0.012)	0.936 (0.008)	0.937 (0.008)
	100	0.811 (0.012)	0.809 (0.012)	0.940 (0.008)	0.929 (0.008)
10	10	0.838 (0.012)	0.816 (0.012)	0.900 (0.009)	0.893 (0.010)
	50	0.874 (0.010)	0.849 (0.011)	0.952 (0.007)	0.946 (0.007)
	100	0.849 (0.011)	0.867 (0.011)	0.941 (0.007)	0.956 (0.006)

5 Discussion

In this article, we establish the asymptotic theory of the ML estimators of random-effects parameters in linear mixed-effects models for unbalanced data, without assuming that $m$ grows to infinity with $N$ . We not only allow the dimensions of both the fixed-effects and random-effects models to go to infinity with $N$ , but also allow both models to be misspecified. In addition, we provide an asymptotic valid confidence interval for the random-effects parameters when $m$ is fixed. These asymptotic results are essential for investigating the asymptotic properties of model-selection methods for linear mixed-effects models, which to the best of our knowledge have only been developed under the assumption of $m\rightarrow\infty$ .

Although it is common to assume the random effects to be uncorrelated as done in model (1), it is also of interest to consider correlated random effects with no structure imposed on $\bm{D}$ . However, the technique developed in this article may not be directly applicable to the latter situation; further research in this direction is thus warranted.

Conditions (A1) and (A2) assume that the covariates are asymptotically uncorrelated. These restrictions can be relaxed. Here is a simple example.

Lemma 1.

Consider the data generated from (2) with $m=1$ , $n_{1}=N$ , $p=q=2$ , and the true parameters given in (6). Suppose that $(\alpha_{0},\gamma_{0})=(\{1,2\},\{1,2\})$ is the smallest true model and $(\alpha_{1},\gamma_{1})=(\{1\},\{1\})$ is a misspecified model defined in (4). Let $\hat{\sigma}_{k}^{2}(\alpha,\gamma)$ and $\hat{v}^{2}(\alpha,\gamma)$ be the ML estimators of $\sigma_{k}^{2}$ and $v^{2}$ based on $(\alpha,\gamma)$ . Assume that (A1)–(A3) hold except that $\bm{z}_{1,1}^{\prime}\bm{z}_{1,2}=c_{1,12}N+o(N)$ and $\bm{x}_{1,1}^{\prime}\bm{x}_{1,2}=d_{1,12}N+o(N)$ , for some constants $c_{1,12},d_{1,12}\in\mathbb{R}$ . Then

	$\displaystyle\hat{v}^{2}(\alpha_{0},\gamma_{0})=$	$\displaystyle~{}v_{0}^{2}+O_{p}(N^{-1/2}),$
	$\displaystyle\hat{\sigma}_{k}^{2}(\alpha_{0},\gamma_{0})=$	$\displaystyle~{}b_{k}^{2}+O_{p}(N^{-1/2});\quad k=1,2,$
	$\displaystyle\hat{v}^{2}(\alpha_{1},\gamma_{1})=$	$\displaystyle~{}v_{0}^{2}+\bigg{(}d_{1,2}-\frac{d_{1,12}^{2}}{d_{1,1}}\bigg{)}\beta_{2,0}^{2}+\bigg{(}c_{1,2}-\frac{c_{1,12}^{2}}{c_{1,1}}\bigg{)}b_{2}^{2}+o_{p}(1),$
	$\displaystyle\hat{\sigma}_{1}^{2}(\alpha_{1},\gamma_{1})=$	$\displaystyle~{}\bigg{(}b_{1}+\frac{c_{1,12}}{c_{1,1}}b_{2}\bigg{)}^{2}+o_{p}(1),$

where $\beta_{2,0}\neq 0$ is the true parameter of $\beta_{2}$ .

From Lemma 1, it is not surprising to see that $\hat{v}^{2}(\alpha_{0},\gamma_{0})\xrightarrow{p}v_{0}^{2}$ . On the other hand, $\hat{v}^{2}(\alpha_{1},\gamma_{1})$ tends to overestimate $v_{0}^{2}$ by $(d_{1,2}-d_{1,12}^{2}/d_{1,1})\beta_{2,0}^{2}+(c_{1,2}-c_{1,12}^{2}/c_{1,1})b_{2}^{2}$ . Since $d_{1,2}-d_{1,12}^{2}/d_{1,1}\geq 0$ and $c_{1,2}-c_{1,12}^{2}/c_{1,1}\geq 0$ , the amount of overestimation is smaller when either $c_{1,12}^{2}$ or $d_{1,12}^{2}$ is larger. In contrast, $\hat{\sigma}_{1}^{2}(\alpha_{1},\gamma_{1})$ tends to be more upward biased when $c_{1,12}^{2}$ is larger, since $\mathrm{E}\big{(}b_{1}+(c_{1,12}/c_{1,1})b_{2}\big{)}^{2}=\sigma_{1}^{2}+(c_{1,12}/c_{1,1})^{2}\sigma_{2}^{2}$ . Lemma 1 demonstrates how the correlations between the two covariates affect the behavior of $\hat{v}^{2}(\alpha_{1},\gamma_{1})$ and $\hat{\sigma}_{1}^{2}(\alpha_{1},\gamma_{1})$ . However, when the number of covariates is larger, the ML estimators of $v^{2}$ and $\{\sigma_{k}^{2}\}$ become much more complicated. We leave this extension of Lemma 1 to the general case for future work.

Acknowledgements

The research of Chih-Hao Chang is supported by ROC Ministry of Science and Technology grant MOST 107-2118-M-390-001.

The research of Hsin-Cheng Huang is supported by ROC Ministry of Science and Technology grant MOST 106-2118-M-001-002-MY3.

The research of Ching-Kang Ing is supported by the Science Vanguard Research Program under the Ministry of Science and Technology, Taiwan, ROC.

References

[1] {barticle}[author] \bauthor\bsnmBell, \bfnmB. A.\binitsB. A., \bauthor\bsnmMorgan, \bfnmG. B.\binitsG. B., \bauthor\bsnmSchoeneberger, \bfnmJ. A.\binitsJ. A., \bauthor\bsnmKromney, \bfnmJ. D.\binitsJ. D. and \bauthor\bsnmFerron, \bfnmJ. M.\binitsJ. M. (\byear2014). \btitleHow low can you go? \bjournalMethodology: European Journal of Research Methods for the Behavioral and Social Sciences \bvolume10 \bpages1–11. \endbibitem
[2] {barticle}[author] \bauthor\bsnmChang, \bfnmC. H.\binitsC. H., \bauthor\bsnmHuang, \bfnmH. C.\binitsH. C. and \bauthor\bsnmIng, \bfnmC. K.\binitsC. K. (\byear2017). \btitleMixed domain asymptotics for a stochastic process model with time trend and measurement error. \bjournalBernoulli \bvolume23 \bpages159-190. \endbibitem
[3] {barticle}[author] \bauthor\bsnmChang, \bfnmC. H.\binitsC. H., \bauthor\bsnmHuang, \bfnmH. C.\binitsH. C. and \bauthor\bsnmIng, \bfnmC. K.\binitsC. K. (\byear2020). \btitleSelection of Linear Mixed-Effects Models with a Small Number of Clusters. \bjournalSubmitted. \endbibitem
[4] {barticle}[author] \bauthor\bsnmCrainiceanu, \bfnmC. M.\binitsC. M. and \bauthor\bsnmRuppert, \bfnmD.\binitsD. (\byear2004). \btitleLikelihood ratio tests in linear mixed models with one variance component. \bjournalJournal of Royal Statistical Society. Series B \bvolume66 \bpages165–185. \endbibitem
[5] {barticle}[author] \bauthor\bsnmFan, \bfnmY.\binitsY. and \bauthor\bsnmLi, \bfnmR.\binitsR. (\byear2012). \btitleVariable selection in linear mixed effects models. \bjournalThe Annals of Statistics \bvolume40 \bpages2043-2068. \endbibitem
[6] {barticle}[author] \bauthor\bsnmGumedze, \bfnmF. N.\binitsF. N. and \bauthor\bsnmDunne, \bfnmT. T.\binitsT. T. (\byear2011). \btitleParameter estimation and inference in the linear mixed model. \bjournalLinear Algebra and Its Applications \bvolume435 \bpages1920-1944. \endbibitem
[7] {barticle}[author] \bauthor\bsnmHartley, \bfnmH. O.\binitsH. O. and \bauthor\bsnmRao, \bfnmJ. N. K.\binitsJ. N. K. (\byear1967). \btitleMaximum likelihood estimation for the mixed analysis of variance model. \bjournalBiometrika \bvolume54 \bpages93-108. \endbibitem
[8] {barticle}[author] \bauthor\bsnmHarville, \bfnmD. A.\binitsD. A. (\byear1977). \btitleMaximum likelihood approaches to variance components estimation and related problems. \bjournalJournal of the American Statistical Association \bvolume72 \bpages320-338. \endbibitem
[9] {barticle}[author] \bauthor\bsnmHuang, \bfnmF.\binitsF. (\byear2018). \btitleUsing cluster bootstrapping to analyze nested data with a few clusters. \bjournalEducational and Psychological Measurement \bvolume78 \bpages297-318. \endbibitem
[10] {barticle}[author] \bauthor\bsnmJiang, \bfnmJ.\binitsJ. (\byear2007). \btitleLinear and Generalized Linear Mixed Models and Their Applications. \bjournalSpringer, New York. \endbibitem
[11] {barticle}[author] \bauthor\bsnmJiang, \bfnmJ.\binitsJ. (\byear2017). \btitleAsymptotic Analysis of Mixed Effects Models: Theory, Applications, and Open Problems. \bjournalSpringer, New York. \endbibitem
[12] {barticle}[author] \bauthor\bsnmLaird, \bfnmN. M.\binitsN. M. and \bauthor\bsnmWare, \bfnmJ. H.\binitsJ. H. (\byear1982). \btitleRandom-effects models for longitudinal data. \bjournalBiometrics \bvolume38 \bpages963-974. \endbibitem
[13] {barticle}[author] \bauthor\bsnmLongford, \bfnmN. T.\binitsN. T. (\byear1993). \btitleRandom Coefficient Models Oxford Statistical Science Series 11. \bjournalOxford University Press, New York. \endbibitem
[14] {barticle}[author] \bauthor\bsnmMaas, \bfnmC. J. M.\binitsC. J. M. and \bauthor\bsnmHox, \bfnmJ. J.\binitsJ. J. (\byear2004). \btitleRobustness issues in multilevel regression analysis. \bjournalStatistica Neerlandica \bvolume58 \bpages127-137. \endbibitem
[15] {barticle}[author] \bauthor\bsnmMardia, \bfnmK. V.\binitsK. V. and \bauthor\bsnmMarshall, \bfnmR. J.\binitsR. J. (\byear1984). \btitleMaximum likelihood estimation of models for residual covariance in spatial regression. \bjournalBiometrika \bvolume71 \bpages135-146. \endbibitem
[16] {barticle}[author] \bauthor\bsnmMcNeish, \bfnmD.\binitsD. and \bauthor\bsnmStapleton, \bfnmL. M.\binitsL. M. (\byear2016). \btitleModeling clustered data with very few clusters. \bjournalMultivariate Behavioral Research \bvolume51 \bpages495-518. \endbibitem
[17] {barticle}[author] \bauthor\bsnmMcNeish, \bfnmD.\binitsD. and \bauthor\bsnmStapleton, \bfnmL. M.\binitsL. M. (\byear2016). \btitleThe effect of small sample size on two-level model estimates: A review and illustration. \bjournalEducational Psychology Review \bvolume28 \bpages295-314. \endbibitem
[18] {barticle}[author] \bauthor\bsnmMiller, \bfnmJ. J.\binitsJ. J. (\byear1977). \btitleAsymptotic properties of maximum likelihood estimates in the mixed model of the analysis of variance. \bjournalThe Annals of Statistics \bvolume5 \bpages746-762. \endbibitem
[19] {barticle}[author] \bauthor\bsnmPeng, \bfnmH.\binitsH. and \bauthor\bsnmLu, \bfnmY.\binitsY. (\byear2012). \btitleModel selection in linear mixed effect models. \bjournalJournal of Multivariate Analysis \bvolume109 \bpages109-129. \endbibitem
[20] {barticle}[author] \bauthor\bsnmPu, \bfnmW.\binitsW. and \bauthor\bsnmNiu, \bfnmX. F.\binitsX. F. (\byear2006). \btitleSelecting mixed-effects models based on a generalized information criterion. \bjournalJournal of Multivariate Analysis \bvolume97 \bpages733-758. \endbibitem
[21] {barticle}[author] \bauthor\bsnmSelf, \bfnmS. G.\binitsS. G. and \bauthor\bsnmLiang, \bfnmK. Y.\binitsK. Y. (\byear1987). \btitleAsymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. \bjournalJournal of the American Statistical Association \bvolume82 \bpages605-610. \endbibitem
[22] {barticle}[author] \bauthor\bsnmStram, \bfnmD. O.\binitsD. O. and \bauthor\bsnmLee, \bfnmJ. W.\binitsJ. W. (\byear1994). \btitleVariance component testing in the longitudinal mixed effects model. \bjournalBiometrics \bvolume50 \bpages1171-1177. \endbibitem
[23] {barticle}[author] \bauthor\bsnmVerbeke, \bfnmG.\binitsG. and \bauthor\bsnmMolenberghs, \bfnmG.\binitsG. (\byear2000). \btitleLinear Mixed Models for Longitudinal Data. \bjournalSpringer, New York. \endbibitem

Supplementary Material

The supplementary materials consist of three appendices that prove all the theoretical results except for Theorem 2, whose proof is straightforward and is hence omitted. Appendix A contains auxiliary lemmas that are required in the proofs. Appendix B provides proofs for Example 1 and Theorems 1 and 3–5. Appendix C gives proofs for all the lemmas.

Appendix A Auxiliary Lemmas

We start with the following matrix identities, which will be repeated applied:

	$\displaystyle\det(\bm{A}+\bm{c}\bm{d}^{\prime})=$	$\displaystyle~{}\det(\bm{A})(1+\bm{d}^{\prime}\bm{A}^{-1}\bm{c}),$		(A.1)
	$\displaystyle(\bm{A}+\bm{c}\bm{d}^{\prime})^{-1}=$	$\displaystyle~{}\bm{A}^{-1}-\frac{\bm{A}^{-1}\bm{c}\bm{d}^{\prime}\bm{A}^{-1}}{1+\bm{d}^{\prime}\bm{A}^{-1}\bm{c}},$		(A.2)

where $\bm{A}$ is an $n\times n$ nonsingular matrix, and $\bm{c}$ and $\bm{d}$ are $n\times 1$ column vectors. Note that (A.2) is applied iteratively to establish the decomposition of the precision matrix $\bm{H}_{i}^{-1}(\gamma,\bm{\theta})$ , where

\displaystyle\bm{H}_{i}(\gamma,\bm{\theta})\equiv

\displaystyle~{}\sum_{k\in\gamma}\theta_{k}\bm{z}_{i,k}\bm{z}_{i,k}^{\prime}+\bm{I}_{n_{i}}.

(A.3)

Heuristically speaking, let $\bm{z}_{i,(s)}$ ; $s=1,\dots,q(\gamma)$ be the $s$ -th column of $\bm{Z}_{i}(\gamma)$ and

\displaystyle\bm{H}_{i,t}(\gamma,\bm{\theta})=\sum_{s=1}^{t}\theta_{(s)}\bm{z}_{i,(s)}\bm{z}_{i,(s)}^{\prime}+\bm{I}_{n_{i}};\quad t=1,\dots,q(\gamma),

(A.4)

where $\theta_{(s)}$ denotes the $s$ -th element of $\bm{\theta}$ ; $s=1,\dots,q(\gamma)$ . Suppose that $q(\gamma)=q$ . Then by (A.2),

\displaystyle\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})=

\displaystyle~{}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})-\frac{\theta_{q}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,q}\bm{z}_{i,q}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})}{1+\theta_{q}\bm{z}_{i,q}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,q}}.

(A.5)

Applying (A.2) iteratively, we obtain the decomposition

\displaystyle\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})=

\displaystyle~{}\bm{I}_{n_{i}}-\sum_{k=1}^{q}\frac{\theta_{k}\bm{H}_{i,k-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}\bm{z}_{i,k}^{\prime}\bm{H}_{i,k-1}^{-1}(\gamma,\bm{\theta})}{1+\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i,k-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}};

(A.6)

note that $\bm{H}_{i,0}(\gamma,\bm{\theta})=\bm{I}_{n_{i}}$ . The proofs of Lemmas 2, 3, and 4 are then based on the induction and the decomposition of (A.6).

The proofs of theorems in Section 3 heavily rely on the asymptotic properties of the quadratic forms, $\bm{x}^{\prime}_{i,j}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j^{*}}$ , $\bm{z}^{\prime}_{i,k}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k^{*}}$ , $\bm{\epsilon}^{\prime}_{i}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}$ , $\bm{x}^{\prime}_{i,j}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}$ , $\bm{x}^{\prime}_{i,j}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}$ , and $\bm{z}^{\prime}_{i,k}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}$ , with $\bm{H}_{i}(\gamma,\bm{\theta})$ defined in (A.3), for $i=1,\dots,m$ ; $j,j^{*}=1,\dots,p$ and $k,k^{*}=1,\dots,q$ . The following lemmas give their convergence rates.

Lemma 2.

Consider the linear mixed-effects model $(\alpha,\gamma)$ of (4). Suppose that (A0)–(A3) hold. Then for $\bm{H}_{i}(\gamma,\bm{\theta})$ defined in (A.3), we have

(i)

For $i=1,\dots,m$ and $j,j^{*}=1,\dots,p$ ,

\displaystyle\sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\big{|}\bm{x}_{i,j}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j^{*}}\big{|}=

\displaystyle~{}\left\{\begin{array}[]{ll}d_{i,j}n_{i}^{\xi}+o(n_{i}^{\xi});&\mbox{if }j=j^{*},\\ o(n_{i}^{\xi-\tau});&\mbox{if }j\neq j^{*}.\end{array}\right.

(ii)

For $i=1,\dots,m$ , $j=1,\dots,p$ and $k\notin\gamma$ ,

\displaystyle\sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\big{|}\bm{x}_{i,j}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}\big{|}=

\displaystyle~{}o(n_{i}^{(\xi+\ell)/2-\tau}).

(iii)

For $i=1,\dots,m$ , $j=1,\dots,p$ and $k\in\gamma$ ,

\displaystyle\begin{split}\sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\theta_{k}\big{|}\bm{x}_{i,j}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}\big{|}=&~{}o_{p}(n_{i}^{(\xi-\ell)/2-\tau}),\\ \sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\big{|}\bm{x}_{i,j}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}\big{|}=&~{}o(n_{i}^{(\xi+\ell)/2-\tau}).\end{split}

Lemma 3.

Consider the linear mixed-effects model $(\alpha,\gamma)$ of (4). Suppose that (A0) and (A2) hold. Then for $\bm{H}_{i}(\gamma,\bm{\theta})$ defined in (A.3), we have

(i)

For $i=1,\dots,m$ and $k,k^{*}\notin\gamma$ ,

\displaystyle\sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\big{|}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k^{*}}\big{|}=

\displaystyle~{}\left\{\begin{array}[]{ll}c_{i,k}n_{i}^{\ell}+o(n_{i}^{\ell});&\mbox{if }k=k^{*},\\ o(n_{i}^{\ell-\tau});&\mbox{if }k\neq k^{*}.\end{array}\right.

(ii)

For $i=1,\dots,m$ and $k\in\gamma$ ,

\displaystyle\begin{split}\sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\big{|}\theta_{k}^{2}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}-\theta_{k}\big{|}=&~{}O(n_{i}^{-\ell}),\\ \sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\big{|}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}\big{|}=&~{}O(n_{i}^{\ell}).\end{split}

(iii)

For $i=1,\dots,m$ and $k,k^{*}\in\gamma$ with $k\neq k^{*}$ ,

\displaystyle\begin{split}\sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\theta_{k}\theta_{k^{*}}\big{|}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k^{*}}\big{|}=&~{}o(n_{i}^{-\ell-\tau}),\\ \sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\theta_{k}\big{|}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k^{*}}\big{|}=&~{}o(n_{i}^{-\tau}),\\ \sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\big{|}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k^{*}}\big{|}=&~{}o(n_{i}^{\ell-\tau}).\end{split}

(iv)

For $i=1,\dots,m$ , $k\in\gamma$ and $k^{*}\notin\gamma$ ,

\displaystyle\begin{split}\sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\theta_{k}\big{|}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k^{*}}\big{|}=&~{}o(n_{i}^{-\tau}),\\ \sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\big{|}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k^{*}}\big{|}=&~{}o(n_{i}^{\ell-\tau}).\end{split}

Lemma 4.

Consider the linear mixed-effects model $(\alpha,\gamma)$ of (4). Suppose that (A0)–(A3) hold. Then for $\bm{H}_{i}(\gamma,\bm{\theta})$ defined in (A.3), we have

(i)

For $i=1,\dots,m$ and $k\in\gamma$ ,

\displaystyle\begin{split}\sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\theta_{k}\big{|}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}\big{|}=&~{}O_{p}(n_{i}^{-\ell/2}),\\ \sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\big{|}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}\big{|}=&~{}O_{p}(n_{i}^{\ell/2}).\end{split}

(ii)

For $i=1,\dots,m$ and $k\notin\gamma$ ,

\displaystyle\sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\big{|}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}\big{|}=

\displaystyle~{}O_{p}(n_{i}^{\ell/2}).

(iii)

For $i=1,\dots,m$ and $j=1,\dots,p$ ,

\displaystyle\sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\big{|}\bm{x}_{i,j}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}\big{|}=

\displaystyle~{}O_{p}(n_{i}^{\xi/2}).

In addition,

\displaystyle\sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\bigg{|}\sum_{i=1}^{m}\bm{x}_{i,j}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}\bigg{|}=

\displaystyle~{}O_{p}\bigg{(}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}^{1/2}\bigg{)}.

(iv)

For $i=1,\dots,m$ ,

\displaystyle\sup_{\bm{\theta}\in[0,\infty)^{q(\gamma)}}\bm{\epsilon}_{i}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}=

\displaystyle~{}\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}+O_{p}(q).

Note that Lemma 2 (i) implies that, for $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ ,

	$\displaystyle\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)=$	$\displaystyle~{}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}\bm{T}(\alpha)+\bigg{\{}o\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi-\tau}\bigg{)}\bigg{\}}_{p(\alpha)\times p(\alpha)}$
	$\displaystyle=$	$\displaystyle~{}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}\bm{T}(\alpha)+\bigg{\{}o\bigg{(}n_{\min}^{-\tau}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}\bigg{\}}_{p(\alpha)\times p(\alpha)}$

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ , where $\{a\}_{k\times j}$ denotes a $k\times j$ matrix with elements equal to $a$ and $\bm{T}(\alpha)$ is a diagonal matrix with diagonal elements bounded away from $0$ and $\infty$ . Hence by (A.2) with $\bm{c},\bm{d}=\{o(n_{\min}^{-\tau/2})\}_{p(\alpha)\times 1}$ and $\bm{A}=\bm{T}(\alpha)$ , we have, for $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ ,

\displaystyle\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}=

\displaystyle~{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}

(A.7)

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ , which plays a key role in proving lemmas for theorems.

The following lemma shows that $\hat{\theta}_{k}$ does not converge to $0$ in probability for $k\in\gamma\cap\gamma_{0}$ , which allows us to restrict the parameter space of $\bm{\theta}$ from $[0,\infty)^{q(\gamma)}$ to

\displaystyle\Theta_{\gamma}=\{\bm{\theta}\in[0,\infty)^{q(\gamma)}:\bm{\theta}(\gamma\cap\gamma_{0})\in(0,\infty)^{q(\gamma\cap\gamma_{0})}\}.

(A.8)

Lemma 5.

Under the assumptions of Theorem 1, let $\bm{\theta}_{0}^{\dagger}$ be $\bm{\theta}$ except that $\{\theta_{k}:k\in\gamma\cap\gamma_{0}\}$ are replaced by $\{\theta_{k,0}:k\in\gamma\cap\gamma_{0}\}$ . Then for any $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ , $v^{2}>0$ , and $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ with $\theta_{k}\rightarrow 0$ for some $k\in\gamma\cap\gamma_{0}$ , we have

\displaystyle-2\log L(\bm{\theta},v^{2};\alpha,\gamma)-\{-2\log L(\bm{\theta}_{0}^{\dagger},v^{2},\alpha,\gamma)\}\xrightarrow{p}\infty

as $N\rightarrow\infty$ , where $-2\log L(\bm{\theta},v^{2};\alpha,\gamma)$ is given in (7).

Based on Lemma 5, the following lemma is needed to develop the convergence rates of components of the likelihood equations given in (B.1) and (B.2), uniformly over $\Theta_{\gamma}$ defined in (A.8).

Lemma 6.

Consider a mixed-effects model $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ with $\bm{H}(\gamma,\bm{\theta})$ defined in (5) and $\Theta_{\gamma}$ defined in (A.8). Suppose that (A0)–(A3) hold. Then

(i)

For $i,i^{*}=1,\dots,m$ , $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ and $k,k^{*}\in\gamma$ ,

\displaystyle\begin{split}\sup_{\bm{\theta}\in\Theta_{\gamma}}\theta_{k}\theta_{k^{*}}\big{|}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{h}_{i^{*},k^{*}}\big{|}=&~{}o\Bigg{(}\frac{n_{i}^{(\xi-\ell)/2}n_{i^{*}}^{(\xi-\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)},\\ \sup_{\bm{\theta}\in\Theta_{\gamma}}\theta_{k}\big{|}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{h}_{i^{*},k^{*}}\big{|}=&~{}o\Bigg{(}\frac{n_{i}^{(\xi-\ell)/2}n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)},\\ \sup_{\bm{\theta}\in\Theta_{\gamma}}\big{|}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{h}_{i^{*},k^{*}}\big{|}=&~{}o\Bigg{(}\frac{n_{i}^{(\xi+\ell)/2}n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}.\end{split}

(ii)

For $i,i^{*}=1,\dots,m$ , $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ , $k\in\gamma$ and $k^{*}\notin\gamma$ ,

\displaystyle\begin{split}\sup_{\bm{\theta}\in\Theta_{\gamma}}\theta_{k}\big{|}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{h}_{i^{*},k^{*}}\big{|}=&~{}o\Bigg{(}\frac{n_{i}^{(\xi-\ell)/2}n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)},\\ \sup_{\bm{\theta}\in\Theta_{\gamma}}\big{|}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{h}_{i^{*},k^{*}}\big{|}=&~{}o\Bigg{(}\frac{n_{i}^{(\xi+\ell)/2}n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}.\end{split}

(iii)

For $i=1,\dots,m$ , $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ and $k\in\gamma$ ,

\displaystyle\begin{split}\sup_{\bm{\theta}\in\Theta_{\gamma}}\theta_{k}\big{|}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}\big{|}=&~{}o_{p}(n_{i}^{-\ell/2}),\\ \sup_{\bm{\theta}\in\Theta_{\gamma}}\big{|}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}\big{|}=&~{}o_{p}(n_{i}^{\ell/2}).\end{split}

(iv)

For $i=1,\dots,m$ , $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times\mathcal{G}$ and $k\in\gamma$ ,

\displaystyle\begin{split}\sup_{\bm{\theta}\in\Theta_{\gamma}}\theta_{k}\big{|}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}(\alpha_{0}\setminus\alpha)\big{|}=&~{}o(n_{i}^{(\xi-\ell)/2-\tau}),\\ \sup_{\bm{\theta}\in\Theta_{\gamma}}\big{|}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}(\alpha_{0}\setminus\alpha)\big{|}=&~{}o(n_{i}^{(\xi+\ell)/2-\tau}).\end{split}

(v)

For $i=1,\dots,m$ and $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ ,

\displaystyle\sup_{\bm{\theta}\in\Theta_{\gamma}}\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}=

\displaystyle~{}O_{p}(p(\alpha)).

(vi)

For $i=1,\dots,m$ , $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ and $k\notin\gamma$ ,

\displaystyle\sup_{\bm{\theta}\in\Theta_{\gamma}}\big{|}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}\big{|}=

\displaystyle~{}o_{p}(n_{i}^{\ell/2}).

(vii)

For $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times\mathcal{G}$ ,

\displaystyle\begin{split}\sup_{\bm{\theta}\in\Theta_{\gamma}}\big{|}\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}(\alpha_{0}\setminus\alpha)\big{|}=o_{p}\bigg{(}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}^{1/2}\bigg{)}.\end{split}

(viii)

For $i,i^{*}=1,\dots,m$ , $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ and $k,k^{*}\notin\gamma$ ,

\displaystyle\sup_{\bm{\theta}\in\Theta_{\gamma}}\big{|}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{h}_{i^{*},k^{*}}\big{|}=

\displaystyle~{}o_{p}\Bigg{(}\frac{n_{i}^{(\xi+\ell)/2}n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}.

(ix)

For $i=1,\dots,m$ , $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times\mathcal{G}$ and $k\notin\gamma$ ,

\displaystyle\begin{split}\sup_{\bm{\theta}\in\Theta_{\gamma}}\big{|}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}(\alpha_{0}\setminus\alpha)\big{|}=o(n_{i}^{(\xi+\ell)/2-\tau}).\end{split}

(x)

For $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times\mathcal{G}$ ,

\displaystyle\begin{split}\sup_{\bm{\theta}\in\Theta_{\gamma}}\big{|}&\bm{\beta}(\alpha_{0}\setminus\alpha)^{\prime}\bm{X}(\alpha_{0}\setminus\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\\ &~{}\times\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}(\alpha_{0}\setminus\alpha)\big{|}=o\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi-\tau}\bigg{)}.\end{split}

Appendix B Theoretical Proofs

B.1 Proof of Theorem 1

We shall focus on the asymptotic properties of $\hat{v}^{2}(\alpha,\gamma)$ and $\big{\{}\hat{\theta}_{k}(\alpha,\gamma):k\in\gamma\big{\}}$ , and derive the asymptotic properties of $\{\hat{\sigma}_{k}^{2}(\alpha,\gamma):k\in\gamma\}$ via $\hat{\sigma}_{k}^{2}(\alpha,\gamma)=\hat{v}^{2}(\alpha,\gamma)\hat{\theta}_{k}(\alpha,\gamma)$ ; $k\in\gamma$ . If $\hat{v}^{2}(\alpha,\gamma)>0$ and $\hat{\theta}_{k}(\alpha,\gamma)>0$ ; $k\in\gamma$ , then we can derive them using the likelihood equations. Differentiating the profile log-likelihood function of (7) with respect to $v^{2}$ and $\{\theta_{k}:k\in\gamma\}$ , we obtain

\frac{\partial}{\partial v^{2}}\{-2\log L(\bm{\theta},v^{2};\alpha,\gamma)\}=\frac{N}{v^{2}}-\frac{\bm{y}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}}{v^{4}}

(B.1)

and

\displaystyle\begin{split}\frac{\partial}{\partial\theta_{k}}\{-2\log L(\bm{\theta},v^{2};\alpha,&\gamma)\}=\sum_{i=1}^{m}\bigg{\{}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}\\ &~{}-\frac{\{\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}\}^{2}}{v^{2}}\bigg{\}}.\end{split}

(B.2)

To derive $\hat{v}^{2}(\alpha,\gamma)$ and $\big{\{}\hat{\theta}_{k}(\alpha,\gamma):k\in\gamma\big{\}}$ , we must study the convergence rate of each term on the right-hand sides of both (B.1) and (B.2) by Lemmas 2–4 and Lemma 6.

We first prove (1) using (B.1). Consider the following decomposition of $\bm{y}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}$ in (B.1):

\displaystyle\begin{split}\bm{y}^{\prime}\bm{H}^{-1}&(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}\\ =&~{}\bm{\mu}^{\prime}_{0}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{\mu}_{0}\\ &~{}+2\bm{\mu}^{\prime}_{0}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})\\ &~{}+(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})\\ &~{}-(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}).\end{split}

(B.3)

The first two terms of (B.3) are zeros because

\displaystyle(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{\mu}_{0}=\bm{0};\quad\alpha\in\mathcal{A}_{0}.

(B.4)

By Lemma 3 (ii)–(iii), Lemma 4 (i), and Lemma 4 (iv), the third term of (B.3) can be written as

	$\displaystyle\sum_{i=1}^{m}(\bm{Z}_{i}(\gamma_{0})$	$\displaystyle\bm{b}_{i}(\gamma_{0})+\bm{\epsilon}_{i})^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})(\bm{Z}_{i}(\gamma_{0})\bm{b}_{i}(\gamma_{0})+\bm{\epsilon}_{i})$
	$\displaystyle=$	$\displaystyle~{}\sum_{i=1}^{m}\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}+O_{p}\bigg{(}\sum_{k\in\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}+o_{p}\bigg{(}\sum_{k\in\gamma_{0}}\frac{m}{\theta_{k}^{2}}\bigg{)}+O_{p}(mq)$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . Note that by the Cauchy–Schwarz inequality,

\displaystyle\bigg{(}\sum_{i=1}^{m}n_{i}^{(\xi-\ell)/2}\bigg{)}^{2}=O\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\sum_{i^{*}=1}^{m}n_{i^{*}}^{-\ell}\bigg{)}.

(B.5)

Hence, by Lemma 6 (i), Lemma 6 (iii), and Lemma 6 (v), the last term of (B.3) can be written as

	$\displaystyle\bigg{\{}\bigg{(}\sum_{i=1}^{m}$	$\displaystyle\sum_{k\in\gamma_{0}}b_{i,k}\bm{h}_{i,k}\bigg{)}+\bm{\epsilon}\bigg{\}}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bigg{\{}\bigg{(}\sum_{i=1}^{m}\sum_{k\in\gamma_{0}}b_{i,k}\bm{h}_{i,k}\bigg{)}+\bm{\epsilon}\bigg{\}}$
	$\displaystyle=$	$\displaystyle~{}o_{p}\bigg{(}\sum_{k,k^{}\in\gamma_{0}}\frac{m}{\theta_{k}\theta_{k^{}}}\bigg{)}+o_{p}\bigg{(}\sum_{k\in\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}+O_{p}(p+mq)$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . Therefore, we can rewrite (B.3) as

	$\displaystyle\bm{y}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})$	$\displaystyle(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}$
	$\displaystyle=$	$\displaystyle~{}\bm{\epsilon}^{\prime}\bm{\epsilon}+o_{p}\bigg{(}\sum_{k,k^{}\in\gamma_{0}}\frac{m}{\theta_{k}\theta_{k^{}}}\bigg{)}+O_{p}\bigg{(}\sum_{k\in\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}+O_{p}(p+mq).$

It follows from (B.1) that for $v^{2}\in(0,\infty)$ ,

\displaystyle\begin{split}v^{4}\bigg{\{}\frac{\partial}{\partial v^{2}}\{-2\log L(\bm{\theta},v^{2};\alpha,\gamma)\}\bigg{\}}=&~{}N\bigg{(}v^{2}-\frac{\bm{\epsilon}^{\prime}\bm{\epsilon}}{N}\bigg{)}+o_{p}\bigg{(}\sum_{k,k^{*}\in\gamma_{0}}\frac{m}{\theta_{k}\theta_{k^{*}}}\bigg{)}\\ &~{}+O_{p}\bigg{(}\sum_{k\in\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}+O_{p}(p+mq)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This and Lemma 5 imply that

\displaystyle\begin{split}\hat{v}^{2}(\alpha,\gamma)=&~{}\frac{\bm{\epsilon}^{\prime}\bm{\epsilon}}{N}+O_{p}\Big{(}\frac{p+mq}{N}\Big{)}.\end{split}

(B.6)

Thus (1) follows by applying the law of large numbers to $\bm{\epsilon}^{\prime}\bm{\epsilon}/N$ . In addition, the asymptotic normality of $\hat{v}^{2}(\alpha,\gamma)$ follows by $p+mq=o(N^{1/2})$ and an application of the central limit theorem to $\bm{\epsilon}^{\prime}\bm{\epsilon}/N$ in (B.6).

Next, we prove (4), for $k\in\gamma\cap\gamma_{0}$ , using (B.2). By Lemma 6 (i) and Lemma 6 (iii), we have, for $k\in\gamma\cap\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}\bm{h}_{i,k}^{\prime}&\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bigg{\{}\bigg{(}\sum_{i^{*}=1}^{m}\sum_{k^{*}\in\gamma_{0}}b_{i^{*},k^{*}}\bm{h}_{i^{*},k^{*}}\bigg{)}+\bm{\epsilon}\bigg{\}}\\ =&~{}o_{p}\bigg{(}\sum_{k^{*}\in\gamma_{0}}\frac{n_{i}^{(\xi-\ell)/2}\sum_{i^{*}=1}^{m}n_{i^{*}}^{(\xi-\ell)/2}}{\theta_{k^{*}}\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}+o_{p}(n_{i}^{-\ell/2})\end{split}

(B.7)

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This and (B.4) imply that for $k\in\gamma\cap\gamma_{0}$ ,

	$\displaystyle\theta_{k}$	$\displaystyle\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}$
	$\displaystyle=$	$\displaystyle~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})$
	$\displaystyle=$	$\displaystyle~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})$
		$\displaystyle~{}+o_{p}\bigg{(}\sum_{k^{}\in\gamma_{0}}\frac{n_{i}^{(\xi-\ell)/2}\sum_{i^{}=1}^{m}n_{i^{}}^{(\xi-\ell)/2}}{\theta_{k^{}}\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}+o_{p}(n_{i}^{-\ell/2})$
	$\displaystyle=$	$\displaystyle~{}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})(\bm{Z}_{i}(\gamma_{0})\bm{b}_{i}(\gamma_{0})+\bm{\epsilon}_{i})$
		$\displaystyle~{}+o_{p}\bigg{(}\sum_{k^{}\in\gamma_{0}}\frac{n_{i}^{(\xi-\ell)/2}\sum_{i^{}=1}^{m}n_{i^{}}^{(\xi-\ell)/2}}{\theta_{k^{}}\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}+o_{p}(n_{i}^{-\ell/2})$
	$\displaystyle=$	$\displaystyle~{}b_{i,k}+O_{p}(n_{i}^{-\ell/2})+o_{p}\bigg{(}\sum_{k^{}\in\gamma_{0}}\frac{n_{i}^{(\xi-\ell)/2}\sum_{i^{}=1}^{m}n_{i^{}}^{(\xi-\ell)/2}}{\theta_{k^{}}\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the last equality follows from Lemma 3 (ii)–(iii) and Lemma 4 (i). Hence, for $k\in\gamma\cap\gamma_{0}$ ,

	$\displaystyle\theta_{k}^{2}$	$\displaystyle\{\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}\}^{2}$
	$\displaystyle=$	$\displaystyle~{}b_{i,k}^{2}+O_{p}(n_{i}^{-\ell/2})+o_{p}\bigg{(}\sum_{k^{}\in\gamma_{0}}\frac{n_{i}^{(\xi-\ell)/2}\sum_{i^{}=1}^{m}n_{i^{}}^{(\xi-\ell)/2}}{\theta_{k^{}}\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This together with Lemma 3 (ii) and (B.2) imply that for $k\in\gamma\cap\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}^{2}&\bigg{\{}\frac{\partial}{\partial\theta_{k}}\{-2\log L(\bm{\theta},v^{2};\alpha,\gamma)\}\bigg{\}}\\ =&~{}m\bigg{(}\theta_{k}-\frac{1}{m}\sum_{i=1}^{m}\frac{b_{i,k}^{2}}{v^{2}}\bigg{)}+O_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{-\ell/2}\bigg{)}\\ &~{}+o_{p}\bigg{(}\sum_{k^{*}\in\gamma_{0}}\frac{\sum_{i=1}^{m}n_{i}^{(\xi-\ell)/2}\sum_{i^{*}=1}^{m}n_{i^{*}}^{(\xi-\ell)/2-\tau}}{\theta_{k^{*}}\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\end{split}

(B.8)

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . By (B.5), Lemma 5 and setting (B.8) to $0$ , we obtain

\hat{\theta}_{k}(\alpha,\gamma)=\frac{1}{m}\sum_{i=1}^{m}\frac{b_{i,k}^{2}}{\hat{v}^{2}(\alpha,\gamma)}+O_{p}\bigg{(}\frac{1}{m}\sum_{i=1}^{m}n_{i}^{-\ell/2}\bigg{)};\quad k\in\gamma\cap\gamma_{0}.

This proves (4), for $k\in\gamma\cap\gamma_{0}$ .

It remains to prove (4), for $k\in\gamma\setminus\gamma_{0}$ . We prove by showing that (B.2) is asymptotically nonnegative, for $\theta_{k}\in\big{(}n_{\max}^{-\ell},\infty\big{)}$ ; $k\in\gamma\setminus\gamma_{0}$ using a recursive argument. Let $\bm{\theta}^{\dagger}$ be $\bm{\theta}$ except that $\{\theta_{k}:k\in\gamma\cap\gamma_{0}\}$ are replaced by $\{\hat{\theta}_{k}(\alpha,\gamma):k\in\gamma\cap\gamma_{0}\}$ . By Lemma 6 (i) and Lemma 6 (iii), we have, for $k\in\gamma\setminus\gamma_{0}$ ,

	$\displaystyle\theta_{k}\bm{h}_{i,k}^{\prime}$	$\displaystyle\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})$
	$\displaystyle=$	$\displaystyle~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger})\bigg{(}\sum_{i^{}=1}^{m}\sum_{k^{}\in\gamma_{0}}b_{i^{},k^{}}\bm{h}_{i^{},k^{}}+\bm{\epsilon}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}o_{p}\bigg{(}\frac{n_{i}^{(\xi-\ell)/2}\sum_{i^{}=1}^{m}n_{i^{}}^{(\xi-\ell)/2}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}+o_{p}(n_{i}^{-\ell/2})$

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ . This and (B.4) imply that for $k\in\gamma\setminus\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}&\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))\bm{y}\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})\\ &~{}+o_{p}\bigg{(}\frac{n_{i}^{(\xi-\ell)/2}\sum_{i^{*}=1}^{m}n_{i^{*}}^{(\xi-\ell)/2}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}+o_{p}(n_{i}^{-\ell/2})\\ =&~{}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta}^{\dagger})\bigg{(}\sum_{k^{*}\in\gamma_{0}}\bm{z}_{i,k^{*}}b_{i,k^{*}}+\bm{\epsilon}_{i}\bigg{)}\\ &~{}+o_{p}\bigg{(}\frac{n_{i}^{(\xi-\ell)/2}\sum_{i^{*}=1}^{m}n_{i^{*}}^{(\xi-\ell)/2}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}+o_{p}(n_{i}^{-\ell/2})\\ =&~{}O_{p}(n_{i}^{-\ell/2})+o_{p}\bigg{(}\frac{n_{i}^{(\xi-\ell)/2}\sum_{i^{*}=1}^{m}n_{i^{*}}^{(\xi-\ell)/2}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\end{split}

(B.9)

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ , where the last equality follows from Lemma 3 (iii) and Lemma 4 (i). Hence by (B.5), Lemma 3 (ii), and (B.2), we have, for $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ and $k\in\gamma\setminus\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}^{2}&\bigg{\{}\frac{\partial}{\partial\theta_{k}}\{-2\log L(\bm{\theta}^{\dagger},v^{2};\alpha,\gamma)\}\bigg{\}}\\ =&~{}m\theta_{k}+O_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{-\ell}\bigg{)}+o_{p}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{(\xi-\ell)/2}\sum_{i^{*}=1}^{m}n_{i^{*}}^{(\xi-\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\\ =&~{}m\theta_{k}+O_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{-\ell}\bigg{)}\\ =&~{}m\theta_{k}+o_{p}(m\log(n_{\min})n_{\min}^{-\ell}).\end{split}

This implies that $-2\log L(\bm{\theta}^{\dagger},v^{2};\alpha,\gamma)$ is an asymptotically nondecreasing function on $\theta_{k}\in(\log(n_{\min})n_{\min}^{-\ell},\infty)$ , for $k\in\gamma\setminus\gamma_{0}$ given other $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ . It follows that $\hat{\theta}_{k}(\alpha,\gamma)\in\big{[}0,\log(n_{\min})n_{\min}^{-\ell}\big{)}$ ; $k\in\gamma\setminus\gamma_{0}$ . The above convergence rate can be recursively improved. Without loss of generality, assume that $n_{\min}=n_{1}\leq n_{2}\leq\cdots\leq n_{m}=n_{\max}$ . We can restrict the parameter space of $\theta_{k}$ in the next step to

\displaystyle\Theta_{\gamma,k,i}=\big{\{}\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}:\theta_{k}\leq\log(n_{\min})n_{i}^{-\ell}\big{\}}

(B.10)

with $i=1$ . Then, by Lemma 6 (i) and Lemma 6 (iii), we have, for $k\in\gamma\setminus\gamma_{0}$ ,

	$\displaystyle\theta_{k}$	$\displaystyle\bm{h}_{1,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})$
	$\displaystyle=$	$\displaystyle~{}o_{p}\bigg{(}\frac{n_{1}^{(\xi-\ell)/2}\sum_{i^{}=1}^{m}n_{i^{}}^{(\xi-\ell)/2}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}+o_{p}(\theta_{k}n_{1}^{\ell/2})$

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in\Theta_{\gamma,k,1}$ . This and (B.4) imply that for $k\in\gamma\setminus\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}\bm{h}_{1,k}^{\prime}\bm{H}^{-1}&(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))\bm{y}\\ =&~{}\theta_{k}\bm{h}_{1,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})\\ =&~{}\theta_{k}\bm{h}_{1,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})\\ &~{}+o_{p}\bigg{(}\frac{n_{1}^{(\xi-\ell)/2}\sum_{i^{*}=1}^{m}n_{i^{*}}^{(\xi-\ell)/2}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}+o_{p}(\theta_{k}n_{1}^{\ell/2})\\ =&~{}\theta_{k}\bm{z}_{1,k}^{\prime}\bm{H}_{1}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{Z}_{1}(\gamma_{0})\bm{b}_{1}(\gamma_{0})+\bm{\epsilon}_{1})\\ &~{}+o_{p}\bigg{(}\frac{n_{1}^{(\xi-\ell)/2}\sum_{i^{*}=1}^{m}n_{i^{*}}^{(\xi-\ell)/2}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}+o_{p}(\theta_{k}n_{1}^{\ell/2})\\ =&~{}O_{p}(\theta_{k}n_{1}^{\ell/2})+o_{p}\bigg{(}\frac{n_{1}^{(\xi-\ell)/2}\sum_{i^{*}=1}^{m}n_{i^{*}}^{(\xi-\ell)/2}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\end{split}

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in\Theta_{\gamma,k,1}$ , where the last equality follows from Lemma 3 (iii) and Lemma 4 (i). Hence by (B.5), Lemma 3 (ii), (B.2), and (B.9), we have

	$\displaystyle\theta_{k}^{2}$	$\displaystyle\bigg{\{}\frac{\partial}{\partial\theta_{k}}\{-2\log L(\bm{\theta}^{\dagger},v^{2};\alpha,\gamma)\}\bigg{\}}$
	$\displaystyle=$	$\displaystyle~{}(m-1)\theta_{k}+O_{p}(\theta_{k}^{2}n_{1}^{\ell})+O_{p}\bigg{(}\sum_{i=2}^{m}n_{i}^{-\ell}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{-\ell}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}(m-1)\theta_{k}+O_{p}(\log(n_{\min})\theta_{k})+O_{p}\bigg{(}\sum_{i=2}^{m}n_{i}^{-\ell}\bigg{)}$

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in\Theta_{\gamma,k,1}$ . Hence, setting the above equation equal to $0$ , we have

\hat{\theta}_{k}(\alpha,\gamma)=\frac{1}{m-1+O_{p}(\log(n_{\min}))}O_{p}\bigg{(}\sum_{i=2}^{m}n_{i}^{-\ell}\bigg{)}=O_{p}(n_{2}^{-\ell}).

Now we can further restrict the parameter space of $\theta_{k}$ to $\Theta_{\gamma,k,2}$ in (B.10). Continuing this procedure, we can recursively obtain $\hat{\theta}_{k}(\alpha,\gamma)=O_{p}(n_{i}^{-\ell})$ ; $k\in\gamma\setminus\gamma_{0}$ , for $i=3,\dots,m$ . This completes the proof of (4), for $k\in\gamma\setminus\gamma_{0}$ . Hence the proof of Theorem 1 is complete.

B.2 Proof of Example 1

Note that for $q=1$ , $\bm{Z}_{i}=\bm{z}_{i,1}$ and $\bm{b}_{i}=b_{i,1}$ . Note that by Lemma 5, we consider the sample space $(\sigma_{1}^{2},v^{2})\in(0,\infty)^{2}$ . We first derive the explicit forms of the ML estimators $\hat{\theta}_{1}$ and $\hat{v}^{2}$ . By (B.2), we have

	$\displaystyle\frac{\partial}{\partial\theta_{1}}\{-2\log L(\theta_{1},v^{2})\}=$	$\displaystyle~{}\sum_{i=1}^{m}\frac{\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}-\frac{1}{v^{2}}\sum_{i=1}^{m}\bigg{\{}\bm{z}_{i,1}^{\prime}\bigg{(}\bm{I}_{n}-\frac{\theta_{1}\bm{z}_{i,1}\bm{z}_{i,1}^{\prime}}{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{)}\bm{y}_{i}\bigg{\}}^{2}$
	$\displaystyle=$	$\displaystyle~{}\sum_{i=1}^{m}\frac{\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}-\frac{1}{v^{2}}\sum_{i=1}^{m}\bigg{\{}\frac{\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}b_{i,1}}{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}+\frac{\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{\}}^{2}$
	$\displaystyle=$	$\displaystyle~{}\sum_{i=1}^{m}\bigg{(}\frac{1}{\theta_{1}}-\frac{1}{\theta_{1}(1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}\bigg{)}$
		$\displaystyle~{}-\frac{1}{v^{2}}\sum_{i=1}^{m}\bigg{\{}\frac{b_{i,1}}{\theta_{1}}-\frac{b_{i,1}}{\theta_{1}(1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}+\frac{\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{\}}^{2}$
	$\displaystyle=$	$\displaystyle~{}\frac{m}{\theta_{1}}-\frac{\sum_{i=1}^{m}b_{i,1}^{2}}{v^{2}\theta_{1}^{2}}+2\sum_{i=1}^{m}\frac{b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{v^{2}\theta_{1}(1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}+R(\sigma_{1}^{2},v^{2}),$

where $\sigma_{1}^{2}=\theta_{1}v^{2}$ and

\displaystyle\begin{split}R(\sigma_{1}^{2},v^{2})=&~{}-\sum_{i=1}^{m}\frac{1}{\theta_{1}(1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}-\sum_{i=1}^{m}\frac{(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{v^{2}\{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}}+\sum_{i=1}^{m}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{v^{2}\theta_{1}\{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}}\\ &~{}+\sum_{i=1}^{m}\frac{2b_{i,1}^{2}}{v^{2}\theta_{1}^{2}(1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}-\sum_{i=1}^{m}\frac{b_{i,1}^{2}}{v^{2}\theta_{1}^{2}\{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}}.\end{split}

(B.11)

Note that ML estimators $\hat{\sigma}_{1}^{2}=\hat{\theta}_{1}\hat{v}^{2}$ and $\hat{v}^{2}$ satisfy

\displaystyle 0=

\displaystyle~{}\frac{m}{\hat{\theta}_{1}}-\frac{\sum_{i=1}^{m}b_{i,1}^{2}}{\hat{v}^{2}\hat{\theta}_{1}^{2}}+\sum_{i=1}^{m}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{\hat{v}^{2}\hat{\theta}_{1}(1+\hat{\theta}_{1}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}))}+R(\hat{\sigma}_{1}^{2},\hat{v}^{2}),

which implies that

\displaystyle\begin{split}\hat{\sigma}_{1}^{2}=&~{}\hat{\theta}_{1}\hat{v}^{2}=\frac{1}{m}\sum_{i=1}^{m}b_{i,1}^{2}+\frac{1}{m}\sum_{i=1}^{m}\frac{2\hat{\theta}_{1}b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{1+\hat{\theta}_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}+\frac{\hat{\theta}_{1}^{2}}{m}R(\hat{\sigma}_{1}^{2},\hat{v}^{2})\\ =&~{}\frac{1}{m}\sum_{i=1}^{m}b_{i,1}^{2}+\frac{1}{m}\sum_{i=1}^{m}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}-\frac{1}{m}\sum_{i=1}^{m}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{(1+\hat{\theta}_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}+\frac{\hat{\theta}_{1}^{2}}{m}R(\hat{\sigma}_{1}^{2},\hat{v}^{2})\\ =&~{}\frac{1}{m}\sum_{i=1}^{m}b_{i,1}^{2}+\frac{1}{m}\sum_{i=1}^{m}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}+R^{*}(\hat{\sigma}_{1}^{2},\hat{v^{2}}),\end{split}

(B.12)

where

\displaystyle R^{*}(\hat{\sigma}_{1}^{2},\hat{v}^{2})=

\displaystyle~{}-\frac{1}{m}\sum_{i=1}^{m}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{(1+\hat{\theta}_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}+\frac{\hat{\theta}_{1}^{2}}{m}R(\hat{\theta}_{1},\hat{v}^{2})

(B.13)

with $R(\sigma_{1}^{2},v^{2})$ defined in (B.11). By (B.12), we have

\displaystyle\begin{split}\sum_{i=1}^{m}b_{i,1}^{2}=&~{}O_{p}(\hat{\sigma}_{1}^{2}),\\ b_{i,1}^{2}=&~{}O_{p}(\hat{\sigma}_{1}^{2}),\\ b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}=&~{}O_{p}(1+\hat{\theta}_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}).\end{split}

(B.14)

By (B.13) and (B.14), we have

\displaystyle\begin{split}R^{*}(\hat{\sigma}_{1}^{2},\hat{v}^{2})=&~{}o_{p}(n^{-1}).\end{split}

(B.15)

Similarly, by (B.1), we have

	$\displaystyle\frac{\partial}{\partial v^{2}}\{-2\log L(\theta_{1},v^{2})\}=$	$\displaystyle~{}\frac{N}{v^{2}}-\frac{1}{v^{4}}\sum_{i=1}^{m}\bm{y}_{i}^{\prime}\bigg{(}\bm{I}_{n}-\frac{\theta_{1}\bm{z}_{i,1}\bm{z}_{i,1}^{\prime}}{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{)}\bm{y}_{i}$
	$\displaystyle=$	$\displaystyle~{}\frac{N}{v^{2}}-\frac{1}{v^{4}}\sum_{i=1}^{m}(\bm{z}_{i,1}b_{i,1}+\bm{\epsilon}_{i})^{\prime}\bigg{(}\bm{I}_{n}-\frac{\theta_{1}\bm{z}_{i,1}\bm{z}_{i,1}^{\prime}}{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{)}(\bm{z}_{i,1}b_{i,1}+\bm{\epsilon}_{i})$
	$\displaystyle=$	$\displaystyle~{}\frac{N}{v^{2}}-\frac{1}{v^{4}}\sum_{i=1}^{m}\bigg{\{}\frac{b_{i,1}^{2}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}+\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}+\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}-\frac{\theta_{1}(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{1+\theta_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{\}}.$

The ML estimators $\hat{\theta}_{1}$ and $\hat{v}^{2}$ satisfy

\displaystyle 0=

\displaystyle~{}\frac{N}{\hat{v}^{2}}-\frac{N}{\hat{v}^{4}}\sum_{i=1}^{m}\bigg{\{}\frac{b_{i,1}^{2}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}{1+\hat{\theta}_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}+\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{1+\hat{\theta}_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}+\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}-\frac{\hat{\theta}_{1}(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{1+\hat{\theta}_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{\}},

which implies that

\displaystyle\hat{v}^{2}=

\displaystyle~{}\frac{1}{N}\sum_{i=1}^{m}\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}+R^{{\dagger}}(\hat{\sigma}_{1}^{2},\hat{v}^{2}),

(B.16)

with

\displaystyle R^{{\dagger}}(\hat{\sigma}_{1}^{2},\hat{v}^{2})=

\displaystyle~{}\frac{1}{N}\sum_{i=1}^{m}\bigg{\{}\frac{b_{i,1}^{2}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}{1+\hat{\theta}_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}+\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{1+\hat{\theta}_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}-\frac{\hat{\theta}_{1}(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{1+\hat{\theta}_{1}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{\}}.

This together with (B.14) yields

\displaystyle\begin{split}R^{{\dagger}}(\hat{\sigma}_{1}^{2},\hat{v}^{2}=&~{}O_{p}(n^{-1}).\end{split}

(B.17)

We are now ready to compare the asymptotic behaviors between the LS predictors and the empirical BLUPs. Note that for $i=1,\ldots,m$ , we have

	$\displaystyle\tilde{b}_{i,1}=$	$\displaystyle~{}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{-1}\bm{z}_{i,1}^{\prime}\bm{y}_{i},$
	$\displaystyle\hat{b}_{i,1}(\sigma_{1}^{2},v^{2})=$	$\displaystyle~{}\sigma_{1}^{2}\bm{z}_{i,1}^{\prime}(\sigma_{1}^{2}\bm{z}_{i,1}\bm{z}_{i,1}^{\prime}+v^{2}\bm{I}_{n})^{-1}\bm{y}_{i}.$

Hence

\displaystyle\bm{z}_{i,1}\big{(}\tilde{b}_{i,1}-b_{i,1}\big{)}=

\displaystyle~{}\frac{\bm{z}_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}},

(B.18)

and

\displaystyle\begin{split}\hat{b}_{i,1}(\hat{\sigma}_{1}^{2},\hat{v}^{2})-b_{i,1}=&~{}\hat{\sigma}_{1}^{2}\bm{z}_{i,1}^{\prime}(\hat{\sigma}_{1}^{2}\bm{z}_{i,1}\bm{z}_{i,1}^{\prime}+\hat{v}^{2}\bm{I}_{n})^{-1}(\bm{z}_{i,1}b_{i,1}+\bm{\epsilon}_{i})-b_{i,1}\\ =&~{}\big{\{}\hat{\sigma}_{1}^{2}\bm{z}_{i,1}^{\prime}(\hat{\sigma}_{1}^{2}\bm{z}_{i,1}\bm{z}_{i,1}^{\prime}+\hat{v}^{2}\bm{I}_{n})^{-1}\bm{z}_{i,1}-1\big{\}}b_{i,1}+\hat{\sigma}_{1}^{2}\bm{z}_{i,1}^{\prime}(\hat{\sigma}_{1}^{2}\bm{z}_{i,1}\bm{z}_{i,1}^{\prime}+\hat{v}^{2}\bm{I}_{n})^{-1}\bm{\epsilon}_{i}\\ =&~{}\bigg{\{}(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bigg{(}\bm{I}_{n}-\frac{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}\bm{z}_{i,1}^{\prime}}{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{)}\bm{z}_{i,1}-1\bigg{\}}b_{i}\\ &~{}+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bigg{(}\bm{I}_{n}-\frac{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}\bm{z}_{i,1}^{\prime}}{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{)}\bm{\epsilon}_{i}\\ =&~{}\frac{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}-b_{i}}{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}},\end{split}

which implies that

\displaystyle\bm{z}_{i,1}\big{(}\hat{b}_{i,1}(\hat{\sigma}_{1}^{2},\hat{v}^{2})-b_{i,1}\big{)}=

\displaystyle~{}\frac{\bm{z}_{i,1}\{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}-b_{i}\}}{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}.

(B.19)

Note that by (B.18),

\displaystyle\sum_{i=1}^{m}\big{\|}\bm{z}_{i,1}\big{(}\tilde{b}_{i,1}-b_{i,1}\big{)}\big{\|}^{2}=

\displaystyle~{}\sum_{i=1}^{m}(\tilde{b}_{i,1}-b_{i,1})^{2}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}=\sum_{i=1}^{m}\frac{(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}},

and by (B.19),

\displaystyle\sum_{i=1}^{m}\big{\|}\bm{z}_{i,1}\big{(}\hat{b}_{i,1}(\hat{\sigma}_{1}^{2},\hat{v}^{2})-b_{i,1}\big{)}\big{\|}^{2}=

\displaystyle~{}\sum_{i=1}^{m}\frac{\{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}-b_{i,1}\}^{2}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}{\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}},

which implies that

\displaystyle\begin{split}D(\hat{\sigma}^{2},\hat{v}^{2})=&~{}\sum_{i=1}^{m}\bigg{(}\frac{(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}-\frac{\{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}-b_{i,1}\}^{2}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}{\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}}\bigg{)}\\ =&~{}\sum_{i=1}^{m}\frac{(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}-\{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}-b_{i,1}\}^{2}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}}{\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}}\\ =&~{}\sum_{i=1}^{m}\frac{(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}+2(\hat{\sigma}_{1}^{2}/\hat{v}^{2})(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}+2(\hat{\sigma}_{1}^{2}/\hat{v}^{2})b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}-b_{i,1}^{2}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}}{\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}}.\end{split}

(B.20)

Note that by (B.20) and

	$\displaystyle\frac{2(\hat{\sigma}_{1}^{2}/\hat{v}^{2})(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}}=$	$\displaystyle~{}\frac{2(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}(\hat{\sigma}_{1}^{2}/\hat{v}^{2})}-\frac{2(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}+4(\hat{\sigma}_{1}^{2}/\hat{v}^{2})(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}(\hat{\sigma}_{1}^{2}/\hat{v}^{2})(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}},$
	$\displaystyle\frac{2(\hat{\sigma}_{1}^{2}/\hat{v}^{2})b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}{\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}}=$	$\displaystyle~{}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}-\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}+4b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}(\hat{\sigma}_{1}^{2}/\hat{v}^{2})(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}{\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}(\hat{\sigma}_{1}^{2}/\hat{v}^{2})(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})},$
	$\displaystyle\frac{b_{i,1}^{2}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}{\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}}=$	$\displaystyle~{}\frac{b_{i,1}^{2}}{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})^{2}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}-\frac{b_{i,1}^{2}+2(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}{\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}(\hat{\sigma}_{1}^{2}/\hat{v}^{2})^{2}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})},$

we have

\displaystyle D(\hat{\sigma}^{2},\hat{v}^{2})=

\displaystyle~{}\sum_{i=1}^{m}\bigg{(}\frac{2(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}(\hat{\sigma}_{1}^{2}/\hat{v}^{2})}+\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}-\frac{b_{i,1}^{2}}{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})^{2}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}\bigg{)}+R^{{\ddagger}}(\hat{\sigma}_{1}^{2},\hat{v}^{2})

(B.21)

with

	$\displaystyle R^{{\ddagger}}(\hat{\sigma}_{1}^{2},\hat{v}^{2})=$	$\displaystyle~{}\sum_{i=1}^{m}\bigg{(}\frac{(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}}-\frac{2(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}+4(\hat{\sigma}_{1}^{2}/\hat{v}^{2})(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}(\hat{\sigma}_{1}^{2}/\hat{v}^{2})(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}}$
		$\displaystyle~{}-\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}+4b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}(\hat{\sigma}_{1}^{2}/\hat{v}^{2})(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}{\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}(\hat{\sigma}_{1}^{2}/\hat{v}^{2})(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}+\frac{b_{i,1}^{2}+2(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}{\{1+(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\}^{2}(\hat{\sigma}_{1}^{2}/\hat{v}^{2})^{2}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}\bigg{)}.$

Note that by (B.14),

\displaystyle\begin{split}R^{{\ddagger}}(\hat{\sigma}_{1}^{2},\hat{v}^{2})=&~{}O_{p}(n^{-3/2}).\end{split}

(B.22)

Further, by (B.12) and (B.16), we have

\displaystyle\begin{split}\frac{2(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}(\hat{\sigma}_{1}^{2}/\hat{v}^{2})}=&~{}\frac{2(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}(\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}/N)}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}(\sum_{k=1}^{m}b_{k,1}^{2}/m)}+\frac{2(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}\hat{\sigma}_{1}^{2}(\sum_{k=1}^{m}b_{k,1}^{2}/m)}\\ &~{}\times\bigg{\{}R^{{\dagger}}(\hat{\sigma}_{1}^{2},\hat{v}^{2})\frac{\sum_{k=1}^{m}b_{k,1}^{2}}{m}-\bigg{(}\frac{\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{N}\bigg{)}\bigg{(}\sum_{i=1}^{m}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{m\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{)}-R^{*}(\hat{\sigma}_{1}^{2},\hat{v}^{2})\frac{\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{N}\bigg{\}}\\ \equiv&~{}\frac{2(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{n(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}\sum_{k=1}^{m}b_{k,1}^{2}}+R_{i,1}(\hat{\sigma}_{1}^{2},\hat{v}^{2}),\end{split}

(B.23)

with

	$\displaystyle R_{i,1}(\hat{\sigma}_{1}^{2},\hat{v}^{2})=$	$\displaystyle~{}\frac{2(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}\hat{\sigma}_{1}^{2}(\sum_{k=1}^{m}b_{k,1}^{2}/m)}\bigg{\{}R^{{\dagger}}(\hat{\sigma}_{1}^{2},\hat{v}^{2})\frac{\sum_{k=1}^{m}b_{k,1}^{2}}{m}$
		$\displaystyle~{}-\bigg{(}\frac{\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{N}\bigg{)}\bigg{(}\sum_{i=1}^{m}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{m\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{)}-R^{*}(\hat{\sigma}_{1}^{2},\hat{v}^{2})\frac{\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{N}\bigg{\}}.$

Similarly,

	$\displaystyle\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}=$	$\displaystyle~{}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{n\{\sum_{k=1}^{m}b_{k,1}^{2}+2\sum_{k=1}^{m}b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}/(\bm{z}_{k,1}^{\prime}\bm{z}_{k,1})\}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}+R_{i,2}(\hat{\sigma}_{1}^{2},\hat{v}^{2}),$		(B.24)
	$\displaystyle\frac{b_{i,1}^{2}}{(\hat{\sigma}_{1}^{2}/\hat{v}^{2})^{2}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}=$	$\displaystyle~{}\frac{b_{i,1}^{2}(\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k})^{2}}{n^{2}(\sum_{k=1}^{m}b_{k,1}^{2})^{2}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}+R_{i,3}(\hat{\sigma}_{1}^{2},\hat{v}^{2})$		(B.25)

with

	$\displaystyle R_{i,2}(\hat{\sigma}_{1}^{2},\hat{v}^{2})=$	$\displaystyle~{}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{\hat{\sigma}_{1}^{2}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}\{\sum_{k=1}^{m}b_{k,1}^{2}/m+2\sum_{k=1}^{m}b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}/(m\bm{z}_{k,1}^{\prime}\bm{z}_{k,1})\}}$
		$\displaystyle~{}\times\bigg{\{}R^{{\dagger}}(\hat{\sigma}_{1}^{2},\hat{v}^{2})\bigg{(}\frac{\sum_{k=1}^{m}b_{k,1}^{2}}{m}+\sum_{k=1}^{m}\frac{2b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}}{m\bm{z}_{k,1}^{\prime}\bm{z}_{k,1}}\bigg{)}-R^{*}(\hat{\sigma}_{1}^{2},\hat{v}^{2})\sum_{k=1}^{m}\frac{\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{N}\bigg{\}},$
	$\displaystyle R_{i,3}(\hat{\sigma}_{1}^{2},\hat{v}^{2})=$	$\displaystyle~{}\frac{b_{i,1}^{2}}{\hat{\sigma}_{1}^{4}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})\{\sum_{k=1}^{m}b_{k,1}^{2}/m\}^{2}}\bigg{(}\hat{v}^{2}\sum_{k=1}^{m}\frac{b_{k,1}^{2}}{m}+\hat{\sigma}_{1}^{2}\sum_{k=1}^{m}\frac{\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{N}\bigg{)}$
		$\displaystyle~{}\times\bigg{\{}R^{{\dagger}}(\hat{\sigma}_{1}^{2},\hat{v}^{2})\frac{\sum_{k=1}^{m}b_{k,1}^{2}}{m}-\bigg{(}\frac{\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{N}\bigg{)}\bigg{(}\sum_{i=1}^{m}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}}{m\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\bigg{)}-R^{*}(\hat{\sigma}_{1}^{2},\hat{v}^{2})\frac{\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{N}\bigg{\}}.$

Hence by (B.14), (B.15), and (B.17), we have

\displaystyle\begin{split}R_{i,1}(\hat{\sigma}_{1}^{2},\hat{v}^{2})=&~{}O_{p}(n^{-3/2}),\quad i=1,2,3.\end{split}

(B.26)

Furthermore, we have

\displaystyle\begin{split}\frac{1}{n}&\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{\{\sum_{k=1}^{m}b_{k,1}^{2}+2\sum_{k=1}^{m}b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}/(m\bm{z}_{k,1}^{\prime}\bm{z}_{k,1})\}(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}\\ =&~{}\frac{1}{n(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})}\bigg{\{}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{\sum_{k=1}^{m}b_{k,1}^{2}}-\frac{4b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\{\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}\}\{\sum_{k=1}^{m}b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}/(m\bm{z}_{k,1}^{\prime}\bm{z}_{k,1})\}}{\{\sum_{k=1}^{m}b_{k,1}^{2}\}^{2}}\\ &~{}+\frac{8b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}\{\sum_{k=1}^{m}b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}/(m\bm{z}_{k,1}^{\prime}\bm{z}_{k,1})\}^{2}}{\{\sum_{k=1}^{m}b_{k,1}^{2}+2\sum_{k=1}^{m}b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}/(m\bm{z}_{k,1}^{\prime}\bm{z}_{k,1})\}\{\sum_{k=1}^{m}b_{k,1}^{2}\}^{2}}\bigg{\}}\\ \equiv&~{}\bigg{\{}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{n(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})\sum_{k=1}^{m}b_{k,1}^{2}}-\frac{4b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\{\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}\}\{\sum_{k=1}^{m}b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}/(m\bm{z}_{k,1}^{\prime}\bm{z}_{k,1})\}}{n(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})\{\sum_{k=1}^{m}b_{k,1}^{2}\}^{2}}\bigg{\}}\\ &~{}+R_{i,4},\end{split}

(B.27)

with

\displaystyle R_{i,4}=

\displaystyle~{}\frac{8b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}\{\sum_{k=1}^{m}b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}/(m\bm{z}_{k,1}^{\prime}\bm{z}_{k,1})\}^{2}}{n(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})\{\sum_{k=1}^{m}b_{k,1}^{2}+2\sum_{k=1}^{m}b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}/(m\bm{z}_{k,1}^{\prime}\bm{z}_{k,1})\}\{\sum_{k=1}^{m}b_{k,1}^{2}\}^{2}}.

Note that

\displaystyle R_{i,4}=O_{p}(n^{-3/2}).

(B.28)

By (B.21), (B.23), (B.24), (B.25), and (B.27), we have

\displaystyle\begin{split}nD(\hat{\sigma}_{1}^{2},\hat{v}^{2})=&~{}A_{n,m}+nR^{{\ddagger}}(\hat{\sigma}_{1}^{2},\hat{v}^{2})+n\sum_{i=1}^{m}\bigg{\{}R_{i,1}(\hat{\sigma}_{1}^{2},\hat{v}^{2})+R_{i,2}(\hat{\sigma}_{1}^{2},\hat{v}^{2})-R_{i,3}(\hat{\sigma}_{1}^{2},\hat{v}^{2})+R_{i,4}\bigg{\}}\\ \equiv&~{}A_{n,m}+O_{p}(n^{-1/2})\end{split}

with

	$\displaystyle A_{n,m}=$	$\displaystyle~{}\sum_{i=1}^{m}\bigg{\{}\frac{2(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}\sum_{k=1}^{m}b_{k,1}^{2}}-\frac{b_{i,1}^{2}(\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k})^{2}}{n(\sum_{k=1}^{m}b_{k,1}^{2})^{2}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})\sum_{k=1}^{m}b_{k,1}^{2}}$
		$\displaystyle~{}-\frac{4b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\{\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}\}\{\sum_{k=1}^{m}b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}/(\bm{z}_{k,1}^{\prime}\bm{z}_{k,1})\}}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})\{\sum_{k=1}^{m}b_{k,1}^{2}\}^{2}}\bigg{\}},$

where the last equality follows from (B.22), (B.26), and (B.28). Note that $\big{(}\sum_{k=1}^{m}b_{i,k}^{2}/\sigma_{1,0}^{2}\big{)}^{-1}$ follows the inverse-chi-squared distribution with $m$ degrees of freedom. We have

\displaystyle\begin{split}\mathrm{E}\bigg{(}\frac{1}{\sum_{i=1}^{m}b_{i,1}^{2}}\bigg{)}=&~{}\frac{1}{(m-2)\sigma_{1,0}^{2}},\quad\mbox{provided }m>2,\\ \mathrm{E}\bigg{(}\frac{b_{i,1}^{2}}{\{\sum_{k=1}^{m}b_{k,1}^{2}\}^{2}}\bigg{)}=&~{}\frac{1}{m(m-2)\sigma_{1,0}^{2}}\quad\mbox{provided }m>4.\end{split}

(B.29)

By (B.29) and

\displaystyle\begin{split}\mathrm{E}\bigg{(}\bigg{\{}\sum_{i=1}^{m}\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}\bigg{\}}^{2}\bigg{)}=&~{}(2mn+m^{2}n^{2})v_{0}^{4},\\ \mathrm{E}\big{(}\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}(\bm{z}_{i,1}\bm{\epsilon}_{i})\big{)}=&~{}0,\\ \mathrm{E}\big{(}\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}(\bm{z}_{i,1}\bm{\epsilon}_{i})^{2}\big{)}=&~{}n^{2}v_{0}^{4}+o(n^{2}),\end{split}

we have, for $m>4$ ,

\displaystyle\begin{split}\mathrm{E}(A_{n,m})=&~{}\mathrm{E}\sum_{i=1}^{m}\bigg{\{}\frac{2(\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i})^{2}\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})^{2}\sum_{k=1}^{m}b_{k,1}^{2}}-\frac{b_{i,1}^{2}(\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k})^{2}}{n(\sum_{k=1}^{m}b_{k,1}^{2})^{2}\bm{z}_{i,1}^{\prime}\bm{z}_{i,1}}+\frac{2b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})\sum_{k=1}^{m}b_{k,1}^{2}}\\ &~{}-\frac{4b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\{\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}\}\{\sum_{k=1}^{m}b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}/(\bm{z}_{k,1}^{\prime}\bm{z}_{k,1})\}}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})\{\sum_{k=1}^{m}b_{k,1}^{2}\}^{2}}\bigg{\}}\\ =&~{}\frac{2m^{2}v_{0}^{4}}{(m-2)\sigma_{1,0}^{2}}-\frac{m^{2}v_{0}^{4}}{(m-2)\sigma_{1,0}^{2}}+o(1)\\ &~{}-\mathrm{E}\bigg{(}\mathrm{E}\bigg{(}\sum_{i=1}^{m}\frac{4b_{i,1}\bm{z}_{i,1}^{\prime}\bm{\epsilon}_{i}\{\sum_{k=1}^{m}\bm{\epsilon}_{k}^{\prime}\bm{\epsilon}_{k}\}\{\sum_{k=1}^{m}b_{k,1}\bm{z}_{k,1}^{\prime}\bm{\epsilon}_{k}/(\bm{z}_{k,1}^{\prime}\bm{z}_{k,1})\}}{(\bm{z}_{i,1}^{\prime}\bm{z}_{i,1})\{\sum_{k=1}^{m}b_{k,1}^{2}\}^{2}}\bigg{|}b_{1,1},\ldots,b_{m,1}\bigg{)}\bigg{)}\\ =&~{}\frac{2m^{2}v_{0}^{4}}{(m-2)\sigma_{1,0}^{2}}-\frac{m^{2}v_{0}^{4}}{(m-2)\sigma_{1,0}^{2}}-\mathrm{E}\bigg{(}\sum_{i=1}^{m}\frac{4mv_{0}^{4}b_{i,1}^{2}}{\{\sum_{k=1}^{m}b_{k,1}^{2}\}^{2}}\bigg{)}+o(1)\\ =&~{}\frac{2m^{2}v_{0}^{4}}{(m-2)\sigma_{1,0}^{2}}-\frac{m^{2}v_{0}^{4}}{(m-2)\sigma_{1,0}^{2}}-\frac{4mv_{0}^{4}}{(m-2)\sigma_{1,0}^{2}}+o(1)\\ =&~{}\frac{m(m-4)v_{0}^{4}}{(m-2)\sigma_{1,0}^{2}}+o(1).\end{split}

This completes the proofs.

B.3 Proof of Theorem 5

In this section, we first prove Theorem 5 to simplify the proofs of Theorems 3 and 4. As with the proof of Theorem 1, we shall focus on the asymptotic properties of $\hat{v}^{2}(\alpha,\gamma)$ and $\{\hat{\theta}_{k}(\alpha,\gamma):k\in\gamma\}$ , and derive them by solving the likelihood equations directly.

We first prove (16) using (B.1). For $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times\mathcal{G}$ , we have

\displaystyle(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{\mu}_{0}=

\displaystyle~{}(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha),

(B.30)

where $\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)$ denotes the sub-vector of $\bm{\beta}_{0}$ corresponding to $\alpha_{0}\setminus\alpha$ . Note that by the Cauchy–Schwarz inequality, we have

\displaystyle\bigg{(}\sum_{i=1}^{m}n_{i}^{(\xi+\ell)/2}\bigg{)}^{2}=O\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\sum_{i^{*}=1}^{m}n_{i}^{\ell}\bigg{)}.

(B.31)

Hence by (B.31) and Lemma 6, we have

\displaystyle\begin{split}\big{(}\bm{X}&(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\\ &~{}\times\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}\\ =&~{}\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)^{\prime}\big{(}\bm{X}(\alpha_{0}\setminus\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)\\ &~{}+\bm{b}(\gamma_{0})^{\prime}\bm{Z}(\gamma_{0})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})\\ &~{}+\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}\\ &~{}+2\bm{b}(\gamma_{0})^{\prime}\bm{Z}(\gamma_{0})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)\\ &~{}+2\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)\\ &~{}+2\bm{b}(\gamma_{0})^{\prime}\bm{Z}(\gamma_{0})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}\\ =&~{}\bigg{(}\sum_{i=1}^{m}\sum_{k\in\gamma_{0}}b_{i,k}\bm{h}_{i,k}\bigg{)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bigg{(}\sum_{i=1}^{m}\sum_{k\in\gamma_{0}}b_{i,k}\bm{h}_{i,k}\bigg{)}\\ &~{}+2\bigg{(}\sum_{i=1}^{m}\sum_{k\in\gamma_{0}}b_{i,k}\bm{h}_{i,k}\bigg{)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)\\ &~{}+2\bigg{(}\sum_{i=1}^{m}\sum_{k\in\gamma_{0}}b_{i,k}\bm{h}_{i,k}\bigg{)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi-\tau}\bigg{)}\\ &~{}+O_{p}(p)\\ \end{split}

\displaystyle\begin{split}=&~{}o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell-\tau}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{(\xi+\ell)/2-\tau}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi-\tau}\bigg{)}+O_{p}(p)\\ =&~{}o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}+O_{p}(p)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This and (B.30) imply

\displaystyle\begin{split}\bm{y}^{\prime}\bm{H}^{-1}&(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}\\ =&~{}\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}^{\prime}\\ &~{}\times\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\\ &~{}\times\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}\\ =&~{}\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\\ &~{}\times\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}\\ &~{}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}+O_{p}(p)\\ =&~{}\sum_{i=1}^{m}\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)^{\prime}\bm{X}_{i}(\alpha_{0}\setminus\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)\\ &~{}+2\sum_{i=1}^{m}\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)^{\prime}\bm{X}_{i}(\alpha_{0}\setminus\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})(\bm{Z}_{i}(\gamma_{0})\bm{b}_{i}(\gamma_{0})+\bm{\epsilon}_{i})\\ &~{}+\sum_{i=1}^{m}\bigg{(}\sum_{k\in\gamma_{0}}\bm{z}_{i,k}b_{i,k}+\bm{\epsilon}_{i}\bigg{)}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bigg{(}\sum_{k\in\gamma_{0}}\bm{z}_{i,k}b_{i,k}+\bm{\epsilon}_{i}\bigg{)}\\ &~{}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}+O_{p}(p)\end{split}

\displaystyle\begin{split}=&~{}\sum_{i=1}^{m}\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}+\sum_{i=1}^{m}\sum_{j\in\alpha_{0}\setminus\alpha}\beta_{j,0}^{2}d_{i,j}n_{i}^{\xi}+\sum_{i=1}^{m}\sum_{k\in\gamma_{0}\setminus\gamma}b_{i,k}^{2}c_{i,k}n_{i}^{\ell}\\ &~{}+o_{p}\bigg{(}\sum_{k,k^{*}\in\gamma\cap\gamma_{0}}\frac{m}{\theta_{k}\theta_{k^{*}}}\bigg{)}+O_{p}\bigg{(}\sum_{k\in\gamma\cap\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}\\ &~{}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}+O_{p}(p+mq)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the last equality follows from (B.5) and Lemmas 2–4. Hence by (B.1), we have, for $v^{2}\in(0,\infty)$ ,

\displaystyle\begin{split}v^{4}&\bigg{\{}\frac{\partial}{\partial v^{2}}\{-2\log L(\bm{\theta},v^{2};\alpha,\gamma)\}\bigg{\}}\\ =&~{}N\bigg{(}v^{2}-\frac{\bm{\epsilon}^{\prime}\bm{\epsilon}}{N}+\frac{1}{N}\sum_{i=1}^{m}\sum_{j\in\alpha_{0}\setminus\alpha}\beta_{j,0}^{2}d_{i,j}n_{i}^{\xi}+\frac{1}{N}\sum_{i=1}^{m}\sum_{k\in\gamma_{0}\setminus\gamma}b_{i,k}^{2}c_{i,k}n_{i}^{\ell}\bigg{)}\\ &~{}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}+O_{p}\bigg{(}\sum_{k,k^{*}\in\gamma\cap\gamma_{0}}\frac{m}{\theta_{k}\theta_{k^{*}}}\bigg{)}\\ &~{}+O_{p}\bigg{(}\sum_{k\in\gamma\cap\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}+O_{p}(p+mq)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This and Lemma 5 imply that

\displaystyle\begin{split}\hat{v}^{2}(\alpha,\gamma)=&~{}\frac{\bm{\epsilon}^{\prime}\bm{\epsilon}}{N}+\frac{1}{N}\sum_{i=1}^{m}\sum_{j\in\alpha_{0}\setminus\alpha}\beta_{j,0}^{2}d_{i,j}n_{i}^{\xi}+\frac{1}{N}\sum_{i=1}^{m}\sum_{k\in\gamma_{0}\setminus\gamma}b_{i,k}^{2}c_{i,k}n_{i}^{\ell}\\ &~{}+o_{p}\bigg{(}\frac{1}{N}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}+o_{p}\bigg{(}\frac{1}{N}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}+O_{p}\Big{(}\frac{p+mq}{N}\Big{)}.\end{split}

(B.32)

Thus (16) follows by applying the law of large numbers to $\bm{\epsilon}^{\prime}\bm{\epsilon}/N$ . In addition, if $(\xi,\ell)\in(0,1/2)\times(0,1/2)$ , the asymptotic normality of $\hat{v}^{2}(\alpha,\gamma)$ follows by $p+mq=o(N^{1/2})$ and an application of the central limit theorem to $\bm{\epsilon}^{\prime}\bm{\epsilon}/N$ in (B.32).

Next, we prove (19), for $k\in\gamma\cap\gamma_{0}$ , using (B.2). By (B.31) and Lemma 6 (i)–(iv), we have, for $k\in\gamma\cap\gamma_{0}$ ,

	$\displaystyle\theta_{k}$	$\displaystyle\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}$
	$\displaystyle=$	$\displaystyle~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})$
		$\displaystyle~{}\times\bigg{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\sum_{i^{}=1}^{m}\sum_{k^{}\in\gamma_{0}}b_{i^{},k^{}}\bm{h}_{i^{},k^{}}+\bm{\epsilon}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}o_{p}\bigg{(}\frac{n_{i}^{(\xi-\ell)/2}\sum_{i^{}=1}^{m}n_{i^{}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}+o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(n_{i}^{-\ell/2})$
	$\displaystyle=$	$\displaystyle~{}o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(1)$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This and (B.30) imply that for $k\in\gamma\cap\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}&\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\\ &~{}\times\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}\\ &~{}+o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(1)\\ =&~{}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bigg{(}\bm{X}_{i}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\sum_{k^{*}\in\gamma_{0}}\bm{z}_{i,k^{*}}b_{i,k^{*}}+\bm{\epsilon}_{i}\bigg{)}\\ &~{}+o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(1)\\ =&~{}b_{i,k}+o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(1)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the last equality follows from Lemma 2 (iii), Lemma 3 (ii)–(iv), and Lemma 4 (i). It follows that for $k\in\gamma\cap\gamma_{0}$ ,

	$\displaystyle\theta_{k}^{2}$	$\displaystyle\{\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}\}^{2}$
	$\displaystyle=$	$\displaystyle~{}b_{i,k}^{2}+o_{p}\bigg{(}n_{i}^{\xi-\ell}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{)}+o_{p}(n_{i}^{\xi-\ell})+o_{p}(1)$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . Hence by Lemma 3 (ii) and (B.2), we have, for $k\in\gamma\cap\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}^{2}&\bigg{\{}\frac{\partial}{\partial\theta_{k}}\{-2\log L(\bm{\theta},v^{2};\alpha,\gamma)\}\bigg{\}}\\ =&~{}m\bigg{(}\theta_{k}-\frac{1}{m}\sum_{i=1}^{m}\frac{b_{i,k}^{2}}{v^{2}}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{(}1+\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{)}+o_{p}(m)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This implies that for $k\in\gamma\cap\gamma_{0}$ ,

\displaystyle\hat{\theta}_{k}(\alpha,\gamma)=

\displaystyle~{}\frac{1}{m}\sum_{i=1}^{m}\frac{b_{i,k}^{2}}{\hat{v}^{2}(\alpha,\gamma)}+o_{p}\bigg{(}\frac{1}{m}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{(}1+\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{)}+o_{p}(1).

This proves (19), for $k\in\gamma\cap\gamma_{0}$ .

It remains to prove (19), for $k\in\gamma\setminus\gamma_{0}$ . Let $\bm{\theta}^{\dagger}$ be $\bm{\theta}$ except that $\{\theta_{k}:k\in\gamma\cap\gamma_{0}\}$ are replaced by $\{\hat{\theta}_{k}(\alpha,\gamma):k\in\gamma\cap\gamma_{0}\}$ . By (B.31) and Lemma 6 (i)–(iv), we have, for $k\in\gamma\setminus\gamma_{0}$ ,

	$\displaystyle\theta_{k}$	$\displaystyle\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger})\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}$
	$\displaystyle=$	$\displaystyle~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger})$
		$\displaystyle~{}\times\bigg{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\sum_{i^{}=1}^{m}\sum_{k^{}\in\gamma_{0}}b_{i^{},k^{}}\bm{h}_{i^{},k^{}}+\bm{\epsilon}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(n_{i}^{(\xi-\ell)/2-\tau})+o_{p}(n_{i}^{-\ell/2})$
	$\displaystyle=$	$\displaystyle~{}o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(1)$

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ . This and (B.30) imply that for $k\in\gamma\setminus\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}&\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))\bm{y}\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)\\ ~{}&+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}\\ &~{}+o_{p}(n_{i}^{(\xi-\ell)/2}n_{\max}^{(\ell-\xi)/2})+o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(1)\\ =&~{}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta}^{\dagger})\bigg{(}\bm{X}_{i}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\sum_{i^{*}=1}^{m}\sum_{k^{*}\in\gamma_{0}}b_{i^{*},k^{*}}\bm{h}_{i^{*},k^{*}}+\bm{\epsilon}_{i}\bigg{)}\\ &~{}+o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(1)\\ =&~{}o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(1)\end{split}

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ , where the last equality follows from Lemma 2 (iii), Lemma 3 (iii)–(iv), and Lemma 4 (i). Therefore,

	$\displaystyle\theta_{k}^{2}$	$\displaystyle\{\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))\bm{y}\}^{2}$
	$\displaystyle=$	$\displaystyle~{}o_{p}\bigg{(}n_{i}^{\xi-\ell}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{)}+o_{p}(n_{i}^{\xi-\ell})+o_{p}(1)$

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ . Hence by Lemma 3 (ii) and (B.2), we have for $k\in\gamma\setminus\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}^{2}&\bigg{\{}\frac{\partial}{\partial\theta_{k}}\{-2\log L(\bm{\theta}^{\dagger},v^{2};\alpha,\gamma)\}\bigg{\}}\\ =&~{}m\theta_{k}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{(}1+\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{)}+o_{p}(m)\end{split}

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ . This implies that for $k\in\gamma\setminus\gamma_{0}$ ,

\hat{\theta}_{k}(\alpha,\gamma)=o_{p}\bigg{(}\frac{1}{m}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{(}1+\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{)}+o_{p}(1).

This completes the proof of (19). Thus the proof of Theorem 5 is complete.

B.4 Proof of Theorem 3

As with the proof of Theorem 1, we shall focus on the asymptotic properties of $\hat{v}^{2}(\alpha,\gamma)$ and $\{\hat{\theta}_{k}(\alpha,\gamma):k\in\gamma\}$ , and derive them by solving the likelihood equations directly.

We first prove (8) using (B.1). Hence by (B.31), Lemma 6 (i)–(iii), Lemma 6 (v)–(vi), and Lemma 6 (viii), we have

\displaystyle\begin{split}(\bm{Z}(\gamma_{0})&\bm{b}(\gamma_{0})+\bm{\epsilon})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})\\ =&~{}\bigg{(}\sum_{i=1}^{m}\sum_{k\in\gamma_{0}}b_{i,k}\bm{h}_{i,k}+\bm{\epsilon}\bigg{)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bigg{(}\sum_{i=1}^{m}\sum_{k\in\gamma_{0}}b_{i,k}\bm{h}_{i,k}+\bm{\epsilon}\bigg{)}\\ =&~{}o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell-\tau}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell/2}\bigg{)}+O_{p}(p)\\ =&~{}o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}+O_{p}(p)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This and (B.4) imply

\displaystyle\begin{split}\bm{y}^{\prime}&\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}\\ =&~{}(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})\\ =&~{}(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}+O_{p}(p)\\ =&~{}\sum_{i=1}^{m}(\bm{Z}_{i}(\gamma_{0})\bm{b}_{i}(\gamma_{0})+\bm{\epsilon}_{i})^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})(\bm{Z}_{i}(\gamma_{0})\bm{b}_{i}(\gamma_{0})+\bm{\epsilon}_{i})+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}\\ &~{}+O_{p}(p)\\ =&~{}\sum_{i=1}^{m}\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}+\sum_{i=1}^{m}\sum_{k\in\gamma_{0}\setminus\gamma}b_{i,k}^{2}c_{i,k}n_{i}^{\ell}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}+O_{p}\bigg{(}\sum_{k\in\gamma\cap\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}\\ &~{}+o_{p}\bigg{(}\sum_{k,k^{*}\in\gamma\cap\gamma_{0}}\frac{m}{\theta_{k}\theta_{k^{*}}}\bigg{)}+O_{p}(p+mq)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the last equality follows from Lemma 3, Lemma 4 (i)–(ii), and Lemma 4 (iv). Hence by (B.1), we have, for $v^{2}\in(0,\infty)$ ,

\displaystyle\begin{split}v^{4}&~{}\bigg{\{}\frac{\partial}{\partial v^{2}}\{-2\log L(\bm{\theta},v^{2};\alpha,\gamma)\}\bigg{\}}\\ =&~{}N\bigg{(}v^{2}-\frac{\bm{\epsilon}^{\prime}\bm{\epsilon}}{N}+\frac{1}{N}\sum_{k\in\gamma_{0}\setminus\gamma}b_{i,k}^{2}c_{i,k}n_{i}^{\ell}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}\\ &~{}+O_{p}\bigg{(}\sum_{k\in\gamma\cap\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}+o_{p}\bigg{(}\sum_{k,k^{*}\in\gamma\cap\gamma_{0}}\frac{m}{\theta_{k}\theta_{k^{*}}}\bigg{)}+O_{p}(p+mq)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This and Lemma 5 imply that for $(\xi,\ell)\in(0,1]\times(0,1]$ ,

\displaystyle\begin{split}\hat{v}^{2}(\alpha,\gamma)=&~{}\frac{\bm{\epsilon}^{\prime}\bm{\epsilon}}{N}+\frac{1}{N}\sum_{i=1}^{m}\sum_{k\in\gamma_{0}\setminus\gamma}b_{i,k}^{2}c_{i,k}n_{i}^{\ell}\\ &~{}+o_{p}\bigg{(}\frac{1}{N}\sum_{i=1}^{m}n_{i}^{\ell}\bigg{)}+O_{p}\bigg{(}\frac{p+mq}{N}\bigg{)}.\end{split}

(B.33)

Thus (8) follows by applying the law of large numbers to $\bm{\epsilon}^{\prime}\bm{\epsilon}/N$ . In addition, if $\ell\in(0,1/2)$ , the asymptotic normality of $\hat{v}^{2}(\alpha,\gamma)$ follows by $p+mq=o(N^{1/2})$ and an application of the central limit theorem to $\bm{\epsilon}^{\prime}\bm{\epsilon}/N$ in (B.33).

Next, we prove (11), for $k\in\gamma\cap\gamma_{0}$ , using (B.2). By (B.31) and Lemma 6 (i)–(iii), we have, for $k\in\gamma\cap\gamma_{0}$ ,

	$\displaystyle\theta_{k}\bm{h}_{i,k}^{\prime}$	$\displaystyle\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})$
	$\displaystyle=$	$\displaystyle~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bigg{(}\sum_{i^{}=1}^{m}\sum_{k^{}\in\gamma_{0}}b_{i^{},k^{}}\bm{h}_{i^{},k^{}}+\bm{\epsilon}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(n_{i}^{-\ell/2})$
	$\displaystyle=$	$\displaystyle~{}o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(1)$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This and (B.4) imply that for $k\in\gamma\cap\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}&\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})+o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(1)\\ =&~{}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bigg{(}\sum_{k^{*}\in\gamma_{0}}\bm{z}_{i,k^{*}}b_{i,k^{*}}+\bm{\epsilon}_{i}\bigg{)}\\ &~{}+o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(1)\\ =&~{}b_{i,k}+o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(1)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the last equality follows from Lemma 3 (ii)–(iv) and Lemma 4 (i). Hence, for $k\in\gamma\cap\gamma_{0}$ ,

	$\displaystyle\theta_{k}^{2}$	$\displaystyle\{\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}\}^{2}$
	$\displaystyle=$	$\displaystyle~{}b_{i,k}^{2}+o_{p}\bigg{(}n_{i}^{\xi-\ell}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{)}+o_{p}(1)$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . Hence by Lemma 3 (ii) and (B.2), we have, for $k\in\gamma\cap\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}^{2}&\bigg{\{}\frac{\partial}{\partial\theta_{k}}\{-2\log L(\bm{\theta},v^{2};\alpha,\gamma)\}\bigg{\}}\\ =&~{}m\bigg{(}\theta_{k}-\frac{1}{m}\sum_{i=1}^{m}\frac{b_{i,k}^{2}}{v^{2}}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{)}+o_{p}(m)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . Hence we have, for $k\in\gamma\cap\gamma_{0}$ ,

\displaystyle\hat{\theta}_{k}(\alpha,\gamma)=

\displaystyle~{}\frac{1}{m}\sum_{i=1}^{m}\frac{b_{i,k}^{2}}{\hat{v}^{2}(\alpha,\gamma)}+o_{p}\bigg{(}\frac{1}{m}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{)}+o_{p}(1).

This completes the proof of (11), for $k\in\gamma\cap\gamma_{0}$ .

It remains to prove (11), for $k\in\gamma\setminus\gamma_{0}$ . Let $\bm{\theta}^{\dagger}$ be $\bm{\theta}$ except that $\{\theta_{k}:k\in\gamma\cap\gamma_{0}\}$ are replaced by $\{\hat{\theta}_{k}(\alpha,\gamma):k\in\gamma\cap\gamma_{0}\}$ . By (B.31) and Lemma 6 (i)–(iii), we have, for $k\in\gamma\setminus\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}&\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger})\bigg{(}\sum_{i^{*}=1}^{m}\sum_{k^{*}\in\gamma_{0}}b_{i^{*},k^{*}}\bm{h}_{i^{*},k^{*}}+\bm{\epsilon}\bigg{)}\\ =&~{}o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(1)\end{split}

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ . This and (B.4) imply that for $k\in\gamma\setminus\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}&\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))\bm{y}\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})+o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(1)\\ =&~{}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta}^{\dagger})\bigg{(}\sum_{k^{*}\in\gamma_{0}}\bm{z}_{i,k^{*}}b_{i,k^{*}}+\bm{\epsilon}_{i}\bigg{)}\\ &~{}+o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(1)\\ =&~{}o_{p}\bigg{(}n_{i}^{(\xi-\ell)/2}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{1/2}\bigg{)}+o_{p}(1)\end{split}

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ , where the last equality follows from Lemma 3 (iii)–(iv) and Lemma 4 (i). Therefore,

\displaystyle\theta_{k}^{2}\{\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))\bm{y}\}^{2}=

\displaystyle~{}o_{p}\bigg{(}n_{i}^{\xi-\ell}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{)}+o_{p}(1)

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ . Hence by Lemma 3 (ii) and (B.2), we have, for $k\in\gamma\setminus\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}^{2}&\bigg{\{}\frac{\partial}{\partial\theta_{k}}\{-2\log L(\bm{\theta}^{\dagger},v^{2};\alpha,\gamma)\}\bigg{\}}=m\theta_{k}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{)}+o_{p}(m)\end{split}

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ . This implies that, for $k\in\gamma\setminus\gamma_{0}$ ,

\displaystyle\hat{\theta}_{k}(\alpha,\gamma)=o_{p}\bigg{(}\frac{1}{m}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\ell}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{)}+o_{p}(1).

This completes the proof of (11). Hence the proof of Theorem 3 is complete.

B.5 Proof of Theorem 4

We first prove (12) using (B.1). By Lemma 6 (i), Lemma 6 (iii)–(v), Lemma 6 (vii), and Lemma 6 (x), we have

	$\displaystyle\big{(}\bm{X}$	$\displaystyle(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})$
		$\displaystyle~{}\times\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}$
	$\displaystyle=$	$\displaystyle~{}\bigg{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\sum_{i=1}^{m}\sum_{k\in\gamma_{0}}b_{i,k}\bm{h}_{i,k}+\bm{\epsilon}\bigg{)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})$
		$\displaystyle~{}\times\bigg{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\sum_{i=1}^{m}\sum_{k\in\gamma_{0}}b_{i,k}\bm{h}_{i,k}+\bm{\epsilon}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}o\bigg{(}\sum_{i=1}^{n}n_{i}^{\xi}\bigg{)}+o_{p}\bigg{(}\sum_{k,k^{}\in\gamma_{0}}\frac{m}{\theta_{k}\theta_{k^{}}}\bigg{)}+o_{p}\bigg{(}\sum_{k\in\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}+O_{p}(p)$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This and (B.30) imply

	$\displaystyle\bm{y}^{\prime}$	$\displaystyle\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}$
	$\displaystyle=$	$\displaystyle~{}\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})$
		$\displaystyle~{}\times(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}$

	$\displaystyle=$	$\displaystyle~{}\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})$
		$\displaystyle~{}\times\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}$
		$\displaystyle~{}+o\bigg{(}\sum_{i=1}^{n}n_{i}^{\xi}\bigg{)}+o_{p}\bigg{(}\sum_{k,k^{}\in\gamma_{0}}\frac{m}{\theta_{k}\theta_{k^{}}}\bigg{)}+o_{p}\bigg{(}\sum_{k\in\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}+O_{p}(p)$
	$\displaystyle=$	$\displaystyle~{}\sum_{i=1}^{m}\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)^{\prime}\bm{X}_{i}(\alpha_{0}\setminus\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)$
		$\displaystyle~{}+2\sum_{i=1}^{m}\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)^{\prime}\bm{X}_{i}(\alpha_{0}\setminus\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})(\bm{Z}_{i}(\gamma_{0})\bm{b}_{i}(\gamma_{0})+\bm{\epsilon}_{i})$
		$\displaystyle~{}+\sum_{i=1}^{m}(\bm{Z}_{i}(\gamma_{0})\bm{b}_{i}(\gamma_{0})+\bm{\epsilon}_{i})^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})(\bm{Z}_{i}(\gamma_{0})\bm{b}_{i}(\gamma_{0})+\bm{\epsilon}_{i})$
		$\displaystyle~{}+o\bigg{(}\sum_{i=1}^{n}n_{i}^{\xi}\bigg{)}+o_{p}\bigg{(}\sum_{k,k^{}\in\gamma_{0}}\frac{m}{\theta_{k}\theta_{k^{}}}\bigg{)}+o_{p}\bigg{(}\sum_{k\in\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}+O_{p}(p)$
	$\displaystyle=$	$\displaystyle~{}\sum_{i=1}^{m}\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}+\sum_{i=1}^{m}\sum_{j\in\alpha_{0}\setminus\alpha}\beta_{j,0}^{2}d_{i,j}n_{i}^{\xi}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}+o_{p}\bigg{(}\sum_{k,k^{}\in\gamma_{0}}\frac{m}{\theta_{k}\theta_{k^{}}}\bigg{)}$
		$\displaystyle~{}+O_{p}\bigg{(}\sum_{k\in\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}+O_{p}(p+mq)$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the last equality follows from Lemma 3 (ii)–(iv) and Lemma 4. Hence by (B.1), we have, for $v^{2}\in(0,\infty)$ ,

\displaystyle\begin{split}v^{4}&\bigg{\{}\frac{\partial}{\partial v^{2}}\{-2\log L(\bm{\theta},v^{2};\alpha,\gamma)\}\bigg{\}}\\ =&~{}N\bigg{(}v^{2}-\frac{\bm{\epsilon}^{\prime}\bm{\epsilon}}{N}+\frac{1}{N}\sum_{i=1}^{m}\sum_{j\in\alpha_{0}\setminus\alpha}\beta_{j,0}^{2}d_{i,j}n_{i}^{\xi}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}\\ &~{}+o_{p}\bigg{(}\sum_{k,k^{*}\in\gamma_{0}}\frac{m}{\theta_{k}\theta_{k^{*}}}\bigg{)}+O_{p}\bigg{(}\sum_{k\in\gamma_{0}}\frac{m}{\theta_{k}}\bigg{)}+O_{p}(p+mq)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This and Lemma 5 imply that for $(\xi,\ell)\in(0,1]\times(0,1]$ ,

\displaystyle\begin{split}\hat{v}^{2}(\alpha,\gamma)=&~{}\frac{\bm{\epsilon}^{\prime}\bm{\epsilon}}{N}+\frac{1}{N}\sum_{i=1}^{m}\sum_{j\in\alpha_{0}\setminus\alpha}\beta_{j,0}^{2}d_{i,j}n_{i}^{\xi}\\ &~{}+o_{p}\bigg{(}\frac{1}{N}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}+O_{p}\bigg{(}\frac{p+mq}{N}\bigg{)}.\end{split}

(B.34)

Thus (12) follows by applying the law of large numbers to $\bm{\epsilon}^{\prime}\bm{\epsilon}/N$ . In addition, if $\xi\in(0,1/2)$ , the asymptotic normality of $\hat{v}^{2}(\alpha,\gamma)$ follows by $p+mq=o(N^{1/2})$ and an application of the central limit theorem to $\bm{\epsilon}^{\prime}\bm{\epsilon}/N$ in (B.34).

Next, we prove (15), for $k\in\gamma\cap\gamma_{0}$ , using (B.2). By Lemma 6 (i) and Lemma 6 (iii)–(iv), we have, for $k\in\gamma\cap\gamma_{0}$ ,

	$\displaystyle\theta_{k}$	$\displaystyle\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}$
	$\displaystyle=$	$\displaystyle~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})$
		$\displaystyle~{}\times\bigg{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\sum_{i^{}=1}^{m}\sum_{k^{}\in\gamma_{0}}b_{i^{},k^{}}\bm{h}_{i^{},k^{}}+\bm{\epsilon}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}o_{p}\bigg{(}\sum_{k^{}\in\gamma_{0}}\frac{n_{i}^{(\xi-\ell)/2}}{\theta_{k^{}}}\bigg{)}+o_{p}(n_{i}^{(\xi-\ell)/2-\tau})+o_{p}(n_{i}^{-\ell/2})$
	$\displaystyle=$	$\displaystyle~{}o_{p}\bigg{(}\sum_{k^{}\in\gamma_{0}}\frac{n_{i}^{(\xi-\ell)/2}}{\theta_{k^{}}}\bigg{)}+o_{p}(1)$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This and (B.30) imply that for $k\in\gamma\cap\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}\bm{h}_{i,k}^{\prime}&\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\\ &~{}\times\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}\\ &~{}+o_{p}\bigg{(}\sum_{k^{*}\in\gamma_{0}}\frac{n_{i}^{(\xi-\ell)/2}}{\theta_{k^{*}}}\bigg{)}+o_{p}(1)\\ =&~{}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bigg{(}\bm{X}_{i}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\sum_{k^{*}\in\gamma_{0}}\bm{z}_{i,k^{*}}b_{i,k^{*}}+\bm{\epsilon}_{i}\bigg{)}\\ &~{}+o_{p}\bigg{(}\sum_{k^{*}\in\gamma_{0}}\frac{n_{i}^{(\xi-\ell)/2}}{\theta_{k^{*}}}\bigg{)}+o_{p}(1)\\ =&~{}b_{i,k}+o_{p}\bigg{(}\sum_{k^{*}\in\gamma_{0}}\frac{n_{i}^{(\xi-\ell)/2}}{\theta_{k^{*}}}\bigg{)}+o_{p}(1)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the last equality follows from Lemma 2 (iii), Lemma 3 (ii)–(iii), and Lemma 4 (i). Hence, for $k\in\gamma\cap\gamma_{0}$ ,

\displaystyle\theta_{k}^{2}\{\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\bm{y}\}^{2}=

\displaystyle~{}b_{i,k}^{2}+o_{p}\bigg{(}\sum_{k,k^{*}\in\gamma_{0}}\frac{n_{i}^{\xi-\ell}}{\theta_{k}\theta_{k^{*}}}\bigg{)}+o_{p}(1)

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . Hence by Lemma 3 (ii) and (B.2), we have, for $k\in\gamma\cap\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}^{2}&\bigg{\{}\frac{\partial}{\partial\theta_{k}}\{-2\log L(\bm{\theta},v^{2};\alpha,\gamma)\}\bigg{\}}\\ =&~{}m\bigg{(}\theta_{k}-\frac{1}{m}\sum_{i=1}^{m}\frac{b_{i,k}^{2}}{v^{2}}\bigg{)}+o_{p}\bigg{(}\sum_{i=1}^{m}\sum_{k,k^{*}\in\gamma_{0}}\frac{n_{i}^{\xi-\ell}}{\theta_{k}\theta_{k^{*}}}\bigg{)}+o_{p}(1)\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This and Lemma 5 imply that for $k\in\gamma\cap\gamma_{0}$ ,

\displaystyle\hat{\theta}_{k}(\alpha,\gamma)=

\displaystyle~{}\frac{1}{m}\sum_{i=1}^{m}\frac{b_{i,k}^{2}}{\hat{v}^{2}(\alpha,\gamma)}+o_{p}\bigg{(}\frac{1}{m}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{)}+o_{p}(1).

This completes the proof of (15) when $k\in\gamma\cap\gamma_{0}$ .

It remains to prove (15), for $k\in\gamma\setminus\gamma_{0}$ . Let $\bm{\theta}^{\dagger}$ be $\bm{\theta}$ except that $\{\theta_{k}:k\in\gamma\cap\gamma_{0}\}$ are replaced by $\{\hat{\theta}_{k}(\alpha,\gamma):k\in\gamma\cap\gamma_{0}\}$ . By Lemma 6 (i) and Lemma 6 (iii)–(iv), we have, for $k\in\gamma\setminus\gamma_{0}$ ,

	$\displaystyle\theta_{k}$	$\displaystyle\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger})\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}$
	$\displaystyle=$	$\displaystyle~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger})$
		$\displaystyle~{}\times\bigg{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\sum_{i^{}=1}^{m}\sum_{k^{}\in\gamma_{0}}b_{i^{},k^{}}\bm{h}_{i^{},k^{}}+\bm{\epsilon}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}o_{p}(n_{i}^{(\xi-\ell)/2-\tau})+o_{p}(n_{i}^{-\ell/2})$
	$\displaystyle=$	$\displaystyle~{}o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(1)$

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ . This and (B.30) imply that for $k\in\gamma\setminus\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}&\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))\bm{y}\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))\\ &~{}\times\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}\\ =&~{}\theta_{k}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})\big{(}\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}\big{)}\\ &~{}+o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(1)\\ =&~{}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta}^{\dagger})\bigg{(}\bm{X}_{i}(\alpha_{0}\setminus\alpha)\bm{\beta}_{0}(\alpha_{0}\setminus\alpha)+\sum_{k^{*}\in\gamma_{0}}\bm{z}_{i,k^{*}}b_{i,k^{*}}+\bm{\epsilon}_{i}\bigg{)}\\ &~{}+o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(1)\\ =&~{}o_{p}(n_{i}^{(\xi-\ell)/2})+o_{p}(1)\end{split}

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ , where the last equality follows from Lemma 2 (iii), Lemma 3 (iii), and Lemma 4 (i). Therefore,

\displaystyle\theta_{k}^{2}\{\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}^{\dagger}))\bm{y}\}^{2}=

\displaystyle~{}o_{p}(n_{i}^{\xi-\ell})+o_{p}(1)

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ . Hence by Lemma 3 (ii) and (B.2), we have, for $k\in\gamma\setminus\gamma_{0}$ ,

\displaystyle\begin{split}\theta_{k}^{2}\bigg{\{}\frac{\partial}{\partial\theta_{k}}\{-2\log L(\bm{\theta}^{\dagger},v^{2};\alpha,\gamma)\}\bigg{\}}=&~{}m\theta_{k}+o_{p}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{)}+o_{p}(m)\end{split}

uniformly over $\bm{\theta}(\gamma\setminus\gamma_{0})\in[0,\infty)^{q(\gamma\setminus\gamma_{0})}$ . This and Lemma 5 imply that for $k\in\gamma\setminus\gamma_{0}$ ,

\displaystyle\hat{\theta}_{k}(\alpha,\gamma)=o_{p}\bigg{(}\frac{1}{m}\sum_{i=1}^{m}n_{i}^{\xi-\ell}\bigg{)}+o_{p}(1).

This completes the proof of (15), for $k\in\gamma\setminus\gamma_{0}$ . Hence the proof of Theorem 4 is complete.

Appendix C Proofs of Auxiliary Lemmas

C.1 Proof of Lemma 2

Let $\bm{z}_{i,(s)}$ ; $s=1,\dots,q(\gamma)$ be the $s$ -th column of $\bm{Z}_{i}(\gamma)$ and $\bm{H}_{i,t}(\gamma,\bm{\theta})$ defined in (A.4). For Lemma 2 (i)–(ii) to hold, it suffices to prove that for $k\notin\gamma$ and $j,j^{*}=1,\dots,p$ ,

$\displaystyle\bm{x}_{i,j}^{\prime}\bm{H}_{i,t}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j}=$	$\displaystyle~{}d_{i,j}n_{i}^{\xi}+o(n_{i}^{\xi})+o(tn_{i}^{\xi-2\tau}),$	(C.1)
$\displaystyle\bm{x}_{i,j}^{\prime}\bm{H}_{i,t}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j^{*}}=$	$\displaystyle~{}o(n_{i}^{\xi-\tau})+o(tn_{i}^{\xi-2\tau}),$	(C.2)
$\displaystyle\bm{x}_{i,j}^{\prime}\bm{H}_{i,t}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}=$	$\displaystyle~{}o(n_{i}^{(\xi+\ell)/2-\tau})+o(tn_{i}^{(\xi+\ell)/2-2\tau})$	(C.3)

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ . We prove (C.1)–(C.3) by induction. For $j=1,\dots,p$ and $t=1$ , by (A.2) and (A1)–(A3), we have

	$\displaystyle\bm{x}_{i,j}^{\prime}\bm{H}_{i,1}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j}=$	$\displaystyle~{}\bm{x}_{i,j}^{\prime}\bm{x}_{i,j}-\frac{\theta_{(1)}\bm{x}_{i,j}^{\prime}\bm{z}_{i,(1)}\bm{z}_{i,(1)}^{\prime}\bm{x}_{i,j}}{1+\theta_{(1)}\bm{z}_{i,(1)}^{\prime}\bm{z}_{i,(1)}}$
	$\displaystyle=$	$\displaystyle~{}d_{i,j}n_{i}^{\xi}+o(n_{i}^{\xi})+o(n_{i}^{\xi-2\tau})$

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ . For $j,j^{*}=1,\dots,p$ , $j\neq j^{*}$ and $t=1$ , by (A.2) and (A1)–(A3), we have

	$\displaystyle\bm{x}_{i,j}^{\prime}\bm{H}_{i,1}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j^{*}}=$	$\displaystyle~{}\bm{x}_{i,j}^{\prime}\bm{x}_{i,j^{}}-\frac{\theta_{(1)}\bm{x}_{i,j}^{\prime}\bm{z}_{i,(1)}\bm{z}_{i,(1)}^{\prime}\bm{x}_{i,j^{}}}{1+\theta_{(1)}\bm{z}_{i,(1)}^{\prime}\bm{z}_{i,(1)}}$
	$\displaystyle=$	$\displaystyle~{}o(n_{i}^{\xi-\tau})+o(n_{i}^{\xi-2\tau})$

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ . For $j=1,\dots,p$ , $k\notin\gamma$ and $t=1$ , by (A.2) and (A1)–(A3), we have

	$\displaystyle\bm{x}_{i,j}^{\prime}\bm{H}_{i,1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}=$	$\displaystyle~{}\bm{x}_{i,j}^{\prime}\bm{z}_{i,k}-\frac{\theta_{(1)}\bm{x}_{i,j}^{\prime}\bm{z}_{i,(1)}\bm{z}_{i,(1)}^{\prime}\bm{z}_{i,k}}{1+\theta_{(1)}\bm{z}_{i,(1)}^{\prime}\bm{z}_{i,(1)}}$
	$\displaystyle=$	$\displaystyle~{}o(n_{i}^{(\xi+\ell)/2-\tau})+o(n_{i}^{(\xi+\ell)/2-2\tau})$

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ . Now suppose that (C.1)–(C.3) hold for $t=r$ . Then for $j=1,\dots,p$ and $t=r+1$ , by (A.2) and (C.1)–(C.3) with $t=r$ , and Lemma 3 (i), we have

	$\displaystyle\bm{x}_{i,j}^{\prime}$	$\displaystyle\bm{H}_{i,r+1}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j}$
	$\displaystyle=$	$\displaystyle~{}\bm{x}_{i,j}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j}-\frac{\theta_{(r+1)}\bm{x}_{i,j}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(r+1)}\bm{z}_{i,(r+1)}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j}}{1+\theta_{(r+1)}\bm{z}_{i,(r+1)}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(r+1)}}$
	$\displaystyle=$	$\displaystyle~{}d_{i,j}n_{i}^{\xi}+o(n_{i}^{\xi})+o(\{r+1\}n_{i}^{\xi-2\tau})$

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ . For $j,j^{*}=1,\dots,p$ , $j\neq j^{*}$ , and $t=r+1$ , by (A.2) and (C.1)–(C.3) with $t=r$ , and Lemma 3 (i), we have

	$\displaystyle\bm{x}_{i,j}^{\prime}$	$\displaystyle\bm{H}_{i,r+1}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j^{*}}$
	$\displaystyle=$	$\displaystyle~{}\bm{x}_{i,j}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j^{}}-\frac{\theta_{(r+1)}\bm{x}_{i,j}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(r+1)}\bm{z}_{i,(r+1)}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j^{}}}{1+\theta_{(r+1)}\bm{z}_{i,(r+1)}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(r+1)}}$
	$\displaystyle=$	$\displaystyle~{}o(n_{i}^{\xi-\tau})+o(\{r+1\}n_{i}^{\xi-2\tau})$

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ . For $j,j^{*}=1,\dots,p$ , $k\notin\gamma$ , and $t=r+1$ , by (A.2) and (C.1)–(C.3) with $t=r$ , and Lemma 3 (i), we have

	$\displaystyle\bm{x}_{i,j}^{\prime}$	$\displaystyle\bm{H}_{i,r+1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}$
	$\displaystyle=$	$\displaystyle~{}\bm{x}_{i,j}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}-\frac{\theta_{(r+1)}\bm{x}_{i,j}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(r+1)}\bm{z}_{i,(r+1)}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}}{1+\theta_{(r+1)}\bm{z}_{i,(r+1)}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(r+1)}}$
	$\displaystyle=$	$\displaystyle~{}o(n_{i}^{(\xi+\ell)/2-\tau})+o(\{r+1\}n_{i}^{(\xi+\ell)/2-2\tau})$

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ . This completes the proofs of (C.1)–(C.3). Hence the proofs of Lemma 2 (i)–(ii) are complete.

We finally prove Lemma 2 (iii). Without loss of generality, we assume that $q(\gamma)=q$ , $t=q$ , and $k=(q)$ . Then by (A.2),

	$\displaystyle\theta_{(q)}\bm{x}_{i,j}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}=$	$\displaystyle~{}\theta_{(q)}\bigg{\{}\bm{x}_{i,j}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}$
		$\displaystyle~{}-\frac{\theta_{(q)}\bm{x}_{i,j}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,q}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}\bigg{\}}$
	$\displaystyle=$	$\displaystyle~{}\frac{\theta_{(q)}\bm{x}_{i,j}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}},$

where we note that $\theta_{(q)}$ can be arbitrarily small and the dominant term of the denominator of the last equation can be equal to (i) $\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}$ or (ii) $1$ . For the case of (i), $\theta_{(q)}n_{i}^{\ell}\rightarrow\infty$ by Lemma 3 (i); hence, using Lemma 2 (ii) and Lemma 3 (i), we have

\displaystyle\theta_{(q)}\bm{x}_{i,j}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}=

\displaystyle~{}o(n_{i}^{(\xi-\ell)/2-\tau}),

and thus

\displaystyle\bm{x}_{i,j}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}=

\displaystyle~{}o(n_{i}^{(\xi+\ell)/2-\tau}).

For the case of (ii), $\theta_{(q)}=O(n_{i}^{-\ell})$ by Lemma 3 (i); hence, using Lemma 3 (i), we have

\displaystyle\theta_{(q)}\bm{x}_{i,j}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}=

\displaystyle~{}o(\theta_{(q)}n_{i}^{(\xi+\ell)/2-\tau}),

which also gives the following two results:

	$\displaystyle\bm{x}_{i,j}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}=$	$\displaystyle~{}o(n_{i}^{(\xi+\ell)/2-\tau}),$
	$\displaystyle\theta_{(q)}\bm{x}_{i,j}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}=$	$\displaystyle~{}o(n_{i}^{(\xi-\ell)/2-\tau}).$

In conclusion, we have

\displaystyle\begin{split}\theta_{(q)}\bm{x}_{i,j}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}=&~{}o(n_{i}^{(\xi-\ell)/2-\tau}),\\ \bm{x}_{i,j}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}=&~{}o(n_{i}^{(\xi+\ell)/2-\tau})\end{split}

(C.4)

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ . This completes the proof.

C.2 Proof of Lemma 3

Let $\bm{z}_{i,(s)}$ ; $s=1,\dots,q(\gamma)$ be the $s$ -th column of $\bm{Z}_{i}(\gamma)$ and $\bm{H}_{i,t}(\gamma,\bm{\theta})$ defined in (A.4). We first prove Lemma 3 (i). By (A.4), it suffices to prove that for $k\notin\gamma$ ,

\displaystyle\bm{z}_{i,k}^{\prime}\bm{H}_{i,t}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}=

\displaystyle~{}c_{i,k}n_{i}^{\ell}+o(n_{i}^{\ell})+o(tn_{i}^{\ell-2\tau}),

(C.5)

and for $k,k^{*}\notin\gamma$ and $k\neq k^{*}$ ,

\displaystyle\bm{z}_{i,k}^{\prime}\bm{H}_{i,t}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k^{*}}=

\displaystyle~{}o(n_{i}^{\ell-\tau})+o(tn_{i}^{\ell-2\tau})

(C.6)

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ by induction. For $t=1$ and $k\notin\gamma$ , by (A.2) and (A2), we have

	$\displaystyle\bm{z}_{i,k}^{\prime}\bm{H}_{i,1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}=$	$\displaystyle~{}\bm{z}_{i,k}^{\prime}\bigg{(}\bm{I}_{n_{i}}-\frac{\theta_{(1)}\bm{z}_{i,(1)}\bm{z}_{i,(1)}^{\prime}}{1+\theta_{(1)}\bm{z}_{i,(1)}^{\prime}\bm{z}_{i,(1)}}\bigg{)}\bm{z}_{i,k}$
	$\displaystyle=$	$\displaystyle~{}\bm{z}_{i,k}^{\prime}\bm{z}_{i,k}-\frac{\theta_{(1)}\bm{z}_{i,k}^{\prime}\bm{z}_{i,(1)}\bm{z}_{i,(1)}^{\prime}\bm{z}_{i,k}}{1+\theta_{(1)}\bm{z}_{i,(1)}^{\prime}\bm{z}_{i,(1)}}$
	$\displaystyle=$	$\displaystyle~{}c_{i,k}n_{i}^{\ell}+o(n_{i}^{\ell})+o(n_{i}^{\ell-2\tau})$

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ . For $k,k^{*}\notin\gamma$ and $k\neq k^{*}$ , by (A.2) and (A2), we have

	$\displaystyle\bm{z}_{i,k}^{\prime}\bm{H}_{i,1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k^{*}}=$	$\displaystyle~{}\bm{z}_{i,k}^{\prime}\bm{z}_{i,k^{}}-\frac{\theta_{(1)}\bm{z}_{i,k}^{\prime}\bm{z}_{i,(1)}\bm{z}_{i,(1)}^{\prime}\bm{z}_{i,k^{}}}{1+\theta_{(1)}\bm{z}_{i,(1)}^{\prime}\bm{z}_{i,(1)}}$
	$\displaystyle=$	$\displaystyle~{}o(n_{i}^{\ell-\tau})+o(n_{i}^{\ell-2\tau})$

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ . Now suppose that (C.5) and (C.6) hold for $t=r$ . Then for $k\notin\gamma$ and $t=r+1$ , by (A.2), and (C.5) and (C.6) with $t=r$ , we have

	$\displaystyle\bm{z}_{i,k}^{\prime}\bm{H}_{i,r+1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}=$	$\displaystyle~{}\bm{z}_{i,k}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}$
		$\displaystyle~{}-\frac{\theta_{(r+1)}\bm{z}_{i,k}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(r+1)}\bm{z}_{i,(r+1)}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k}}{1+\theta_{(r+1)}\bm{z}_{i,(r+1)}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(r+1)}}$
	$\displaystyle=$	$\displaystyle~{}c_{i,k}n_{i}^{\ell}+o(n_{i}^{\ell})+o(\{r+1\}n_{i}^{\ell-2\tau})$

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ . For $k,k^{*}\notin\gamma$ and $t=r+1$ , by (A.2), and (C.5) and (C.6) with $t=r$ , we have

	$\displaystyle\bm{z}_{i,k}^{\prime}\bm{H}_{i,r+1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k^{*}}=$	$\displaystyle~{}\bm{z}_{i,k}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k^{*}}$
		$\displaystyle~{}-\frac{\theta_{(r+1)}\bm{z}_{i,k}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(r+1)}\bm{z}_{i,(r+1)}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,k^{*}}}{1+\theta_{(r+1)}\bm{z}_{i,(r+1)}^{\prime}\bm{H}_{i,r}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(r+1)}}$
	$\displaystyle=$	$\displaystyle~{}o(n_{i}^{\ell-\tau})+o(\{r+1\}n_{i}^{\ell-2\tau})$

uniformly over $\bm{\theta}\in[0,\infty)^{q(\gamma)}$ . This completes the proof of (C.5) and (C.6). Hence Lemma 3 (i) follows from (C.5), (C.6) with $t=q(\gamma)$ and $q=o(n_{\min}^{\tau})$ . This completes the proof of Lemma 3 (i).

We now prove Lemma 3 (ii). Without loss of generality, we assume that $q(\gamma)=q$ and $k=(q)$ . Then by Lemma 3 (i) and (A.2),

	$\displaystyle\theta_{(q)}^{2}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}=$	$\displaystyle~{}\theta_{(q)}^{2}\bigg{\{}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}$
		$\displaystyle~{}-\frac{\theta_{(q)}(\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)})^{2}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}\bigg{\}}$
	$\displaystyle=$	$\displaystyle~{}\frac{\theta_{(q)}^{2}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}=O(\theta_{(q)}^{2}n_{i}^{\ell})$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ . Again, by Lemma 3 (i), we have

	$\displaystyle\theta_{(q)}^{2}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}=$	$\displaystyle~{}\frac{\theta_{(q)}^{2}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}$
	$\displaystyle=$	$\displaystyle~{}\theta_{(q)}-\frac{\theta_{(q)}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}$
	$\displaystyle=$	$\displaystyle~{}\theta_{(q)}+O(n_{i}^{-\ell})$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ . This completes the proof of Lemma 3 (ii).

We now prove Lemma 3 (iii). Without loss of generality, we assume that $q(\gamma)=q$ , $k=(q)$ , and $k^{*}=(q-1)$ . Then by (A.2),

	$\displaystyle\theta_{(q)}$	$\displaystyle\theta_{(q-1)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}$
	$\displaystyle=$	$\displaystyle~{}\theta_{(q)}\theta_{(q-1)}\bigg{\{}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}$
		$\displaystyle~{}-\frac{\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}\bigg{\}}$
	$\displaystyle=$	$\displaystyle~{}\frac{\theta_{(q)}\theta_{(q-1)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}$
	$\displaystyle=$	$\displaystyle~{}\frac{\theta_{(q)}\theta_{(q-1)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-2}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}}{(1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)})(1+\theta_{(q-1)}\bm{z}_{i,(q-1)}^{\prime}\bm{H}_{i,q-2}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)})},$

where we note that $\theta_{(q)}$ and $\theta_{(q-1)}$ can be arbitrarily small and the dominant term of the denominator of the last equation can be equal to

(i)

$\theta_{(q)}\theta_{(q-1)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}\bm{z}_{i,(q-1)}^{\prime}\bm{H}_{i,q-2}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}$ ;
(ii)

$\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}+\theta_{(q-1)}\bm{z}_{i,(q-1)}^{\prime}\bm{H}_{i,q-2}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}$ ;
(iii)

$1$ .

For the case of (i), $\theta_{(q)}n_{i}^{\ell}\rightarrow\infty$ and $\theta_{(q-1)}n_{i}^{\ell}\rightarrow\infty$ by Lemma 3 (i); hence, using Lemma 3 (i), we have

\displaystyle\theta_{(q)}\theta_{(q-1)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=

\displaystyle~{}o_{p}(n_{i}^{-\ell-\tau}),

which also gives the following two results:

	$\displaystyle\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=$	$\displaystyle~{}o_{p}(n_{i}^{-\tau}),$
	$\displaystyle\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=$	$\displaystyle~{}o_{p}(n_{i}^{\ell-\tau}).$

For the case of (ii), $\theta_{(q)}n_{i}^{\ell}\rightarrow\infty$ and $\theta_{(q)}=O(n_{i}^{-\ell})$ (or vice versa) by Lemma 3 (i); hence, using Lemma 3 (i), we have

\displaystyle\theta_{(q)}\theta_{(q-1)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=

\displaystyle~{}o_{p}(\theta_{(q-1)}n_{i}^{-\tau}),

which gives the following three results:

	$\displaystyle\theta_{(q)}\theta_{(q-1)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=$	$\displaystyle~{}o_{p}(n_{i}^{-\ell-\tau}),$
	$\displaystyle\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=$	$\displaystyle~{}o_{p}(n_{i}^{-\tau}),$
	$\displaystyle\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=$	$\displaystyle~{}o_{p}(n_{i}^{\ell-\tau}).$

For the case of (iii), $\theta_{(q)}=O(n_{i}^{-\ell})$ and $\theta_{(q)}=O(n_{i}^{-\ell})$ by Lemma 3 (i); hence, using Lemma 3 (i), we have

\displaystyle\theta_{(q)}\theta_{(q-1)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=

\displaystyle~{}o_{p}(\theta_{(q)}\theta_{(q-1)}n_{i}^{\ell-\tau}),

which also gives the following three results:

	$\displaystyle\theta_{(q)}\theta_{(q-1)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=$	$\displaystyle~{}o_{p}(n_{i}^{-\ell-\tau}),$
	$\displaystyle\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=$	$\displaystyle~{}o_{p}(n_{i}^{-\tau}),$
	$\displaystyle\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=$	$\displaystyle~{}o_{p}(n_{i}^{\ell-\tau}).$

In conclusion, we have

\displaystyle\begin{split}\theta_{(q)}\theta_{(q-1)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=&~{}o_{p}(n_{i}^{-\ell-\tau}),\\ \theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=&~{}o_{p}(n_{i}^{-\tau}),\\ \bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}=&~{}o_{p}(n_{i}^{\ell-\tau})\end{split}

(C.7)

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ . This completes the proof of Lemma 3 (iii).

We finally prove Lemma 3 (iv). Without loss of generality, it suffices to prove Lemma 3 (iv) by replacing $\bm{H}_{i}(\gamma,\bm{\theta})$ with $\bm{H}_{i,q-1}(\gamma,\bm{\theta})$ with $q(\gamma)=q$ , $k=(q-1)$ , and $k^{*}=(q)$ . Then by (A.2),

	$\displaystyle\theta_{(q-1)}$	$\displaystyle\bm{z}_{i,(q-1)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}$
	$\displaystyle=$	$\displaystyle~{}\theta_{(q-1)}\bigg{\{}\bm{z}_{i,(q-1)}^{\prime}\bm{H}_{i,q-2}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}$
		$\displaystyle~{}-\frac{\theta_{(q-1)}\bm{z}_{i,(q-1)}^{\prime}\bm{H}_{i,q-2}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}\bm{z}_{i,(q-1)}^{\prime}\bm{H}_{i,q-2}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}{1+\theta_{(q-1)}\bm{z}_{i,(q-1)}^{\prime}\bm{H}_{i,q-2}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}}\bigg{\}}$
	$\displaystyle=$	$\displaystyle~{}\frac{\theta_{(q-1)}\bm{z}_{i,(q-1)}^{\prime}\bm{H}_{i,q-2}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}{1+\theta_{(q-1)}\bm{z}_{i,(q-1)}^{\prime}\bm{H}_{i,q-2}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q-1)}}.$

Hence, Lemma 3 (iv) follows from Lemma 3 (i) and arguments similar to the proof of (C.4). This completes the proof.

C.3 Proof of Lemma 4

Note that for $k=1,\ldots,q$ and $j=1,\ldots,p$ ,

	$\displaystyle\bm{\epsilon}_{i}^{\prime}\bm{z}_{i,k}=$	$\displaystyle~{}O_{p}(n_{i}^{\ell/2}),$
	$\displaystyle\bm{\epsilon}_{i}^{\prime}\bm{x}_{i,j}=$	$\displaystyle~{}O_{p}(n_{i}^{\xi/2}).$

Lemma 4 (ii)–(iii) then follow arguments similarly from the induction and the proofs of Lemma 2 (iii) are hence omitted.

We next prove Lemma 4 (iv). Let $\bm{z}_{i,(s)}$ be the $s$ -th column of $\bm{Z}_{i}(\gamma)$ and $\bm{H}_{i,t}(\gamma,\bm{\theta})$ be defined in (A.4). Without loss of generality, we assume $q(\gamma)=q$ . Hence by (A.6), Lemma 3 (i), and Lemma 4 (ii), we have

	$\displaystyle\bm{\epsilon}_{i}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}=$	$\displaystyle~{}\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}-\sum_{k=1}^{q}\frac{\theta_{(k)}\bm{\epsilon}_{i}^{\prime}\bm{H}_{i,k-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(k)}\bm{z}_{i,(k)}^{\prime}\bm{H}_{i,k-1}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}}{1+\theta_{(k)}\bm{z}_{i,(k)}^{\prime}\bm{H}_{i,k-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(k)}}$
	$\displaystyle=$	$\displaystyle~{}\bm{\epsilon}_{i}^{\prime}\bm{\epsilon}_{i}+O_{p}(q)$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ . This completes the proof of Lemma 4 (iv).

It remains to prove Lemma 4 (i). Again, without loss of generality, it suffices to prove Lemma 4 (i) for $q(\gamma)=q$ and $k=(q)$ . Then by (A.2),

\displaystyle\theta_{(q)}\bm{\epsilon}_{i}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}=

\displaystyle~{}\frac{\theta_{(q)}\bm{\epsilon}_{i}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}.

Hence, Lemma 4 (i) follows from Lemma 3 (i), Lemma 4 (ii), and arguments similar to the proof of (C.4). This completes the proof.

C.4 Proof of Lemma 5

We show the lemma for $(\alpha,\gamma)\in\mathcal{A}_{0}\times\mathcal{G}_{0}$ , where the proofs with respect to the remaining models are similar and are hence omitted.

Let $\bm{z}_{i,(s)}$ be the $s$ -th column of $\bm{Z}_{i}(\gamma)$ and $\bm{H}_{i,t}(\gamma,\bm{\theta})$ be defined in (A.4). Without loss of generality, we assume that $q(\gamma)=q$ and $\bm{Z}_{i}(\gamma_{0})\bm{b}_{i}(\gamma_{0})=\sum_{s=q-q_{0}+1}^{q}\bm{z}_{i,(s)}b_{i,(s)}$ . It then suffices to prove that for $(\alpha,\gamma)\in\mathcal{A}_{0}\times\mathcal{G}_{0}$ and $v^{2}>0$

\displaystyle-2\log L(\bm{\theta},v^{2};\alpha,\gamma)-\{-2\log L(\bm{\theta}_{0}^{\dagger},v^{2};\alpha,\gamma)\}\xrightarrow{p}\infty,

(C.8)

as both $N\rightarrow\infty$ and $\theta_{(k)}\rightarrow 0$ for some $k\in\{q-q_{0}+1,\dots,q\}$ , where $\bm{\theta}_{0}^{\dagger}\equiv(0,\dots,0,\theta_{(q-q_{0}+1),0},\dots,\theta_{(q),0})^{\prime}$ , and $\theta_{(s),0}$ being the true value of $\theta_{(s)}$ ; $s=q-q_{0}+1,\dots,q$ . Note that by (A.3) and (A.1), we have

	$\displaystyle\det(\bm{H}_{i}(\gamma,\bm{\theta}))=$	$\displaystyle~{}\det\bigg{(}\bm{I}_{n_{i}}+\sum_{s=1}^{q}\theta_{(s)}\bm{z}_{i,(s)}\bm{z}_{i,(s)}^{\prime}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}\det(\bm{H}_{i,q-1}(\gamma,\bm{\theta})+\theta_{(q)}\bm{z}_{i,(q)}\bm{z}_{i,(q)}^{\prime})$
	$\displaystyle=$	$\displaystyle~{}\det(\bm{H}_{i,q-1}(\gamma,\bm{\theta}))(1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}).$

Continuously expanding the above equation by (A.1) yields

\displaystyle\begin{split}\log\det(\bm{H}_{i}(\gamma,\bm{\theta}))=&~{}\log\bigg{\{}\prod_{s=1}^{q}(1+\theta_{(s)}\bm{z}_{i,(s)}^{\prime}\bm{H}_{i,s-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(s)})\bigg{\}}\\ =&~{}\sum_{s=1}^{q}\log(1+\theta_{(s)}\bm{z}_{i,(s)}^{\prime}\bm{H}_{i,s-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(s)}),\end{split}

where $\bm{H}_{i,0}(\gamma,\bm{\theta})=\bm{I}_{n_{i}}$ . This together with (7) and (B.4) yields for $(\alpha,\gamma)\in\mathcal{A}_{0}\times\mathcal{G}_{0}$ and fixed $v^{2}>0$ ,

\displaystyle\begin{split}-2&\log L(\bm{\theta},v^{2};\alpha,\gamma)\\ =&~{}N\log(2\pi)+N\log(v^{2})+\log\det(\bm{H}(\gamma,\bm{\theta}))+\frac{\bm{y}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{A}(\alpha,\gamma;\bm{\theta})\bm{y}}{v^{2}}\\ =&~{}N\log(2\pi)+N\log(v^{2})+\sum_{i=1}^{m}\sum_{s=1}^{q}\log(1+\theta_{(s)}\bm{z}_{i,(s)}^{\prime}\bm{H}_{i,s-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(s)})\\ &~{}+\frac{(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})}{v^{2}}.\end{split}

Hence, we have, for $(\alpha,\gamma)\in\mathcal{A}_{0}\times\mathcal{G}_{0}$ ,

	$\displaystyle-2\log$	$\displaystyle L(\bm{\theta},v^{2};\alpha,\gamma)-\{-2\log L(\bm{\theta}_{0}^{\dagger},v^{2};\alpha,\gamma)\}$
	$\displaystyle=$	$\displaystyle~{}\sum_{i=1}^{m}\bigg{\{}\sum_{s=q-q_{0}+1}^{q}\log\bigg{(}\frac{1+\theta_{(s)}\bm{z}_{i,(s)}^{\prime}\bm{H}_{i,s-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(s)}}{1+\theta_{(s),0}\bm{z}_{i,(s)}^{\prime}\bm{H}_{i,s-1}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{z}_{i,(s)}}\bigg{)}\bigg{\}}$
		$\displaystyle~{}+\frac{1}{v^{2}}(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})^{\prime}\big{\{}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))$
		$\displaystyle~{}-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger}))\big{\}}(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon}),$

where

	$\displaystyle(\bm{Z}(\gamma_{0})\bm{b}$	$\displaystyle(\gamma_{0})+\bm{\epsilon})^{\prime}\big{\{}\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))$
		$\displaystyle~{}-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger}))\big{\}}(\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+\bm{\epsilon})$
	$\displaystyle=$	$\displaystyle~{}\bm{b}(\gamma_{0})^{\prime}\bm{Z}(\gamma_{0})^{\prime}\{\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))$
		$\displaystyle~{}-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger}))\}\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})$
		$\displaystyle~{}+2\bm{b}(\gamma_{0})^{\prime}\bm{Z}(\gamma_{0})^{\prime}\{\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))$
		$\displaystyle~{}-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger}))\}\bm{\epsilon}$
		$\displaystyle~{}+\bm{\epsilon}^{\prime}\{\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))$
		$\displaystyle~{}-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger}))\}\bm{\epsilon}.$

Hence, for (C.8) to hold, it suffices to prove

\displaystyle\begin{split}\bm{\epsilon}^{\prime}&\{\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\\ &~{}-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger}))\}\bm{\epsilon}=O_{p}(m)\end{split}

(C.9)

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ and

\displaystyle\begin{split}\sum_{i=1}^{m}&~{}\bigg{\{}\sum_{s=q-q_{0}+1}^{q}\log\bigg{(}\frac{1+\theta_{(s)}\bm{z}_{i,(s)}^{\prime}\bm{H}_{i,s-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(s)}}{1+\theta_{(s),0}\bm{z}_{i,(s)}^{\prime}\bm{H}_{i,s-1}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{z}_{i,(s)}}\bigg{)}\bigg{\}}\\ &~{}+\frac{1}{v^{2}}\bigg{(}\bm{b}(\gamma_{0})^{\prime}\bm{Z}(\gamma_{0})^{\prime}\{\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))\\ &~{}-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger}))\}\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})\bigg{)}+O_{p}(m)\xrightarrow{p}\infty,\end{split}

(C.10)

as both $N\rightarrow\infty$ and $\theta_{(k)}\rightarrow 0$ for some $k\in\{q-q_{0}+1,\dots,q\}$ . Before proving (C.9) and (C.10), we prove the following equations, for $\bm{h}_{i,k}$ being defined in (5) and $k=q-q_{0}+1,\dots,q$ :

	$\displaystyle\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{h}_{i,(k)}\bm{h}_{i,(k)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}=$	$\displaystyle~{}O_{p}(1),$		(C.11)
	$\displaystyle\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{h}_{i,(k)}\bm{h}_{i,(k)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}=$	$\displaystyle~{}o_{p}(1),$		(C.12)

and

	$\displaystyle\begin{split}\bm{\epsilon}^{\prime}&\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}\\ &~{}\times\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{h}_{i,(k)}\bm{h}_{i,(k)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}=o_{p}(1),\end{split}$			(C.13)
	$\displaystyle\begin{split}\bm{\epsilon}^{\prime}&\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha))^{-1}\\ &~{}\times\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{h}_{i,(k)}\bm{h}_{i,(k)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)\\ &~{}\times(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{\epsilon}=o_{p}(1)\end{split}$			(C.14)

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ . It suffices to prove (C.11)–(C.14) for $k=q$ . For (C.11) with $k=q$ , we have

	$\displaystyle\bm{\epsilon}^{\prime}$	$\displaystyle\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{h}_{i,(q)}\bm{h}_{i,(q)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\{\bm{\epsilon}_{i}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{z}_{i,(q)}\}\{\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}\}$
	$\displaystyle=$	$\displaystyle~{}\{\bm{\epsilon}_{i}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{z}_{i,(q)}\}\bigg{(}\frac{\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}\{O_{p}(n_{i}^{-\ell/2})\}_{1\times 1}\{O_{p}(n_{i}^{\ell/2})\}_{1\times 1}$
	$\displaystyle=$	$\displaystyle~{}O_{p}(1)$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ , where the second last equality follows from Lemma 3 (i) and Lemma 4 (i)–(ii). For (C.12) with $k=q$ , we have

	$\displaystyle\bm{\epsilon}^{\prime}$	$\displaystyle\bm{H}^{-1}(\gamma,\bm{\theta})\bm{h}_{i,(q)}\bm{h}_{i,(q)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\{\bm{\epsilon}_{i}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}\}\bm{h}_{i,(q)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\bigg{(}\frac{\bm{\epsilon}_{i}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}\bigg{)}\bigg{(}\frac{\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}_{i}(\alpha)}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}$
		$\displaystyle~{}\times\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}\{O_{p}(n_{i}^{\ell/2})\}_{1\times 1}\{o(n_{i}^{-\ell/2-\tau})\}_{1\times p(\alpha)}\{\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\}$
		$\displaystyle~{}\times\{O_{p}(1)\}_{p(\alpha)\times 1}$
	$\displaystyle=$	$\displaystyle~{}o_{p}(1)$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ , where the second equality follows from (9) and (A.5) and the third equality follows from (A.7), Lemma 2 (iii), and Lemma 4 (ii)–(iii). For (C.13) with $k=q$ , we have

	$\displaystyle\bm{\epsilon}^{\prime}$	$\displaystyle\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}$
		$\displaystyle~{}\times\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{h}_{i,(q)}\bm{h}_{i,(q)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\bigg{(}\frac{\sum_{i=1}^{m}\bm{\epsilon}_{i}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}_{i}(\alpha)}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}$
		$\displaystyle~{}\times\bigg{(}\frac{\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{z}_{i,(q)}}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}\bigg{(}\frac{\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}\{O_{p}(1)\}_{1\times p(\alpha)}\{\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\}\{o(n_{i}^{-\ell/2-\tau})\}_{p(\alpha)\times 1}$
		$\displaystyle~{}\times\{O_{p}(n_{i}^{\ell/2})\}_{1\times 1}$
	$\displaystyle=$	$\displaystyle~{}o_{p}(1)$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ , where the second equality follows from (A.7), Lemma 2 (iii), and Lemma 4 (ii)–(iii). For (C.14) with $k=q$ ,

	$\displaystyle\bm{\epsilon}^{\prime}$	$\displaystyle\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{h}_{i,(q)}$
		$\displaystyle~{}\times\bm{h}_{i,(q)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\bigg{(}\frac{\sum_{i=1}^{m}\bm{\epsilon}_{i}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}_{i}(\alpha)}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}$
		$\displaystyle~{}\times\bigg{(}\frac{\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}(1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)})}\bigg{)}\bigg{(}\frac{\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}_{i}(\alpha)}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}$
		$\displaystyle~{}\times\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{\epsilon}_{i}}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}\{O_{p}(1)\}_{1\times p(\alpha)}\{\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\}\{o(n_{i}^{\ell/2-\tau})\}_{p(\alpha)\times 1}$
		$\displaystyle~{}\times\{o(n_{\min}^{-\tau})\}_{1\times p(\alpha)}\{\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\}\{O_{p}(1)\}_{p(\alpha)\times 1}$
	$\displaystyle=$	$\displaystyle~{}o_{p}(1)$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ , where the second equality follows from (A.7), Lemma 2 (ii)–(iii), and Lemma 4 (iii). This completes the proofs of (C.11)–(C.14). We now prove (C.9). Note that

\displaystyle\begin{split}\bm{\epsilon}^{\prime}&\{\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger})-\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\}\bm{\epsilon}\\ =&~{}\bm{\epsilon}^{\prime}\{\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger})-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta})\\ &~{}+\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta})-\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\}\bm{\epsilon}\\ =&~{}\bm{\epsilon}^{\prime}\{\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger})-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta})\}\bm{\epsilon}+o_{p}(m)\\ =&~{}\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\{\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger})\\ &~{}-\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\\ &~{}+\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\\ &~{}-\bm{M}(\alpha,\gamma;\bm{\theta})\}\bm{\epsilon}+o_{p}(m)\\ =&~{}\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\{\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger})\\ &~{}-\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\}\bm{\epsilon}+o_{p}(m)\\ =&~{}o_{p}(m)\end{split}

(C.15)

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ , where the second equality follows from (C.12) that

	$\displaystyle\bm{\epsilon}^{\prime}$	$\displaystyle\{\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})-\bm{H}^{-1}(\gamma,\bm{\theta})\}\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\{\bm{H}(\gamma,\bm{\theta})-\bm{H}(\gamma,\bm{\theta}_{0}^{\dagger})\}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\sum_{i=1}^{m}\sum_{k=q-q_{0}+1}^{q}(\theta_{(k)}-\theta_{(k),0})\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{h}_{i,(k)}\bm{h}_{i,(k)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}o_{p}(m)$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ , the second last equality follows from (C.13) that

	$\displaystyle\bm{\epsilon}^{\prime}$	$\displaystyle\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\{\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})$
		$\displaystyle~{}-\bm{M}(\alpha,\gamma;\bm{\theta})\}\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}$
		$\displaystyle~{}\times\{\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})-\bm{H}^{-1}(\gamma,\bm{\theta})\}\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\sum_{i=1}^{m}\sum_{k=q-q_{0}+1}^{q}(\theta_{(k)}-\theta_{(k),0})\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha))^{-1}$
		$\displaystyle~{}\times\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{h}_{i,(k)}\bm{h}_{i,(k)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)$
	$\displaystyle=$	$\displaystyle~{}o_{p}(m)$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ , and the last equality follows from (C.14) that

	$\displaystyle\bm{\epsilon}^{\prime}$	$\displaystyle~{}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\{\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger})$
		$\displaystyle~{}-\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\}\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)\{(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha))^{-1}$
		$\displaystyle~{}-(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}\}\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}\{\bm{H}^{-1}(\gamma,\bm{\theta})$
		$\displaystyle~{}-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\}\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\sum_{i=1}^{m}\sum_{k=q-q_{0}+1}^{q}(\theta_{(k),0}-\theta_{(k)})\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha))^{-1}$
		$\displaystyle~{}\times\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{h}_{i,(k)}\bm{h}_{i,(k)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{X}(\alpha)(\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{X}(\alpha))^{-1}$
		$\displaystyle~{}\times\bm{X}(\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}o_{p}(m)$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ . Also, by (C.11),

	$\displaystyle\bm{\epsilon}^{\prime}$	$\displaystyle\{\bm{H}^{-1}(\gamma,\bm{\theta})-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\}\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\{\bm{H}(\gamma,\bm{\theta}_{0}^{\dagger})-\bm{H}(\gamma,\bm{\theta})\}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\sum_{i=1}^{m}\sum_{k=q-q_{0}+1}^{q}\{\theta_{(k),0}-\theta_{(k)}\}\bm{\epsilon}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{h}_{i,(k)}\bm{h}_{i,(k)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}O_{p}(m)$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ . This together with (C.15) gives (C.9). We now prove (C.10). As with the proof of (C.15), we have

	$\displaystyle\bm{b}(\gamma_{0})^{\prime}\bm{Z}(\gamma_{0})^{\prime}$	$\displaystyle\{\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger})$
		$\displaystyle~{}-\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\}\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})=o_{p}(m)$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ . Hence

	$\displaystyle\bm{b}(\gamma_{0})^{\prime}$	$\displaystyle\bm{Z}(\gamma_{0})^{\prime}\{\bm{H}^{-1}(\gamma,\bm{\theta})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}))$
		$\displaystyle~{}-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})(\bm{I}_{N}-\bm{M}(\alpha,\gamma;\bm{\theta}_{0}^{\dagger}))\}\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})$
	$\displaystyle=$	$\displaystyle~{}\bm{b}(\gamma_{0})^{\prime}\bm{Z}(\gamma_{0})^{\prime}\{\bm{H}^{-1}(\gamma,\bm{\theta})-\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\}\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+o_{p}(m)$
	$\displaystyle=$	$\displaystyle~{}\sum_{i=1}^{m}\sum_{s=q-q_{0}+1}^{q}(\theta_{(s),0}-\theta_{(s)})\bm{b}(\gamma_{0})^{\prime}\bm{Z}(\gamma_{0})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{h}_{i,(s)}$
		$\displaystyle~{}\times\bm{h}_{i,(s)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})+o_{p}(m)$

uniformly over $\bm{\theta}\in[0,\infty)^{q}$ . Hence, for (C.10) to hold, it suffices to prove that for $k=q-q_{0}+1,\dots,q$ and $i=1,\dots,m$ ,

\displaystyle\begin{split}\log\bigg{(}&~{}\frac{1+\theta_{(k)}\bm{z}_{i,(k)}^{\prime}\bm{H}_{i,k-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(k)}}{1+\theta_{(k),0}\bm{z}_{i,(k)}^{\prime}\bm{H}_{i,k-1}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{z}_{i,(k)}}\bigg{)}\\ =&~{}o_{p}\bigg{(}\bm{b}(\gamma_{0})^{\prime}\bm{Z}(\gamma_{0})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{h}_{i,(k)}\bm{h}_{i,(k)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})\bigg{)},\end{split}

(C.16)

as both $N\rightarrow\infty$ and $\theta_{(k)}\rightarrow 0$ for some $k\in\{q-q_{0}+1,\ldots,q\}$ . It suffices to prove (C.16) for $k=q$ . By Lemma 3 (ii)–(iii), we have

	$\displaystyle\bm{b}(\gamma_{0})^{\prime}$	$\displaystyle\bm{Z}(\gamma_{0})^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{h}_{i,(q)}\bm{h}_{i,(q)}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{Z}(\gamma_{0})\bm{b}(\gamma_{0})$
	$\displaystyle=$	$\displaystyle~{}\bigg{(}\frac{b_{i,(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}{\{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}\}}\bigg{)}\{\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{Z}_{i}(\gamma_{0})\bm{b}_{i}(\gamma_{0})\}$
	$\displaystyle=$	$\displaystyle~{}\bigg{(}\frac{b_{i,(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}\bigg{)}\bigg{(}\frac{b_{i,(q)}}{\theta_{(q),0}}+o_{p}(n_{i}^{-\ell-\tau})\bigg{)}.$

Hence, for (C.16) with $k=q$ to hold, it suffices to prove that

	$\displaystyle\log\bigg{(}$	$\displaystyle\frac{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}{1+\theta_{(q),0}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta}_{0}^{\dagger})\bm{z}_{i,(q)}}\bigg{)}\bigg{(}\frac{\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}{1+\theta_{(q)}\bm{z}_{i,(q)}^{\prime}\bm{H}_{i,q-1}^{-1}(\gamma,\bm{\theta})\bm{z}_{i,(q)}}\bigg{)}^{-1}$
		$\displaystyle~{}\rightarrow 0,$

as both $N\rightarrow\infty$ and $\theta_{(q)}\rightarrow 0$ , which follows from Lemma 3 (i) and L’Hospital’s rule. This completes the proof of (C.16). This completes the proof.

C.5 Proof of Lemma 6

We first prove Lemma 6 (i). For $i,i^{*}=1,\ldots,m$ , $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ and $k,k^{*}\in\gamma$ , we have

\displaystyle\begin{split}\theta_{k}\theta_{k^{*}}&\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{h}_{i^{*},k^{*}}\\ =&~{}\big{(}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)\big{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\\ &~{}\times\bigg{(}\frac{\theta_{k^{*}}\bm{X}_{i^{*}}(\alpha)^{\prime}\bm{H}_{i^{*}}^{-1}(\gamma,\bm{\theta})\bm{z}_{i^{*},k^{*}}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\\ =&~{}\{o(n_{i}^{(\xi-\ell)/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\\ &~{}\times\bigg{\{}o\bigg{(}\frac{n_{i^{*}}^{(\xi-\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{\}}_{p(\alpha)\times 1}\\ =&o\Bigg{(}\frac{n_{i}^{(\xi-\ell)/2}n_{i^{*}}^{(\xi-\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the second equality follows from (A.7) and Lemma 2 (iii). Similarly, by (A.7) and Lemma 2 (iii), we have

\displaystyle\begin{split}\theta_{k}\bm{h}_{i,k}^{\prime}&\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{h}_{i^{*},k^{*}}\\ =&~{}\big{(}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)\big{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\\ &~{}\times\bigg{(}\frac{\bm{X}_{i^{*}}(\alpha)^{\prime}\bm{H}_{i^{*}}^{-1}(\gamma,\bm{\theta})\bm{z}_{i^{*},k^{*}}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\\ =&~{}\{o(n_{i}^{(\xi-\ell)/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\\ &~{}\times\Bigg{\{}o\Bigg{(}\frac{n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}\Bigg{\}}_{p(\alpha)\times 1}\\ =&~{}o\Bigg{(}\frac{n_{i}^{(\xi-\ell)/2}n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . Further, by (A.7) and Lemma 2 (iii), we have

\displaystyle\begin{split}\bm{h}_{i,k}^{\prime}&\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{h}_{i^{*},k^{*}}\\ =&~{}\big{(}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)\big{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\\ &~{}\times\bigg{(}\frac{\bm{X}_{i^{*}}(\alpha)^{\prime}\bm{H}_{i^{*}}^{-1}(\gamma,\bm{\theta})\bm{z}_{i^{*},k^{*}}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\\ =&~{}\{o(n_{i}^{(\xi+\ell)/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\\ &~{}\times\Bigg{\{}o\Bigg{(}\frac{n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}\Bigg{\}}_{p(\alpha)\times 1}\\ =&~{}o\Bigg{(}\frac{n_{i}^{(\xi+\ell)/2}n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This completes the proof of Lemma 6 (i).

We now prove Lemma 6 (ii). For $i,i^{*}=1,\ldots,m$ , $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ , $k\in\gamma$ and $k^{*}\notin\gamma$ ,

	$\displaystyle\theta_{k}$	$\displaystyle\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{h}_{i^{},k^{}}$
	$\displaystyle=$	$\displaystyle~{}\big{(}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)\big{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}$
		$\displaystyle~{}\times\bigg{(}\frac{\bm{X}_{i^{}}(\alpha)^{\prime}\bm{H}_{i^{}}^{-1}(\gamma,\bm{\theta})\bm{z}_{i^{},k^{}}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}\{o(n_{i}^{(\xi-\ell)/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}$
		$\displaystyle~{}\times\bigg{\{}o\bigg{(}\frac{n_{i}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{\}}_{1\times p(\alpha)}$
	$\displaystyle=$	$\displaystyle~{}o\Bigg{(}\frac{n_{i}^{(\xi-\ell)/2}n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the second equality follows from Lemma 2 (ii)–(iii) and (A.7). Similarly, by (A.7) and Lemma 2 (ii)–(iii), we have

	$\displaystyle\bm{h}_{i,k}^{\prime}$	$\displaystyle\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{h}_{i^{},k^{}}$
	$\displaystyle=$	$\displaystyle~{}\big{(}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)\big{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}$
		$\displaystyle~{}\times\bigg{(}\frac{\bm{X}_{i^{}}(\alpha)^{\prime}\bm{H}_{i^{}}^{-1}(\gamma,\bm{\theta})\bm{z}_{i^{},k^{}}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}\{o(n_{i}^{(\xi+\ell)/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}$
		$\displaystyle~{}\times\Bigg{\{}o\Bigg{(}\frac{n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}\Bigg{\}}_{p(\alpha)\times 1}$
	$\displaystyle=$	$\displaystyle~{}o\Bigg{(}\frac{n_{i}^{(\xi+\ell)/2}n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This completes the proof of Lemma 6 (ii).

We now prove Lemma 6 (iii). For $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ and $k\in\gamma$ ,

\displaystyle\begin{split}\theta_{k}&\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}\\ =&~{}\bigg{(}\frac{\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\\ &~{}\times\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}\\ =&~{}\{o(n_{i}^{-\ell/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\{O_{p}(1)\}_{p(\alpha)\times 1}\\ =&~{}o_{p}(n_{i}^{-\ell/2})\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the second equality follows from (A.7), Lemma 2 (iii), and Lemma 4 (iii). Similarly, by (A.7), Lemma 2 (iii), and Lemma 4 (iii), we have

\displaystyle\begin{split}\bm{h}_{i,k}^{\prime}&\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}\\ =&~{}\bigg{(}\frac{\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\\ &~{}\times\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}\\ =&~{}\{o(n_{i}^{\ell/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\{O_{p}(1)\}_{p(\alpha)\times 1}\\ =&~{}o_{p}(n_{i}^{\ell/2})\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This completes the proof of Lemma 6 (iii).

We now prove Lemma 6 (iv). For $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times\mathcal{G}$ , $k\in\gamma$ ,

\displaystyle\begin{split}\theta_{k}&\bm{h}_{i,k}^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}(\alpha_{0}\setminus\alpha)\\ =&~{}\big{(}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)\big{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\\ &~{}\times\bigg{(}\frac{\sum_{i=1}^{m}\sum_{j\in\gamma_{0}\setminus\alpha}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j}\beta_{j,0}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\\ =&~{}\{o(n_{i}^{(\xi-\ell)/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\bigg{\{}o\bigg{(}\frac{\sum_{i=1}^{m}n_{i}^{\xi-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\bigg{\}}_{p(\alpha)\times 1}\\ =&~{}\{o(n_{i}^{(\xi-\ell)/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times 1}\\ =&~{}o(n_{i}^{(\xi-\ell)/2-\tau})\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the second equality follows from (A.7), Lemma 2 (i), and Lemma 2 (iii). Similarly, by (A.7) and Lemma 2 (i) and (iii), we have

\displaystyle\begin{split}\bm{h}_{i,k}^{\prime}&\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}(\alpha_{0}\setminus\alpha)\\ =&~{}\big{(}\theta_{k}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)\big{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\\ &~{}\times\bigg{(}\frac{\sum_{i=1}^{m}\sum_{j\in\alpha_{0}\setminus\alpha}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j}\beta_{j,0}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\\ =&~{}\{o(n_{i}^{(\xi+\ell)/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times 1}\\ =&~{}o_{p}(n_{i}^{(\xi+\ell)/2-\tau})\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ . This completes the proof of Lemma 6 (iv).

We now prove Lemma 6 (v). For $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ , we have

\displaystyle\begin{split}\bm{\epsilon}^{\prime}&\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}\\ =&~{}\bigg{(}\frac{\sum_{i=1}^{m}\bm{\epsilon}_{i}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\\ &~{}\times\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}\\ =&~{}\{O_{p}(1)\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\{O_{p}(1)\}_{p(\alpha)\times 1}\\ =&~{}O_{p}(p(\alpha))\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the second equality follows from (A.7) and Lemma 4 (iii). This completes the proof of Lemma 6 (v).

We now prove Lemma 6 (vi). For $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ and $k\notin\gamma$ , we have

	$\displaystyle\bm{h}_{i,k}^{\prime}$	$\displaystyle\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{\epsilon}$
	$\displaystyle=$	$\displaystyle~{}\bigg{(}\frac{\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}$
		$\displaystyle~{}\times\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{\epsilon}_{i}}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}\{o(n_{i}^{\ell/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\{O_{p}(1)\}_{p(\alpha)\times 1}$
	$\displaystyle=$	$\displaystyle~{}o_{p}(n_{i}^{\ell/2})$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the second equality follows from (A.7), Lemma 2 (ii), and Lemma 4 (iii). This completes the proof of Lemma 6 (vi).

We now prove Lemma 6 (vii). For $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times\mathcal{G}$ , we have

\displaystyle\begin{split}\bm{\epsilon}^{\prime}&\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}(\alpha_{0}\setminus\alpha)\\ =&~{}\bigg{(}\frac{\sum_{i=1}^{m}\bm{\epsilon}_{i}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\\ &~{}\times\bigg{(}\frac{\sum_{i=1}^{m}\sum_{j\in\alpha_{0}\setminus\alpha}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j}\beta_{j,0}}{(\sum_{i=1}^{m}n_{i}^{\xi})^{1/2}}\bigg{)}\\ =&~{}\{O_{p}(1)\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\\ &~{}\times\bigg{\{}o\bigg{(}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}^{1/2}n_{\min}^{-\tau}\bigg{)}\bigg{\}}_{p(\alpha)\times 1}\\ =&~{}o_{p}\bigg{(}\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi}\bigg{)}^{1/2}\bigg{)}\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the second equality follows from (A.7), Lemma 2 (i), and Lemma 4 (iii). This completes the proof of Lemma 6 (vii).

We now prove Lemma 6 (viii). For $i,i^{*}=1,\ldots,m$ , $(\alpha,\gamma)\in\mathcal{A}\times\mathcal{G}$ and $k,k^{*}\notin\gamma$ , we have

	$\displaystyle\bm{h}_{i,k}^{\prime}$	$\displaystyle\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{h}_{i^{},k^{}}$
	$\displaystyle=$	$\displaystyle~{}\big{(}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)\big{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}$
		$\displaystyle~{}\times\bigg{(}\frac{\bm{X}_{i^{}}(\alpha)^{\prime}\bm{H}_{i^{}}^{-1}(\gamma,\bm{\theta})\bm{z}_{i^{},k^{}}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}$
	$\displaystyle=$	$\displaystyle~{}\{o(n_{i}^{(\xi+\ell)/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}$
		$\displaystyle~{}\times\Bigg{\{}o_{p}\Bigg{(}\frac{n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}\Bigg{\}}_{p(\alpha)\times 1}$
	$\displaystyle=$	$\displaystyle~{}`o_{p}\Bigg{(}\frac{n_{i}^{(\xi+\ell)/2}n_{i^{*}}^{(\xi+\ell)/2-\tau}}{\sum_{i=1}^{m}n_{i}^{\xi}}\Bigg{)}$

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the second equality follows from (A.7) and Lemma 2 (ii). This completes the proof of Lemma 6 (viii).

We now prove Lemma 6 (ix). For $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times\mathcal{G}$ , $k\notin\gamma$ , we have

\displaystyle\begin{split}\bm{h}_{i,k}^{\prime}\bm{H}^{-1}&(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}(\alpha_{0}\setminus\alpha)\\ =&~{}\big{(}\bm{z}_{i,k}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)\big{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\\ &~{}\times\bigg{(}\frac{\sum_{i=1}^{m}\sum_{j\in\alpha_{0}\setminus\alpha}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{x}_{i,j}\beta_{j,0}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\\ =&~{}\{o(n_{i}^{(\xi+\ell)/2-\tau})\}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\\ &~{}\times\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times 1}\\ =&~{}o(n_{i}^{(\xi+\ell)/2-\tau})\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the second equality follows from (A.7) and Lemma 2 (i)–(ii). This completes the proof of Lemma 6 (ix).

We finally prove Lemma 6 (x). For $(\alpha,\gamma)\in(\mathcal{A}\setminus\mathcal{A}_{0})\times\mathcal{G}$ , we have

\displaystyle\begin{split}\bm{\beta}&(\alpha_{0}\setminus\alpha)^{\prime}\bm{X}(\alpha_{0}\setminus\alpha)^{\prime}\bm{H}^{-1}(\gamma,\bm{\theta})\bm{M}(\alpha,\gamma;\bm{\theta})\bm{X}(\alpha_{0}\setminus\alpha)\bm{\beta}(\alpha_{0}\setminus\alpha)\\ =&~{}\bigg{(}\sum_{i=1}^{m}\sum_{j\in\alpha_{0}\setminus\alpha}\beta_{j,0}\bm{x}_{i,j}^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)\bigg{)}\bigg{(}\frac{\sum_{i=1}^{m}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}^{-1}\\ &~{}\times\bigg{(}\frac{\sum_{i=1}^{m}\sum_{j\in\alpha_{0}\setminus\alpha}\bm{X}_{i}(\alpha)^{\prime}\bm{H}_{i}^{-1}(\gamma,\bm{\theta})\bm{X}_{i}(\alpha)\bm{x}_{i,j}}{\sum_{i=1}^{m}n_{i}^{\xi}}\bigg{)}\\ =&~{}\bigg{\{}o\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi-\tau}\bigg{)}\bigg{\}}_{1\times p(\alpha)}\bigg{\{}\bm{T}^{-1}(\alpha)+\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times p(\alpha)}\bigg{\}}\{o(n_{\min}^{-\tau})\}_{p(\alpha)\times 1}\\ =&~{}o\bigg{(}\sum_{i=1}^{m}n_{i}^{\xi-\tau}\bigg{)}\end{split}

uniformly over $\bm{\theta}\in\Theta_{\gamma}$ , where the second equality follows from (A.7) and Lemma 2 (i). This completes the proof.