High-dimensional Asymptotic Theory of Bayesian Multiple Testing Procedures Under General Dependent Setup and Possible Misspecification

The area of multiple hypotheses testing has gained much importance and popularity, particularly in this era of big data, where often very large number of hypotheses need to be tested simultaneously. There are applications abound in the fields of statistical genetics, spatio-temporal statistics, brain imaging, to name a few. On the theoretical side, it is important to establish validity of the multiple testing procedure in the sense that the procedure controls the false discovery rate $(FDR)$ at some pre-specified level or attains oracle, as the number of tests grows to infinity.

Although there is considerable literature addressing these issues, the important factor of dependence among the tests seem to have attained less attention. Indeed, realistically, the test statistics or the parameters can not be expected to be independent. In this regard, Chandra and Bhattacharya (2019) introduced a novel Bayesian multiple testing procedure that coherently accounts for such dependence and yields joint decision rules that are functions of appropriate joint posterior probabilities. As demonstrated in Chandra and Bhattacharya (2019) and Chandra and Bhattacharya (2020), the new Bayesian method significantly outperforms existing popular multiple testing methods by proper utilization of the dependence structures. Since in the new method the decisions are taken jointly, the method is referred to as Bayesian non-marginal multiple testing procedure.

Chandra and Bhattacharya (2020) investigated in detail the asymptotic theory of the non-marginal procedure, and indeed general Bayesian multiple testing methods under additive loss, for fixed number of hypotheses, when the sample size tends to infinity. In particular, they provided sufficient conditions for strong consistency of such procedures and also showed that the asymptotic convergence rates of the versions of $FDR$ and false non-discovery rate ($FNR$) are directly related to the Kullback-Leibler (KL) divergence from the true model. Interestingly, their results continue to hold even under misspecifications, that is, if the class of postulated models does not include the true model. In this work, we investigate the asymptotic properties of the Bayesian non-marginal procedure in particular, and Bayesian multiple testing methods under additive loss in general, when the sample size, as well as the number of hypotheses, tend to infinity.

As mentioned earlier, asymptotic works in multiple testing when the number of hypotheses grows to infinity, are not rare. Xie et al. (2011) have proposed an asymptotic optimal decision rule for short range dependent data with dependent test statistics. Bogdan et al. (2011) studied the oracle properties and Bayes risk of several multiple testing methods under sparsity in Bayesian decision-theoretic setup. Datta and Ghosh (2013) studied oracle properties for horse-shoe prior when the number of tests grows to infinity. However, in the aforementioned works, the test statistics are independent and follow Gaussian distribution. Fan et al. (2012) proposed a method of dealing with correlated test statistics where the covariance structure is known. Their method is based on principal eigen values of the covariance matrix, which they termed as principal factors. Using those principal factors their method dilutes the association between correlated statistics to deal with an arbitrary dependence structure. They also derived an approximate consistent estimator for the false discovery proportion (FDP) in large-scale multiple testing. Fan and Han (2017) extended this work when the dependence structure is unknown. In these approaches, the decision rules are marginal and the test statistics jointly follow multivariate Gaussian distribution. Chandra and Bhattacharya (2019) argue that when the decision rules corresponding to different hypotheses are marginal, the full potential of the dependence structure is not properly exploited. Results of extensive simulation studies reported in Chandra and Bhattacharya (2019) and Chandra and Bhattacharya (2020), demonstrating superior performance of the Bayesian non-marginal method compared to popular marginal methods, even for large number of hypotheses, seem to vindicate this issue. This makes asymptotic analysis of the Bayesian non-marginal method with increasing number of hypotheses all the more important.

To be more specific, we investigate the asymptotic theory of the Bayesian non-marginal procedure in the general dependence setup, without any particular model assumption, when the sample size ($n$) and the number of hypotheses ($m_{n}$, which may be a function of $n$), both tend to infinity. We establish strong consistency of the procedure and show that even in this setup, the convergence rates of versions of $FDR$ and $FNR$ are directly related to the KL-divergence from the true model. We show that our results continue to hold for general Bayesian procedures under the additive loss function. In the Bayesian non-marginal context we illustrate the theory with the time-varying covariate selection problem, where the number of covariates tends to infinity with the sample size $n$. We distinguish between the two setups: ultra high-dimensional case, that is, where $\frac{m_{n}}{n}\rightarrow\infty$ (or some constant), as $n\rightarrow\infty$, and the high-dimensional but not ultra high-dimensional case, that is, $m_{n}\rightarrow\infty$ and $\frac{m_{n}}{n}\rightarrow 0$, as $n\rightarrow\infty$. We particularly focus on the ultra high-dimensional setup because of its much more challenging nature.

Let $\boldsymbol{X}_{n}=\{X_{1},\ldots,X_{n}\}$ denote the available data set. Suppose the data is modelled by the family of distributions $P_{\boldsymbol{X}_{n}|\boldsymbol{\theta}}$ (which may also be non-parametric). For $M>1$, let us denote by $\boldsymbol{\Theta}=\Theta_{1}\times\cdots\times\Theta_{M}$ the relevant parameter space associated with $\boldsymbol{\theta}=(\theta_{1},\ldots,\theta_{M})$, where we allow $M$ to be infinity as well. Let $P_{\boldsymbol{\theta}|\boldsymbol{X}_{n}}(\cdot)$ and $E_{\boldsymbol{\theta}|\boldsymbol{X}_{n}}(\cdot)$ denote the posterior distribution and expectation respectively of $\boldsymbol{\theta}$ given $\boldsymbol{X}_{n}$ and let $P_{\boldsymbol{X}_{n}}(\cdot)$ and $E_{\boldsymbol{X}_{n}}(\cdot)$ denote the marginal distribution and expectation of $\boldsymbol{X}_{n}$ respectively. Let us consider the problem of testing $m$ hypotheses simultaneously corresponding to the actual parameters of interest, where $1<m\leq M$.

Without loss of generality, let us consider testing the parameters associated with $\Theta_{i}$; $i=1,\ldots,m$, formalized as:

where $\Theta_{0i}\bigcap\Theta_{1i}=\emptyset\mbox{ and }\Theta_{0i}\bigcup\Theta_{1i}=\Theta_{i},\mbox{ for $i=1,\cdots,m$}.$

Let

Following Chandra and Bhattacharya (2019) we define $G_{i}$ to be the set of hypotheses, including the $i$-th one, which are highly dependent, and define

If, for any $i\in\{1,\ldots,m\}$, $G_{i}=\{i\}$, a singleton, then we define $z_{i}=1$. Chandra and Bhattacharya (2019) maximize the posterior expectation of the number of true positives

subject to controlling the posterior expectation of the error term

which is actually the posterior mean of the sum of three error terms $E_{1}=\sum_{i=1}^{m}d_{i}(1-r_{i})z_{i}$, $E_{2}=\sum_{i=1}^{m}d_{i}(1-r_{i})(1-z_{i})$ and $E_{3}=\sum_{i=1}^{m}d_{i}r_{i}(1-z_{i})$. For detailed discussion regarding these, see Chandra and Bhattacharya (2019).

It follows that the decision configuration can be obtained by minimizing the function

with respect to all possible decision configurations of the form $\boldsymbol{d}=\{d_{1},\ldots,d_{m}\}$, where $\lambda_{n}>0$, and

is the posterior probability of the decision configuration $\{d_{1},\ldots,d_{i-1},1,d_{i+1},\ldots,d_{m}\}$ being correct. Letting $\beta_{n}=\lambda_{n}/(1+\lambda_{n})$, one can equivalently maximize

with respect to $\boldsymbol{d}$ and obtain the optimal decision configuration.

For detailed discussion regarding the choice of $G_{i}$s in (2.3), see Chandra and Bhattacharya (2019) and Chandra and Bhattacharya (2020). In particular, Chandra and Bhattacharya (2020) show that asymptotically, the Bayesian non-marginal method is robust with respect to $G_{i}$s in the sense that it is consistent with respect to any choice of the grouping structure. As will be shown in this article, the same holds even in the high-dimensional asymptotic setup.

Following Shalizi (2009) we consider a probability space $(\Omega,\mathcal{F},P)$, and a sequence of random variables $X_{1},X_{2},\ldots$, taking values in some measurable space $(\Xi,\mathcal{X})$, whose infinite-dimensional distribution is $P$. The natural filtration of this process is $\sigma(\boldsymbol{X}_{n})$.

We denote the distributions of processes adapted to $\sigma(\boldsymbol{X}_{n})$ by $P_{\boldsymbol{X}_{n}|\boldsymbol{\theta}}$, where $\boldsymbol{\theta}$ is associated with a measurable space $(\boldsymbol{\Theta},\mathcal{T})$, and is generally infinite-dimensional. For the sake of convenience, we assume, as in Shalizi (2009), that $P$ and all the $P_{\boldsymbol{X}_{n}|\boldsymbol{\theta}}$ are dominated by a common reference measure, with respective densities $p$ and $f_{\boldsymbol{\theta}}$. The usual assumptions that $P\in\boldsymbol{\Theta}$ or even $P$ lies in the support of the prior on $\boldsymbol{\Theta}$, are not required for Shalizi’s result, rendering it very general indeed. We put the prior distribution $\pi(\cdot)$ on the parameter space $\boldsymbol{\Theta}$.

In this section we show that the non-marginal procedure is asymptotically consistent under any general dependency model satisfying the conditions in Section 3.1. Since one of our main goals is to allow for misspecification, we must define consistency of multiple testing methods encompassing misspecification, while also allowing for $m_{n}$ hypotheses where $m_{n}/n\rightarrow c$, where $c\geq 0$ or $\infty$. We formalize this below by introducing appropriate notions.

For a fixed number of hypotheses $m$, Chandra and Bhattacharya (2020) investigated convergence of different versions of $FDR$ as the sample size $n$ tends to infinity. They show that show that the convergence rates of the posterior error measures $mFDR_{\boldsymbol{X}_{n}}$ and $FDR_{\boldsymbol{X}_{n}}$ are directly associated with the KL-divergence from the true model. Indeed, they were able to obtain the exact limits of $\frac{1}{n}\log mFDR_{\boldsymbol{X}_{n}}$ and $\frac{1}{n}\log FDR_{\boldsymbol{X}_{n}}$ in terms of the relevant $m$-dimensional KL-divergence rate.

In the current high-dimensional setup, however, such exact KL-divergence rate can not be expected to be available since the number of hypotheses $m_{n}$ is not fixed. As $m_{n}\rightarrow\infty$, it is plausible to expect that the convergence rates depend upon the infinite-dimensional KL-divergence $J$. We show that this is indeed the case, but the exact limit is not available, which is again to be expected, since $m_{n}$ approaches infinity, not equal to infinity. Here, in the high-dimensional setup we obtain $-J$ as an upper bound of the limit supremums. It is easy to observe that the limits in the finite-dimensional setup are bounded above by $-J$, thus providing evidence of internal consistency as we move from fixed-dimensional to high-dimensional setup.

We also show that $mpBFDR$ and $pBFDR$ approach zero, even though the rates of convergence are not available. Recall that even in the fixed-dimensional setup, the convergence rates of $mpBFDR$ and $pBFDR$ were not available. As in the consistency result, these results too do not require any restriction on the growth rate of $m_{n}$, except that required for Shalizi’s conditions to hold.

We present our results below, the proofs of which are presented in the supplement.

The above theorem shows that the convergence rate of $mFDR_{\boldsymbol{X}_{n}}$ and $FDR$ to 0 for arbitrarily large number of hypotheses is at exponential rate, for arbitrary growth rate of $m_{n}$ with respect to $n$. However, again Shalizi’s conditions would require restriction on the growth rate of $m_{n}$.

High-dimensional asymptotic treatments of versions of $FNR$ are similar to those for versions of $FDR$. In particular, limit supremums of both $\frac{1}{n}\log mFNR_{\boldsymbol{X}_{n}}$ and $\frac{1}{n}\log FNR_{\boldsymbol{X}_{n}}$ are bounded above by $-J$, and that both $mpBFNR$ and $pBFNR$ converge to zero. The proofs of these results are also similar to those for the respective $FDR$ versions. Internal consistency of these results is again evident as the limits of $\frac{1}{n}\log mFNR_{\boldsymbol{X}_{n}}$ and $\frac{1}{n}\log FNR_{\boldsymbol{X}_{n}}$ in the finite dimensional setups are bounded above by $-J$ and $mpBFNR$ and $pBFNR$ converge to zero for fixed number of hypotheses. In the latter cases, convergence rates are not available for either fixed or high-dimensional cases. Below we provide the relevant results on versions of $FNR$, with proofs in the supplement.

The above theorem shows that the convergence rate of $mFNR_{\boldsymbol{X}_{n}}$ and $FNR$ to 0 for arbitrarily large number of hypotheses is at exponential rate, for arbitrary growth rate of $m_{n}$ with respect to $n$. However, again Shalizi’s conditions would require restriction on the growth rate of $m_{n}$.

It has been proved in Chandra and Bhattacharya (2019) for the non-marginal multiple testing procedure and additive loss-function based methods, $mpBFDR$ and $pBFDR$ are continuous and non-increasing in $\beta$. Consequently, for suitable values of $\beta$ any $\alpha\in(0,1)$ can be achieved by these errors. For suitably chosen positive values of $\alpha$, one can hope to reduce the corresponding $BFNR$. This is standard practice even in the single hypothesis testing literature, where the Type-I error is controlled at some positive value so that a reduced Type-II error may be incurred. However, as shown in Chandra and Bhattacharya (2020) in the fixed-dimensional setup, for the non-marginal multiple testing procedure and additive loss-function based methods, values of $\alpha$ that are as close to 1 as desired, can not be attained by versions of $FDR$ as the sample size $n$ tends to infinity. This is not surprising, however, since consistent procedures are not expected to incur large errors asymptotically, at least when the number of hypothesis is fixed. Indeed, in the fixed-dimensional setup, Chandra and Bhattacharya (2020) provided an interval of the form $(a,b)$ where $0<a<b<1$, in which maximum values of the versions of $FDR$ can lie asymptotically and obtained asymptotic results for $FNR$ for such $\alpha$-controlled versions of $FDR$.

In this section we investigate the asymptotic theory for $\alpha$-control in the high-dimensional context, that is, when $m_{n}\rightarrow\infty$ as $n\rightarrow\infty$. Although none of our previous high-dimensional results did not require any explicit restrictions on the growth rate of $m_{n}$ given that the posterior convergence result of Shalizi holds, here we need a very mild condition on $m_{n}$ that it grows slower than the exponential rate in $n$. We also need to fix the proportion ($p$) of true alternatives as $m_{n}\rightarrow\infty$, and the proportion ($q$) of groups associated with at least one false null hypothesis. As we show, these two proportions define an interval of the form $(0,b)$, with $b=\frac{1-q}{1+p-q}<1$, in which the maximum of the versions of $FDR$ lie, as $m_{n}\rightarrow\infty$ with $n$. In contrast with the fixed-dimensional asymptotics of Chandra and Bhattacharya (2020), the lower bound of the interval is zero for high dimensions, not strictly positive. To explain, for fixed dimension $m$, the lower bound was $a=\frac{1}{\sum_{i=1}^{m}d^{t}_{i}+1}$. Intuitively, replacing $a$ and $m$ with $a_{m_{n}}$ and $m_{n}$ respectively, dividing both numerator and denominator of $a$ by $m_{n}$, taking the limit, replacing the denominator with $p$, we obtain $a_{m_{n}}\rightarrow 0$, as $n\rightarrow\infty$. Similar intuition can be used to verify that the upper bound $b$ in the fixed dimensional case converges to $\frac{1-q}{1+p-q}$ in the high-dimensional setup. As in our previous results, these provide a verification of internal consistency in the case of transition from fixed-dimensional to high-dimensional situations.

Our results regarding asymptotic $\alpha$ control of versions of $FDR$ and corresponding convergence of versions of $FNR$ are detailed in Sections 7.1 and 7.2.