The Impossibility of Testing for Dependence Using Kendall’s $\tau$ Under Missing Data of Unknown Form

Testing for statistical dependence between two random variables is an important facet of theoretical and empirical statistical research, and arises as a problem of interest in various areas of the natural and social sciences. Applications in social science include the study of the relationship between health outcomes and insurance levels (e.g., Cameron and Trivedi, 1993), survey analysis (e.g., Yu et al., 2016), stress-testing risk-management models (e.g., Asimit et al., 2016), and stock market co-movements (e.g., Horváth and Rice, 2015; Cameron and Trivedi, 1993). In the natural sciences, applications arise in contexts as diverse as cancerous somatic alteration co-occurrences (e.g., Canisius et al., 2016) and the movement of animals across time (e.g., Swihart and Slade, 1985).

Tests for dependence based on Kendall’s $\tau$ (Kendall, 1938) constitute a relevant tool in empirical practice to detect monotonic dependence between two random variables. The interested reader may refer, for instance, to the monographs Nelsen (2006), Bagdonavicius et al. (2011), and the references therein. The strength of such testing procedures is that $\tau$ is a distribution-free measure of association between paired continuous random variables. In particular, let $(X,Y)$ be a pair of continuous random variables having joint distribution $H$ and marginal distributions $F$ and $G$, respectively. In moment form, this measure of association for the random vector $(X,Y)$ is defined as

where $C$ is the copula of $H$, and $E_{H}$ denotes the expectation operator with respect to the distribution $H$. The hypothesis testing problem for detecting monotonic dependence using $\tau$ in (1.1) has the form

The null hypothesis in (1.2) posits no monotonic dependence between the two random variables, and the alternative hypothesis is the negation of the null.

Statistical procedures for the hypothesis test problem (1.2) are predicated on the assumption that the random vector $(X,Y)$ is observable. However, this assumption is violated in empirical practice because datasets can have missing values. For example, missing data can arise in the form of nonresponse, as in self-reported cross-sectional and longitudinal surveys, which is inevitable, or at follow-up in clinical studies. See, for example, Dutz et al. (2021) for a discussion on the prevalence of nonresponse in economics research. Missing data are also universal in ecological and evolutionary data, as in other branches of science; see, for example, the monograph Fox et al. (2015) and the references therein. Imputation methods are commonly used to address the missing data problem and enable testing with a complete dataset. However, the validity of such tests hinges on the correct specification of the imputation procedure, which can lead to biased inferences if misspecified. Another approach in the literature that addresses the missing data problem imposes assumptions on the missingness-generating process (MGP) that point-identify $\tau$ in (1.1). In the context of Kendall’s rank-correlation test, see, for example, Alvo and Cabilio (1995) who assume the MGP is either missing completely at random or weakly exogenous, and Ma (2012) who assumes that it is either missing at random or missing completely at random. While practical, such tests also ignore misspecification of the MGP, which weakens the credibility of any derived inferences (Manski, 2003).

Consequently, we ask if it is possible to conduct non-parametric inference for dependence using $\tau$ under missing data of unknown form. The results of this paper imply that such robust inference is impossible. Reasoning from first principles, any sensible testing procedure of this sort must be based on $\tau$’s identified set because it characterizes the information about this parameter contained in the observables. The identified set for this parameter is an interval $[\underaccent{\bar}{\tau},\bar{\tau}]\subseteq[-1,1]$ whose bounds depend on observables. Therefore, the testing problem for inferring statistical dependence using this information must have the form

Under $H_{1}$ in (1.3), the identified set is a subset of either $[-1,0)$ or $(0,1]$. Since $\underaccent{\bar}{\tau}\leq\tau\leq\bar{\tau}$ holds by definition, this hypothesis implies that either $\tau<0$ or $\tau>0$ holds, so that $X$ and $Y$ are statistically dependent. We show the bounds satisfy the inequalities $\underaccent{\bar}{\tau}\leq 0\leq\bar{\tau}$, for all joint distributions of $(X,Y)$ and MGPs. These inequalities show the null hypothesis in (1.3) always holds, implying that one cannot partition the underlying probability model into two submodels that are compatible with the assertions of the null and alternative hypotheses. Therefore, the worst-case bounds are useless in detecting dependence between $X$ and $Y$ through the testing problem (1.3). We prove that this property of the bounds holds in the setup where the marginal distributions are known to the practitioner, which implies that it holds in the setup where those distributions are unknown. The reason is that the bounds in the case where the marginal distributions are unknown must be less informative, and hence, weakly wider than their counterparts under known marginal distributions, implying that they must also satisfy this property. A critical step in our theoretical derivations is an innovative use of results on extremal dependence described in Puccetti and Wang (2015).

This paper contributes to the literature on impossible inference, which has a rich history that started with the classic paper of Bahadur and Savage (1956). The recent paper by Bertanha and Moreira (2020) connects this literature and presents a taxonomy of the types of impossible inferences. Our result falls under Type A in their taxonomy, as the alternative is indistinguishable from the null. However, it should be noted that our result is not a consequence of the model of the null hypotheses being dense in the set of all likely models with respect to the total-variation distance, which is the essential characteristic of Type A impossible inferences. Rather, it flows from the fact that the bounds $\underaccent{\bar}{\tau}$ and $\bar{\tau}$ are uninformative because they do not define a partition of the underlying probability model into two submodels that are compatible with the assertions of $H_{0}$ and $H_{1}$ in (1.3).

The idea of using bounds to account for missing data problems started with the seminal paper of Manski (1989) and gained popularity with the important paper of Horowitz and Manski (1995). Since then, there has been a growing influential literature on partial identification that has shaped empirical practice; see, for example, Canay and Shaikh (2016) for a recent survey of this literature and the references therein. Inference on bounds to account for missing data in moment inequality models have been considered in a variety of settings, such as distributional analyses (e.g., Blundell et al., 2007), treatment effect (e.g., Lee, 2009), and stochastic dominance testing (e.g., Fakih et al., 2021). In contrast to those works, this paper shows that such an approach is futile in testing for dependence under missing data of unknown form using Kendall’s $\tau$ and its worst-case bounds. We also discuss how to obtain informative partitions of the underlying probability model through restrictions on the dependence between $X$ and $Y$ and/or the MGPs.

There is also a strand in the partial identification literature focusing on parameters that depend on the joint distribution of two random variables with point-identified marginal distributions; see, for example, Fan and Patton (2014) for a survey of this strand and the references therein. However, to the best of our knowledge, this literature strand has not considered partial identification of those parameters arising from the missing data problem. While convenient, the point-identification of the margins can be untenable in applications with missing data and can create challenges in the inference for such parameters — the results of our paper exemplify this point.

The rest of this paper is organized as follows. Section 2 introduces the statistical setup of this paper and preliminary results on extremal dependence that we utilize in the proof of our results. Section 3 presents our results, Section 4 discusses the scope of the results and implications for empirical practice. Section 5 concludes. All proofs are relegated to the Appendix.

Consider the random vector $(X,Y,Z)$ having joint distribution $P$, where $X$ and $Y$ are the random variables of interest, which are continuous, and $Z$ is a categorical variable supported on $\{1,2,3,4\}$ indicating missingness on $X$ and $Y$. In this setup,

where $\ast$ denotes the missing variable. For simplicity, we assume that the marginal cumulative distribution functions (CDFs) of $X$ and $Y$, denoted by $F$ and $G$, respectively, are known by the practitioner. We derive the worst-case bounds on $\tau$ under the following probability model.

In this setup, a MGP is specified through restrictions on the joint distribution of $(X,Y,Z)$. The model $\mathcal{P}_{F,G}$ does not place any restrictions on the dependence between $(X,Y)$ and $Z$ beyond the existence of a density. To account for the missing data problem, we exhibit $\tau$ as a functional of $P\in\mathcal{P}_{F,G}$. For each $P\in\mathcal{P}_{F,G}$, an application of the Law of Total probability shows the corresponding value of $\tau$ has the following representation:

where $C_{P}$ is the copula of the joint CDF $P(X\leq\cdot,Y\leq\cdot)$.¹¹1The existence and uniqueness of $C_{P}$ in our setup is a result due to Sklar (1959). This representation of $\tau$ is useful since it clarifies the situation faced by the practitioner in our setup. In particular, it shows that $\tau$ can be calculated for each $P\in\mathcal{P}_{F,G}$ using the following parameters: the copula $C_{P}$; the conditional CDFs, $P(X\leq\cdot,Y\leq\cdot\mid Z=z)$ for $z=1,2,3,4$; and the marginal probabilities of $Z$, $\left\{P(Z=z)\right\}_{z=1}^{4}$. From sampling, asymptotically the practitioner can recover $\left\{P(Z=z)\right\}_{z=1}^{4}$ and $P(X\leq\cdot,Y\leq\cdot\mid Z=1)$ but not $C_{P}$ and $\left\{P(X\leq\cdot,Y\leq\cdot\mid Z=z)\right\}_{z=2}^{4}$, as the data alone do not contain any information on the latter. Consequently, a MGP can be characterized in terms of a specification of the conditional CDFs $\left\{P(X\leq\cdot,Y\leq\cdot\mid Z=z)\right\}_{z=1}^{4}$.

The above analysis shows $\tau$ is partially identified in the missing data setting when were are agnostic about the MGP. The identified set of $\tau$ in this case is a closed interval subset of $[-1,1]$ whose boundary corresponds to the worst-case bounds on $\tau$. These bounds permit the entire spectrum of MGPs, which is especially useful when the data have a large number of missing values, as there can be a diversity of explanations for it.

The next section describes the worst-case bounds on $\tau$ in (2.2) and raises the statistical issues concerning testing for dependence between $X$ and $Y$ in a manner that is robust to the MGP. In developing our results we make use of the Fréchet-Hoeffding copula bounds and two results on extreme values of means of supermodular functions from Puccetti and Wang (2015). To describe these results, denote by $\mathcal{C}$ the set of all bivariate copulas on the unit square $[0,1]^{2}$. The Fréchet-Hoeffding bounds are $\max\{u+v-1,0\}\leq C(u,v)\leq\min\{u,v\}$ for all $(u,v)\in[0,1]^{2}$, which hold for all $C\in\mathcal{C}$.

A function $s:\mathbb{R}^{2}\to\mathbb{R}$ is called supermodular if for all $x_{1}\leq x_{2}$ and $y_{1}\leq y_{2}$,

important examples of which are copulas. This point is important as the bounds on $\tau$ are characterized in terms of copulas of $(X,Y)$’s joint distribution. The results of Puccetti and Wang (2015) that we utilize are Theorems 2.1 and 3.1 in their paper and we restate them in the following lemma, but in a form that is more suitable for the derivation of our results.

The first result characterizes the worst-case bounds on Kendall’s $\tau$ in the case where the marginal CDFs of $(X,Y)$ are known.

To test for statistical dependence using $\tau$ in a manner that is robust to the form of the MGP, one can only consider tests that depend on observables through $\tau$’s identified set. This means positing the hypothesis testing problem

where $P_{0}$ is the true distribution of $(X,Y,Z)$, $\mathcal{P}^{0}_{F,G}=\{P\in\mathcal{P}_{F,G}:\bar{\tau}(P)\geq 0\;\text{and}\;\underaccent{\bar}{\tau}(P)\leq 0\}$ and $\mathcal{P}^{1}_{F,G}=\{P\in\mathcal{P}_{F,G}:\bar{\tau}(P)<0\;\text{or}\;\underaccent{\bar}{\tau}(P)>0\}$. Notice that $\mathcal{P}^{1}_{F,G}$ is the relative complement of $\mathcal{P}^{0}_{F,G}$ in $\mathcal{P}_{F,G}$; that is, $\mathcal{P}^{1}_{F,G}=\mathcal{P}_{F,G}-\mathcal{P}^{0}_{F,G}$, which implies that either $\tau(P_{0})<0$ or $\tau(P_{0})>0$ holds. The next result implies that $\mathcal{P}^{1}_{F,G}=\emptyset$, meaning the null hypothesis in (3.1) is always true.

The result of Theorem 2 shows the worst-case bounds of $\tau$ are not informative in the sense that they do not simultaneously take negative or positive values when the joint distribution of $(X,Y)$ exhibits negative or positive dependence, respectively. This property creates an impossibility in testing for dependence on the basis of $\tau$ that is robust to missingness of any form, as in (3.1), since it implies that $\mathcal{P}^{1}_{F,G}=\emptyset$.

The results of Theorem 1 and 2 have assumed that the marginal CDFs of $X$ and $Y$ are known to the practitioner. In the scenario where this is not the case, neither of those distributions would be point-identified when we are agnostic about the nature of the MGP. By an application of the Law of Total Probability to $F$ and $G$, we can obtain pointwise bounds on these marginal CDFs as $\underline{F}(x)\leq F(x)\leq\overline{F}(x)$ and $\underline{G}(y)\leq G(y)\leq\overline{G}(y)$ for all $(x,y)\in\mathcal{X}\times\mathcal{Y}$, with the boundaries themselves CDFs, given by

and

Denoting by $\mathcal{F}$ and $\mathcal{G}$ the sets of all CDFs of $X$ and $Y$ that satisfy the respective bounds described above, the probability model is $\mathcal{P}=\bigcup_{F\in\mathcal{F},G\in\mathcal{G}}\mathcal{P}_{F,G}$. Thus, one has bounds on $\tau(P)$ that depend on hypothetical values of the margins $F$ and $G$, and extremizing these bounds with respect to the margins over $\mathcal{F}$ and $\mathcal{G}$ yields the worst-case upper and lower bounds, $\sup_{(F,G)\in\mathcal{F}\times\mathcal{G}}\bar{\tau}(P)$ and $\inf_{(F,G)\in\mathcal{F}\times\mathcal{G}}\underaccent{\bar}{\tau}(P)$, respectively. Therefore, the conclusion of Theorem 2 also holds for these worst-case bounds since they are wider than their counterparts in the scenario where the marginal CDFs of $X$ and $Y$ are known to the practitioner.²²2Scrutinizing the expressions of $\bar{\tau}(P)$ and $\underaccent{\bar}{\tau}(P)$, observe that the worst-case bounds in this larger model can be obtained in closed-form. For the upper bound, replace $F$ and $G$ in $\bar{\tau}(P)$ with $\overline{F}$ and $\overline{G}$, respectively; and for the lower bound, replace $F$ and $G$ in $\underaccent{\bar}{\tau}(P)$ with $\underline{F}$ and $\underline{G}$, respectively.

This section discusses the implications of our results. The model $\mathcal{P}_{F,G}$ is large, which is the reason why the bounds $\bar{\tau}$ and $\underaccent{\bar}{\tau}$ do not yield a partition of $\mathcal{P}_{F,G}$ that is compatible with the hypotheses in (3.1). Note that the model is non-parametric and permits (i) the entire spectrum of MGPs, and (ii) all bivariate absolutely continuous copulas in modelling the statistical dependence between $X$ and $Y$. This point raises the following question: does restricting $\mathcal{P}_{F,G}$ give rise to an identified set for $\tau$ whose bounds are informative in detecting dependence between $X$ and $Y$? The answer is in the affirmative. Restrictions on $\mathcal{P}_{F,G}$ can be motivated by many considerations based on the application at hand. For example, they can be arise from the possession of side-information or by restrictions implied by economic theory as in the partial identification approach in econometrics (e.g., Tamer, 2010). We elaborate on this point with an example of the former utilizing results in Nelsen et al. (2001).

Let $P_{0}\in\mathcal{P}_{F,G}$ denote the true distribution of $(X,Y,Z)$. Suppose we possess side-information that $P_{0}\left(X\leq\tilde{x},Y\leq\tilde{y}\right)=\theta,$ where $\tilde{x}$ and $\tilde{y}$ are the medians of $X$ and $Y$, and $\theta\in[0,1/2]$. Accordingly, we must have $C_{P_{0}}(1/2,1/2)=\theta$, which represents the side-information in terms of the copula. Theorem 1 of Nelsen et al. (2001) provides the bounds on the copula under this restriction, which are given by

for all $(u,v)\in[0,1]^{2}$, where $a^{+}=\max\{a,0\}$. Thus, the bounds on the joint distribution of $(X,Y)$ are

which hold for all $P\in\mathcal{P}_{F,G,\theta}$ where the probability model $\mathcal{P}_{F,G,\theta}=\left\{P\in\mathcal{P}_{F,G}:P\left(X\leq\tilde{x},Y\leq\tilde{y}\right)=\theta\right\}$ accounts for the side-information. We can apply identical steps as in the proof of Theorem 1 to obtain the corresponding bounds on $\tau$ under the model $\mathcal{P}_{F,G,\theta}$, but replacing the Fréchet-Hoeffding lower and upper bounds with $\underaccent{\bar}{C}_{\theta}$ and $\bar{C}_{\theta}$, respectively. For brevity, we omit these details. The bounds on $\tau(P)$ are given by

and satisfy $\left[\underaccent{\bar}{\tau}_{\theta}(P),\bar{\tau}_{\theta}(P)\right]\subset\left[\underaccent{\bar}{\tau}(P),\bar{\tau}(P)\right]$, for $P\in\mathcal{P}_{F,G,\theta}$.

In contrast to the worst-case bounds on $\tau$, the bounds $\bar{\tau}_{\theta}$ and $\underaccent{\bar}{\tau}_{\theta}$ are informative, in the sense that $\exists P_{1},P_{2},P_{3}\in\mathcal{P}_{F,G,\theta}$ such that

We demonstrate this point using a numerical example in which we specify $X$ and $Y$ being uniformly distributed on $[0,1]$ for simplicity. Furthermore, we set $\theta=0.4$ and derive the MGP from a multinomial logit specification for the propensity probabilities; i.e.,

Finally, to complete the specification of $P$ we must designiate the copula of $(X,Y)$, $C_{P}$, as the marginal probabilities of $Z$ can be obtained by integrating the propensity scores with respect to this copula. Then, by Bayes’ Theorem, the MGP is given by the conditional probability density functions

We set $C_{P}$ as the bivariate Gaussian copula, with standard normal margins, and construct $P_{1}$, $P_{2}$, and $P_{3}$ through setting the correlation coefficient $\rho$. As $\bar{\tau}_{\theta}(P_{1}),\underaccent{\bar}{\tau}_{\theta}(P_{2}),\bar{\tau}_{\theta}(P_{3})$, and $\underaccent{\bar}{\tau}_{\theta}(P_{3})$ are linear combinations of moments, we calculated them using Monte Carlo simulations with $10^{8}$ random draws from the corresponding bivariate Gaussian copula.

The parameter specification for $P_{1}$ are as follows: $\gamma_{1,1}=\gamma_{2,1}=2$; $\gamma_{1,2}=-5,\gamma_{2,2}=1/4$;$\gamma_{1,3}=5,\gamma_{2,3}=-1/4$;$\gamma_{1,4}=\gamma_{2,4}=-5$; and $\rho=-0.999$. This yields $\tau(P_{1})<0$ and $\bar{\tau}_{\theta}(P_{1})\approx-0.0108$. The parameter specification for $P_{2}$ are as follows: $\gamma_{1,1}=\gamma_{2,1}=1/2$; $\gamma_{1,2}=3,\gamma_{2,2}=1/2$;$\gamma_{1,3}=1/2,\gamma_{2,3}=-2$;$\gamma_{1,4}=\gamma_{2,4}=2$; and $\rho=0.99$. This specification has $\tau(P_{2})>0$ and gives rise to $\underaccent{\bar}{\tau}_{\theta}(P_{1})\approx 0.034$. Finally, the parameter specification for $P_{3}$ are identical to that under $P_{2}$ except that now $\rho=0$. This specification has $\tau(P_{3})=0$ and gives rise to $\underaccent{\bar}{\tau}_{\theta}(P_{3})\approx-0.32$ and $\bar{\tau}_{\theta}(P_{3})\approx 0.63$.

The numerical results demonstrate the refined bounds can be informative in the detection of dependence. In such a situation, the practitioner can consider the following testing problem

where $\mathcal{P}_{F,G,\theta}^{0}=\{P\in\mathcal{P}_{F,G,\theta}:\bar{\tau}_{\theta}(P)\geq 0\,\text{and}\,\underaccent{\bar}{\tau}_{\theta}(P)\leq 0\}$ and $\mathcal{P}_{F,G,\theta}^{1}=\{P\in\mathcal{P}_{F,G,\theta}:\bar{\tau}_{\theta}(P)<0\,\text{or}\,\underaccent{\bar}{\tau}_{\theta}(P)>0\}$ form a partition of $\mathcal{P}_{F,G,\theta}$. The bounds are linear combinations of moments but with unknown coefficients being the marginal probabilities $P[Z=z]$. As these marginal probabilities are typically estimable at the $\sqrt{n}$-rate, one can adapt moment inequality testing procedures for this situation, which are abundant and well-established (e.g., Andrews and Soares, 2010; Canay, 2010; Romano et al., 2014). Developing the details of such a testing procedure goes beyond the intended scope of the paper, and is left for future research.

As this side-information only restricts the dependence between $X$ and $Y$, any valid testing procedure that rejects $H_{0}$ in favour of $H_{1}$ in (4.3) would be robust to the nature of the MGP. This robustness, however, comes at the expense of an ambiguity under the null. Specifically, $\exists P\in\mathcal{P}_{F,G,\theta}$ such that $\tau(P)\neq 0$ and $\underaccent{\bar}{\tau}_{\theta}(P)\leq 0\leq\bar{\tau}_{\theta}(P)$. The uninformative nature of $H_{0}$ in (4.3) is a consequence of circumventing assumptions on the MGP, which are unverifiable in practice. If one fails to reject the null, then, unfortunately, one cannot conclude anything informative about the dependence between $X$ and $Y$. In such a situation, we recommend empirical researchers perform a sensitivity analysis of this empirical conclusion (i.e., non-rejection of $H_{1}$) with respect to plausible assumptions on the MGP. The virtue of this type of analysis is that it establishes, in a transparent way, clear links between empirical outcomes and different assumptions made on the MGP. Such an analysis would reveal non-trivial links between assumptions on the MGP and inferences made. See, for example,  Blundell et al. (2007) and Lee (2009) who refine worst-case distributional bounds using economic theory and develop testable implications based on them in the contexts of distributional analyses and treatment effect, respectively. See also Fakih et al. (2021) who discuss the refinement of worst-case distributional bounds of ordinal variables in the context of stochastic dominance testing by positing assumptions on the form of nonresponse in self-reported surveys.

This paper establishes the impossibility of performing inference for dependence between two continuous random variables using Kendall’s $\tau$ under missing data of unknown form. The crux of the issue is that its identified set always includes zero, implying that the sign of $\tau$ is not identified. We show how refining this identified set using additional information can address this problem, creating a pathway for robust inference based on statistical procedures from the moment inequality testing literature.

We thank Brendan K. Beare and Christopher D. Walker for helpful feedback and comments.