A Basic Treatment of the Distance Covariance

The distance covariance, a measure of dependence between multivariate random variables $X$ and $Y$, was introduced by Székely, Rizzo, and Bakirov [24] and has since received extensive attention in the statistical literature. A crucial feature of the distance covariance is that it equals zero if and only if $X$ and $Y$ are mutually independent. Hence the distance covariance is sensitive to arbitrary dependencies; this is in contrast to the classical covariance, which is generally capable of detecting only linear dependencies. This property is illustrated in Figure 1, which illustrates that tests based on the distance covariance are able to detect numerous types of non-linear associations even when tests based on the classical covariance may fail to detect many such statistical relationships.

While the dependencies illustrated in Figure 1 clearly represent purely illustrative examples, the sensitivity of the distance covariance to arbitrary dependencies can be very useful for applications. This is demonstrated in Figure 2, where we show three dependencies between expression values genes in the breast cancer data set by Van De Vijver, et al. [26]; all these dependencies can be detected by the distance covariance but not by the classical covariance.

For comparisons of the distance covariance and classical covariance in applications to data, see the examples given by Székely and Rizzo [22, Section 5.2] and Dueck, et al. [3, Section 5]; for extensive numerical experiments and fast algorithms for computing the distance covariance, see Huo and Székely [11, Section 5]. We also refer to Sejdinovic, et al. [19], Dueck, et al. [3], Székely and Rizzo [22, 23], Huo and Székely [11], and Edelmann, et al. [6], representing only a few of the many authors who have given further theoretical results on the distance covariance and distance correlation coefficients; and to Zhou [28], Fiedler [7], and Edelmann, et al. [5] as among the applications to time series analyses. Many applications to data analysis of the distance correlation coefficient and the distance covariance are now available, including: Kong, et al. [12] on data in sociology, Martínez-Gómez, textitet al. [14] and Richards, et al. [18] on astrophysical databases and galaxy clusters, Dueck, et al. [3] on time series analyses of wind vectors at electricity-generating facilities, Richards [17] on the relationship between the strength of gun control laws and firearm-related homicides, Zhang, et al. [27] for remote sensing applications, and Ohana-Levi, et al. [15] on grapevine transpiration.

The original papers of Székely, et al. [24] and Székely and Rizzo [22] are now widely recognized as seminal and important contributions to measuring dependence between sets of random variables; however, the exposition therein includes some ingenious arguments that may make the material challenging to readers not having an advanced background in mathematical statistics. With the benefit of hindsight, we are able to provide in this article a simpler, albeit mathematically rigorous, introduction to the distance covariance that can be taught even in an undergraduate-level course covering the basic theory of U-statistics. Other than standard U-statistics theory and some well-known properties of characteristic functions, the requirements for our treatment are a knowledge of multidimensional integration and trigonometric inequalities, as covered in a course on undergraduate-level advanced calculus. Consequently, we hope that this treatment will prove to be beneficial to non-mathematical statisticians.

Our presentation introduces the distance covariance as an important alternative to the classical covariance. Moreover, the distance covariance constitutes a particularly interesting example of a U-statistic since it includes both the “non-degenerate” and “first-order degenerate” cases of the asymptotic distribution theory of U-statistics, these corresponding to the situations in which $X$ and $Y$ are dependent, leading to the non-degenerate case, or $X$ and $Y$ are independent, leading to the first-order degenerate case of the asymptotic theory.

Throughout the exposition, $\|\cdot\|$ denotes the Euclidean norm and $\langle\cdot,\cdot\rangle$ the corresponding inner product. Also, we denote by $|\cdot|$ the modulus in $\mathbb{C}$ or the absolute value in $\mathbb{R}$, and the imaginary unit is $i=\sqrt{-1}$.

Following Székely, et al. [24], we first establish a closed-form expression for an integral that plays a central work in this article, leading to the equivalence of two crucial expressions for the distance covariance. The first expression displays the distance covariance as an integrated distance between the joint characteristic function of $(X,Y)$ and the product of the marginal characteristic functions of $X$ and $Y$; we will deduce from this expression that the distance covariance equals zero if and only if $X$ and $Y$ are independent. The second expression allows us to derive consistent distance covariance estimators that are expressible as polynomials in the distances between random samples.

Since the ability to characterize independence and the existence of easily computable estimators are arguably the most important properties of the distance covariance, we will refer to this integral as the fundamental integral of distance covariance.

Denote by $I_{p}$ the integral in (2.1). For $p=1$, replacing $t$ by $t/x$ yields

Denoting the latter integral in (2.2) by $c_{1}$, it follows by integration-by-parts that

the last equality being classical in calculus (Spivak [20, Chapter 19, Problem 43]).

For general $p$, note that $I_{p}$ is invariant under orthogonal transformations $H$ of $x$:

where the first equality follows from the transformation $t\mapsto Ht$, which leaves the Lebesgue measure ${{\hskip 1.0pt}\rm{d}}t$ unchanged; and the second equality holds because the norm and the inner product are orthogonally invariant. Therefore, in evaluating $I_{p}$ we may replace $x$ by $\|x\|(1,0,\ldots,0)$; letting $t=(t_{1},\ldots,t_{p})$, we obtain

the last equality obtained by replacing $t_{j}$ by $t_{j}/\|x\|$, $j=1,\ldots,p$.

Denoting by $c_{p}$ the latter integral in (2.4), we substitute in that integral $t_{j}=v_{j}$, $j=1,\ldots,p-1$, and $t_{p}=p^{-1/2}(v_{1}^{2}+\cdots+v_{p-1}^{2})^{1/2}v_{p}$. As the Jacobian of this transformation is $p^{-1/2}(v_{1}^{2}+\cdots+v_{p-1}^{2})^{1/2}$, we obtain

As the remaining integral in (2) is the familiar normalizing constant of the Student’s $t$-distribution on $p$ degrees-of-freedom, we obtain

Starting with $c_{1}=\pi$, we solve this recursive equation for $c_{p}$, obtaining (2.1). ∎

We now introduce the representations of the distance covariance mentioned above. Following Székely, et al. [24], we define the distance covariance through its characteristic function representation. For jointly distributed random vectors $X\in\mathbb{R}^{p}$ and $Y\in\mathbb{R}^{q}$, let $\phi_{X,Y}(s,t)=\mathbb{E}e^{i\langle s,X\rangle+i\langle t,Y\rangle}$ be the joint characteristic function of $(X,Y)$ and $\phi_{X}(s)=\phi_{X,Y}(s,0)$ and $\phi_{Y}(t)=\phi_{X,Y}(0,t)$ be the corresponding marginal characteristic functions.

As the integrand in (3.1) is nonnegative, it follows that ${\mathcal{V}}^{2}(X,Y)\geq 0$. Further, we will show in Corollary 3.4 that ${\mathcal{V}}^{2}(X,Y)<\infty$ whenever $X$ and $Y$ have finite first moments.

An advantage of the representation (3.1) is that it directly implies one of the most important properties of the distance covariance, viz., the characterization of independence.

Conversely, if $X$ and $Y$ are not independent then the functions $\phi_{X,Y}(s,t)$ and $\phi_{X}(s)\phi_{Y}(t)$ are not identical (Van der Vaart [25, Lemma 2.15]). Since characteristic functions are continuous then there exists an open set $\mathcal{A}\subseteq\mathbb{R}^{p}\times\mathbb{R}^{q}$ such that $|\phi_{X,Y}(s,t)-\phi_{X}(s)\phi_{Y}(t)|^{2}>0$ for all $(s,t)\in\mathcal{A}$. Hence, by (3.1), ${\mathcal{V}}^{2}(X,Y)>0$. ∎

For the purpose of deriving estimators for ${\mathcal{V}}^{2}(X,Y)$, we now apply Lemma 2.1 to obtain a second representation of the distance covariance.

For $j,k,\in\{1,2,3,4\}$, we apply to (3.7) the elementary identity,

then we obtain

which is obtained by decomposing all summands on the right-hand side using Eq. (3) and observing that all terms which are not of the form $\mathbb{E}[\cos{\langle s,X_{i}-X_{j}\rangle}\,\cos{\langle t,Y_{l}-Y_{k}\rangle}]$ cancel each other. By applying the Fubini-Tonelli Theorem and the linearity of expectation and integration, we obtain

The proof is completed by applying Lemma 2.1 to calculate these three integrals. ∎

Before establishing estimators for ${\mathcal{V}}^{2}(X,Y)$, we remark briefly on the assumptions necessary for the existence of the distance covariance.

Using the representation of the distance covariance given in (3.2), it is straightforward to derive a U-statistic estimator for ${\mathcal{V}}^{2}(X)$. Specifically, we define the symmetric kernel function

where the sum is over all $i,j,k,l\in\{1,2,3,4\}$ such that $i$, $j$, $k$, and $l$ are distinct.

It follows from the representation (3.2) that each of the $24$ summands in (4.1) has expectation ${\mathcal{V}}^{2}(X,Y)$. Therefore,

Letting $(X_{1},Y_{1}),\ldots,(X_{n},Y_{n})$ be a random sample from $(X,Y)$, we find that an unbiased estimator of ${\mathcal{V}}^{2}(X,Y)$ is

We can now derive the consistency and asymptotic distribution of this estimator using standard U-statistic theory (Lee [13]). For this purpose, let us define

and

The preceding formulas and a classical result on U-statistics (Hoeffding [9, Theorem 7.1]) leads immediately to a proof of the following result.

Except for pathological examples, Theorem 4.1 provides the asymptotic distribution of ${\mathcal{V}}^{2}(X,Y)$ if $X$ and $Y$ are dependent. For the crucial case of independent $X$ and $Y$, however, the asymptotic distribution of $\sqrt{n}(\widehat{\Omega}-{\mathcal{V}}(X,Y)^{2})$ is degenerate; in this case, the asymptotic distribution can be derived using results on first-order degenerate U-statistics (Lee [13, Section 3.2.2]).

The proof follows by elementary, but lengthy, transformations and may be left as an exercise to students. A complete proof is provided by Huang and Ho [10, Appendices B.6 and B.7].

Finally, the following result follows directly from Lemma 4.2 and classical results on the distributions of first-order degenerate U-statistics (Lee [13, Section 3.2.2]).

In this article, we have derived under minimal technical requirements the most important statistical properties of the distance covariance. From this starting point, there are several additional interesting topics that can be explored, e.g., as instructional assignments:

We also remark on the fundamental integral, (2.1), that underpins the entire distance covariance and distance correlation theory. As noted by Dueck, et al. [4], the fundamental integral and variants of it have appeared in functional analysis (Gelfand and Shilov [8, pp. 192–195]), in Fourier analysis (Stein [21, pp. 140 and 263]), and in the theory of fractional Brownian motion on generalized random fields (Chil‘es and Delfiner [2, p. 266]; Reed, et al. [16]).

The fundamental integral also extends further. For $m\in\mathbb{N}$ and $v\in\mathbb{R}$, define

the truncated Maclaurin expansion of the cosine function. Dueck, et al. [4] proved that for $\alpha\in\mathbb{C}$,

with absolute convergence if and only if $2(m-1)<\Re(\alpha)<2m$. For $m=1$ and $\alpha=1$, (5.3) reduces to (2.1). Further, for $m=1$ and $0<\alpha<2$, the integral (5.3) provides the Lévy-Khintchine representation of the negative definite function $\|x\|^{\alpha}$, thereby linking the fundamental integral to the probability theory of the stable distributions.

In conclusion, the statistical analysis of data through distance covariance and distance correlation theory, by means of the fundamental integral, is seen to be linked closely to many areas of the mathematical sciences.