On the Le Cam distance between multivariate hypergeometric
and multivariate normal experiments

Let $d\in\mathbb{N}$. The $d$-dimensional (unit) simplex and its interior are defined by

where $\|\boldsymbol{x}\|_{1}\vcentcolon=\sum_{i=1}^{d}|x_{i}|$ denotes the $\ell^{1}$ norm on $\mathbb{R}^{d}$. Given a set of probability weights $\boldsymbol{p}\in N^{-1}\,\mathbb{N}_{0}^{d}\cap\mathrm{Int}\hskip 0.56905pt(\mathcal{S}_{d})$, the probability mass function of the multivariate hypergeometric distribution, $\mathrm{Hypergeometric}\hskip 0.56905pt(N,n,\boldsymbol{p})$, is defined, by Johnson et al., [6, Chapter 39], as

where $p_{d+1}\vcentcolon=1-\|\boldsymbol{p}\|_{1}>0$, $k_{d+1}\vcentcolon=n-\|\boldsymbol{k}\|_{1}$, $n,N\in\mathbb{N}$ with $n\leq N$, and

This distribution represents the first $d$ components of the vector of categorical sample counts when randomly sorting a random sample of $n$ objects from a finite population of $N$ objects into $d+1$ categories, where $p_{i},~{}1\leq i\leq d+1$, is the probability of any given object to be sorted in the $i$-th category.

Our first main goal in this paper is to develop a local approximation for the log-ratio of the multivariate hypergeometric probability mass function (1.2) over the $\mathrm{Multinomial}\hskip 0.56905pt(n,\boldsymbol{p})$ probability mass function, namely

This latter distribution represents the exact same as (1.2) above, except that the population from which the $n$ objects are drawn is infinite ($N=\infty$). Another way of distinguishing $P_{N,n,\boldsymbol{p}}$ and $Q_{n,\boldsymbol{p}}$ in a finite population of $N$ objects is to say that we sample the $n$ objects without replacement and with replacement, respectively. In both cases, the categorical probabilities $(\boldsymbol{p},p_{d+1})$ are the same. For good general references on normal approximations, we refer the reader to Bhattacharya & Ranga Rao, [2] and Kolassa, [7].

Our second main goal is to prove an upper bound on the total variation between the probability measure on $\mathbb{R}^{d}$ induced by a random vector distributed according to $P_{N,n,\boldsymbol{p}}$ and jittered by a uniform on $(-1/2,1/2)^{d}$, and the probability measure on $\mathbb{R}^{d}$ induced by a multivariate normal random vector with the same mean and covariances as a random vector distributed according to $Q_{n,\boldsymbol{p}}$, namely $n\boldsymbol{p}$ and $n(\mathrm{diag}(\boldsymbol{p})-\boldsymbol{p}\boldsymbol{p}^{\top})$. The proof makes use of the total variation bound from Ouimet, [14, Lemma 3.1] (which improved Lemma 2 in [4]) on the total variation between the probability measure on $\mathbb{R}^{d}$ induced by a multinomial vector distributed according to $Q_{n,\boldsymbol{p}}$ and jittered by a uniform on $(-1/2,1/2)^{d}$, and the probability measure on $\mathbb{R}^{d}$ induced by a multivariate normal random vector with the same mean and covariances. As pointed out by Mattner & Schulz, [11, p.732], the univariate case here would be much simpler since Morgenstern, [12, p.62-63] showed that the hypergeometric probability mass function can be written as a ratio of three binomial probability mass functions, and local limit theorems are well-known for the binomial distribution, see, e.g., Prokhorov, [15] and Govindarajulu, [5].

The deficiency between a given statistical experiment and another measures the loss of information from carrying inferences to the second setting using information from the first setting. This loss of information goes in both directions, but the deficiency is not necessarily symmetric. The maximum of the two deficiencies is called the Le Cam distance (or $\Delta$-distance in [8]). The usefulness of this notion comes from the fact that seemingly completely different statistical experiments can result in asymptotically equivalent inferences using Markov kernels to carry information from one setting to another. For instance, it was famously shown by Nussbaum, [13] that the density estimation problem and the Gaussian white noise problem are asymptotically equivalent in the sense that the Le Cam distance between the two experiments goes to $0$ as the number of observations goes to infinity. The main idea was that the information we get from sampling observations from an unknown density function and counting the observations that fall in the various boxes of a fine partition of the density’s support can be encoded using the increments of a properly scaled Brownian motion with drift $t\mapsto\int_{0}^{t}\sqrt{f(s)}\hskip 1.42262pt{\rm d}s$, and vice versa. An alternative (simpler) proof of this asymptotic equivalence was shown by Brown et al., [3] who combined a Haar wavelet cascade scheme with coupling inequalities relating the binomial and univariate normal distributions at each step (a similar argument was developed previously by Carter, [4] to derive a multinomial/multivariate coupling inequality). Not only Brown et al., [3] streamlined the proof of the asymptotic equivalence originally shown by Nussbaum, [13], but their results hold for a larger class of densities and the asymptotic equivalence was also extended to Poisson processes. Our third main result in the present paper extends the multinomial/multivariate normal comparison from [4] (revisited and improved by Ouimet, [14], who removed the inductive part of the argument) to the multivariate hypergeometric/multivariate normal comparison (recall from (1.4) that the multinomial distribution is just the limiting case $N=\infty$ of the multivariate hypergeometric distribution). For an excellent and concise review on Le Cam’s theory for the comparison of statistical models, we refer the reader to Mariucci, [10].

The three results we have just described are presented in Section 2, and the related proofs are gathered in Section 3. Here are now some motivations for these results. First, we believe that the first two results (the local expansion of the log-ratio and the total variation bound) could help in developing asymptotic Berry-Esseen type bounds for the symmetric multivariate hypergeometric distribution and the symmetric multinomial distribution, similar to the exact optimal bounds proved recently by Mattner & Schulz, [11] in the univariate setting. Second, there might be a way to use the Le Cam distance upper bound between multivariate hypergeometric and multivariate normal experiments to extend the results on the asymptotic equivalence between the density estimation problem and the Gaussian white noise problem shown by Nussbaum, [13] and Brown et al., [3].

Our first main result is an asymptotic expansion for the log-ratio of the multivariate hypergeometric probability mass function (1.2) over the corresponding multinomial probability mass function (1.4).

The local limit theorem above together with the total variation bound in [4, 14] between jittered multinomials and the corresponding multivariate normals allow us to derive an upper bound on the total variation between the probability measure on $\mathbb{R}^{d}$ induced by a multivariate hypergeometric random vector jittered by a uniform random vector on $(-1/2,1/2)^{d}$ and the probability measure on $\mathbb{R}^{d}$ induced by a multivariate normal random vector with the same mean and covariances as the multinomial distribution in (1.4).

Since the Le Cam distance is a pseudometric and the Markov kernel that jitters a random vector by a uniform on $(-1/2,1/2)^{d}$ is easily inverted (round off each component of the vector to the nearest integer), then we find, as a consequence of the total variation bound in Theorem 2, an upper bound on the Le Cam distance between multivariate hypergeometric and multivariate normal experiments.

Now, consider the following multivariate normal experiments with independent components

where $\boldsymbol{1}\vcentcolon=(1,1,\dots,1)^{\top}$, then Carter, [4, Section 7] showed, using a variance stabilizing transformation, that

with proper adjustments to the definition of the deficiencies in (2.6).