Posterior Convergence of Nonparametric Binary and Poisson Regression Under Possible Misspecifications

The situation for applicability of nonparametric regression is frequently encountered in many practical scenarios where no parametric model fits the data. In particular, non-parametric regression for binary dependent variables is very common for various branches of statistics like medical and spatial statistics, whereas nonparametric version of Poisson regression is being used recently in many non-trivial scenerios such as for analyzing the likelihood and severity of vehicle crashes (Ye et al. (2018)). Interestingly, despite vast applicability of both the binary as well as Poisson regression, it seems that the available literature on nonparametric Poisson regression is scarce in comparison to the available literature on nonparametric binary regression. The Bayesian approach to nonparametric binary regression problem has been accounted for in Diaconis and Freedman (1993). An account of posterior consistency for Gaussian process prior in nonparametric binary regression modeling can be found in Ghosal and Roy (2006), where the authors suggested that similar consistency results should hold for nonparametric Poisson regression model setup. Literature on consistency results for nonparametric Poisson regression is very limited. Pillai et al. (2007) have obtained consistency results for Poisson regression using an approach similar to that of Ghosal and Roy (2006) under certain assumptions, but so far without explicit specifications and detail on prior. On the other hand, our approach will be based on results on Shalizi (2009), which is much different from Ghosal and Roy (2006) and capable of handling model misspecification. Unlike the previous works, the approach of Shalizi (2009) also enables us to investigate the rate at which the posterior converges, which turns out to be the Kullback-Leibler (KL) divergence rate, and also the traditional posterior convergence rate.

In this article, we investigate posterior convergence of nonparametric binary and Poisson regression where the nonparametric regression is modeled as some suitable stochastic process. In the binary situation, we consider a similar setup as that of Ghosal and Roy (2006), where the authors have considered binary observations with response probability as an unknown smooth function of a set of covariates, which was modeled using Gaussian process. Here we will consider a binary response variable $Y$ and a $d$-dimensional covariate $x$ belonging to a compact subset. The probability function is given by $p(x)=P(Y=1|X=x)$ along with a prior for $p$ induced by some appropriate stochastic process $\eta(x)$ with the relation $p(x)=H\left(\eta(x)\right)$ for a known, non-decreasing and continuously differentiable cumulative distribution function $H(\cdot)$. We will establish a posterior convergence theory for nonparametric binary regression under possible misspecifications based on the general theory of posterior convergence of Shalizi (2009). Our theory also includes the case of misspecified models, that is, if the true regression function is not even supported by the prior. This approach to Bayesian asymptotics also permits us to show that the relevant posterior probabilities converge at the KL divergence rate, and that the posterior convergence rate with respect to KL-divergence is just slower than $\frac{1}{n}$, where $n$ denotes the number of observations. We further show that even in the case of misspecification, the posterior predictive distribution can approximate the best possible predictive distribution adequately, in the sense that the Hellinger distance, as well as the total variation distance between the two distributions can tend to zero.

For nonparametric Poisson regression, given $x$ in the compact space of covariates, we model the mean function $\lambda(x)$ as $\lambda(x)=H(\eta(x))$, where $H$ is a continuously differentiable function. Again, we investigate the general theory of posterior convergence, including misspecifications, rate of convergence of the posterior distribution and the usual posterior convergence rate, in Shalizi’s framework.

The rest of our paper is structured as follows. In Section 2 we provide a brief overview and intuitive explanation of the main assumptions and results of Shalizi (2009) suitable for our approach. The basic prenises for nonparametric binary and Poisson regression are provided in Sections 3 and 4, respectively. The required assumptions and their discussions are provided in Section 5. In Section 6, our main results on posterior convergence of binary and Poisson regression are provided, while Section 8 details the consequences of misspecifications. Concluding remarks are provided in Section 9.

The technical details are presented in the Appendix. Specifically, details of the necessary assumptions and results of Shalizi (2009) are provided in Appendix A. The detailed proofs of verification of Shalizi’s assumptions are provided in Appendix B and Appendix C for binary and Poisson regression setups, respectively.

Let the set of random variables for the response be denoted by $\mathbf{Y_{n}}=\left(Y_{1},Y_{2},\ldots,Y_{n}\right)$. For a given parameter space $\Theta$, let $f_{\theta}(\mathbf{Y_{n}})$ be the observed likelihood and $f_{\theta_{0}}(\mathbf{Y_{n}})$ be the true likelihood. We assume $\theta\in\Theta$ but the truth $\theta_{0}$ need not be in $\Theta$, thus allowing possible misspecification.

The KL divergence $KL(f,g)=\int f\log(\frac{f}{g})$ is a measure of divergence between two probability densities $f$ and $g$. The KL divergence is related to likelihood ratios, since by the Strong Law of Large Numbers (SLLN) for independent and identical ($iid$) situations,

$$\frac{1}{n}\displaystyle\sum_{i=1}^{n}\log\left[\dfrac{f(Y_{i})}{g(Y_{i})}\right]\rightarrow KL(f,g).$$

For every $\theta\in\Theta$, the KL divergence rate is given by:

$$h(\theta)=\underset{n\rightarrow\infty}{\lim}~{}\dfrac{1}{n}E\left[\log\left\{\dfrac{f_{\theta_{0}}(\mathbf{Y_{n}})}{f_{\theta}(\mathbf{Y_{n}})}\right\}\right].$$

(2.1)

The key ingredient associated with the approach of Shalizi (2009) for proving convergence of the posterior distribution of $\theta$ is to show that the asymptotic equipartition property holds. To illustrate, let us consider the following likelihood ratio:

$$R_{n}(\theta)=\dfrac{f_{\theta}(\mathbf{Y_{n}})}{f_{\theta_{0}}(\mathbf{Y_{n}})}.$$

(2.2)

If we think of the $iid$ setup, $h(\theta)$ reduces to the KL divergence between the true and the hypothesized model. For each $\theta\in\Theta$, “asymptotic equipartition” property is as follows:

$$\underset{n\rightarrow\infty}{\lim}~{}\frac{1}{n}\log\left[R_{n}(\theta)\right]=-h(\theta),$$

(2.3)

Here “asymptotic equipartition” refers to dividing up $\log\left[R_{n}(\theta)\right]$ into $n$ factors for large $n$ such that all the factors are asymptotically equal. For illustration, in the $iid$ scenario, each factor converges to the same KL divergence between the true and the postulated model. The purpose of asymptotic equipartition is to ensure that relative to the true distribution, the likelihood of each $\theta$ decreases to zero exponentially fast, with rate being the KL divergence rate.

for $A\subseteq\Theta$, let

	$\displaystyle h\left(A\right)$	$\displaystyle=\underset{\theta\in A}{\mbox{ess~{}inf}}~{}h(\theta);$		(2.4)
	$\displaystyle J(\theta)$	$\displaystyle=h(\theta)-h(\Theta);$		(2.5)
	$\displaystyle J(A)$	$\displaystyle=\underset{\theta\in A}{\mbox{ess~{}inf}}~{}J(\theta),$		(2.6)

where $h(A)$ roughly represent the minimum KL-divergence between the postulated and the true model over the set A. If $h(\Theta)>0$, it indicates model misspecification. However, as we shall show, model misspecification need not always imply that $h(\Theta)>0$. One such counter example is also given in Chatterjee and Bhattacharya (2020).

Observe that, for $A\subset\Theta$, $J(A)>0$. For the prior, it is required to construct an appropriate sequence of sieve sets $\mathcal{G}_{n}\rightarrow\Theta$ as $n\rightarrow\infty$ such that:

1. (1)

   $h\left(\mathcal{G}_{n}\right)\rightarrow h\left(\Theta\right)$, as $n\rightarrow\infty$.

2. (2)

   $\pi\left(\mathcal{G}_{n}\right)\geq 1-\alpha\exp\left(-\beta n\right),~{}\mbox{for some}~{}\alpha>0,~{}\beta>2h(\Theta)$;

The sets $\mathcal{G}_{n}$ can be interpreted as the sieves in the sense that, the behaviour of the likelihood ratio and the posterior on the sets $\mathcal{G}_{n}$ essentially carries over to $\Theta$.

Let $\pi(\cdot|\mathbf{Y}_{n})$ denote the posterior distribution of $\theta$ given $\mathbf{Y}_{n}$. Then with the above notions, verification of (2.3) along with several other technical conditions (details given in Appendix A) ensure that any $A\subseteq\Theta$ for which $\pi(A)>0$,

$$\underset{n\rightarrow\infty}{\lim}~{}\pi(A|\mathbf{Y}_{n})=0,$$

(2.7)

almost surely, provided that $h(A)>h(\Theta)$. The latter $h(A)>h(\Theta)$ implies positive KL-divergence in $A$, even if $h(\Theta)=0$. That is, $A$ is the set in which the postulated model fails to capture the true model in terms of the KL-divergence. Hence, expectedly, the posterior probability of that set converges to zero.

Under mild assumptions, it also holds that

$$\underset{n\rightarrow\infty}{\lim}~{}\frac{1}{n}\log\pi(A|\mathbf{Y}_{n})=-J(A),$$

(2.8)

almost surely. This result shows that the rate at which the posterior probability of $A$ converges to zero is about $\exp(-nJ(A))$. From the above results it is clear that the posterior concentrates on sets of the form $N_{\epsilon}=\left\{\theta:h(\theta)\leq h(\Theta)+\epsilon\right\}$, for any $\epsilon>0$.

Shalizi addressed the rate of posterior convergence as follows. Letting $N_{\epsilon_{n}}=\left\{\theta:h(\theta)\leq h(\Theta)+\epsilon_{n}\right\}$, where $\epsilon_{n}\rightarrow 0$ such that $n\epsilon_{n}\rightarrow\infty$, Shalizi showed, under an additional technical assumption, that almost surely,

$$\underset{n\rightarrow\infty}{\lim}~{}\pi\left(N_{\epsilon_{n}}|\mathbf{Y}_{n}\right)=1.$$

(2.9)

Moreover, it was shown by Shalizi that the squares of the Hellinger and the total variation distances between the posterior predictive distribution and the best possible predictive distribution under the truth, are asymptotically almost surely bounded above by $h(\Theta)$ and $4h(\Theta)$, respectively. That is, if $h(\Theta)=0$, then this allows very accurate approximation of the true predictive distribution by the posterior predictive distribution.

Let $Y\in\{0,1\}$ be a binary outcome variable and X a vector of covariates. Suppose $Y_{1},Y_{2},\ldots,Y_{n}\in\{0,1\}^{n}$ are some independent binary responses conditional on unobserved covariates $X_{1},X_{2},\ldots,X_{n}\in\mathfrak{X}\subset\Re^{d}$. We assume that the covariate space $\mathfrak{X}$ is compact. Let $\mathbf{Y}_{n}=(Y_{1},Y_{2},\ldots,Y_{n})^{T}$ be the binary response random variables against the covariate vector $\mathbf{X}_{n}=(X_{1},X_{2},\ldots,X_{n})^{T}$. The corresponding observed values will be denoted by $\mathbf{y}_{n}=(y_{1},y_{2},\ldots,y_{n})$ and $\mathbf{x}_{n}=(x_{1},x_{2},\ldots,x_{n})$ respectively. Let the model be specified as follows: for $i=1,2,\ldots,n$:

	$\displaystyle Y_{i}|X_{i}\displaystyle\sim Binomial\left(1,p(X_{i})\right)$		(3.1)
	$\displaystyle p(x)=H\left(\eta(x)\right)$		(3.2)
	$\displaystyle\eta(\cdot)\sim\pi_{\eta},$		(3.3)

where $\pi_{\eta}$ is the prior for some suitable stochastic process. Note that the prior for $p$ is induced by the prior for $\eta$. Our concern is to infer about the success probability function $p(x)=P(Y=1|X=x)$ when the number of observations goes to infinity. We will assume that the functions $\eta$ have continuous first partial derivatives. We denote this class of functions by $\mathcal{C^{\prime}}(\mathfrak{X})$. We do not assume the truth $\eta_{0}$ in $\mathcal{C^{\prime}}(\mathfrak{X})$, allowing misspecification. The link function $H$ is a known, non-decreasing, continuously differentiable cumulative distribution function on the real line $\Re$. It is widely accepted to assume the function $H(\cdot)$ to be known as part of model assumption. For example, in logistic regression we choose the standard logistic cumulative distribution function as the link function, whereas in probit regression $H$ is chosen to be the standard normal cumulative distribution function $\phi$. More discussion on link function along with several other examples can be found in Choudhuri et al. (2007), Newton et al. (1996), Gelfand and Kuo (1991). A Bayesian method for estimation of $p$ has been provided in Choudhuri et al. (2007). In has been shown in Ghosal and Roy (2006) that the sample paths of the Gaussian processes can well approximate a large class of functions and hence it is not essential to consider additional uncertainty in the link function $H$.

Let $\mathfrak{C}$ be the counting measure on $\{0,1\}$. Then according to the model assumption, the conditional density of $y$ given $x$ with respect to $\mathfrak{C}$ will be represented by the density function $f$ as follows:

$$f(y|x)=p(x)^{y}\left(1-p(x)\right)^{1-y}.$$

(3.4)

The prior for $f$ will be denoted by $\pi$. Let $f_{0}$ and $p_{0}$ denote truth density and success probability, respectively. Then under the truth, the joint density is:

$$f_{0}(y|x)=p_{0}(x)^{y}\left(1-p_{0}(x)\right)^{1-y}.$$

(3.5)

One of the main objectives of this article is to show consistency of the posterior distribution of $p$ treated as parameter arising from the parameter space $\Theta$ specified as follows:

$$\Theta=\left\{p(\cdot):p(x)=H\left(\eta(x)\right),\eta\in\mathcal{C^{\prime}}(\mathfrak{X})\right\},$$

(3.6)

or simply, $\Theta=\mathcal{C^{\prime}}(\mathfrak{X})$.

For Poisson regression model set up, let $Y\in\mathbb{N}$ be a count outcome variable and $X$ a vector of covariates. Here $\mathbb{N}$ denote the set of non negative integers. Suppose $Y_{1},Y_{2},\ldots,Y_{n}\in\mathbb{N}^{n}$ are some independent responses conditional on covariates $X_{1},X_{2},\ldots,X_{n}\in\mathfrak{X}\subset\Re^{d}$. We assume that the covariate space $\mathfrak{X}$ is compact. Let $\mathbf{Y}_{n}=(Y_{1},Y_{2},\ldots,Y_{n})^{T}$ be the response random variables against the covariate vector $\mathbf{X}_{n}=(X_{1},X_{2},\ldots,X_{n})^{T}$. The corresponding observed values will be denoted by $\mathbf{y}_{n}=(y_{1},y_{2},\ldots,y_{n})$ and $\mathbf{x}_{n}=(x_{1},x_{2},\ldots,x_{n})$ respectively. Let the parameter space be specified as follows:

$$\Lambda=\left\{\lambda(\cdot):\lambda(x)=H\left(\eta(x)\right),\eta\in\mathcal{C^{\prime}}(\mathfrak{X})\right\}.$$

(4.1)

The link function $H$ is a known, non-negative continuously differentiable function on $\Re$. We equivalently define the parameter space as $\Theta=\mathcal{C^{\prime}}(\mathfrak{X})$. Thus, in what follows, we shall use both $\Lambda$ and $\Theta$ to denote the parameter space, depending on convenience. Then the model is specified as follows: for $i=1,2,\ldots,n$,

	$\displaystyle Y_{i}|X_{i}\sim\exp\left(-\lambda(X_{i})\right)\dfrac{(\lambda(X_{i}))^{y}}{y!}$		(4.2)
	$\displaystyle\lambda(x)=H\left(\eta(x)\right);$		(4.3)
	$\displaystyle\eta(\cdot)\sim\pi_{\eta}.$		(4.4)

Similar to binary regression, here our concern will be to infer about $\lambda(x)$ when the number of observations goes to infinity. We do not assume the truth $\eta_{0}$ in $\mathcal{C^{\prime}}(\mathfrak{X})$ as before, allowing misspecification.

Now, suppose $\mathfrak{C}$ be the counting measure on $\mathbb{N}$. According to the model assumption for Poisson regression, the conditional density of $y$ given $x$ with respect to $\mathfrak{C}$ will be represented by density function $f$ as follows:

$$f(y|x)=\exp\left(-\lambda(x)\right)\dfrac{(\lambda(x))^{y}}{y!}.$$

(4.5)

The prior for $f$ will be denoted by $\Pi$. Let $f_{0}$ and $\lambda_{0}$ denote truth density and true mean function, respectively. Again, one of our main aims is to establish consistency of the posterior distribution of $\lambda$ treated as parameter arising from $\Lambda$.

We need to make some appropriate assumptions for establishing convergence of both the binary and Poisson regression models equipped with stochastic process prior. The latter also requires suitable assumptions. Many of the assumptions are similar to those taken in Chatterjee and Bhattacharya (2020). Hence the purpose of such assumptions will be as discussed in Chatterjee and Bhattacharya (2020), which we shall briefly touch upon here.

We treat the covariates as either random (observed or unobserved) or non-random (observed). Accordingly, in Assumption 4 below we provide conditions pertaining to these aspects.

Here we will state a summary of our main results regarding posterior convergence of nonparametric binary regression and Poisson regression. The key results associated with the asymptotic equipartition property are provided in Theorems 1 – 4, proofs of which are provided in Appendix B (for binary regression) and in Appendix C (for Poisson regression).

Theorems 1 and 3 for binary regression and Theorems 2 and 4 for Poisson regression ensure that conditions (S1) to (S3) of Shalizi (2009) hold, and (S4) holds for both binary and Poisson regression because of compactness of $\mathfrak{X}$ and continuity of $H$ and $\eta$. The detailed proofs are presented in Appendix B.4 and Appendix C.4, respectively.

We construct the sieves $\mathcal{G}_{n}$ for binary regression model set up as follows:

	$\displaystyle\mathcal{G}_{n}={}$	$\displaystyle\{\eta\in\mathcal{C}^{\prime}(\mathfrak{X}):\|\eta\|\leq\exp(\left(\beta n\right)^{1/4}),$		(6.6)
		$\displaystyle\|\eta_{j}^{\prime}\|\leq\exp(\left(\beta n\right)^{1/4});j=1,2,\ldots,d\}$

It follows that $\mathcal{G}_{n}\rightarrow\Theta$ as $n\rightarrow\infty$, where the parameter space $\Theta$ is given by (3.6).

In a similar manner, we construct the sieves $\mathbb{G}_{n}$ for binary regression as follows:

	$\displaystyle\mathbb{G}_{n}={}$	$\displaystyle\{\lambda(\cdot):\ \ \lambda(x)=H(\eta(x)),\eta\in\mathcal{C}^{\prime}(\mathfrak{X}),\|\eta\|\leq\exp(\left(\beta n\right)^{1/4}),$		(6.7)
		$\displaystyle\|\eta_{j}^{\prime}\|\leq\exp(\left(\beta n\right)^{1/4});j=1,2,\ldots,d\}.$

Then similarly it will also follow that $\mathbb{G}_{n}\rightarrow\Lambda$ as $n\rightarrow\infty$, where the parameter space $\Lambda$ is given by (4.1).

Assumption 3 ensures that for binary regression, $\Pi\left(\mathcal{G}^{c}_{n}\right)\leq\alpha\exp(-\beta n)$ for some $\alpha>0$ and similarly $\Pi\left(\mathbb{G}^{c}_{n}\right)\leq\alpha\exp(-\beta n)$ for Poisson regression. Now, these results, continuity of $h(\theta)$, $h(\lambda)$ (the proofs of continuity of $h(p)$ and $h(\lambda)$ follows using the same techniques as in Appendices B.1 and C.1), compactness of $\mathcal{G}_{n}$, $\mathbb{G}_{n}$ and the uniform convergence results of Theorems 3 and 4, together ensure (S5) for both the model setups.

Now, as pointed out in Chatterjee and Bhattacharya (2020), we observe that the aim of assumption (S6) is to ensure that (see the proof of Lemma 7 of Shalizi (2009)) for every $\epsilon>0$ and for all sufficiently large $n$,

$$\dfrac{1}{n}\log\int_{\mathcal{G}_{n}}R_{n}(p)\ d\pi(p)\leq h(\mathcal{G}_{n})+\epsilon,\ \ \text{almost surely}.$$

(6.8)

As $h(\mathcal{G}_{n})\rightarrow h(\Theta)$ as $n\rightarrow\infty$, it is enough to verify that for every $\epsilon>0$ and for all $n$ sufficiently large,

$$\dfrac{1}{n}\log\int_{\mathcal{G}_{n}}R_{n}(p)\ d\pi(p)\leq h(\Theta)+\epsilon,\ \ \text{almost surely}.$$

(6.9)

First we observe that

$$\dfrac{1}{n}\log\int_{\mathcal{G}_{n}}R_{n}(p)\ d\pi(p)\leq\dfrac{1}{n}\sup_{p\in\mathcal{G}_{n}}\log R_{n}(p).$$

(6.10)

For large enough $\kappa>h(\Theta)$, consider $S=\{p:h(p)\leq\kappa\}$.

In a very similar manner, the following lemma also holds for Poisson model set up.

Using compactness of $S$, in the same way as in Chatterjee and Bhattacharya (2020), condition (S6) of Shalizi can be shown to be equivalent to (6.11) and (6.12) in Theorems 5 and 6 below, corresponding to binary and Poisson cases. In the supplement we show that these equivalent conditions are satisfied in our model setups.

Assumption (S7) of Shalizi also holds for both the model setups because of continuity of $h(p)$ and $h(\lambda)$. Hence, all the assumptions (S1)–(S7) stated in Appendix A are satisfied for binary and Poisson regression setups.

Overall, our results lead to the following theorems.

Consider a sequence of positive reals $\epsilon_{n}$ such that $\epsilon_{n}\rightarrow 0$ while $n\epsilon_{n}\rightarrow\infty$ as $n\rightarrow\infty$ and the set $N_{\epsilon_{n}}=\{p:h(p)\leq h(\Theta)+\epsilon_{n}\}$. Then the following result of Shalizi holds.

To investigate the rate of convergence in our cases (and also for the case of Chatterjee and Bhattacharya (2020)), it has been proved in Chatterjee and Bhattacharya (2020) that $\epsilon_{n}$ will be the rate of convergence for $\epsilon_{n}\rightarrow 0$, $n\epsilon_{n}\rightarrow\infty$ as $n\rightarrow\infty$, if we can show that the following hold:

$$\dfrac{1}{n}\log\int_{\mathcal{G}_{n}\cap N_{\epsilon_{n}}^{c}}R_{n}(p)\ d\pi(p)\leq-h(\Theta)+\epsilon,$$

(7.3)

$$\dfrac{1}{n}\log\int_{\mathbb{G}_{n}\cap N_{\epsilon_{n}}^{c}}R_{n}(\lambda)\ d\pi(\lambda)\leq-h(\Lambda)+\epsilon,$$

(7.4)

for any $\epsilon>0$ and all $n$ sufficiently large.

Following similar arguments of Chatterjee and Bhattacharya (2020), we find that the posterior rate of convergence with respect to KL-divergence is just slower than $n^{-1}$. To put it another way, it is just slower that $n^{-\frac{1}{2}}$ with respect to Hellinger distance for the model setups we consider. Our results can be formally stated in Theorem 10 for Binary regression and in Theorem 11 for Poisson regression.

Suppose that the true function $\eta_{0}$ consists of countable number of discontinuities but has continuous first order partial derivatives at all other points. Then $\eta_{0}\not\in\mathcal{C^{\prime}}(\mathfrak{X})$. However, there exists some $\tilde{\eta}\in\mathcal{C^{\prime}}(\mathfrak{X})$ such that $\tilde{\eta}(x)=\eta_{0}(x)$ for all $x\in\mathfrak{X}$ where $\eta_{0}$ is continuous. Similar to this kind of situation is mentioned in Chatterjee and Bhattacharya (2020). Observe that, if the probability measure $Q$ of $X_{i}$ is dominated by the Lebesgue measure, then from Theorem 1 we have $h(\Theta)=0$. Then the posterior of $\eta$ concentrates around $\tilde{\eta}$, which is the same as $\eta_{0}$ except at the countable number of discontinuities of $\eta_{0}$. Corresponding $\tilde{p}=H(\tilde{\eta})$ and $\tilde{\lambda}=H(\tilde{\eta})$ will also differ from $p_{0}$ and $\lambda_{0}$. If $p_{0}$ and $\lambda_{0}$ are such that $0<h(\Theta)<\infty$ and $0<h(\Lambda)<\infty$ respectively then the posteriors concentrate around the minimizers of $h(p)$ and $h(\lambda)$, provided such minimizers exist in $\Theta$ and $\Lambda$, respectively.

In this paper we attempted to address posterior convergence of nonparametric binary and Poisson regression, along with the rate of convergence, while also allowing for misspecification, using the approach of Shalizi (2009). We also have shown that, even in the case of misspecification, the posterior predictive distribution can be quite accurate asymptotically, which should be a point of interest from subjective Bayesian viewpoint. The asymptotic equipartition property plays a central role here. It is one of the crucial assumptions and yet relatively easy to establish under mild conditions. It actually brings forward the KL property of the posterior, which in turn characterizes the posterior convergence, and also the rate of posterior convergence and misspecification.