Pólya urn with memory kernel and
asymptotic behaviors of autocorrelation function

Econophysics and socio-physics are fascinating research fields in statistical physicsMantegna and Stanley (2007); Galam (2008); Castellano et al. (2009). In particular, herding or the tendency to follow the majority has attracted many researchers as it plays a crucial role in the understanding of the phenomenaArthur (1989); Bikhchandani et al. (1992); Cont and Bouchaud (2000); Mori et al. (2016). Many types of probabilistic models have been proposed to model herding and one example is the ant recruitment model, which describes the intermittent oscillation of the ants faced with two identical food sourcesKirman (1993). It incorporates a simple herding mechanism where a randomly chosen ant chooses one of the two food sources with a probability that is proportional to the numbers of ants who have chosen the food sources.

A variant of the ant recruitment model has been adopted as a model for the intermittent switching phenomena of trading strategy, trend-follower and contrarian, in the modeling of the time-series data of stock and foreign exchange marketsLux (1995). Another application is the collective bankruptcy of assetsHisakado and Mori (2020). In the modeling of the time-series data of defaults of assets, the estimation of the default probability (PD) and the default correlation is crucial as it affects the risk estimation of the investment. In addition to the default correlation for the defaults in the same year, it is necessary to model the correlation of the defaults in different years. The beta-binomial distribution model with memory kernel is a model for both correlations and it is tractable compared to the established model based on the correlation of asset valuesPluto and Tasche (2011); Hisakado and Mori (2021).

The fascinating point of the beta-binomial distribution model with memory kernel is its phase transition, and its essence is grasped by a variant of the Pólya urnG.Pólya (1931), Pólya urn with memory kernelHisakado and Mori (2020). If the memory kernel decays with the power law, the power-law index $\gamma$ determines the phases of the model. If $\gamma>1$, the autocorrelation function (ACF) shows power-law decay with exponent $\delta=\gamma$. For $\gamma<1$, the process becomes nonstationary and $\delta=0$. If $\gamma$ approaches 1, $\delta$ changes discontinuously to a value $\delta_{c}$. The order parameter of the phase transition is the limit value of the ACF. In the nonstationary phase with $c>0$ for $\gamma<1$, the memory of the history remains forever and the estimation of the PD and other parameters becomes impossible even if the length of the time series is infinite.

This study addresses the phase transition of the Pólya urn with an arbitrary memory kernel using the stochastic differential equation (SDE). The original Pólya urn is a Markov process as the probability that the color of a newly added ball is red depends only on the present number of red balls in the urn. The Pólya urn with exponential decay memory kernel can also be described as a Markov process when one focuses on the probability that the color of a newly added ball is red. However, if the memory kernel is a power-law decay one, it is necessary to introduce an infinite number of auxiliary variables to make the process Markov. The auxiliary field obeys the SDE with a common Wiener process. Using the formulation, we derive the self-consistent equation for the ACF. By studying the equation, we succeed to derive the power-law exponents of the leading and subleading terms of the ACF.

Our analysis is very similar to the one that studies the Hawkes process using a stochastic differential equationKanazawa and D.Sornette (2020a, b). The Hawkes process is a non-Markov self-excited point processHawkes (1971). The Hawkes process shows a stationary–nonstationary phase transition when the branching ratio exceeds one. The critical and near-critical behaviors of the probability density function of the intensity have been studied by solving the Master equationKanazawa and D.Sornette (2020a).

In our study of the Pólya urn with memory kernel, we focus on the phase transition. Two universality classes are known for the phase transitions of non-linear Pólya urnsMori and Hisakado (2015a); Nakayama and Mori (2021), which have the same scaling structure of order parameter as that of the absorption state phase transitionsHinrichsen (2000). Our goal is to understand the nature of the phase transition of the ”linear” Pólya urn with power-law memory kernel based on the behaviors of the order parameter. The organization of the paper is as follows. Section II defines the model. We review some results of the Pólya urn. In section III, we study the model using the SDE. We start from the exponential decay memory kernel model. The ACF decays exponentially and the process is stationary. Next we study the power-law decay memory kernel model. We introduce an auxiliary field to make the process Markovian and we derive the self-consistent equation for the ACF. Section IV shows the results about the asymptotic behavior of the ACF for the power-law decay memory kernel model. We study the power-law exponents of the leading and sub-leading terms of the ACF. In section V, we summarize the results.

The Pólya urn is a binary stochastic process $X(t)\in\{0,1\},t=1,2,\cdots$ and the probability that $X(t+1)$ takes the value 1 is

$\alpha,\beta>0$ are parameters and can be interpreted as the initial numbers of red (1) and blue (0) balls in the urnG.Pólya (1931). They are parameters and are not necessarily restricted to be integers. We denote $\alpha+\beta$ as $\theta$.

We define the Pólya urn with an arbitrary memory kernel $d(t),t=0,1,\cdots$Hisakado and Mori (2020). We denote the cumulative sum of $d(t)$ as $D_{d}(t)$

and the weighted sum of $X(t)$ with weights $d(t)$ as $S_{d}(t)$,

$Z_{d}(t)$ is then defined as

The probability that $X(t+1)$ takes the value 1 is

If one chooses $d(t)=1,t=0,1,\cdots$, $D_{d}(t)=t$ and $S_{d}(t)=\sum_{s=1}^{t}X(s)$, so that (2) reduces to (1). The probabilistic rule of (2) is linear in $X(s)$, the expected value of $X(t)$ is $E(X(t))=\alpha/\theta$.

We are interested in the response of $X(t+1)$ by the change in the initial value $X(1)$. The ACF $C(t)$ is defined as the covariance of $X(1)$ and $X(t+1)$ divided by the variance of $X(1)$ as,

We have another expression for $C(t)$ as Mori and Hisakado (2015a),

$X(2)$ depends on $X(1)$ and $E(X(2)|X(1))=(\alpha+d(0)X(1))/(\theta+d(0))$, we have

The order parameter $c$ of the phase transition of the stochastic process is defined as the limit value of $C(t)$, $c=\lim_{t\to\infty}C(t)$. The conditional expected value of $X(t+1)$ under the condition for $X(1)$ is the average probability that $X(t+1)$ takes the value 1 under the same condition.

As $C(t)=E(X(t+1)|X(1)=1)-E(X(t+1)|X(1)=0)$, we have the recursive equation for $C(t)$ as,

One can compute $C(t),t\geq 2$ using this equation and the initial condition $C(1)=d(0)/(\theta+d(0))$.

There are several choices for the memory kernel $d(t),t=0,\cdots$.

1. 1.

   Pólya urn, $d(t)=1,t\geq 0$.

   As we have noted before, if one choose $d(t)=1,t=0,\cdots$, the model is the Pólya urn. We denote $Z_{d}(t)$ for this case as $Z_{\infty}(t)$. $tZ_{\infty}(t)$ obeys a binomial distribution with shape parameters $\alpha,\beta$G.Pólya (1931). The expected value of $X(t)$ is $E(X(t))=\alpha/\theta$. $Z_{\infty}(\infty)$ obeys the beta distribution and the variance is $\frac{1}{\theta+1}\cdot\frac{\alpha\beta}{\theta^{2}}$. When one is interested in the dynamics of $X(t)$ for the initial condition $X(1)=x$, it is only necessary to alter the parameters $\alpha,\beta$ and $\theta$ as $\alpha+x,\beta+(1-x)$ and $\theta+1$, respectively. The conditional expected value $E(X(t+1)|X(1)=x)$ is $(\alpha+x)/(\theta+1)$ for $t\geq 1$ and ACF $C(t)$ becomes constant.

   $$C(t)=\frac{1}{\theta+1},t\geq 1.$$

   (4)

   The process is not stationary and not mean-reverting. The conditional variance of $Z_{\infty}(\infty)$ for $X(1)=x$ is

   $$V(Z_{\infty}(\infty)|X(1)=x)=\frac{1}{\theta+2}\cdot\frac{(\alpha+x)(\beta+(1-x))}{(\theta+1)^{2}}.$$

   (5)

2. 2.

   Finite memory kernel, $d(t)=1,t<r$ and $d(t)=0,t\geq r$.

   There is a cutoff $r$ in the memory kernel, and $d(t)=1$ for $0\leq t<r$, and $d(t)=0$ for $t\geq r$. For $t>r$, $X(t)$ is influenced by the most recent $r$ variables $X(s),s=t-1,\cdots,t-r$. For $t\leq r$, $X(t)$ are influenced from all past variables $X(s),s=1,\cdots,t-1$. $Z_{d}(t)$ is defined as

   $$Z_{d}(t)=\left\{\begin{array}[]{cc}\frac{1}{r}\sum_{s=t-r+1}^{t}X(s)&t\geq r,\\ \frac{1}{t}\sum_{s=1}^{t}X(s)&t<r.\end{array}\right.$$

   The probability that $Z_{d}(t),t>r$ changes by $\pm 1/r$ for the condition $Z_{d}(t)=z_{d}$ is given as

   	$\displaystyle P(\Delta Z_{d}(t)=+1/r|Z_{d}(t)=z_{d})$	$\displaystyle=$	$\displaystyle P(X(t-r+1)=0|Z_{d}(t)=z_{d})\cdot P(X(t+1)=1|Z_{d}(t)=z_{d}),$
   	$\displaystyle P(\Delta Z_{d}(t)=-1/r|Z_{d}(t)=z_{d})$	$\displaystyle=$	$\displaystyle P(X(t-r+1)=1|Z_{d}(t)=z_{d})\cdot P(X(t+1)=0|Z_{d}(t)=z_{d}).$

   If one adopts the mean field approximation as $P(X(t-r+1)=1|Z_{d}(t)=z_{d})=z_{d}(t)$ and $P(X(t-r+1)=0|Z_{d}(t)=z_{d})=1-z_{d}(t)$, the probabilities can be written as

   	$\displaystyle P(\Delta Z_{d}(t)=+1/r|Z_{d}(t)=z_{d})$	$\displaystyle=$	$\displaystyle(1-z_{d})\cdot\frac{\alpha+rz_{d}(t)}{\theta+r}$
   	$\displaystyle P(\Delta Z_{d}(t)=-1/r|Z_{d}(t)=z_{d})$	$\displaystyle=$	$\displaystyle z_{d}\cdot\frac{\beta+r(1-z_{d}(t))}{\theta+r}$

   By the approximation, the model becomes the Kirman’s ant colony modelKirman (1993). In the Kirman’s ant colony model, there are two food sources: $0,1$ and $r$ ants. The numbers of ants that have chosen food sources 1 and 0 are $n_{1}$ and $n_{0}$, respectively. A randomly selected ant chooses food sources 1 and 0 with a probability that is linear in $n_{1}$ and $n_{0}$ among $r-1$ ants. The probability that the ratio $z=n_{1}/r$ changes $\pm 1/r$ is given as

   	$\displaystyle P(\Delta z=+1|n_{1}(t)/r=z)$	$\displaystyle=$	$\displaystyle(1-z)\cdot\frac{\alpha+(r-1)z}{\theta+(r-1)},$
   	$\displaystyle P(\Delta z=-1/r|n_{1}(t)/r=z)$	$\displaystyle=$	$\displaystyle z\cdot\frac{\beta+(r-1)(1-z)}{\theta+(r-1)}.$

   The prefactors in the right hand side of the equations are the probabilities that the randomly selected ants are from food sources 0 and 1, respectively. The remaining terms are the probabilities that the selected ant chooses food sources 1 and 0, respectively. The $\alpha,\beta$ are noises in the decision of the ants. The Pólya urn with finite memory kernel corresponds to the ant colony model in the mean filed approximation.

   The stationary distribution of $n_{1}$ is a beta binomial distribution, with shape parameters $\alpha,\beta$. $C(t)$ decays exponentially as $C(t)\sim\exp(-t/\xi)$, which defines the correlation length $\xi$. $\xi$ depends on the length of the memory $r$ and diverges in the limit $r\to\infty$Mori and Hisakado (2015b).

3. 3.

   Exponential decay memory kernel, $d(t)=e^{-rt}$.

   A previous study reveals that $C(t)$ decays exponentially as $C(t)\sim e^{-t/\xi}$ for $t>>1$Hisakado and Mori (2020). The correlation length $\xi$ is determined by $r$,

   $$1/\xi=\ln\left(\frac{\theta(1-e^{-r})+1}{\theta e^{-r}(1-e^{-r})+1}\right)>0,$$

   and $\xi$ diverges in the limit $r\to 0$. In the limit $r\to\infty$, we have $d(t)=0$ for $t>0$. $X(t+1)$ depends only on $X(t)$, which corresponds to the correlated random walkB$\ddot{o}$hm (2000).

4. 4.

   Power-law decay memory kernel, $d(t)=(1+t)^{-\gamma}$.

   The model shows a phase transition at $\gamma=1$Hisakado and Mori (2020). By the numerical study of the ACF $C(t)$, it was conjectured that the asymptotic behavior of $C(t)$ is

   $$C(t)\sim\left\{\begin{array}[]{cc}ct^{-\gamma}&\gamma>1,\\ ct^{-\delta_{c}}&\gamma=1,\\ ct^{0}&\gamma<1.\end{array}\right.$$

   The critical power-law exponent $\delta_{c}$ at $\gamma=1$ is given as the solution of the next equation,

   $$\theta=\lim_{t\to\infty}\left(\int_{1/(t+1)}^{t/(t+1)}x^{-\delta_{c}}(1-x)^{-1}dx-\ln t\right).$$

   (6)

We study the Pólya urn with memory kernel using the method of stochastic approximation Gardiner (2009); Renlund (2010). We start from the exponential decay memory kernel model. For the reader’s convenience, we summarize the results of the analysis of the original Pólya urn using the SDE in Appendix A.