Population Level Activity in Large Random Neural Networks

James MacLaurin, Moshe Silverstein, Pedro Vilanova

Abstract

We determine limiting equations for large asymmetric ‘spin glass’ networks. The initial conditions are not assumed to be independent of the disordered connectivity: one of the main motivations for this is that allows one to understand how the structure of the limiting equations depends on the energy landscape of the random connectivity. The method is to determine the convergence of the double empirical measure (this yields population density equations for the joint distribution of the spins and fields). An additional advantage to utilizing the double empirical measure is that it yields a means of obtaining accurate finite-dimensional approximations to the dynamics.

1 Introduction

This paper concerns the high-dimensional dynamics of asymmetric random neural networks of the form, for $j\in I_{N}=\{1,2,\ldots,N\}$ .

dx^{j}_{t}=\big{(}-x^{j}_{t}/\tau+\beta N^{-1/2}\sum_{k=1}^{N}J^{jk}\lambda(x^{k}_{t})\big{)}dt+\sigma_{t}dW^{j}_{t},

(1)

where $\lambda$ is a Lipschitz function, $\tau$ is a constant, and $\{J^{jk}\}_{j,k\in I_{N}}$ are sampled independently from a centered normal distribution of variance $1$ , $\{W^{j}_{t}\}_{j\in I_{N}}$ are Brownian Motions. We study the convergence of the double empirical measure

\displaystyle N^{-1}\sum_{j\in I_{N}}\delta_{(\mathbf{z}^{j}_{[0,T]},\mathbf{G}^{j}_{[0,T]})},

(2)

where $G^{j}_{t}=N^{-1/2}\sum_{k=1}^{N}J^{jk}\lambda(x^{k}_{t})\big{)}$ . The dynamics of high-dimensional recurrent neural networks have many applications. They have been heavily applied to neuroscientific problems: many scholars think that they can be used to explain how the brain balances excitation and inhibition [7, 32, 19, 28, 12]. They have been used to study spatially-extended patterns in the brain [33, 34]. Most recently, it has been recognized that they are of fundamental importance to data science [6, 2, 22, 35]. For more applications, see the mongraph of Helias and Dahmen [26] and the recent survey in [13].

There exist limiting ‘correlation equations’ [14, 26] that describe the effective dynamics of high dimensional random neural networks. These constitute delayed integro-differential equations that have proven very difficult to analyze, particularly over short timescales. A related problem is that the correlation equations have only been determined from initial conditions that are independent of the connectivity. This means that they may not be accurate over longer timescales that diverge with $N$ . For example, many scholars are interested in understanding the nature of the limiting dynamics after the system attains a particular state (such as, if it enters an ‘energy well’ of specified characteristics, does it escape?). To address this question, one needs to start the dynamics at a particular point in the energy landscape of the connectivity (and therefore the initial condition is disorder-dependent).

The literature concerning large $N$ limiting equations for random neural networks has a complex history. Sompolinsky, Crisanti and Sommers anticipated that Path Integral methods would yield limiting dynamical equations [36] - the derivation was published in a later work [14]. We refer the reader to the excellent discussion in the monograph of Helias and Dahmen [26].

Path Integral methods (as practiced by physicists) yield population density equations by determining where the probability measure for the $N$ -dimensional system concentrates. In the probability literature, one of the most powerful means of addressing this question is the theory of Large Deviations [18]. Large Deviations theory was used to determine spin glass dynamics in the pioneering papers of Ben Arous and Guionnet [3, 5, 24]; they obtained the first rigorous results concerning the large $N$ limit of random neural networks. After this work, Grunwald employed Large Deviations theory to obtain correlation / response equations for random neural networks whose spins flip randomly between discrete states [23]. Moynot and Samuelides studied the non-Gaussian case [31]. Faugeras and MacLaurin extended the work of Ben Arous and Guionnet to include correlations in the connectivity [20]. Touboul and Cabana determined limiting equations for spatially-extended systems [10, 11]. Faugeras, Soret and Tanre [21] determined novel integral equations to describe the state of these systems.

On a related note, correlation-response equations for symmetric random neural networks were first derived by Crisanti, Horner Sommers [15] and Cugliandolo and Kurchan [16]. Ben Arous, Dembo and Guionnet [4] proved the accuracy of these correlation / response equations for symmetric random neural networks, employing concentration inequalities.

Broadly-speaking, this paper follows the approach of Ben Arous and Guionnet [3]. We employ the theory of Large Deviations to determine the large $N$ limit of the empirical measure. However the main novelties of our approach are:

•

We employ a general class of connectivity-dependent initial conditions. This unsurprisingly yields a different limiting dynamics as $N\to\infty$ . Connectivity-dependent initial conditions were employed in the papers of Ben Arous and Guionnet [5] (who studied dynamics started at the equilibrium distribution, in the high temperature regime) and Dembo and Subag [17].
•

We study the double empirical measure, that includes information about both the spins and the fields. This has several advantages: it facilitates finite-dimensional approximations to the dynamics that are very accurate, and it facilitates a broader class of disorder-dependent initial condition. For spin-glass dynamics, the Large Deviations of the double empirical measure was determined by Grunwald [23] for jump-Markov systems.
•

We include Replicas (i.e. $M$ copies of the system with the same connectivity, but independent Brownian Motions). This broadens the class of admissible disorder-dependent initial conditions.
•

The function $\lambda$ can be unbounded and the diffusion coefficient $\sigma_{t}$ can vary in time. The time-varying nature of $\sigma_{t}$ is essential for studying how periodic environmental noise in the brain shapes the dynamics of random neural networks.

Notation: Let $I_{N}=\{1,2,\ldots,N\}$ be the set of neuron indices. For any Polish space $\mathcal{X}$ , let $\mathcal{P}(\mathcal{X})$ denote all probability measures over $\mathcal{X}$ . The space $\mathcal{C}([0,T],\mathbb{R})$ is always endowed with the supremum topology (unless indicated otherwise), i.e.

\left\|x_{[0,T]}\right\|=\sup_{t\in[0,T]}|x_{t}|

For $\mathbf{y}\in\mathbb{R}^{N}$ , $\left\|\mathbf{y}\right\|$ is the Euclidean norm. For any probability measures $\mu$ and $\nu$ over a Polish Space, let $\mathcal{R}(\mu||\nu)$ denotes the relative entropy of measure $\mu$ with respect to $\nu$ . For any two measures on the same metric space with metric $d$ , $d_{W}(\cdot,\cdot)$ indicates the Wasserstein distance, i.e.

d_{W}(\mu,\nu)=\inf_{\zeta}\mathbb{E}^{\zeta}\big{[}d(x,y)\big{]},

(3)

where the infimum is taken over all $\zeta$ on the product space such that the marginal of the first variable is equal to $\mu$ and the marginal of the second variable is equal to $\nu$ . In the particular case that $\mu,\nu\in\mathcal{C}([0,T],\mathbb{R}^{M})$ , the distance is (unless otherwise indicated) $d(x,y)=\sup_{t\in[0,T]}\sup_{p\in I_{M}}\big{|}x^{p}_{t}-y^{p}_{t}\big{|}$ .

For any $\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$ , we write $\mu^{(1)},\mu^{(2)}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}$ to be the marginals over (respectively) the first $M$ variables and last $M$ variables.

2 Outline of Model and Main Results

We are going to rigorously determine the limiting dynamics of multiple replicas (with identical connections $\mathbf{J}$ , but with independent initial conditions and independent Brownian Motions). We let the superscript $a$ denote replica $a\in I_{M}=\{1,2,\ldots,M\}$ , and consider the system

	$\displaystyle dz^{a,j}_{t}=$	$\displaystyle\big{(}-z^{a,j}_{t}/\tau+G^{a,j}_{t}\big{)}dt+\sigma_{t}dW^{a,j}_{t}\text{ where }$		(4)
	$\displaystyle G^{a,j}_{t}=$	$\displaystyle N^{-1/2}\sum_{k\in I_{N}}J^{jk}\lambda(z^{a,k}_{t}).$		(5)

We assume that $\lambda\in\mathcal{C}^{2}(\mathbb{R})$ : this means in particular that there is a constant $C_{\lambda}$ such that $|\lambda(x)-\lambda(y)|\leq C_{\lambda}|x-y|$ . The noise intensity $t\to\sigma_{t}$ is taken to be continuous and non-random, and such that for constants $\underline{\sigma}$ and $\bar{\sigma}$ ,

0<\underline{\sigma}\leq\sigma_{t}\leq\bar{\sigma}.

(6)

Our major motivation for time-varying diffusivity lies in neuroscience: often synaptic noise exhibits particular rhythms. It has been of major interest how these rhythms shape pattern formation [8].

The connectivities $\{J^{jk}\}$ are taken to be independent centered Gaussian variables, with variance

\mathbb{E}\big{[}J^{jk}J^{lm}\big{]}=\delta(j,l)\delta(k,m).

Let $\gamma^{N}\in\mathcal{P}\big{(}\mathbb{R}^{N^{2}}\big{)}$ be their joint probability law. There are two cases for the initial conditions $\{z^{a,j}_{0}\}_{j\in I_{N},a\in I_{M}}$ that are considered in this paper.

2.1 Assumptions on the Initial Conditions

2.1.1 Case 1: Connectivity-Dependent Initial Conditions

The probability law of the initial conditions is assumed to be such that for any measurable set $\mathcal{A}\subset\mathbb{R}^{MN}$ ,

\displaystyle\mathbb{P}\big{(}\mathbf{Z}_{0}\in\mathcal{A}\big{)}=\int_{\mathcal{A}}\rho^{N}_{\mathbf{J}}(\mathbf{x})d\mathbf{x},

(7)

where the probability density $\rho^{N}_{\mathbf{J}}:\mathbb{R}^{MN}\to\mathbb{R}^{+}$ is defined as follows.

Let $\mu^{N}$ be the uniform Lebesgue Measure over the set $\mathbb{R}^{MN}$ . Conditionally on a realization $\mathbf{J}$ of the random connections, let the probability density $\rho^{N}_{\mathbf{J}}:\mathbb{R}^{MN}\to\mathbb{R}^{+}$ be such that for $\delta_{N}>0$ (with $\delta_{1}=1$ and $\delta_{N}$ decreasing to $0$ as $N\to\infty$ ), there exists some $\kappa\in\mathcal{P}\big{(}\mathbb{R}^{2M}\big{)}$ such that

$\displaystyle\rho^{N}_{\mathbf{J}}(\mathbf{g})=$	$\displaystyle\chi\big{\{}d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{G}_{0}),\kappa\big{)}\leq\delta_{N}\big{\}}/Z^{N}_{\mathbf{J}}\text{ where }$	(8)
$\displaystyle Z^{N}_{\mathbf{J}}=$	$\displaystyle\mu^{N}\big{(}d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{G}_{0}),\kappa\big{)}\leq\delta_{N}\big{)}\text{ and }$	(9)
$\displaystyle\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{G}_{0})=$	$\displaystyle N^{-1}\sum_{j\in I_{N}}\delta_{\mathbf{z}^{j}_{0},\mathbf{G}^{j}_{0}}\in\mathcal{P}\big{(}\mathbb{R}^{2M}\big{)}$	(10)

Roughly speaking, we need to assume that as $N\to\infty$ , the law of $\mathbf{z}^{N}_{0}$ behaves like its annealed average. Its assumed that (i) $\kappa$ has a finite second moment in each of its variables, and (ii) we have the bound

\displaystyle\underset{N\to\infty}{\underline{\lim}}N^{-1}\log\mathbb{E}[Z^{N}_{\mathbf{J}}]>-\infty,

(11)

It is also assumed that for any $\epsilon>0$ ,

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\big{|}N^{-1}\log Z^{N}_{\mathbf{J}}-\mathbb{E}[Z^{N}_{\mathbf{J}}]\big{|}\geq\epsilon\big{)}<0.

(12)

Define $\mathfrak{V}_{0},\tilde{\mathfrak{V}}_{0}\in\mathbb{R}^{M\times M}$ to be the covariance matrix with entries, $p,q\in I_{M}$ ,

	$\displaystyle\mathfrak{V}_{0}^{pq}$	$\displaystyle=\mathbb{E}^{\kappa}\big{[}\lambda(z^{p}_{0})\lambda(z^{q}_{0})\big{]}$		(13)
	$\displaystyle\tilde{\mathfrak{V}}_{0}^{pq}$	$\displaystyle=\mathbb{E}^{\kappa}\big{[}z^{p}_{0}z^{q}_{0}\big{]}.$		(14)

It is also assumed that both $\mathfrak{V}_{0}$ and $\tilde{\mathfrak{V}}_{0}$ are invertible.

2.1.2 Case 2: Connectivity-Independent Initial Conditions

One can also assume that the initial conditions $(z^{j}_{0})_{j\in I_{N}}$ are (i) independent of the connectivity, and (ii) sampled independently from a $\mathbb{R}^{M}$ -valued probabilistic distribution of bounded variance. This distribution is written as $\hat{\kappa}\in\mathcal{P}(\mathbb{R}^{M})$ .

2.2 Main Result

Our main result is that the empirical measure converges to a fixed point of a mapping $\Phi:\mathcal{U}\to\mathcal{U}$ . Here $\mathcal{U}\subset\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$ is defined in (54), consisting of (i) a broad class of measures with nice regularity properties, and (ii) such that the empirical measure inhabits $\mathcal{U}$ with unit probability.

For any $\mu\in\mathcal{U}$ , in the case of connectivity-dependent initial conditions $\Phi(\mu)$ is specified as follows. It is defined to be the law of Gaussian random variables $\big{(}z^{p}_{t},G^{p}_{t}\big{)}_{p\in I_{M}\fatsemi t\in[0,T]}$ such that (i) $(z^{p}_{0},G^{p}_{0})_{p\in I_{M}}$ are distributed according to $\kappa$ , and (ii) conditionally on the initial conditions, $\big{(}G^{p}_{t}\big{)}_{p\in I_{M}\fatsemi t\in[0,T]}$ is a Gaussian system such that $\mathbb{E}[G^{p}_{s}]=\mathfrak{m}^{p}_{s}(\mu,\mathbf{G}_{0})$ and the conditional variance is

\displaystyle\mathbb{E}\big{[}\big{(}G^{p}_{s}-\mathfrak{m}^{p}_{s}(\mu,\mathbf{G}_{0})\big{)}\big{(}G^{q}_{t}-\mathfrak{m}^{q}_{t}(\mu,\mathbf{G}_{0})\big{)}\;|\;\mathbf{G}_{0},\mathbf{z}_{0}\big{]}=\mathfrak{W}^{\mu,pq}_{st}.

(15)

Here

$\displaystyle\mathfrak{W}^{\mu,pq}_{st}$	$\displaystyle=\sum_{a,b\in I_{M}}\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{s})\lambda(z^{a}_{0})\big{]}\mathbb{E}^{\mu}\big{[}\lambda(z^{q}_{t})\lambda(z^{b}_{0}\big{]}\big{(}\mathfrak{V}_{\mu,0}^{-1}\big{)}^{ab}$	(16)
$\displaystyle\mathfrak{V}_{\mu,0}^{pq}$	$\displaystyle=\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{0})\lambda(z^{q}_{0})\big{]}$	(17)
$\displaystyle\mathfrak{m}^{p}_{s}(\mu,\mathbf{g})$	$\displaystyle=\sum_{a,b\in I_{M}}\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{s})\lambda(z^{a}_{0}\big{]}\big{(}\mathfrak{V}_{\mu,0}^{-1}\big{)}^{ab}g^{b}$	(18)

Letting $\big{(}W^{p}_{[0,T]}\big{)}_{p\in I_{M}}$ be Brownian Motions that are independent of $\mathbf{G}^{\mu}$ , we define $(z^{p}_{t})_{p\in I_{M}\fatsemi t\in[0,T]}$ to be the strong solution to the stochastic differential equation

\displaystyle dz^{p}_{t}=\big{(}-\tau^{-1}z^{p}_{t}+G^{\mu,p}_{t}\big{)}dt+\sigma_{t}dW^{p}_{t}.

(19)

In the case of connectivity-independent initial conditions, $\Phi$ is defined as follows. One first defines $\big{(}G^{p}_{t}\big{)}_{p\in I_{M}\fatsemi t\in[0,T]}$ to be a centered Gaussian system such that

\mathbb{E}\big{[}G^{p}_{t}G^{q}_{s}\big{]}=\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{t})\lambda(z^{q}_{s})\big{]}.

$(z^{p}_{0})_{p\in I_{M}}$ is independent of $\big{(}G^{p}_{t}\big{)}_{p\in I_{M}\fatsemi t\in[0,T]}$ and distributed according to $\hat{\kappa}$ . For Brownian Motions $\big{(}W^{p}_{[0,T]}\big{)}_{p\in I_{M}}$ , that are independent of $\mathbf{G}^{\mu}$ , $z^{p}_{t}$ is the strong solution of (19).

Theorem 1.

The mapping $\Phi$ is well-defined for all $\mu\in\mathcal{U}$ . Furthermore there exists a unique probability measure $\xi\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$ such that with unit probability,

\lim_{N\to\infty}N^{-1}\sum_{j\in I_{N}}\delta_{(\mathbf{z}^{j}_{[0,T]},\mathbf{G}^{j}_{[0,T]})}=\xi.

(20)

$\xi$ is the unique measure such that $\Phi(\xi)=\xi$ . Furthermore,

\displaystyle\xi=\lim_{n\to\infty}\xi^{(n)},

(21)

where $\xi^{(n+1)}=\Phi(\xi^{(n)})$ and $\xi^{(1)}$ is any measure in $\mathcal{U}$ .

Remark. This theorem is useful because it also implies a means to efficiently determine the large $N$ limiting equations, through repeated application of the mapping $\Phi$ . Because the limiting system is Gaussian, one only needs to solve for its covariance matrix. See the discussion in Helias and Dahmen [26] for an alternative formulation of the limiting covariance function in terms of a PDE.

2.3 An Example System that Satisfies the Conditions of Section 2.1.1

We now outline a general system that satisfies the conditions of Section 2.1.1. Suppose that $\lambda:\mathbb{R}\to\mathbb{R}$ is an odd function. Define $\iota:\mathcal{P}\big{(}\mathbb{R}^{2}\times\mathbb{R}^{2}\big{)}\to\mathbb{R}^{+}$ to be

\displaystyle\iota(\mu)=\mathcal{R}(\mu||p).

(22)

Here $p\in\mathcal{P}\big{(}\mathbb{R}^{2}\times\mathbb{R}^{2}\big{)}$ is centered and Gaussian, the law of random variables $(z^{1}_{0},g^{1}_{0},z^{2}_{0},g^{2}_{0})$ that are such that $\mathbb{E}[(z^{p}_{0})^{2}]=1$ , $\mathbb{E}[z^{p}_{0}g^{q}_{0}]=0$ and $\mathbb{E}[g^{p}_{0}g^{q}_{0}]=\mathbb{E}[\lambda(z^{p}_{0})\lambda(z^{q}_{0})]$ . Let $\Xi:\mathcal{C}\big{(}\mathbb{R}^{2}\big{)}\to\mathbb{R}$ be bounded.

Lemma 2.

Suppose that there is a unique $\eta\in\mathcal{P}\big{(}\mathbb{R}^{2}\times\mathbb{R}^{2}\big{)}$ such that

\displaystyle\sup_{\mu\in\mathcal{P}\big{(}\mathbb{R}^{2}\times\mathbb{R}^{2}\big{)}}\big{\{}\mathbb{E}^{\mu}[\Xi(z^{1},g^{1})]-\iota(\mu)\big{\}}=\mathbb{E}^{\eta}[\Xi(z^{1},g^{1})]-\iota(\eta).

(23)

Suppose also that

\sup\big{\{}\mathbb{E}^{\mu}[\Xi(z^{1},g^{1})+\Xi(z^{2},g^{2})]-\iota(\mu)\;:\;\mu\in\mathcal{P}\big{(}\mathbb{R}^{2}\times\mathbb{R}^{2}\big{)}\text{ and }\mu^{(1)}=\eta^{(1)}\text{ and }\mu^{(2)}=\eta^{(1)}\big{\}}\\ =2\big{\{}\mathbb{E}^{\eta}[\Xi(z^{1},g^{1})]-\iota(\eta)\big{\}}.

(24)

Then the conditions of Section 2.1.1 are satisfied, substituting $M=1$ , $\kappa=\eta^{(1)}$ and $\delta_{N}=(\log N)^{-1}$ .

Proof.

We show that

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\big{\{}\log\mathbb{E}^{\gamma}\big{[}(Z^{N}_{\mathbf{J}})^{2}\big{]}-2\log\mathbb{E}^{\gamma}\big{[}Z^{N}_{\mathbf{J}}\big{]}\big{\}}=0.

(25)

It is immediate from (25) that (12) must be satisfied. Let $\{y^{p,j}_{0}\}_{j\in I_{N}\fatsemi 1\leq p\leq 2}$ be iid $\mathcal{N}(0,1)$ random variables. Form the empirical measure $\hat{\mu}^{N}_{0}:=N^{-1}\sum_{j\in I_{N}}\delta_{\mathbf{y}^{j},\mathbf{G}^{j}_{0}}\in\mathcal{P}\big{(}\mathbb{R}^{4}\big{)}$ .

Define

\displaystyle\mathcal{A}_{N}=\chi\bigg{\{}d_{W}\big{(}\eta^{(1)},N^{-1}\sum_{j\in I_{N}}\delta_{y^{1,j},g^{1,j}}\big{)}\leq(\log N)^{-1}\;,\;d_{W}\big{(}\eta^{(2)},N^{-1}\sum_{j\in I_{N}}\delta_{y^{2,j},g^{2,j}}\big{)}\leq(\log N)^{-1}\bigg{\}}.

We will first demonstrate that

\lim_{N\to\infty}N^{-1}\log\mathbb{E}\big{[}\mathcal{A}_{N}\exp\big{(}N\mathbb{E}^{\hat{\mu}^{N}}[\Xi(z^{1},g^{1})+\Xi(z^{2},g^{2})]\big{)}\big{]}=\\ \sup\big{\{}\mathbb{E}^{\mu}[\Xi(z^{1},g^{1})+\Xi(z^{2},g^{2})]-\iota(\mu)\;:\;\mu\in\mathcal{P}\big{(}\mathbb{R}^{2}\times\mathbb{R}^{2}\big{)}\text{ and }\mu^{(1)}=\eta^{(1)}\text{ and }\mu^{(2)}=\eta^{(1)}\big{\}}.

(26)

It is straightforward to prove that for any $a>0$ there exists a compact subset $\mathfrak{U}_{a}\subset\mathcal{P}\big{(}\mathbb{R}^{4}\big{)}$ such that

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}_{0}\notin\mathfrak{U}_{a}\big{)}\leq-a.

(27)

A simple Large Deviations estimate yields that, for any $\mu\in\mathcal{P}\big{(}\mathbb{R}^{4}\big{)}$ , and $\epsilon\ll 1$ ,

\displaystyle N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}_{0}\in B_{\epsilon}(\mu)\big{)}=-\iota(\mu)+O(\sqrt{\epsilon}).

(28)

(In fact this could also be proved using Corollary 11). Our choice that $\delta_{N}$ goes to zero sufficiently slowly (i.e. $\delta_{N}=(\log N)^{-1}$ ) ensures that the rate function in (28) dominates the asymptotic estimate of the probability as $N\to\infty$ . Thus discretizing $\mathfrak{U}_{a}$ into $\epsilon$ balls, and then taking $\epsilon\to 0^{+}$ , one obtains (26).

Now (26) also implies that

\displaystyle\lim_{N\to\infty}N^{-1}\log\mathbb{E}\big{[}\mathcal{A}_{N}\big{]}=-2\iota(\eta).

(29)

This holds because since $\mu\to\mathbb{E}^{\mu}[\Xi(y^{1},g^{1}+\Xi(y^{2},g^{2})]$ is continuous, it becomes almost constant as $N\to\infty$ , as the radius of $\mathcal{A}_{N}$ shrinks. (29) in turn implies (25). ∎

3 Proof Outline

The main goal of this paper is to prove Theorem 1 employing the theory of Large Deviations [18]. The method - similarly to the original work by Ben Arous and Guionnet [3] - is to (i) prove a Large Deviations Principle for the uncoupled system, and then (ii) perform an exponential change-of-measure using Girsanov’s Theorem to obtain the Large Deviations Principle for the coupled system, before (iii) proving that the rate function has a unique zero.

The three main differences between this paper and the early papers of Ben Arous and Guionnet is that we (i) study the convergence of the double empirical measure (2) (whereas Ben Arous and Guionnet study the convergence of the annealed empirical measure in their earlier papers [3]) (in the later works [25, 4] quenched asymptotics are determined) (ii) we employ disorder-dependent initial conditions and (iii) we employ replicas.

Our main focus is on proving Case 1 (i.e. the connectivity-dependent initial conditions). The proofs are broadly similar, however Case 1 is more difficult because it requires a uniform Large Deviations Principle for the conditioned probability laws.

3.1 Large Deviations of the Uncoupled System

We start by noting a Large Deviation Principle for the uncoupled system. Because we are employing general disorder-dependent initial conditions, we must determine a Large Deviations Principle for the conditioned probability law. To this end, we must first define the set $\mathcal{Y}^{N}$ of all ‘valid initial conditions’ (basically the set of all initial points such that the empirical measure at time $0$ is close to its limit). More precisely, we define $\mathcal{Y}^{N}\subset\mathbb{R}^{NM}\times\mathbb{R}^{NM}$ to be such that

\displaystyle\mathcal{Y}^{N}=\big{\{}(\mathbf{z}_{0},\mathbf{g}_{0})\;:\;d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{g}_{0}),\kappa\big{)}\leq\tilde{\delta}_{N}\big{\}},

(30)

where $\tilde{\delta}_{N}=\max\big{\{}\delta_{N},(\log N)^{-1}\big{\}}$ and $\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{g}_{0})=N^{-1}\sum_{j\in I_{N}}\delta_{\mathbf{z}^{j}_{0},\mathbf{g}^{j}_{0}}\in\mathcal{P}(\mathbb{R}^{2M})$ . Define the uncoupled dynamics,

\displaystyle y^{p,j}_{t}=z^{p,j}_{0}+\int_{0}^{t}\sigma_{s}dW_{s}^{p,j},

(31)

and let $P^{N}_{\mathbf{z}_{0}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$ be the law of $\{y^{j}_{[0,T]}\}_{j\in I_{N}}$ , conditioned on the values at time $0$ . Write

\displaystyle\tilde{G}^{p,j}_{t}=N^{-1/2}\sum_{k\in I_{N}}J^{jk}\lambda(y^{p,k}_{t}).

(32)

Define $\gamma^{N}_{\mathbf{g}_{0}}\in\mathcal{P}\big{(}\mathbb{R}^{N^{2}}\big{)}$ to be the law of the connections $\{J^{jk}\}_{j,k\in I_{N}}$ , conditioned on the event

\displaystyle\big{\{}\tilde{G}^{p,j}_{0}=g^{p,j}_{0}\text{ for each }j\in I_{N},\;p\in I_{M}\big{\}}.

(33)

We note that $\gamma^{N}_{\mathbf{g}_{0}}$ is Gaussian, but no longer of zero mean (in general). The mean of $\tilde{G}^{p,j}_{t}$ is a function of the empirical measure and $\mathbf{g}^{j}_{0}$ (explicit formulae are outlined further below). Let $Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\in\mathcal{P}\big{(}\mathbb{R}^{N^{2}}\times\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$ be the joint law of the uncoupled system, i.e.

Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}=\gamma^{N}_{\mathbf{g}_{0}}\otimes P^{N}_{\mathbf{z}_{0}}.

(34)

The first main result is a uniform Large Deviations Principle for the conditioned system.

Theorem 3.

Let $\mathcal{A},\mathcal{O}\in\mathcal{B}\big{(}\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\big{)}$ , such that $\mathcal{O}$ is open and $\mathcal{A}$ closed. Then

	$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}_{[0,T]})\in\mathcal{A}\big{)}$	$\displaystyle\leq-\inf_{\mu\in\mathcal{A}}\mathcal{I}(\mu)$		(35)
	$\displaystyle\underset{N\to\infty}{\underline{\lim}}\inf_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}_{[0,T]})\in\mathcal{O}\big{)}$	$\displaystyle\geq-\inf_{\mu\in\mathcal{O}}\mathcal{I}(\mu).$		(36)

Here the rate function $\mathcal{I}:\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\to\mathbb{R}^{+}$ is such that $\mathcal{I}(\mu)=\infty$ if $\mu\notin\mathcal{U}$ , else otherwise

\displaystyle\mathcal{I}(\mu)=\tilde{I}_{\mu}(\mu),

(37)

where, $\tilde{I}_{\mu}(\mu)$ is defined in (117). Furthermore $\mathcal{I}$ is lower-semi-continuous and has compact level sets.

We will prove Theorem 3 by locally freezing the dependence of the fields $\{\tilde{G}^{p,j}_{t}\}$ on the empirical measure. In order that we may do this, we must first define a regular subset $\mathcal{Q}_{\mathfrak{a}}$ (for a positive integer $\mathfrak{a}\gg 1$ ) which is such that (i) the empirical measure $\hat{\mu}^{N}(\mathbf{y})=N^{-1}\sum_{j\in I_{N}}\delta_{\mathbf{y}^{j}_{[0,T]}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}$ inhabits with high probability, and (ii) there exist uniform bounds on the fluctuations in time. To this end, writing $\mathcal{K}_{\mathfrak{a}}$ to be the compact set specified in Lemma 24, define the set

\mathcal{Q}_{\mathfrak{a}}=\bigg{\{}\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}\;:\;\mu\in\mathcal{K}_{\mathfrak{a}}\text{ and }\sup_{p\in I_{M}}\mathbb{E}^{\mu}[\sup_{t\in[0,T]}(y^{p}_{t})^{2}\big{]}\leq\mathfrak{a}\text{ and }\\ \text{ For all integers }m\geq\mathfrak{a}\text{ it holds that }\sup_{0\leq i\leq m}\mathbb{E}^{\mu}\big{[}\sup_{p\in I_{M}}(w^{p}_{t^{(m)}_{i+1}}-w^{p}_{t^{(m)}_{i}})^{2}\big{]}\leq\Delta_{m}^{1/4}\bigg{\}}

(38)

where $\Delta_{m}=T/m$ and $t^{(m)}_{i}=iT/m$ . Write

\displaystyle\mathfrak{Q}=\bigcup_{\mathfrak{a}\geq 1}\mathfrak{Q}_{\mathfrak{a}}.

(39)

Lemma 4.

For any $L>0$ , there exists $a>0$ such that

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{Q}_{a}\big{)}\leq-L

(40)

The above lemma is proved in the Appendix. Next, for any $\nu\in\mathcal{Q}$ , we define a centered Gaussian law $\beta_{\nu}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R})^{M}\big{)}$ as follows. We stipulate that $\beta_{\nu}$ is the law of Gaussian random variables $\{G^{\nu,p}_{t}\}_{t\in[0,T],p\in I_{M}}$ with covariance structure

\displaystyle\mathbb{E}^{\beta_{\nu}}\big{[}G^{\nu,p}_{s}G^{\nu,q}_{t}\big{]}=\mathbb{E}^{\nu}\big{[}\lambda(x^{p}_{s})\lambda(x^{q}_{t})\big{]}

(41)

This definition will be useful because for any $j\in I_{N}$ , the law of $\tilde{G}^{j}_{[0,T]}$ under $\gamma^{N}$ is $\beta_{\hat{\mu}^{N}(\mathbf{y})}$ . In the following Lemma we collect some regularity estimates for the Gaussian Law $\beta_{\nu}$ .

Lemma 5.

(i) $\beta_{\nu}$ is a well-defined Gaussian probability law. (ii) Furthermore, the map $t\to G^{\nu,p}_{t}$ is ‘uniformly continuous’ for all measures in $\mathcal{U}_{a}$ , in the following sense. For any $a>0$ , and any $\epsilon>0$ , there exists $\delta(a,\epsilon)$ such that for all $\nu\in\mathcal{U}_{a}$ ,

\displaystyle\sup_{\nu\in\mathcal{U}_{a}}\sup_{p\in I_{M}}\mathbb{E}^{\beta_{\nu}}\bigg{[}\sup_{s,t\in[0,T]\fatsemi|s-t|\leq\delta(a,\epsilon)}\big{|}G^{\nu,p}_{s}-G^{\nu,p}_{t}\big{|}\bigg{]}\leq\epsilon

(42)

In order that we may make sense of the disorder-dependent initial condition, we also require an understanding of the distribution of $\beta_{\nu}$ , conditioned on the value of $\tilde{G}_{0}$ . To this end, for any $\nu\in\mathcal{U}$ and any $\mathbf{g}\in\mathbb{R}^{M}$ , let $\beta_{\nu,\mathbf{g}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}$ be the probability law $\beta_{\nu}$ conditioned on the event $\mathbf{G}^{\nu}_{0}=\mathbf{g}$ . Standard Gaussian identities [29] imply that $\beta_{\nu,\mathbf{g}}$ is also Gaussian, with the following mean and variance,

	$\displaystyle\mathbb{E}^{\beta_{\nu,\mathbf{g}}}\big{[}G^{\nu,p}_{s}\big{]}=$	$\displaystyle\mathfrak{m}^{p}_{s}(\nu,\mathbf{g})$		(43)
	$\displaystyle\mathbb{E}^{\beta_{\nu,\mathbf{g}}}\big{[}\big{(}G^{\nu,p}_{s}-\mathfrak{m}^{p}_{s}(\nu,\mathbf{g})\big{)}\big{(}G^{\nu,q}_{t}-\mathfrak{m}^{q}_{t}(\nu,\mathbf{g}\big{)}\big{]}=$	$\displaystyle\mathfrak{W}^{\nu,pq}_{st},$		(44)

where

$\displaystyle\mathfrak{W}^{\mu,pq}_{st}$	$\displaystyle=\sum_{a,b\in I_{M}}\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{s})\lambda(z^{a}_{0})\big{]}\mathbb{E}^{\mu}\big{[}\lambda(z^{q}_{t})\lambda(z^{b}_{0}\big{]}\big{(}\mathfrak{V}_{\mu,0}^{-1}\big{)}^{ab}$	(45)
$\displaystyle\mathfrak{m}^{p}_{s}(\mu,\mathbf{g})$	$\displaystyle=\sum_{a,b\in I_{M}}\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{s})\lambda(z^{a}_{0}\big{]}\big{(}\mathfrak{V}_{\mu,0}^{-1}\big{)}^{ab}g^{b}$	(46)
$\displaystyle\mathfrak{V}_{\mu,0}^{pq}$	$\displaystyle=\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{0})\lambda(z^{q}_{0})\big{]}.$	(47)

Corollary 6.

(i) For any $a>0$ , and any $\epsilon>0$ , there exists $\tilde{\delta}(a,\epsilon)$ such that for all $\nu\in\mathcal{U}_{a}$ , and all $\mathbf{g}\in\mathbb{R}^{M}$ ,

\displaystyle\sup_{\nu\in\mathcal{U}_{a}}\sup_{p\in I_{M}}\mathbb{E}^{\beta_{\nu,\mathbf{g}}}\bigg{[}\sup_{s,t\in[0,T]\fatsemi|s-t|\leq\tilde{\delta}(a,\epsilon)}\big{|}G^{\nu,p}_{s}-\mathfrak{m}^{p}_{s}(\nu,\mathbf{g})-G^{\nu,p}_{t}+\mathfrak{m}^{p}_{t}(\nu,\mathbf{g})\big{|}\bigg{]}\leq\epsilon

(48)

(ii) For any $\epsilon,a>0$ , there exists a compact set $\mathcal{C}_{\epsilon,a}\subset\mathcal{C}([0,T],\mathbb{R}^{M})$ such that for all $\nu\in\mathcal{Q}_{a}$ ,

\displaystyle\beta_{\nu}(\mathcal{C}_{\epsilon,a})\geq 1-\epsilon,

(49)

and for all $\mathbf{g}_{0}\in\mathbb{R}^{M}$ such that $\left\|\mathbf{g}_{0}\right\|\leq a$ ,

\displaystyle\beta_{\nu,\mathbf{g}_{0}}(\mathcal{C}_{\epsilon,a})\geq 1-\epsilon.

(50)

(iii) For $1\leq j\leq N$ and any $\mathbf{g}_{0}\in\mathbb{R}^{M}$ , the law of $\tilde{G}^{j}_{[0,T]}$ under $\gamma^{N}_{\mathbf{g}_{0}}$ is identical to $\beta_{\hat{\mu}^{N}(\mathbf{y}),\mathbf{g}_{0}}$ .

3.2 Exponential Tightness

To prove a Large Deviation Principle, one requires that the empirical measure inhabits a compact set with arbitrarily high probability. For any $\mathbf{y}\in\mathcal{C}([0,T],\mathbb{R}^{M})^{N}$ , write $\tilde{\gamma}^{N}_{\mathbf{y}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$ to be the law of the random variables $(\tilde{G}^{p,j}_{t})_{j\in I_{N}\fatsemi p\in I_{M}\fatsemi t\in[0,T]}$ , and write $\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}$ to be $\tilde{\gamma}^{N}_{\mathbf{y}}$ conditioned on the event in (33).

The following lemmas are needed for this proof.

Lemma 7.

For any $L>0$ , there exists a compact set $\tilde{\mathcal{C}}_{L}\subset\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}$ such that the following holds. For any $N\geq 1$ , and any $\{\mathbf{y}^{j}_{[0,T]}\}_{j\in I_{N}}$ such that $\hat{\mu}^{N}(\mathbf{y})\in\mathcal{Q}_{L}$ ,

\displaystyle N^{-1}\log\tilde{\gamma}^{N}_{\mathbf{y}}\big{(}\hat{\mu}^{N}(\tilde{\mathbf{G}})\notin\tilde{\mathcal{C}}_{L}\big{)}\leq-L.

(51)

Also, as long as $(\mathbf{y}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$ ,

\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{y}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}(\tilde{\mathbf{G}})\notin\tilde{\mathcal{C}}_{L}\big{)}\leq-L.

(52)

For $\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R})^{M}\times\mathcal{C}([0,T],\mathbb{R})^{M}\big{)}$ , write $\mu^{(1)}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R})^{M}\big{)}$ to be the marginal of $\mu$ over its first $M$ variables, and $\mu^{(2)}$ to be the marginal of $\mu$ over its last $M$ variables. Next, define the set

\mathcal{U}_{a}=\bigg{\{}\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R})^{M}\times\mathcal{C}([0,T],\mathbb{R})^{M}\big{)}\;:\;\mu^{(1)}\in\mathcal{Q}_{a},\;\mu^{(2)}\in\tilde{\mathcal{C}}_{a}\text{ and }\\ \sup_{t\in[0,T]}\sup_{p\in I_{M}}\mathbb{E}^{\mu}[(G^{p}_{t})^{2}]\leq C_{\lambda}^{2}a,\;\text{for all }0\leq s,t\leq T,\;\sup_{p\in I_{M}}\mathbb{E}^{\mu}[(G^{p}_{t}-G^{p}_{s})^{2}]\leq aC_{\lambda}^{2}|t-s|^{1/2}\bigg{\}},

(53)

and let

\displaystyle\mathcal{U}=\bigcup_{a\geq 0}\mathcal{U}_{a}.

(54)

It follows immediately from the above definition that $d_{W}(\mu,\nu)<\infty$ for any $\mu,\nu\in\mathcal{U}$ . We can now prove an ‘exponential tightness’ result.

Lemma 8.

For any $a\geq 0$ , $\mathcal{U}_{a}$ is compact. For any $L>0$ , there exists $a>0$ such that

\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\notin\mathcal{U}_{a}\big{)}\leq-L.

(55)

Proof.

Since the sets $\mathcal{Q}_{a}$ and $\tilde{\mathcal{C}}_{a}$ are compact, this follows almost immediately from Lemmas 4 and 7. ∎

3.3 The Coupled System (with connectivity-dependent initial conditions)

Define $\mathcal{J}:\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\to\mathbb{R}$ to be such that $\mathcal{J}(\mu)=\infty$ if $\mu\notin\mathcal{U}$ , or if the marginal of $\mu$ at time $0$ is not $\kappa$ . Else otherwise, for any $\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$ and $\mathbf{z}_{0},\mathbf{g}_{0}\in\mathbb{R}^{M}$ , writing $\mu_{\mathbf{z}_{0},\mathbf{g}_{0}}$ for $\mu$ conditioned on the values of its variables at time $0$ , define

\displaystyle\mathcal{J}(\mu)=\mathbb{E}^{\kappa}\bigg{[}\mathcal{R}\big{(}\mu_{\mathbf{z}_{0},\mathbf{g}_{0}}||S_{\mu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{)}\bigg{]}

(56)

Here $S_{\mu,\mathbf{z}_{0},\mathbf{g}_{0}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$ is defined to be the probability law of $(\mathbf{z},\mathbf{G}^{\mu})$ , where $\mathbf{G}^{\mu}$ is distributed according to $\beta_{\mu,\mathbf{g}_{0}}$ , and for Brownian Motions $\big{(}W^{p}_{[0,T]}\big{)}_{p\in I_{M}}$ that are independent of $\mathbf{G}^{\nu}$

\displaystyle dz^{p}_{t}=\big{(}-\tau^{-1}z^{p}_{t}+G^{\mu,p}_{t}\big{)}dt+\sigma_{t}dW^{p}_{t}

(57)

Define $\Phi:\mathcal{U}\to\mathcal{U}$ to be the following map. For some $\mu\in\mathcal{U}$ , write $\Phi(\mu)$ to be the law of the following random variables $(\mathbf{x},\mathbf{h})$ . First, it is stipulated that $(\mathbf{x}_{0},\mathbf{h}_{0})$ have probability law $\kappa$ . Second, conditionally on $(\mathbf{x}_{0},\mathbf{h}_{0})$ , the distribution of $(\mathbf{x}_{[0,T]},\mathbf{h}_{[0,T]})$ is given by $S_{\mu,\mathbf{x}_{0},\mathbf{g}_{0}}$ .

Lemma 9.

The probability law $S_{\mu,\mathbf{z}_{0},\mathbf{g}_{0}}$ is well-defined. Furthermore, there exists a unique zero $\xi$ of the rate function $\mathcal{J}$ . $\xi$ is the unique measure in $\mathcal{U}$ such that $\Phi(\xi)=\xi$ .

Proof.

We have already proved in Lemma 5 that $\big{(}G^{\mu,p}_{[0,T]}\big{)}_{p\in I_{M}}$ is well-defined. It is straightforward to check that for any path $\big{(}G^{\mu,p}_{[0,T]}\big{)}_{p\in I_{M}}$ , there exists a unique strong solution to the stochastic differential equation (57). Thus the probability law is well-defined.

It is well-known that the Relative Entropy is zero if and only if its two arguments are identical [9]. Thus, from the form of $\mathcal{J}$ in (56), any zero must be a fixed point of $\Phi$ . Furthermore, there must exist at least one zero of the rate function (if not, the total probability mass could not be one as $N\to\infty$ ). The uniqueness of the zero is proved in Lemma 22. ∎

Theorem 10.

For any $\epsilon>0$ ,

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}d_{W}(\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\xi)\geq\epsilon\big{)}<0.

(58)

Thus with unit probability,

\lim_{N\to\infty}\hat{\mu}^{N}(\mathbf{z},\mathbf{G})=\xi.

(59)

Furthermore,

\displaystyle\xi=\lim_{n\to\infty}\xi^{(n)},

(60)

where $\xi^{(n+1)}=\Phi(\xi^{(n)})$ and $\xi^{(1)}$ is any measure in $\mathcal{U}$ .

3.4 Connectivity-Independent Initial Conditions

The above reasoning can be adapted to prove a Large Deviation Principle for the unconditioned system. This is needed for proving the main theorem for Case 2 (connectivity-independent initial conditions). Write $Q^{N}=\gamma^{N}\otimes P^{N}$ to be the law of the random variables $(\mathbf{y},\mathbf{G})$ (with no conditioning), and for any $\nu\in\mathfrak{Q}$ , define $S_{\nu}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$ to be $S_{\nu}=P\otimes\beta_{\nu}$ . In the following corollary to Theorem 3, we prove the Large Deviation Principle for the unconditioned and uncoupled system.

Corollary 11.

Let $\mathcal{A},\mathcal{O}\in\mathcal{B}\big{(}\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\big{)}$ , such that $\mathcal{O}$ is open and $\mathcal{A}$ closed. Then

	$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log Q^{N}\big{(}\hat{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}_{[0,T]})\in\mathcal{A}\big{)}$	$\displaystyle\leq-\inf_{\mu\in\mathcal{A}}\mathcal{R}\big{(}\mu\|\|S_{\mu^{(1)}}\big{)}$		(61)
	$\displaystyle\underset{N\to\infty}{\underline{\lim}}N^{-1}\log Q^{N}\big{(}\hat{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}_{[0,T]})\in\mathcal{O}\big{)}$	$\displaystyle\geq-\inf_{\mu\in\mathcal{O}}\mathcal{R}\big{(}\mu\|\|S_{\mu^{(1)}}\big{)}.$		(62)

Here the rate function $\mu\to\mathcal{R}\big{(}\mu||S_{\mu^{(1)}}\big{)}$ is lower semi-continuous and has compact level sets.

We now specify the operator $\tilde{\Phi}:\mathcal{U}\to\mathcal{U}$ . Fix $\mu\in\mathcal{U}$ and defined $\tilde{\Phi}(\mu)$ to be the law of processes $\big{(}z^{p}_{[0,T]},G^{p}_{[0,T]}\big{)}_{p\in I_{M}\fatsemi t\in[0,T]}$ . One first defines $\big{(}G^{p}_{t}\big{)}_{p\in I_{M}\fatsemi t\in[0,T]}$ to be centered Gaussian system such that

\mathbb{E}\big{[}G^{p}_{t}G^{q}_{s}\big{]}=\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{t})\lambda(z^{q}_{s})\big{]}.

$(z^{p}_{0})_{p\in I_{M}}$ is independent of $\big{(}G^{p}_{t}\big{)}_{p\in I_{M}\fatsemi t\in[0,T]}$ and distributed according to $\hat{\kappa}$ . Letting $\big{(}W^{p}_{[0,T]}\big{)}_{p\in I_{M}}$ be Brownian Motions that are independent of $\mathbf{G}^{\mu}$ , we define $(z^{p}_{t})_{p\in I_{M}\fatsemi t\in[0,T]}$ to be the strong solution to the stochastic differential equation

\displaystyle dz^{p}_{t}=\big{(}-\tau^{-1}z^{p}_{t}+G^{\mu,p}_{t}\big{)}dt+\sigma_{t}dW^{p}_{t}.

(63)

Theorem 12.

Assume the connectivity-independent initial conditions (Case 2). For any $\epsilon>0$ ,

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}d_{W}(\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\xi)\geq\epsilon\big{)}<0.

(64)

Thus with unit probability,

\lim_{N\to\infty}\hat{\mu}^{N}(\mathbf{z},\mathbf{G})=\xi.

(65)

Furthermore,

\displaystyle\xi=\lim_{n\to\infty}\xi^{(n)},

(66)

where $\xi^{(n+1)}=\tilde{\Phi}(\xi^{(n)})$ and $\xi^{(1)}$ is any measure in $\mathcal{U}$ .

4 Proofs

We have divided the proofs into three main sections. In Section 4.1, we prove general regularity properties of the stochastic processes. In Section 4.2, we prove the LDP for the uncoupled system. In Section 4.3, we determine the limiting dynamics of the coupled system.

4.1 Regularity Estimates and Compactness

We first prove Lemma 5.

Proof.

We first check that the covariance function is positive definite (when restricted to a finite set of times). Let $\{t_{i}\}_{1\leq i\leq m}\subset[0,T]$ be a finite set of times. Then evidently for any constants $\{\alpha^{p}_{i}\}_{p\in I_{M},1\leq i\leq m}$ , it must be that

\displaystyle\sum_{p,q\in I_{M}}\sum_{1\leq i,j\leq m}\alpha^{p}_{i}\alpha^{q}_{j}\mathbb{E}^{\nu}\big{[}\lambda\big{(}x^{p}_{t_{i}}\big{)}\lambda\big{(}x^{q}_{t_{j}}\big{)}\big{]}=\mathbb{E}^{\nu}\big{[}\big{(}\sum_{p\in I_{M}}\sum_{1\leq i\leq m}\alpha^{p}_{i}\lambda\big{(}x^{p}_{t_{i}}\big{)}\big{)}^{2}\big{]}\geq 0.

(67)

This means that there exists a finite set of centered Gaussian variables $\{G^{\nu,p}_{t^{(m)}_{i}}\}_{p\in I_{M}\fatsemi 1\leq i\leq m}$ such that (41) holds. It then follows from the Komolgorov Extension Theorem that $\beta_{\nu}$ is well-defined on any countably dense subset of times of $[0,T]$ . It remains for us to demonstrate continuity, i.e. that there exists a Gaussian probability law such that (41) holds for all time. We do this using standard theory for the continuity of Gaussian Processes (following Chapter 2 of [1]).

First, we notice that

\displaystyle\sup_{p\in I_{M}}\sup_{t\in[0,T]}\mathbb{E}\big{[}(G^{\nu,p}_{t})^{2}\big{]}<\infty.

(68)

Now define the canonical metric,

	$\displaystyle\bar{d}_{p}(s,t)=$	$\displaystyle\mathbb{E}\big{[}\big{(}G^{\nu,p}_{s}-G^{\nu,p}_{t}\big{)}^{2}\big{]}^{\frac{1}{2}}=\mathbb{E}^{\nu}\big{[}\big{(}\lambda(x^{p}_{s})-\lambda(x^{p}_{t})\big{)}^{2}\big{]}^{\frac{1}{2}}$		(69)
	$\displaystyle\leq$	$\displaystyle\rm{Const}\sup_{p\in I_{M}}\mathbb{E}^{\nu}\big{[}\big{\|}x^{p}_{s}-x^{p}_{t}\big{\|}^{2}\big{]}^{\frac{1}{2}}\leq a\;(t-s)^{\frac{1}{4}}$		(70)

thanks to properties of the set $\mathcal{Q}_{a}$ , for all $s,t$ such that $|s-t|$ is smaller than some constant depending on $a$ . It follows from Theorem 1.4.1 of [1] that the Gaussian Process is almost-surely continuous.

Write $B_{t}(\epsilon)=\big{\{}s\in[0,T]:\bar{d}(s,t)\leq\epsilon\big{\}}$ to be the $\epsilon$ -ball about $t$ , and let $\mathcal{N}(\epsilon)$ denote the smallest number of such balls that cover $T$ . We see that there exists a constant $\mathfrak{c}_{a}>0$ such that

\displaystyle\mathcal{N}(\epsilon)\leq\mathfrak{c}_{a}\epsilon^{-4}.

(71)

Writing $H(\epsilon)=\log\mathcal{N}(\epsilon)$ , it follows from Theorem 1.3.5 in [1] that there exist $M$ Gaussian Processes $(G^{\nu,p}_{t})_{t\in[0,T]}$ such that $t\to G^{\nu,p}_{t}$ is almost-surely continuous, and there exists a universal constant $\mathfrak{K}>0$ and a random $\eta>0$ such that for all $\delta<\eta$ ,

	$\displaystyle\sup_{p\in I_{M}\fatsemi s,t\leq T\fatsemi\bar{d}(s,t)\leq\delta}\big{\|}G^{\nu,p}_{s}-G^{\nu,p}_{t}\big{\|}$	$\displaystyle\leq\mathfrak{K}\int_{0}^{\delta}H^{1/2}(\epsilon)d\epsilon$		(72)
		$\displaystyle\leq\mathfrak{K}\int_{0}^{\delta}\big{(}4\log\big{(}\epsilon^{-1}\big{)}+\log\mathfrak{c}_{a}\big{)}^{\frac{1}{2}}d\epsilon,$		(73)

and we note that the above goes to 0 as $\delta\to 0^{+}$ . This also implies (42). ∎

We next prove Corollary 6.

Proof.

The proof of (48) is analogous to the proof of (42). Notice that $\mathfrak{m}^{p}_{t}(\mu,\mathbf{g})$ depends continuously on $\mathbf{g}$ .

(42) and (48) imply (respectively) (49) and (50) . ∎

We can now prove Lemma 7.

Proof.

We prove (52) only. The other proof is very similar.

It follows from Lemma 5 that for any $\epsilon>0$ , there exists a compact set $\mathcal{C}_{\epsilon}\subset\mathcal{C}([0,T],\mathbb{R}^{M})$ such that for any $\mu\in\mathcal{Q}_{L}$ , and all $\mathbf{g}_{0}\in\mathbb{R}^{M}$ such that $\left\|\mathbf{g}_{0}\right\|\leq\epsilon^{-1}$ ,

\displaystyle\beta_{\mu,\mathbf{g}_{0}}\big{(}G^{\mu}_{[0,T]}\notin\mathcal{C}_{\epsilon}\big{)}\leq\epsilon.

(74)

It has already been noted above that for any $\{\mathbf{y}_{[0,T]}^{j}\}_{j\in I_{N}}\subset\mathcal{C}([0,T],\mathbb{R}^{M})$ , $\{\tilde{G}^{j}_{[0,T]}\}_{j\in I_{N}}$ are independent, and the probability law of $\tilde{G}^{j}_{[0,T]}$ is $\beta_{\hat{\mu}^{N}(\mathbf{y}),\mathbf{g}_{0}}$ . Thus as long as $\hat{\mu}^{N}(\mathbf{y})\in\mathcal{Q}_{L}$ , the estimate in (74) holds for any $\tilde{G}^{j}_{[0,T]}$ such that $\|\mathbf{g}^{j}_{0}\|\leq\epsilon^{-1}$ .

Define

\displaystyle\theta_{N,\epsilon}(\mathbf{g}_{0})=N^{-1}\sum_{j\in I_{N}}\chi\{\|\mathbf{g}^{j}_{0}\|>\epsilon^{-1}\}.

(75)

Our construction of $\mathcal{Y}^{N}$ implies that

\displaystyle\lim_{\epsilon\to 0^{+}}\lim_{N\to\infty}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}\theta_{N,\epsilon}(\mathbf{g}_{0})=0.

(76)

Write $I_{N,\epsilon}=\big{\{}j\;:\|\mathbf{g}^{j}_{0}\|\leq\epsilon^{-1}\big{\}}$ . Next, for some $\delta>\epsilon/2$ , and $b>0$ , by Chernoff’s Inequality, for any $N\geq 1$ and any $(\mathbf{y}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$ ,

	$\displaystyle\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}\bigg{(}N^{-1}\sum_{j\in I_{N}}\{G^{j}_{[0,T]}\notin\mathcal{C}_{\epsilon}\}\geq\delta\bigg{)}$	$\displaystyle\leq\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}\bigg{(}N^{-1}\sum_{j\in\tilde{I}_{N,\epsilon}}\{G^{j}_{[0,T]}\notin\mathcal{C}_{\epsilon}\}\geq\delta/2\bigg{)}$
		$\displaystyle\leq\mathbb{E}^{\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}}\bigg{[}\exp\bigg{(}b\sum_{j\in\tilde{I}_{N,\epsilon}}\{G^{j}_{[0,T]}\notin\mathcal{C}_{\epsilon}\}-bN\delta/2\bigg{)}\bigg{]}$
		$\displaystyle\leq\big{\{}\epsilon\exp(b)+1-\epsilon\big{\}}^{N}\exp\big{(}-Nb\delta/2\big{)}$

We thus find that (by taking small enough $\epsilon$ , and $1\ll b\ll-\log\epsilon$ ), for any integer $n$ , that there must exist a compact set $\mathcal{C}^{(n)}$ such that for all $N\geq 1$ , all $(\mathbf{y}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$ and all $\mathbf{y}$ such that $\hat{\mu}^{N}(\mathbf{y})\in\mathcal{Q}_{L}$

\displaystyle N^{-1}\log\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}\big{(}N^{-1}\sum_{j\in I_{N}}\{G^{j}_{[0,T]}\notin\mathcal{C}^{(n)}\}\geq n^{-1}\big{)}\leq-n^{2}.

(77)

This motivates us to define the compact set $\tilde{\mathcal{C}}_{L}\subset\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$ to consist of all measures $\mu$ such that for all $n\geq m$ ,

\displaystyle\mu\big{(}\mathcal{C}^{(n)}\big{)}\geq 1-n^{-1}.

(78)

Thus using a union-of-events bound,

$\displaystyle\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{G})\notin\tilde{\mathcal{C}}_{L}\big{)}$	$\displaystyle\leq\sum_{n\geq m}\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}\bigg{(}N^{-1}\sum_{j\in I_{N}}\{G^{j}_{[0,T]}\notin\mathcal{K}^{(n)}\}\geq n^{-1}\bigg{)}$	(79)
	$\displaystyle\leq\sum_{n\geq m}\exp\big{(}-Nn^{2}\big{)}$	(80)
	$\displaystyle\leq\exp(-NL),$	(81)

for all $N\geq 1$ , as long as $m$ is large enough.

∎

The following bound on the operator norm of the connectivity matrix is well-known (and the proof is omitted).

Lemma 13.

For any $L>0$ , there exists $\ell$ such that

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\geq\ell\big{)}\leq-L,

(82)

where $\mathcal{J}_{N}\in\mathbb{R}^{N\times N}$ has $(j,k)$ entry

\mathcal{J}_{N,jk}=N^{-1/2}J^{jk}

Lemma 14.

For any $\ell>0$ , there exists $L>0$ such that for all $p\in I_{M}$ and all $N\geq 1$ ,

\displaystyle N^{-1}\log\mathbb{P}\big{(}\mathcal{A}_{c}\;,\;\sup_{t\in[0,T]}\sum_{j\in I_{N}}(z^{p,j}_{t})^{2}\geq N\ell\big{)}\leq-L

(83)

where

\mathcal{A}_{c}=\bigg{\{}\left\|\mathcal{J}_{N}\right\|\leq c\;,\;\sup_{p\in I_{M}}\sum_{j\in I_{N}}(z^{p,j}_{0})^{2}\leq N\mathbb{E}^{\kappa}[(z^{p}_{0})^{2}]+N\bigg{\}}.

Proof.

Write

u_{t}=N^{-1}\sum_{j\in I_{N}}(z^{p,j}_{t})^{2}.

If the event $\mathcal{A}_{c}$ holds, then thanks to Ito’s Lemma it must be that

	$\displaystyle du_{t}=$	$\displaystyle\big{\{}-2\tau^{-1}u_{t}+1+N^{-1}\sum_{j\in I_{N}}z^{p,j}_{t}G^{p,j}_{t}\big{\}}dt+N^{-1}\sum_{j\in I_{N}}z^{p,j}_{t}dW^{p,j}_{t}$		(84)
	$\displaystyle\leq$	$\displaystyle\big{\{}-2\tau^{-1}u_{t}+1+cC_{\lambda}u_{t}\big{\}}dt+N^{-1}\sum_{j\in I_{N}}z^{p,j}_{t}dW^{p,j}_{t},$		(85)

since $N^{-1}\sum_{j\in I_{N}}\lambda(z^{p,j}_{t})^{2}\leq C_{\lambda}^{2}u_{t}$ . Write

\displaystyle v_{t}=\sup_{s\in[0,t]}N^{-1}\bigg{|}\sum_{j\in I_{N}}\int_{0}^{t}z^{p,j}_{s}dW^{p,j}_{s}\bigg{|},

(86)

and define the stopping time, for a constant $A>0$ ,

\displaystyle\tau_{A}=\inf\big{\{}t\geq 0\;:\;v_{t}\geq\exp(At)+A\big{\}}.

(87)

Gronwall’s Inequality implies that for all $t\leq\tau_{A}$ ,

\displaystyle u_{t}\leq\big{(}A+u_{0}+t\big{)}\exp\big{(}\tilde{c}t\big{)}

where $\tilde{c}=A+cC_{\lambda}-2\tau^{-1}$ . The quadratic variation of $x(t):=N^{-1}\sum_{j\in I_{N}}\int_{0}^{t}z^{p,j}_{s}dW^{p,j}_{s}$ is

\displaystyle(QV)_{t}^{N}=N^{-2}\sum_{j\in I_{N}}\int_{0}^{t}(z^{p,j}_{s})^{2}ds.

(88)

For all $t\leq\tau_{A}$ ,

\displaystyle(QV)_{t}^{N}

\displaystyle\leq N^{-1}\tilde{c}^{-1}\big{(}A+u_{0}+t\big{)}\exp\big{(}\tilde{c}t\big{)}:=N^{-1}h_{t},

(89)

and notice that $h_{t}$ is independent of the Brownian Motions. Now define the stochastic process $w(t)$ to be such that

	$\displaystyle w(t)=$	$\displaystyle x\big{(}\alpha^{N}_{t}\big{)}\text{ where }$		(90)
	$\displaystyle\alpha^{N}_{t}=$	$\displaystyle\inf\big{\{}s\geq 0\;:\;(QV)_{s}^{N}=t\big{\}}$		(91)

Thanks to the time-rescaled representation of a stochastic integral, $w(t)$ is a Brownian Motion [27]. Writing $f(t)=\exp(At)+A$ , it follows that

	$\displaystyle\mathbb{P}\bigg{(}\text{ There exists }$	$\displaystyle s\leq T\text{ such that }\big{\|}x(s)\big{\|}\geq f(s)\bigg{)}$
	$\displaystyle\leq$	$\displaystyle\mathbb{P}\bigg{(}\text{ There exists }s\leq T\text{ such that }\big{\|}w(N^{-1}h_{s})\big{\|}\geq f(s)\bigg{)}$
	$\displaystyle\leq$	$\displaystyle\mathbb{P}\bigg{(}\text{ There exists }s\leq T\text{ such that }\big{\|}w(N^{-1}h_{s^{(m)}})\big{\|}\geq f(s_{(m)})\bigg{)}$

and we have written

	$\displaystyle s^{(m)}$	$\displaystyle=\inf\big{\{}t^{(m)}_{a}\;:t^{(m)}_{a}\geq s\big{\}}$		(92)
	$\displaystyle s_{(m)}$	$\displaystyle=\sup\big{\{}t^{(m)}_{a}\;:t^{(m)}_{a}\leq s\big{\}}.$		(93)

and we recall that $t^{(m)}_{a}=Ta/m$ . Employing a union-of-events bound,

\mathbb{P}\bigg{(}\text{ There exists }s\leq T\text{ such that }\big{|}w(h_{s^{(m)}})\big{|}\geq f(s_{(m)})\bigg{)}\leq\\ \sum_{a=0}^{m-1}\bigg{\{}\mathbb{P}\bigg{(}w\big{(}N^{-1}h_{t_{a+1}^{(m)}}\big{)}\geq f\big{(}t_{a}^{(m)}\big{)}\bigg{)}+\mathbb{P}\bigg{(}w\big{(}N^{-1}h_{t_{a+1}^{(m)}}\big{)}\leq-f\big{(}t_{a}^{(m)}\big{)}\bigg{)}\bigg{\}}

(94)

Now since $w(t)$ is centered and Gaussian, with variance of $t$ ,

	$\displaystyle N^{-1}\log\mathbb{P}\bigg{(}w\big{(}N^{-1}h_{t_{a+1}^{(m)}}\big{)}\geq f\big{(}t_{a}^{(m)}\big{)}\bigg{)}=$	$\displaystyle-\frac{N}{2}f\big{(}t_{a}^{(m)}\big{)}^{2}\big{(}h_{t_{a+1}^{(m)}}\big{)}^{-1}+O\big{(}\log N\big{)}$		(95)
	$\displaystyle N^{-1}\log\mathbb{P}\bigg{(}w\big{(}N^{-1}h_{t_{a+1}^{(m)}}\big{)}\leq-f\big{(}t_{a}^{(m)}\big{)}\bigg{)}=$	$\displaystyle-\frac{N}{2}f\big{(}t_{a}^{(m)}\big{)}^{2}\big{(}h_{t_{a+1}^{(m)}}\big{)}^{-1}+O\big{(}\log N\big{)}.$		(96)

We fix $m=A$ , and take $A$ to be arbitrarily large. Then

\lim_{A\to\infty}\inf_{0\leq a\leq m-1}f\big{(}t_{a}^{(m)}\big{)}^{2}\big{(}h_{t_{a+1}^{(m)}}\big{)}^{-1}=\infty.

We thus find that, for large enough $A$ ,

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\bigg{(}\mathcal{A}_{c},\text{ There exists }s\leq T\text{ such that }\big{|}x(s)\big{|}\geq f(s)\bigg{)}\leq-L.

(97)

We have already demonstrated in the course of the proof that if the event $\mathcal{A}_{c}$ holds, and $\sup_{s\in[0,T]}|x(s)|\leq f(s)$ , then there exists a constant such that $\sup_{t\in[0,T]}u_{t}\leq\rm{Const}$ . We have thus established the Lemma. ∎

The following $L^{2}$ -Wasserstein distance provides a very useful way of controlling the dependence of the fields $(G^{\nu}_{t})$ on the measure $\nu$ . Define $d^{(2)}_{t}(\cdot,\cdot)$ to be such that for any $\mu,\nu\in\mathcal{U}$ ,

\displaystyle d^{(2)}_{t}(\mu,\nu)=\inf_{\zeta}\mathbb{E}^{\zeta}\bigg{[}\sum_{p\in I_{M}}\int_{0}^{t}\big{\{}(y^{p}_{s}-\tilde{y}^{p}_{s})^{2}+(G^{p}_{s}-\tilde{G}^{p}_{s})^{2}\big{\}}ds\bigg{]}^{1/2},

(98)

where the infimum is over all $\zeta\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{2M})\times\mathcal{C}([0,T],\mathbb{R}^{2M}\big{)}$ , such that the law of the first $2M$ processes is given by $\mu$ , and the law of the last $2M$ processes is given by $\nu$ . Let $d^{(2)}(\mu,\nu):=d^{(2)}_{T}(\mu,\nu)$ .

Lemma 15.

For any $a>0$ , $d^{(2)}(\cdot,\cdot)$ metrizes weak convergence in $\mathcal{U}_{a}$ . Furthermore,

\displaystyle\lim_{\epsilon\to 0^{+}}\sup\big{\{}d_{W}(\mu,\nu)\;:\mu,\nu\in\mathcal{U}_{a}\text{ and }d^{(2)}(\mu,\nu)\leq\epsilon\big{\}}=0.

(99)

Proof.

Since $\mathcal{U}_{a}$ is compact, Prokhorov’s Theorem implies that for any $\tilde{\epsilon}>0$ , there exists a compact set $\mathcal{D}_{\epsilon}\subset\mathcal{C}([0,T],\mathbb{R}^{M})^{2}$ such that for all $\mu\in\mathcal{U}_{a}$ ,

\displaystyle\mu\big{(}\mathcal{D}_{\epsilon}\big{)}\geq 1-\tilde{\epsilon}.

(100)

Since $\mathcal{D}_{\epsilon}$ is compact, it follows from the Arzela-Ascoli Theorem that for any $\delta>0$ , there exists $\upsilon(\epsilon,\delta)$ such that for all $f,g\in\mathcal{D}_{\epsilon}$ such that for all $p\in I_{2M}$ ,

\displaystyle\int_{0}^{T}(f^{p}(t)-g^{p}(t))^{2}dt\leq\upsilon(\epsilon,\delta),

(101)

it necessarily holds that

\displaystyle\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}f^{p}(t)-g^{p}(t)\big{|}\leq\delta.

(102)

Let $\zeta$ be any measure that is within $\eta\ll 1$ of realizing the infimum in (98). Then, writing

\mathcal{A}_{\epsilon}=\chi\big{\{}\text{ For each }p\in I_{M},\;\;y^{p},\tilde{y}^{p},g^{p},\tilde{g}^{p}\in\mathcal{D}_{\epsilon}\big{\}},

we have the bound

\mathbb{E}^{\zeta}\bigg{[}\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{]}\\ \leq\mathbb{E}^{\zeta}\bigg{[}\bigg{(}\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{)}\mathcal{A}_{\epsilon}\bigg{]}+\\ \mathbb{E}^{\zeta}\bigg{[}\bigg{(}\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{)}\big{(}1-\mathcal{A}_{\epsilon}\big{)}\bigg{]}

(103)

Now we take $d^{(2)}(\mu,\nu)\to 0^{+}$ , and $\eta\to 0^{+}$ too. Since $\mathcal{A}_{\epsilon}$ is closed, thanks to the Portmanteau Theorem, we thus find that for any $\epsilon>0$ ,

\displaystyle\mathbb{E}^{\zeta}\bigg{[}\mathcal{A}_{\epsilon}\sum_{p\in I_{M}}\int_{0}^{T}\big{\{}(y^{p}_{s}-\tilde{y}^{p}_{s})^{2}+(G^{p}_{s}-\tilde{G}^{p}_{s})^{2}\big{\}}ds\bigg{]}\to 0.

(104)

which in turn implies that (making use of the uniform convergence over $\mathcal{D}_{\epsilon}$ in (102))

\mathbb{E}^{\zeta}\bigg{[}\bigg{(}\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{)}\mathcal{A}_{\epsilon}\bigg{]}\to 0.

(105)

For the other term on the RHS of (103), for $b>0$ , write

\displaystyle\mathcal{B}_{b}=\chi\bigg{\{}\text{ For each }p\in I_{M},\;\sup_{t\in[0,T]}\big{|}y^{p}_{t}\big{|}\leq b\;,\;\sup_{t\in[0,T]}\big{|}\tilde{y}^{p}_{t}\big{|}\leq b\;,\;\sup_{t\in[0,T]}\big{|}g^{p}_{t}\big{|}\leq b\;,\;\sup_{t\in[0,T]}\big{|}\tilde{g}^{p}_{t}\big{|}\leq b\bigg{\}}

Then,

\mathbb{E}^{\zeta}\bigg{[}\bigg{(}\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{)}\big{(}1-\mathcal{A}_{\epsilon}\big{)}\bigg{]}\\ \leq\mathbb{E}^{\zeta}\bigg{[}\bigg{(}\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{)}\big{(}1-\mathcal{A}_{\epsilon}\big{)}\mathcal{B}_{b}\bigg{]}\\ +\mathbb{E}^{\zeta}\bigg{[}\bigg{(}\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{)}\big{(}1-\mathcal{A}_{\epsilon}\big{)}\big{(}1-\mathcal{B}_{b}\big{)}\bigg{]}.

(106)

Thanks to the fact that, for all $\mu\in\mathcal{U}_{a}$ ,

\sup_{p\in I_{M}}\mathbb{E}^{\mu}\big{[}\sup_{t\in[0,T]}(y^{p}_{t})^{2}\big{]}\leq a,

one finds that the second term on the RHS of (106) goes to $0$ as $b\to\infty$ , uniformly for all $\epsilon>0$ and all $\mu,\nu\in\mathcal{U}_{a}$ . For any fixed $b\gg 1$ , the first term on the RHS of (106) must go to zero as $\epsilon\to 0^{+}$ , thanks to (100). We have thus proved the Lemma. ∎

For $\mu,\in\mathfrak{Q}$ , we define $d^{(2)}_{t}(\mu,\nu)$ analogously to (98).

Lemma 16.

There exists a constant $\mathfrak{C}>0$ such that for all $\mu,\nu\in\mathfrak{Q}$ and all $t\in[0,T]$ ,

\displaystyle d^{(2)}_{t}(\beta_{\nu},\beta_{\mu})

\displaystyle\leq\mathfrak{C}d^{(2)}_{t}(\nu,\mu).

(107)

Also for all $\mu,\nu\in\mathfrak{Q}$ such that for some $b>0$ , $\det(\mathfrak{V}_{\mu,0}),\det(\mathfrak{V}_{\nu,0})\geq b>0$ , there exists a constant $\mathfrak{C}_{b}$ such that

\displaystyle d^{(2)}_{t}\big{(}\beta_{\nu,\mathbf{g}},\beta_{\mu,\mathbf{g}}\big{)}

\displaystyle\leq\tilde{\mathfrak{C}}_{b}(1+\left\|\mathbf{g}\right\|)d^{(2)}_{t}(\nu,\mu),

(108)

and $\left\|\cdot\right\|$ is the Euclidean norm on $\mathbb{R}^{M}$ .

Proof.

We start by proving (107). Recalling the definition of the distance $d^{(2)}_{t}$ in (98), let $\zeta_{\epsilon}$ be such that

\displaystyle d^{(2)}_{t}(\mu,\nu)^{2}\geq\mathbb{E}^{\zeta_{\epsilon}}\bigg{[}\sum_{p\in I_{M}}\int_{0}^{t}(z^{p}_{s}-y^{p}_{s})^{2}ds\bigg{]}+\epsilon.

(109)

Furthermore define centered Gaussian processes $G^{(\epsilon),\mu,p}_{s},G^{(\epsilon),\nu,p}_{s}$ to be such that for any $p,q\in I_{M}$ and any $s,t\in[0,T]$ ,

$\displaystyle\mathbb{E}\big{[}G^{(\epsilon),\mu,p}_{s}G^{(\epsilon),\nu,q}_{t}\big{]}$	$\displaystyle=\mathbb{E}^{\zeta_{\epsilon}}\big{[}\lambda(z^{p}_{s})\lambda(y^{q}_{t})\big{]}$	(110)
$\displaystyle\mathbb{E}\big{[}G^{(\epsilon),\mu,p}_{s}G^{(\epsilon),\mu,q}_{t}\big{]}$	$\displaystyle=\mathbb{E}^{\zeta_{\epsilon}}\big{[}\lambda(z^{p}_{s})\lambda(z^{q}_{t})\big{]}$	(111)
$\displaystyle\mathbb{E}\big{[}G^{(\epsilon),\nu,p}_{s}G^{(\epsilon),\nu,q}_{t}\big{]}$	$\displaystyle=\mathbb{E}^{\zeta_{\epsilon}}\big{[}\lambda(y^{p}_{s})\lambda(y^{q}_{t})\big{]}.$	(112)

This definition is possible thanks to a trivial modification of Lemma 5 (switching $M\to 2M$ ). We thus find that

$\displaystyle\lim_{\epsilon\to 0^{+}}\mathbb{E}\bigg{[}\sum_{p\in I_{M}}\int_{0}^{r}(G^{(\epsilon),\mu,p}_{t}-G^{(\epsilon),\nu,p}_{t})^{2}dt\bigg{]}\leq$	$\displaystyle\lim_{\epsilon\to 0^{+}}\mathbb{E}^{\zeta_{\epsilon}}\bigg{[}\sum_{p\in I_{M}}\int_{0}^{r}\big{\{}\lambda(z^{p}_{t})-\lambda(y^{p}_{t})\big{\}}^{2}dt\bigg{]}$	(113)
$\displaystyle\leq$	$\displaystyle\lim_{\epsilon\to 0^{+}}C_{\lambda}^{2}\mathbb{E}^{\zeta_{\epsilon}}\bigg{[}\sum_{p\in I_{M}}\int_{0}^{r}\big{\{}z^{p}_{t}-y^{p}_{t}\big{\}}^{2}dt\bigg{]}$	(114)
$\displaystyle=$	$\displaystyle C_{\lambda}^{2}d_{r}^{(2)}(\mu,\nu).$	(115)

Now as $\epsilon\to 0^{+}$ , the LHS of (113) must decrease to $d^{(2)}(\beta_{\mu},\beta_{\nu})$ . (108) follows analogously.

The proof of (108) is analogous, since the mean and variance functions in (45) and (46) are such that for all $\mu,\nu\in\mathcal{U}$ , there is a constant $C_{b}>0$ such that

	$\displaystyle\sup_{p,q\in I_{M}\fatsemi s.t\in[0,T]}\big{\|}\mathfrak{W}^{\mu,pq}_{st}-\mathfrak{W}^{\nu,pq}_{st}\big{\|}$	$\displaystyle\leq C_{b}\lim_{\epsilon\to 0^{+}}\sup_{r\in I_{M}\fatsemi s\in[0,T]}\mathbb{E}^{\zeta_{\epsilon}}\big{[}\big{(}\lambda(z^{r}_{s})-\lambda(y^{r}_{s})\big{)}^{2}\big{]}$
	$\displaystyle\sup_{p,q\in I_{M}\fatsemi s.t\in[0,T]}\big{\|}\mathfrak{m}^{p}_{s}(\mu,\mathbf{g})-\mathfrak{m}^{p}_{s}(\nu,\mathbf{g})\big{\|}$	$\displaystyle\leq C_{b}\big{(}1+\\|\mathbf{g}\\|\big{)}\lim_{\epsilon\to 0^{+}}\sup_{r\in I_{M}\fatsemi s\in[0,T]}\mathbb{E}^{\zeta_{\epsilon}}\big{[}\big{(}\lambda(z^{r}_{s})-\lambda(y^{r}_{s})\big{)}^{2}\big{]}$

We have also employed the fact that $\det(\mathfrak{V}_{\mu,0})$ is uniformly lower-bounded by a positive constant $b$ (as noted in the statement of the Lemma). ∎

4.2 Large Deviations of the Uncoupled System

Our first aim is to prove a Large Deviation Principle in the case of fields with a frozen interaction structure (in Lemma 17 below). This would ordinarily be a trivial application of Sanov’s Theorem. However the proof is slightly complicated by the need for the LDP to be uniform with respect to the variables $(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathbb{R}^{MN}\times\mathbb{R}^{MN}$ that the probability laws are conditioned on.

For any $\mathbf{g}_{0}\in\mathbb{R}^{MN}$ and $\nu\in\mathcal{U}$ , define $\tilde{\gamma}^{N}_{\nu,\mathbf{g}_{0}}:=\otimes_{j=1}^{N}\beta_{\nu,\mathbf{g}_{0}^{j}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$ . In other words, $\tilde{\gamma}^{N}_{\nu,\mathbf{g}_{0}}$ is the law of $N$ independent $\mathcal{C}([0,T],\mathbb{R}^{M})$ -valued Gaussian variables $\{\tilde{G}^{\nu,j}\}_{j\in I_{N}}$ . The mean and variance of these variables is specified in (43) and (44).

Let $Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\times\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$ be the joint law of the uncoupled system, i.e.

Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}=\gamma^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\otimes P^{N}_{\mathbf{z}_{0}}.

(116)

For $\nu\in\mathcal{U}$ , define $\tilde{I}_{\nu}:\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\to\mathbb{R}$ as follows. We specify that $\tilde{I}_{\nu}(\mu)=\infty$ if either the marginal of $\mu$ at time $0$ is not equal to $\kappa$ , and / or $\mu\notin\mathcal{U}$ . Otherwise, for any $\zeta\in\mathcal{U}$ , writing $\zeta_{\mathbf{z}_{0},\mathbf{g}_{0}}$ to be the law of $\zeta$ , conditioned on the values of its variables at time $0$ , define

\displaystyle\tilde{I}_{\nu}(\zeta)=\mathbb{E}^{\kappa}\big{[}\mathcal{R}\big{(}\zeta_{\mathbf{z}_{0},\mathbf{g}_{0}}||P_{\mathbf{z}_{0}}\otimes\tilde{\gamma}_{\nu,\mathbf{g}_{0}}\big{)}\big{]}.

(117)

Define the empirical measure $\tilde{\mu}^{N}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R})^{M}\times\mathcal{C}([0,T],\mathbb{R})^{M}\big{)}$ to be

\displaystyle\tilde{\mu}^{N}=N^{-1}\sum_{j\in I_{N}}\delta_{\mathbf{y}^{j}_{[0,T]},\tilde{G}^{\nu,j}_{[0,T]}},

(118)

where we recall that

\displaystyle y^{p}_{t}=z^{p}_{0}+\int_{0}^{t}\sigma_{s}dW_{s}^{p}.

(119)

Lemma 17.

Fix some $\nu\in\mathcal{U}$ . Let $\mathcal{A},\mathcal{O}\subseteq\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\big{)}$ , such that $\mathcal{O}$ is open and $\mathcal{A}$ closed. Then

	$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{A}\big{)}$	$\displaystyle\leq-\inf_{\mu\in\mathcal{A}}\tilde{I}_{\nu}(\mu)$		(120)
	$\displaystyle\underset{N\to\infty}{\underline{\lim}}\inf_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{O}\big{)}$	$\displaystyle\geq-\inf_{\mu\in\mathcal{O}}\tilde{I}_{\nu}(\mu).$		(121)

Furthermore $\tilde{I}_{\nu}(\cdot)$ is lower semi-continuous, and has compact level sets.

Proof.

First, fix any sequence $\big{(}\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}\big{)}_{N\geq 1}$ , such that $\big{(}\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}\big{)}\in\mathcal{Y}^{N}$ . Necessarily, thanks to the definition of $\mathcal{Y}^{N}$ , it must be that

\displaystyle\hat{\mu}^{N}\big{(}\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}\big{)}\to\kappa\in\mathcal{P}\big{(}\mathbb{R}^{2M}\big{)}.

(122)

It follows from (122) that

	$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log Q^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{A}\big{)}$	$\displaystyle\leq-\inf_{\mu\in\mathcal{A}}\tilde{I}_{\nu}(\mu)$		(123)
	$\displaystyle\underset{N\to\infty}{\underline{\lim}}N^{-1}\log Q^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{O}\big{)}$	$\displaystyle\geq-\inf_{\mu\in\mathcal{O}}\tilde{I}_{\nu}(\mu).$		(124)

See for instance [30] for a proof of this fact. Furthermore $\tilde{I}_{\nu}:\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\to\mathbb{R}^{+}$ is lower-semi-continuous and has compact level sets.

We next have to show that the convergence is uniform over $\mathcal{Y}^{N}$ (as in the statement of the Theorem). To do this, we first wish to show that for any measurable set $\mathcal{E}\subset\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$ and any $\epsilon>0$ , for all large enough $N$ ,

\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{E}\big{)}\leq\inf_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{E}^{(\epsilon)}\big{)}

(125)

and $\mathcal{E}^{(\epsilon)}$ is the closed $\epsilon$ -blowup of $\mathcal{E}$ with respect to $d_{W}$ . To do this, we are going to compare the conditioned probability to the conditioned probability induced by any other sequence in $\mathcal{Y}^{N}$ . This comparison is facilitated by using the following permutation-averaged probability law.

Define the set

\mathfrak{S}^{N}=\bigg{\{}(\mathbf{y},\mathbf{g})\in\mathcal{C}\big{(}[0,T],\mathbb{R}^{M}\big{)}^{N}\times\mathcal{C}\big{(}[0,T],\mathbb{R}^{M}\big{)}^{N}:\\ N^{-1}\sum_{j\in I_{N}}\big{\{}\sup_{p\in I_{M}}\big{|}y^{p,j}_{0}-z^{(N),p,j}_{0}\big{|}+\sup_{p\in I_{M}}\big{|}g^{p,j}_{0}-g^{(N),p,j}_{0}\big{|}\big{\}}\leq 2\tilde{\delta}_{N}\bigg{\}},

(126)

and we recall that $(\tilde{\delta}_{N})_{N\geq 1}$ is a sequence that decreases to $0$ , as defined in (30). We endow $\mathfrak{S}^{N}$ with the topology that it inherits from $\mathcal{C}\big{(}[0,T],\mathbb{R}^{M}\big{)}^{N}\times\mathcal{C}\big{(}[0,T],\mathbb{R}^{M}\big{)}^{N}$ . Write $\mathfrak{P}^{N}$ to be the set of all permutations on $I_{N}$ , and define the measure $\bar{Q}^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\in\mathcal{P}\big{(}\mathfrak{S}^{N}\big{)}$ to be the average over all permutations, i.e. for any measurable $\mathcal{A}\subseteq\mathfrak{S}^{N}$ ,

\displaystyle\bar{Q}^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}(\mathcal{A})=\big{|}\mathfrak{P}^{N}\big{|}^{-1}\sum_{\pi\in\mathfrak{P}^{N}}Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\pi\cdot\mathcal{A}\big{)}

(127)

and here we denote $\pi:\mathcal{C}\big{(}[0,T],\mathbb{R}^{M}\big{)}^{N}\times\mathcal{C}\big{(}[0,T],\mathbb{R}^{M}\big{)}^{N}$ to be the permutation,

\displaystyle\big{(}\pi\cdot(\mathbf{y},\mathbf{g})\big{)}^{j}=(\mathbf{y}^{\pi(j)},\mathbf{g}^{\pi(j)}).

(128)

Since the empirical measure is invariant under any permutation of its arguments, for any measurable $\mathcal{A}\subset\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$

\displaystyle Q^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{A}\big{)}=\bar{Q}^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{A}\big{)}.

(129)

We can without loss of generality take $\big{(}\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}\big{)}\in\mathfrak{S}^{N}$ . Now consider any other sequence $\big{(}\grave{\mathbf{z}}^{(N)}_{0},\grave{\mathbf{g}}^{(N)}_{0}\big{)}\in\mathfrak{S}^{N}\cap\mathcal{Y}^{N}$ . Let $d_{W}$ be the $1$ -Wasserstein Distance on $\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\times\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$ induced by the norm

\displaystyle\left\|(\mathbf{y},\mathbf{G})-(\bar{\mathbf{y}},\bar{\mathbf{G}})\right\|=N^{-1}\sum_{j\in I_{N}}\big{\{}\sup_{p\in I_{M}\fatsemi t\in[0,T]}\big{|}y^{p,j}_{t}-\bar{y}^{p,j}_{t}\big{|}+\sup_{p\in I_{M}\fatsemi t\in[0,T]}\big{|}G^{p,j}_{t}-\bar{G}^{p,j}_{t}\big{|}\big{\}}.

(130)

We claim that

$\displaystyle d_{W}\big{(}\bar{Q}^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}},\bar{Q}^{N}_{\nu,\grave{\mathbf{z}}^{(N)}_{0},\grave{\mathbf{g}}^{(N)}_{0}}\big{)}\leq$	$\displaystyle N^{-1}\sum_{j\in I_{N}}\big{\{}\sup_{p\in I_{M}}\big{\|}z^{(N),p,j}_{0}-\grave{z}^{(N),p,j}_{0}\big{\|}$
	$\displaystyle+\sup_{p\in I_{M}\fatsemi t\in[0,T]}\big{\|}\mathfrak{m}^{p}_{t}(\nu,\mathbf{g}^{(N),j}_{0})-\mathfrak{m}^{p}_{t}(\nu,\grave{\mathbf{g}}^{(N),j}_{0})\big{\|}\big{\}}$	(131)
$\displaystyle:=$	$\displaystyle f^{N}\big{(}\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0},\grave{\mathbf{z}}^{(N)}_{0},\grave{\mathbf{g}}^{(N)}_{0}\big{)}$	(132)

This identity follows from the fact that $\tilde{z}^{p,j}_{t}:=z^{p,j}_{t}-z^{p,j}_{0}$ and $\tilde{g}^{p,j}_{t}:=G^{p,j}_{t}-\mathfrak{m}^{p}_{t}(\nu,\mathbf{G}^{j}_{0})$ are identically distributed, both (i) for all $j\in I_{N}$ , and (ii) with respect to both probability laws $\bar{Q}^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}}$ and $\bar{Q}^{N}_{\nu,\grave{\mathbf{z}}^{(N)}_{0},\grave{\mathbf{g}}^{(N)}_{0}}$ .

It follows from the definition of $\mathcal{Y}^{N}$ that

\displaystyle\lim_{N\to\infty}\sup_{(\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}),(\grave{\mathbf{z}}^{(N)}_{0},\grave{\mathbf{g}}^{(N)}_{0})\in\mathfrak{S}^{N}\cap\mathcal{Y}^{N}}f^{N}\big{(}\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0},\grave{\mathbf{z}}^{(N)}_{0},\grave{\mathbf{g}}^{(N)}_{0}\big{)}=0.

(133)

We have thus proved (125).

To now must prove the Large Deviations bounds in the statement of the theorem. We start with the upper-bound (120). It follows from (123) and (125) that for any $\epsilon>0$ ,

\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{A}\big{)}\leq-\inf_{\mu\in\mathcal{A}^{(\epsilon)}}\tilde{I}_{\nu}(\mu).

(134)

The lower-semi-continuity of $\tilde{I}_{\nu}(\mu)$ dictates that

\lim_{\epsilon\to 0^{+}}\inf_{\mu\in\mathcal{A}^{(\epsilon)}}\tilde{I}_{\nu}(\mu)=\inf_{\mu\in\mathcal{A}}\tilde{I}_{\nu}(\mu),

and we have proved the upperbound. For the lower-bound, let $\mathcal{O}$ be open, and for any $\mu\in\mathcal{O}$ , take $\epsilon>0$ to be such that $B_{2\epsilon}(\mu)\subset\mathcal{O}$ . Then

$\displaystyle\underset{N\to\infty}{\underline{\lim}}\inf_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}$	$\displaystyle N^{-1}\log Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{O}\big{)}$	(135)
	$\displaystyle\geq\underset{N\to\infty}{\underline{\lim}}\inf_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in B_{2\epsilon}(\mu)\big{)}$	(136)
	$\displaystyle\geq\underset{N\to\infty}{\underline{\lim}}N^{-1}\log Q^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in B_{\epsilon}(\mu)\big{)}$	(137)
	$\displaystyle\geq-\inf_{\zeta\in B_{\epsilon}(\mu)}\tilde{I}_{\nu}(\zeta),$	(138)

using the Large Deviations estimate (124). Taking $\epsilon\to 0^{+}$ , it must be that for any $\mu\in\mathcal{O}$ ,

\displaystyle\underset{N\to\infty}{\underline{\lim}}\inf_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{O}\big{)}\geq-\tilde{I}_{\nu}(\mu).

(139)

Since $\mu\in\mathcal{O}$ is arbitrary, (121) follows immediately. ∎

We can now state the proof of Theorem 3.

Proof.

We start with the upper bound (35). We write $\hat{\mu}^{N}:=\hat{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}_{[0,T]})$ . Using a union-of-events bound, for any $a>0$ ,

\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{A}\big{)}\leq\\ \max\bigg{\{}\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{A}\cap\mathcal{U}_{a}\big{)},\\ \underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\notin\mathcal{U}_{a}\big{)}\bigg{\}}\\ \leq\max\bigg{\{}\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{A}\cap\mathcal{U}_{a}\big{)},-L\bigg{\}},

(140)

for any $L>0$ , as long as $a$ is sufficiently large, thanks to the exponential tightness proved in Lemma 8. By taking $a\to\infty$ , it thus suffices that we prove that for arbitrary $\mathcal{U}_{a}$ such that $\mathcal{A}\cap\mathcal{U}_{a}\neq\emptyset$ ,

\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{A}\cap\mathcal{U}_{a}\big{)}=-\inf_{\mu\in\mathcal{A}\cap\mathcal{U}_{a}}\tilde{I}_{\mu}(\mu).

(141)

Since $\mathcal{A}\cap\mathcal{U}_{a}$ is compact, for any $\epsilon>0$ we can always find an open covering of the form, for some positive integer $\mathcal{N}_{\epsilon}$ , $\{\zeta_{i}\}_{1\leq i\leq\mathcal{N}_{\epsilon}}\subseteq\mathcal{A}\cap\mathcal{U}_{a}$ ,

\displaystyle\mathcal{A}\cap\mathcal{U}_{a}\subseteq\bigcup_{i=1}^{\mathcal{N}_{\epsilon}}B_{\epsilon}(\zeta_{i}).

(142)

We thus find that

\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{A}\cap\mathcal{U}_{a}\big{)}\\ \leq\sup_{1\leq i\leq\mathcal{N}_{\epsilon}}\bigg{\{}\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in B_{\epsilon}(\zeta_{i})\big{)}\bigg{\}}.

(143)

Thus, employing Lemma 5 in the third line below,

$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}$	$\displaystyle N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in B_{\epsilon}(\zeta_{i})\big{)}$	(144)
	$\displaystyle=\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\gamma^{N}_{\mathbf{y},\mathbf{g}_{0}}\bigg{(}\hat{\mu}^{N}\in B_{\epsilon}(\zeta_{i})\bigg{)}\bigg{]}$	(145)
	$\displaystyle=\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\tilde{\gamma}^{N}_{\hat{\mu}^{N}(\mathbf{y}),\mathbf{g}_{0}}\bigg{(}\hat{\mu}^{N}\in B_{\epsilon}(\zeta_{i})\bigg{)}\bigg{]}$	(146)
	$\displaystyle\leq\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\sup_{\nu\in B_{\epsilon}(\zeta_{i})}\gamma^{N}_{\nu^{(1)},\mathbf{g}_{0}}\bigg{(}\hat{\mu}^{N}\in B_{\epsilon}(\zeta_{i})\bigg{)}\bigg{]}$	(147)
	$\displaystyle=-\inf_{\mu,\nu\in B_{\epsilon}(\zeta_{i})}\tilde{I}_{\nu}(\mu),$	(148)

thanks to Theorem 17. We thus find that

\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{A}\cap\mathcal{U}_{a}\big{)}

\displaystyle\leq-\inf_{1\leq i\leq\mathcal{N}_{\epsilon}}\inf_{\nu,\mu\in B_{\epsilon}(\zeta_{i})}\tilde{I}_{\nu}(\mu).

(149)

Now, it is proved in Lemma 16 that $\nu\to Q_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}$ is continuous. Since the Relative Entropy is lower-semi-continuous in both of its arguments, we thus find that the following map is lower-semi-continuous,

(\nu,\mu)\to\tilde{I}_{\nu}(\mu).

Thus taking $\epsilon\to 0^{+}$ , we obtain that

\displaystyle\lim_{\epsilon\to 0^{+}}\inf_{1\leq i\leq\mathcal{N}_{\epsilon}}\inf_{\nu,\mu\in B_{\epsilon}(\zeta_{i})}\tilde{I}_{\nu}(\mu)=\inf_{\mu\in\mathcal{A}\cap\mathcal{U}_{a}}\tilde{I}_{\mu}(\mu),

(150)

and we have proved (141).

Turning to the lower bound (36), consider an arbitrary open set $\mathcal{O}$ . If $\mathcal{O}\cap\mathcal{U}=\emptyset$ , then

\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{O}\big{)}=-\infty=-\inf_{\mu\in\mathcal{O}}\mathcal{I}(\mu),

since $\mathcal{I}$ is identically $\infty$ outside of $\mathcal{U}$ . In this case, its clear that (121) holds.

We can thus assume that $\mathcal{O}\cap\mathcal{U}\neq\emptyset$ . Let $\mu\in\mathcal{O}$ be such that $\mu$ is in the interior of $\mathcal{U}_{a}$ , for some $a>0$ . We can thus find a sequence of neighborhoods $\{\mathcal{N}_{i}\}_{i\geq 1}$ of $\mu$ such that $\mathcal{N}_{j}\subseteq\mathcal{O}\cap\mathcal{U}_{a}\cap B_{j^{-1}}(\mu)$ . We thus find that for any $j\geq 1$ ,

\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{O}\big{)}\geq\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{N}_{j}\big{)}.

(151)

Similarly to the bound for the closed sets, we obtain that

\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{N}_{j}\big{)}\geq-\sup_{\nu\in\mathcal{N}_{j}}\inf_{\mu\in\mathcal{N}_{j}}\tilde{I}_{\nu}(\mu).

(152)

Taking $j\to\infty$ , since $(\nu,\mu)\to\tilde{I}_{\nu}(\mu)$ is lower semicontinuous, it must be that

\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{N}_{j}\big{)}\geq-\tilde{I}_{\mu}(\mu).

(153)

Since $\mu\in\mathcal{O}$ is arbitrary, it must be that

\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{O}\big{)}\geq-\inf_{\mu\in\mathcal{O}}\tilde{I}_{\mu}(\mu).

(154)

∎

4.2.1 Uncoupled System (with no conditioning)

In this subsection we prove Corollary 11.

For some $\nu\in\mathfrak{Q}$ , let $Q^{N}_{\nu}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\times\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$ be the joint law of the uncoupled system (with no conditioning), i.e.

Q^{N}_{\nu}=\big{(}\beta_{\nu}\otimes P_{\mathbf{z}}\big{)}^{\otimes N}.

(155)

We reach a corollary to Lemma 17.

Corollary 18.

	$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log Q^{N}_{\nu}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{A}\big{)}$	$\displaystyle\leq-\inf_{\mu\in\mathcal{A}}\mathcal{R}(\mu\|\|S_{\nu})$		(156)
	$\displaystyle\underset{N\to\infty}{\underline{\lim}}N^{-1}\log Q^{N}_{\nu}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{O}\big{)}$	$\displaystyle\geq-\inf_{\mu\in\mathcal{O}}\mathcal{R}(\mu\|\|S_{\nu}).$		(157)

Furthermore $\mu\to\mathcal{R}(\mu||S_{\nu})$ is lower semi-continuous, and has compact level sets.

Proof.

This is a consequence of Sanov’s Theorem. ∎

The proof of Corollary 11 now follows analogously to the proof of 3.

4.3 Coupled System

Girsanov’s Theorem implies that

\displaystyle\frac{dP^{N}_{\mathbf{J},\mathbf{z}_{0}}}{dP^{N}_{\mathbf{z}_{0}}}\bigg{|}_{\mathcal{F}_{T}}(\mathbf{y})=\exp\big{(}N\Gamma^{N}_{\mathbf{J},T}(\mathbf{y})\big{)}

(158)

where $\Gamma^{N}_{\mathbf{J},T}:\mathbb{R}^{MN}\to\mathbb{R}$ is

\Gamma^{N}_{\mathbf{J},T}(\mathbf{y})=N^{-1}\sum_{j\in I_{N}\fatsemi p\in I_{M}}\int_{0}^{T}\sigma_{s}^{-2}\big{(}\tilde{G}^{p,j}_{s}-\tau^{-1}y^{p,j}_{s}\big{)}dy^{p,j}_{s}-\frac{1}{2}\sigma_{s}^{-2}\big{(}\tilde{G}^{p,j}_{s}-\tau^{-1}y^{p,j}_{s}\big{)}^{2}ds.

(159)

We wish to specify a map $\Gamma:\mathcal{U}\to\mathbb{R}$ with (i) as nice regularity properties as possible, and (ii) such that with unit probability

\displaystyle\Gamma^{N}_{\mathbf{J},T}(\mathbf{y})=\Gamma\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}.

(160)

It is well-known that the stochastic integral is not a continuous function of the driving Brownian motion. Thus we define the map $\Gamma$ to be a limit of time-discretized approximations, and we will show that this limit must always converge for any measure in $\mathcal{U}$ .

Our precise definition of $\Gamma:\mathcal{U}\to\mathbb{R}$ is as follows. We first define a time-discretized approximation to $\Gamma$ . $\Gamma^{(m)}:\mathcal{U}\to\mathbb{R}^{+}$ ,

\Gamma^{(m)}(\mu)=\sum_{p\in I_{M}}\sum_{a=0}^{m-1}\mathbb{E}^{\mu}\bigg{[}\sigma_{t^{(m)}_{a}}^{-2}\big{(}G^{p}_{t_{a}^{(m)}}-\tau^{-1}z^{p}_{t_{a}^{(m)}}\big{)}\big{(}z^{p}_{t^{(m)}_{a+1}}-z^{p}_{t^{(m)}_{a}}+\Delta_{m}\tau^{-1}z^{p}_{t^{(m)}_{a}}\big{)}\\ -\frac{1}{2}\sigma_{t^{(m)}_{a}}^{-2}\Delta_{m}\big{(}G^{p}_{t_{a}^{(m)}}-\tau^{-1}z^{p}_{t_{a}^{(m)}}\big{)}^{2}\bigg{]}.

(161)

We now define $\Gamma:\mathcal{U}\to\mathbb{R}$ to be such that (in the case that the following limit exists)

\displaystyle\Gamma(\mu)=\lim_{j\to\infty}\Gamma^{(m_{j,j})}(\mu),

(162)

where $m_{j,j}$ is a positive integer defined further below in Lemma 20. If the above limit does not exist, then we define $\Gamma(\mu)=0$ (in fact we will see that the limit always exists if $\mu\in\mathcal{U}$ ). It may be observed that $\Gamma$ is a well-defined measurable function.

Lemma 19.

For every $N\geq 1$ , every $(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$ , and for $Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}$ almost every $(\mathbf{y},\tilde{\mathbf{G}})$ , the following limit exists

\lim_{j\to\infty}\Gamma^{(m_{j,j})}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}

(163)

With unit probability, the Radon-Nikodym Derivative in (158) is such that

\displaystyle\frac{dP^{N}_{\mathbf{J},\mathbf{z}_{0}}}{dP^{N}_{\mathbf{z}_{0}}}\bigg{|}_{\mathcal{F}_{T}}=\exp\big{(}\Gamma\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}\big{)}

(164)

Also for any $\epsilon,L>0$ , there exists $k\in\mathbb{Z}^{+}$ such that for all $N\geq 1$ ,

\displaystyle\sup_{j\geq k}\sup_{\mathbf{z}_{0},\mathbf{g}_{0}\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\bigg{(}\big{|}\Gamma^{(m_{j,j})}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}-\Gamma\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}\big{|}\geq\epsilon\bigg{)}\leq-L

(165)

Proof.

Define the set

\displaystyle\mathcal{A}_{j}=\big{\{}\mu\in\mathcal{U}:\big{|}\Gamma^{(m_{j,j})}(\mu)-\Gamma^{(m_{j+1,j+1})}(\mu)\big{|}\geq 2^{1-j}\big{\}}

(166)

Thanks to a union-of-events bound, for any $N\geq 1$ , and using the bound in Lemma 20,

\displaystyle\sup_{\mathbf{z}_{0},\mathbf{g}_{0}\in\mathcal{Y}^{N}}Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\bigg{(}\hat{\mu}^{N}\in\bigcup_{j\geq k}\mathcal{A}_{j}\bigg{)}\leq\sum_{j=k}^{\infty}\exp\big{(}-N2^{j}\big{)}.

(167)

It thus follows from the Borel-Cantelli Lemma that there exists a random $k$ such that $\hat{\mu}^{N}\notin\mathcal{A}_{j}$ for all $j\geq k$ , and so the limit in (163) exists (almost surely). (165) follows analogously.

∎

Lemma 20.

(i) $\Gamma^{(m)}:\mathcal{U}\to\mathbb{R}$ is continuous. (ii) Moreover, for any $a,j\in\mathbb{Z}^{+}$ , there exists $m_{a,j}$ such that for all $m\geq m_{a,j}$ and all $n\geq m$ ,

\displaystyle\sup_{\mathbf{z}_{0},\mathbf{g}_{0}\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\big{|}\Gamma^{(m)}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}-\Gamma^{(n)}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}\geq 2^{-j}\big{)}\leq-2^{a}.

(168)

Proof.

(i) The continuity of $\Gamma^{(m)}$ is almost immediate from the definition.

(ii) For any $t\in[0,T]$ , write $t^{(m)}=\sup\{t^{(m)}_{b}\;:t^{(m)}_{b}\leq t\}$ . Starting with the discrete approximation to the stochastic integral, we can thus write

\displaystyle\sum_{b=0}^{m-1}\sigma_{t^{(m)}_{a}}^{-2}\big{(}G^{p}_{t_{b}^{(m)}}-\tau^{-1}z^{p}_{t_{b}^{(m)}}\big{)}\big{(}z^{p}_{t^{(m)}_{b+1}}-z^{p}_{t^{(m)}_{b}}\big{)}=\int_{0}^{T}\sigma^{-2}_{t^{(m)}}\big{(}G^{p}_{t^{(m)}}-\tau^{-1}z^{p}_{t^{(m)}}\big{)}dz^{p}_{t}.

(169)

Hence,

\sum_{b=0}^{m-1}\mathbb{E}^{\mu}\bigg{[}\sigma_{t^{(m)}_{b}}^{-2}\big{(}G^{p}_{t_{b}^{(m)}}-\tau^{-1}z^{p}_{t_{b}^{(m)}}\big{)}\big{(}z^{p}_{t^{(m)}_{b+1}}-z^{p}_{t^{(m)}_{a}}\big{)}\bigg{]}-\\ \sum_{ab=0}^{n-1}\mathbb{E}^{\mu}\bigg{[}\sigma_{t^{(n)}_{b}}^{-2}\big{(}G^{p}_{t_{b}^{(n)}}-\tau^{-1}z^{p}_{t_{b}^{(n)}}\big{)}\big{(}z^{p}_{t^{(n)}_{b+1}}-z^{p}_{t^{(n)}_{b}}\big{)}\bigg{]}\\ =\mathbb{E}^{\mu}\bigg{[}\int_{0}^{T}\bigg{\{}\sigma^{-2}_{t^{(m)}}\big{(}G^{p}_{t^{(m)}}-\tau^{-1}z^{p}_{t^{(m)}}\big{)}-\sigma^{-2}_{t^{(n)}}\big{(}G^{p}_{t^{(n)}}-\tau^{-1}z^{p}_{t^{(n)}}\big{)}\bigg{\}}dz^{p}_{t}\bigg{]}\\ =\mathbb{E}^{\mu}\bigg{[}\int_{0}^{T}\sum_{p\in I_{M}}(f^{p}_{t^{(m)}}-f^{p}_{t^{(n)}})dz_{t}^{p}\bigg{]}

(170)

where $f^{p}_{t}=\sigma_{t}^{-2}\big{(}G^{p}_{t}-\tau^{-1}z^{p}_{t}\big{)}$ . Writing

\displaystyle f^{p,j}_{t}=\sigma_{t}^{-2}\big{(}G^{p,j}_{t}-\tau^{-1}z^{p,j}_{t}\big{)},

(171)

we obtain that

\displaystyle\mathbb{E}^{\hat{\mu}^{N}}\bigg{[}\int_{0}^{T}\sum_{p\in I_{M}}(f^{p}_{t^{(m)}}-f^{p}_{t^{(n)}})dz_{t}^{p}\bigg{]}=N^{-1}\sum_{j\in I_{N}\fatsemi p\in I_{M}}\int_{0}^{T}\big{(}f^{p,j}_{t^{(m)}}-f^{p,j}_{t^{(n)}}\big{)}dy^{p,j}_{t}.

(172)

The quadratic variation of this stochastic integral is

\displaystyle(QV)^{(m,n),N}_{t}=N^{-2}\sum_{j\in I_{N}\fatsemi p\in I_{M}}\int_{0}^{t}\big{(}f^{p,j}_{s^{(m)}}-f^{p,j}_{s^{(n)}}\big{)}^{2}\sigma_{s}^{2}ds

(173)

By definition of the set $\mathcal{U}_{a}$ , if $\hat{\mu}^{N}\in\mathcal{U}_{a}$ , then for any $\delta>0$ , one can find $m_{\delta}$ such that as long as $m,n\geq m_{\delta}$ , necessarily

(QV)^{(m,n),N}_{T}\leq N^{-1}\delta.

Then writing $w(\cdot)$ to be a standard Brownian Motion, using the Dambin-Dubins-Schwarz [27] time-rescaled representation of the stochastic integral, as long as $(m,n)\geq m_{\delta}$ ,

	$\displaystyle\mathbb{P}\bigg{(}\hat{\mu}^{N}\in\mathcal{U}_{a}\;,\;\bigg{\|}\int_{0}^{T}\sum_{p\in I_{M}}(f^{p}_{t^{(m)}}-f^{p}_{t^{(n)}})dz_{t}^{p}\bigg{\|}\geq\frac{\epsilon}{2}\bigg{)}\leq$	$\displaystyle\mathbb{P}\big{(}\big{\|}w\big{(}N^{-1}\delta\big{)}\big{\|}\geq\epsilon\big{)}$		(174)
	$\displaystyle=$	$\displaystyle\exp\big{(}-N\epsilon^{2}/(8\delta)\big{)}\leq\exp(-NL),$		(175)

as long as we choose $\delta$ sufficiently small.

The other terms in

\Gamma^{(m)}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}-\Gamma^{(n)}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}

are treated similarly (observe that they are just Riemann Sums, so its straightforward to control their difference from the limiting integral).

∎

We now prove Theorems 10 and 12.

Proof.

In the case of connectivity-independent initial conditions (Case 2 of the Assumptions), the theorem follows from Corollary 11. Since the relative entropy is only zero when its two arguments are identical, any zero must be a fixed point of the operator $\Phi$ . It is proved in the following Lemma that there is a unique zero.

For the rest of this proof, we prove the theorem in the case of connectivity-dependent initial conditions. We start by proving that for any $\epsilon>0$ , there must exist a measure $\mu\in\mathcal{U}$ such that

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}d_{W}(\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu)\leq\epsilon\big{)}=0.

(176)

Write $\mathfrak{U}=\mathcal{U}_{a}$ , where $a$ is large enough that

\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\in\mathcal{U}_{a}\big{)}<-C.

where $C$ is the upperbound for $\Gamma$ in Lemma 21. This is possible thanks to the Exponential Tightness. Thanks to the Radon-Nikodym derivative identity in (160), we thus find that

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}(\mathbf{z},\tilde{\mathbf{G}})\notin\mathfrak{U}\big{)}<0.

(177)

Thus for (176) to hold, it suffices that we prove that there exists $\mu\in\mathfrak{U}$ such that

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}(\mathbf{z},\mathbf{G})\in\mathfrak{U}\;,\;d_{W}(\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu)\leq\epsilon\big{)}=0.

(178)

Since $\mathfrak{U}$ is compact, for any $\epsilon>0$ , we can obtain a finite covering of $\mathfrak{U}$ of the form

\displaystyle\mathfrak{U}\subseteq\bigcup_{i=1}^{\mathcal{N}_{\epsilon}}B_{\epsilon}(\mu_{i}),

(179)

where $\mu_{i}\in\mathfrak{U}$ . By a union of events bound,

	$\displaystyle 0=$	$\displaystyle\lim_{N\to\infty}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}(\mathbf{z},\tilde{\mathbf{G}})\in\mathfrak{U}\big{)}$		(180)
	$\displaystyle\leq$	$\displaystyle\max_{1\leq i\leq\mathcal{N}_{\epsilon}}\bigg{\{}\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}(\mathbf{z},\tilde{\mathbf{G}})\in B_{\epsilon}(\mu_{i})\big{)}\bigg{\}}$		(181)

If our proposition in (178) were to be false, then (181) would be strictly negative, which would be a contradiction.

Write $\mu_{(k)}\in\mathfrak{U}$ to be such that

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}d_{W}(\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu_{(k)})\geq k^{-1}\big{)}=0.

(182)

Let $\mu\in\mathfrak{U}$ be any measure such that for some subsequence $(p_{k})_{k\geq 1}$ , $\lim_{k\to\infty}\mu_{(p_{k})}=\mu$ (this must be possible because $\mathfrak{U}$ is compact).

We next claim that

\displaystyle\lim_{\epsilon\to 0^{+}}\underset{N\to\infty}{\underline{\lim}}N^{-1}\log\mathbb{P}\big{(}d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu\big{)}\leq\epsilon\big{)}=-\mathcal{I}(\mu)+\Gamma(\mu)

(183)

Indeed writing $\mathcal{A}_{\epsilon}=\big{\{}d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu\big{)}\leq\epsilon\big{\}}$ ,

$\displaystyle\mathbb{P}\big{(}d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu\big{)}\leq\epsilon\big{)}=$	$\displaystyle\mathbb{E}^{\gamma}\bigg{[}\int_{\mathbb{R}^{MN}}P^{N}_{\mathbf{J},\mathbf{x}}(\mathcal{A}_{\epsilon})\rho^{N}_{\mathbf{J}}(\mathbf{x})d\mathbf{x}\bigg{]}$	(184)
$\displaystyle=$	$\displaystyle\mathbb{E}^{\gamma}\bigg{[}\int_{\mathbb{R}^{MN}}\mathbb{E}^{P^{N}_{\mathbf{x}}}\big{[}\exp\big{(}N\Gamma(\hat{\mu}^{N})\big{)}\chi\{\mathcal{A}_{\epsilon}\}\big{]}\rho^{N}_{\mathbf{J}}(\mathbf{x})d\mathbf{x}\bigg{]}$	(185)
$\displaystyle=$	$\displaystyle\int_{\mathbb{R}^{MN}}\mathbb{E}^{\gamma}\bigg{[}\mathbb{E}^{P^{N}_{\mathbf{x}}}\big{[}\exp\big{(}N\Gamma(\hat{\mu}^{N})\big{)}\chi\{\mathcal{A}_{\epsilon}\}\big{]}\rho^{N}_{\mathbf{J}}(\mathbf{x})\bigg{]}d\mathbf{x}$	(186)
$\displaystyle=$	$\displaystyle\int_{\mathbb{R}^{MN}}\mathbb{E}^{\gamma}\bigg{[}\mathbb{E}^{\gamma}\bigg{[}\mathbb{E}^{P^{N}_{\mathbf{x}}}\big{[}\exp\big{(}N\Gamma(\hat{\mu}^{N})\big{)}\chi\{\mathcal{A}_{\epsilon}\}\big{]}\rho^{N}_{\mathbf{J}}(\mathbf{x})\;\bigg{\|}\;\mathbf{G}_{0}\bigg{]}\bigg{]}d\mathbf{x}$	(187)

and in this last step we first perform the conditional expectation, for $\gamma$ conditioned on the values of $\{G^{p,j}_{0}\}_{j\in I_{N}\fatsemi p\in I_{M}}$ .

Now, recall that

\rho^{N}_{\mathbf{J}}(\mathbf{z}_{0})=(Z^{N}_{\mathbf{J}})^{-1}\chi\big{\{}\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{G}_{0})\in B_{\delta_{N}}(\kappa)\big{\}}.

Furthermore, writing

u_{N}=N^{-1}\log\mathbb{E}[Z^{N}_{\mathbf{J}}],

our assumption on the initial condition dictates that for any $\delta>0$ ,

\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\big{|}N^{-1}\log Z^{N}_{\mathbf{J}}-u_{N}\big{|}\geq\delta\big{)}<0.

(188)

Next, we claim that

\displaystyle\lim_{\epsilon\to 0^{+}}\inf_{\nu\in\mathfrak{U}\cap\mathcal{A}_{\epsilon}}\Gamma(\nu)=\Gamma(\mu).

(189)

Indeed (189) is a consequence of Lemma 19: this Lemma implies that $\Gamma$ can be approximated arbitrarily well by continuous functions over $\mathfrak{U}$ .

We thus obtain that

	$\displaystyle\lim_{\epsilon\to 0^{+}}\underset{N\to\infty}{\underline{\lim}}\inf_{(\mathbf{z}_{0},\mathbf{G}_{0})}N^{-1}\log\big{(}d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu\big{)}\leq\epsilon\big{)}$	(190)
$\displaystyle=$	$\displaystyle\Gamma(\mu)+\lim_{\epsilon\to 0^{+}}\underset{N\to\infty}{\underline{\lim}}\big{\{}-u_{N}+N^{-1}\log\int_{\mathbb{R}^{MN}}\mathbb{E}^{\gamma}\big{[}Q^{N}_{\mathbf{x},\mathbf{G}_{0}}\big{(}\mathcal{A}_{\epsilon}\big{)}\big{]}\chi\big{\{}\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{G}_{0})\in B_{\delta_{N}}(\kappa)\big{\}}d\mathbf{x}\big{\}}$	(191)
$\displaystyle=$	$\displaystyle\Gamma(\mu)-\lim_{\epsilon\to 0^{+}}\inf_{\nu\in\mathcal{A}_{(\epsilon)}}\mathcal{I}(\nu),$	(192)

since (by definition)

N^{-1}\log\int_{\mathbb{R}^{MN}}\mathbb{E}^{\gamma}\big{[}\chi\big{\{}\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{G}_{0})\in B_{\delta_{N}}(\kappa)\big{]}d\mathbf{z}_{0}=u_{N},

and we have employed the uniform lower bound in (36). The lower semi-continuity of $\mathcal{I}$ implies that

\lim_{\epsilon\to 0^{+}}\inf_{\nu\in\mathcal{A}_{(\epsilon)}}\mathcal{I}(\nu)=\mathcal{I}(\mu).

We thus obtain (183), as required.

Next, we must show that $\mathcal{J}(\mu)=\mathcal{I}(\mu)-\Gamma(\mu)$ (recall the definition of $\mathcal{J}(\mu)$ in (56). Now

\displaystyle\frac{dS_{\mu,\mathbf{z}_{0},\mathbf{g}_{0}}}{dQ_{\mu,\mathbf{z}_{0},\mathbf{g}_{0}}}\bigg{|}_{\mathcal{F}_{T}}=\exp\bigg{(}\sum_{p\in I_{M}}\int_{0}^{T}\sigma_{s}^{-2}\big{(}g^{p}_{s}-\tau^{-1}z^{p}_{s}\big{)}dz^{p}_{s}-\frac{1}{2}\sigma_{s}^{-2}\big{(}g^{p}_{s}-\tau^{-1}z^{p}_{s}\big{)}^{2}ds\bigg{)}

(193)

Substituting this identity into the proposed rate function definition in (56),

\displaystyle\mathbb{E}^{\kappa}\bigg{[}\mathcal{R}\big{(}\mu_{\mathbf{z}_{0},\mathbf{g}_{0}}||S_{\mu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{)}\bigg{]}=\mathcal{I}(\mu)-\Gamma(\mu),

(194)

as required.

The above reasoning dictates that there must be at least one $\xi$ such that $\mathcal{J}(\xi)=0$ . In fact, it is proved in Lemma 22 that there can only be one measure $\xi$ such that $\mathcal{J}(\xi)=0$ . Furthermore it follows from Lemma 22, over small enough time increments, the mapping $\Phi_{t}$ must be a contraction. This implies (66). ∎

Lemma 21.

There exists a constant $C>0$ such that

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq C\big{)}<0.

(195)

Proof.

For any $\ell>0$ ,

\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\bigg{(}\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq C\bigg{)}\leq\max\bigg{\{}\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|>\ell\big{)},\\ \underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\leq\ell,\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq C\big{)}\bigg{\}}

(196)

Thanks to Lemma 13, $\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|>\ell\big{)}$ converges to $-\infty$ as $\ell\to\infty$ . It thus suffices that we prove that, for abitrary $\ell>0$ , there exists $C_{\ell}>0$ such that

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\leq\ell,\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq C_{\ell}\big{)}<0.

(197)

Now, leaving out the negative-semi-definite terms, we find that

\displaystyle\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\leq N^{-1}\sum_{j\in I_{N}\fatsemi p\in I_{M}}\int_{0}^{T}\sigma_{s}^{-2}\big{(}\tilde{G}^{p,j}_{s}-\tau^{-1}y^{p,j}_{s}\big{)}dy^{p,j}_{s}

(198)

Furthermore, writing $h^{p}_{s}=\sigma_{s}^{-2}\big{(}\tilde{G}^{p,j}_{s}-\tau^{-1}y^{p,j}_{s}\big{)}$ , and assuming that $\left\|\mathcal{J}_{N}\right\|\leq\ell$ , one finds that

$\displaystyle\sum_{j\in I_{N}}(h^{p,j}_{s})^{2}\leq$	$\displaystyle 2\sigma_{s}^{-4}\sum_{j\in I_{N}}\big{\{}(\tilde{G}^{p,j}_{s})^{2}+\tau^{-2}(y^{p,j}_{s})^{2}\big{\}}$	(199)
$\displaystyle\leq$	$\displaystyle 2\sigma_{s}^{-4}\sum_{j\in I_{N}}\big{\{}\ell\lambda(y^{p,j}_{s})^{2}+\tau^{-2}(y^{p,j}_{s})^{2}\big{\}}$	(200)
$\displaystyle\leq$	$\displaystyle 2\sigma_{s}^{-4}\sum_{j\in I_{N}}\big{\{}\ell C_{\lambda}^{2}(y^{p,j}_{s})^{2}+\tau^{-2}(y^{p,j}_{s})^{2}\big{\}}.$	(201)

We thus find that, thanks to Lemma , for any $L>0$ there exists a constant $\bar{C}_{L}>0$ such that

\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\sup_{p\in I_{M}}\sum_{j\in I_{N}}(h^{p,j}_{s})^{2}\geq N\bar{C}_{L}\big{)}\leq-L.

(202)

Write

\mathcal{H}_{N}=\bigg{\{}\sup_{p\in I_{M}}\sum_{j\in I_{N}}(h^{p,j}_{s})^{2}\leq N\bar{C}_{L}\bigg{\}}.

We thus find that,

\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\leq\ell\;,\;\mathcal{H}_{N}\;,\;\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq C_{\ell}\big{)}\\ \leq\max\bigg{\{}\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\mathcal{H}_{N}^{c}\big{)},\\ \underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\leq\ell\;,\;\mathcal{H}_{N}\;,\;\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq C_{\ell}\big{)}\bigg{\}}\\ \leq\max\bigg{\{}-L,\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\leq\ell\;,\;\mathcal{H}_{N}\;,\;\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq C_{\ell}\big{)}\bigg{\}}

(203)

Furthermore, using the Dambins-Dubins Schwarz Theorem [27], and writing $w(t)$ to be 1D Brownian Motion,

\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\leq\ell\;,\;\mathcal{H}_{N}\;,\;\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq C_{\ell}\big{)}\\ \leq\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\sup_{s\in[0,T]}\big{|}w\big{(}\bar{C}N^{-1}s\big{)}\big{|}\geq C_{\ell}\big{)}\leq-L,

(204)

as long as $C_{\ell}$ is sufficiently large, using standard properties of Brownian Motion. ∎

We now prove that the rate function $\mathcal{J}$ has a unique minimizer (i.e. we prove Lemma 9).

Lemma 22.

There exists a unique fixed point $\xi$ of $\Phi$ in $\mathcal{U}$ . Furthermore $\xi$ is such that for any $\mu\in\mathcal{U}$ , writing $\xi_{(1)}=\mu$ and $\xi_{(n+1)}=\Phi(\xi_{(n)})$ , it holds that

\displaystyle\xi=\lim_{n\to\infty}\xi_{(n)}

(205)

Proof.

We start by considering the following restricted map $\tilde{\Phi}:\mathfrak{Q}\to\mathfrak{Q}$ . This is such that $\tilde{\Phi}(\mu)=\nu^{(1)}$ , where $\nu=\Phi(\alpha)$ for any $\alpha\in\mathcal{U}$ such that $\alpha^{(1)}=\mu$ . Define $d^{(2)}_{t}:\mathfrak{Q}\times\mathfrak{Q}\to\mathbb{R}^{+}$ analogously.

We are going to demonstrate that there is a constant $c>0$ such that for all $\mu,\nu\in\mathfrak{Q}$ ,

\displaystyle d^{(2)}_{t}\big{(}\tilde{\Phi}_{t}(\mu),\tilde{\Phi}_{t}(\nu)\big{)}\leq c\sqrt{t}d^{(2)}_{t}(\mu,\nu).

(206)

For any $\mu,\nu\in\mathfrak{Q}$ , we construct a particular $\zeta$ that is within $\eta\ll 1$ of realizing the infimum in the definition of the Wasserstein distance in (98). To do this, we employ the construction of Lemma 16. Let $\mathbf{G}^{\mu},\mathbf{G}^{\nu}$ be $\mathcal{C}([0,T],\mathbb{R}^{M})$ -valued random variables (in the same probability space), with joint probability law $\beta_{\mu,\nu}$ . Then for Brownian motions $\big{(}W^{p}_{t}\big{)}_{p\in I_{M}}$ , define

	$\displaystyle dz^{\nu,p}_{t}$	$\displaystyle=\big{(}-\tau^{-1}z^{\nu,p}_{t}+G^{\nu,p}_{t}\big{)}dt+\sigma_{t}dW^{p}_{t}$		(207)
	$\displaystyle dz^{\mu,p}_{t}$	$\displaystyle=\big{(}-\tau^{-1}z^{\mu,p}_{t}+G^{\mu,p}_{t}\big{)}dt+\sigma_{t}dW^{p}_{t}.$		(208)

The initial conditions are identical: $z^{\nu,p}_{0}=z^{\mu,p}_{0}$ . We immediately see that

\displaystyle\frac{d}{dt}\big{(}z^{\nu,p}_{t}-z^{\mu,p}_{t}\big{)}=-\tau^{-1}\big{(}z^{\nu,p}_{t}-z^{\mu,p}_{t}\big{)}+G^{\nu,p}_{t}-G^{\mu,p}_{t},

(209)

and hence

	$\displaystyle\frac{d}{dt}\big{(}z^{\nu,p}_{t}-z^{\mu,p}_{t}\big{)}^{2}$	$\displaystyle=-2\tau^{-1}\big{(}z^{\nu,p}_{t}-z^{\mu,p}_{t}\big{)}^{2}+2\big{(}z^{\nu,p}_{t}-z^{\mu,p}_{t}\big{)}\big{(}G^{\nu,p}_{t}-G^{\mu,p}_{t}\big{)}\text{ and thus }$		(210)
	$\displaystyle\big{(}z^{\nu,p}_{t}-z^{\mu,p}_{t}\big{)}^{2}$	$\displaystyle=\int_{0}^{t}\exp\big{(}2(s-t)/\tau\big{)}2\big{(}z^{\nu,p}_{s}-z^{\mu,p}_{s}\big{)}\big{(}G^{\nu,p}_{s}-G^{\mu,p}_{s}\big{)}ds.$		(211)

It follows from this that there exists a constant $c>0$ such that for all $t\in[0,T]$ ,

	$\displaystyle d^{(2)}_{t}\big{(}\tilde{\Phi}_{t}(\mu),\tilde{\Phi}_{t}(\nu)\big{)}$	$\displaystyle\leq ctd^{(2)}_{t}\big{(}\beta_{\mu},\beta_{\nu}\big{)}$		(212)
		$\displaystyle\leq cC_{\lambda}td^{(2)}_{t}(\mu,\nu),$		(213)

using Lemma 16. Thus for small enough $t$ , there is a unique fixed point of $\tilde{\Phi}_{t}$ (the mapping upto time $t$ ). Iterating this argument, we find a unique fixed point for $\tilde{\Phi}$ . The uniqueness for $\tilde{\Phi}$ in turn implies uniqueness for $\Phi$ , thanks to the identity in Lemma 16.

To see why (205) holds. First consider arbitrary $\nu_{(1)}\in\mathfrak{Q}$ , and define $\nu_{(n+1)}=\tilde{\Phi}(\nu_{(n)})$ . The above bound in (213) implies that necessarily $\big{(}\nu_{(n)}\big{)}_{n\geq 1}$ is Cauchy. It then immediate follows that for any $\xi_{(1)}\in\mathcal{U}$ with first marginal equal to $\nu_{(1)}$ , and writing $\xi_{(n+1)}=\Phi(\xi_{(n)})$ , it must be that $\big{(}\xi_{(n)}\big{)}_{n\geq 1}$ is Cauchy.

Finally we note that $d^{(2)}$ metrizes weak convergence, thanks to Lemma 15.

∎

Appendix A Bounding Fluctuations of the Noise

For the processes $(\mathbf{y}^{j}_{[0,T]})_{j\in I_{N}}$ that are defined in (31), define the empirical measure

\displaystyle\hat{\mu}^{N}(\mathbf{y})=N^{-1}\sum_{j\in I_{N}}\delta_{\mathbf{y}^{j}_{[0,T]}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}.

(214)

Next, we bound the probability of the empirical being in the set $\mathfrak{Q}_{a}$ , defined in (38), which we recall

\mathcal{Q}_{\mathfrak{a}}=\bigg{\{}\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}\;:\;\sup_{m\geq\mathfrak{a}}\sup_{0\leq i\leq m}\mathbb{E}^{\mu}\big{[}\sup_{M\in I_{M}}(w^{p}_{t^{(m)}_{i+1}}-w^{p}_{t^{(m)}_{i}})^{2}\big{]}>\Delta_{m}^{1/4}\text{ and }\\ \mu\in\mathcal{K}_{\mathfrak{a}}\text{ and }\sup_{p\in I_{M}}\mathbb{E}^{\mu}[\sup_{t\in[0,T]}(y^{p}_{t})^{2}\big{]}\leq\mathfrak{a}\bigg{\}}

(215)

where $\Delta_{m}=T/m$ and $t^{(m)}_{i}=iT/m$ . The main result of this section is the following.

Lemma 23.

For any $L>0$ , there exists $\mathfrak{a}\in\mathbb{Z}^{+}$ such that for all $N\geq 1$ ,

\displaystyle\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})}N^{-1}\log P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{Q}_{\mathfrak{a}}\big{)}\leq-L.

(216)

Proof.

Employing a union-of-events bound, or any $(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$ ,

N^{-1}\log P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{Q}_{\mathfrak{a}}\big{)}\leq N^{-1}\log\bigg{\{}P^{N}_{\mathbf{z}_{0}}\bigg{(}\sup_{p\in I_{M}}N^{-1}\sum_{j\in I_{N}}\sup_{t\in[0,T]}(y^{p,j}_{t})^{2}>\mathfrak{a}\bigg{)}\\ +P^{N}_{\mathbf{z}_{0}}\bigg{(}\sup_{0\leq t\leq\Delta_{m}}\sum_{j\in I_{N}}\sup_{0\leq i\leq m-1}\sup_{p\in I_{M}}\big{|}y^{p,j}_{t+t^{(m)}_{i}}-y^{p,j}_{t^{(m)}_{i}}\big{|}^{2}\geq Na\Delta_{m}\bigg{)}+P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{K}_{\mathfrak{a}}\big{)}\bigg{\}}.

(217)

With a view to bounding the first term on the RHS, since $y^{p,j}_{0}=z^{p,j}_{0}$ ,

(y^{p,j}_{t})^{2}\leq 2\big{(}y^{p,j}_{t}-y^{p,j}_{0}\big{)}^{2}+2(z^{p,j}_{0})^{2}.

Thus for a positive constant $b>0$ ,

\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\exp\bigg{(}b\sup_{p\in I_{M}}\sum_{j\in I_{N}}\sup_{t\in[0,T]}(y^{p,j}_{t})^{2}\bigg{)}\bigg{]}\\ \leq\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\exp\bigg{(}2b\sup_{p\in I_{M}}\sum_{j\in I_{N}}(z^{p,j}_{0})^{2}+2b\sum_{j\in I_{N}}\sup_{t\in[0,T]}(y^{p,j}_{t}-z^{p,j}_{0})^{2}\bigg{)}\bigg{]}.

(218)

Thus, thanks to Chernoff’s Inequality,

	$\displaystyle N^{-1}\log P^{N}_{\mathbf{z}_{0}}\bigg{(}\sup_{p\in I_{M}}N^{-1}\sum_{j\in I_{N}}\sup_{t\in[0,T]}(y^{p,j}_{t})^{2}>\mathfrak{a}\bigg{)}\leq\frac{2b}{N}\sup_{p\in I_{M}}\sum_{j\in I_{N}}(z^{p,j}_{0})^{2}$		(219)
	$\displaystyle+N^{-1}\log\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\exp\bigg{(}2b\sum_{j\in I_{N}}\sup_{t\in[0,T]}(y^{p,j}_{t}-z^{p,j}_{0})^{2}\bigg{)}\bigg{]}-b\mathfrak{a}.$		(220)

The first term on the RHS is bounded for all $N$ and all $(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$ . For the second term on the RHS, standard theory on stochastic processes implies that the exponential moment exists, as long as $b$ is small enough. Thus, taking $\mathfrak{a}\to\infty$ , the RHS can be made arbitrarily small. We thus find that

\displaystyle\lim_{\mathfrak{a}\to\infty}\sup_{N\geq 1}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log P^{N}_{\mathbf{z}_{0}}\bigg{(}\sup_{p\in I_{M}}N^{-1}\sum_{j\in I_{N}}\sup_{t\in[0,T]}(y^{p,j}_{t})^{2}>\mathfrak{a}\bigg{)}=-\infty.

(221)

The Lemma now follows from applying (221), Lemma 25 and Lemma 24 to (217). ∎

The following result is well-known. Nevertheless we sketch a quick proof for clarity.

Lemma 24.

For any $L>0$ , there exists a compact set $\mathcal{K}_{L}$ such that for all $N\geq 1$ ,

\displaystyle\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{K}_{L}\big{)}\leq-L

(222)

Proof.

The following property follows straightforwardly from properties of the stochastic integral (noting that the diffusion coefficient is uniformly bounded): for any $\epsilon>0$ , there exists a compact set $\mathcal{C}_{\epsilon}\subset\mathcal{C}([0,T],\mathbb{R}^{M})$ such that for all $j\in I_{N}$ such that $\|z_{0}^{j}\|\leq\epsilon^{-1}$ ,

\displaystyle\sup_{j\in I_{N}}P^{N}_{\mathbf{z}_{0}}\big{(}y^{j}_{[0,T]}\notin\mathcal{C}_{\epsilon}\big{)}\leq\epsilon.

(223)

Write

\displaystyle u^{N}_{\epsilon}=\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\sum_{j\in I_{N}}\chi\{\|\mathbf{y}^{j}_{0}\|\geq\epsilon^{-1}\},

(224)

and note that our assumptions on $\mathcal{Y}^{N}$ dictates that

\displaystyle\lim_{\epsilon\to 0^{+}}\lim_{N\to\infty}u_{\epsilon}^{N}=0.

(225)

For any $m\in\mathbb{Z}^{+}$ , define the set $\mathcal{L}_{m,\delta}\subset\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}$ to be such that

\displaystyle\mathcal{L}_{m,\delta}=\big{\{}\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}\;:\;\mu(\mathcal{C}_{m^{-1}})\geq\delta\big{)}\big{\}}

(226)

We claim that for any $m\geq 1$ , there exists $\delta_{m}>0$ such that

\displaystyle\sup_{N\geq 1}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}\big{(}\mathbf{y}\big{)}\notin\mathcal{L}_{m,\delta_{m}}\big{)}\leq-m

(227)

To see this, employing a Chernoff Inequality, for a constant $b>0$ , for any $(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$ ,

$\displaystyle N^{-1}\log P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}\big{(}\mathbf{y}\big{)}\notin\mathcal{L}_{m,\delta}\big{)}\leq$	$\displaystyle\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\exp\bigg{(}b\sum_{j\in I_{N}}\chi\{\mathbf{y}^{j}_{[0,T]}\notin\mathcal{C}_{m^{-1}}\}-Nb\delta\bigg{)}\bigg{]}$	(228)
$\displaystyle\leq$	$\displaystyle-b\delta+N^{-1}\log\big{\{}(\epsilon+u^{N}_{\epsilon})\big{(}\exp(b)-1\big{)}+1\big{\}}^{N}$	(229)
$\displaystyle=$	$\displaystyle-b\delta+\log\big{\{}(\epsilon+u^{N}_{\epsilon})\big{(}\exp(b)-1\big{)}+1\big{\}}$	(230)

Taking $\epsilon$ to be sufficiently small, and $b$ sufficiently large, we obtain (227).

Now, for an integer $m_{L}$ to be specified further below, define $\mathcal{K}_{L}=\bigcap_{m\geq m_{L}}\mathcal{L}_{m,\delta_{m}}$ . Prokhorov’s Theorem implies that $\mathcal{K}_{L}$ is compact. Employing a union-of-events bound, we obtain that

	$\displaystyle P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{K}_{L}\big{)}$	$\displaystyle\leq\sum_{m\geq m_{L}}\exp(-mN)$		(231)
		$\displaystyle\leq\exp(-m_{L}N)\sup_{n\geq 1}\sum_{j=0}^{\infty}\exp(-jN).$		(232)

We thus find that, for large enough $m_{L}$ ,

\sup_{N\geq 1}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{K}_{L}\big{)}\leq-L,

as required. ∎

Lemma 25.

There exists a constant $\mathfrak{C}$ such that for any positive integer $m$ and any $a>0$ , writing $\Delta_{m}=Tm^{-1}$ and $t^{(m)}_{i}=Ti/m$ , for any $N\geq 1$ ,

\displaystyle\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log P^{N}_{\mathbf{z}_{0}}\bigg{(}\sup_{0\leq t\leq\Delta_{m}}\sum_{j\in I_{N}}\sup_{0\leq i\leq m-1}\sup_{p\in I_{M}}\big{|}y^{p,j}_{t+t^{(m)}_{i}}-y^{p,j}_{t^{(m)}_{i}}\big{|}^{2}\geq Na\Delta_{m}\bigg{)}\leq\mathfrak{C}+\log m-\frac{a}{4}

(233)

Proof.

Define, for $t\in[0,\Delta_{m})$ ,

f^{N}_{t}=\sum_{j\in I_{N}}\sup_{0\leq i\leq m-1}\sup_{p\in I_{M}}\big{(}y^{p,j}_{t+t^{(m)}_{i}}-y^{p,j}_{t^{(m)}_{i}}\big{)}^{2}

Notice that $t\to f^{N}_{t}$ is a submartingale. Thus, writing $a=(4\Delta_{m})^{-1}$ , $\exp\big{(}af^{N}_{t}\big{)}$ is a submartingale. Therefore, thanks to Doob’s Submartingale Inequality,

$\displaystyle\mathbb{P}\big{(}f^{N}_{t}\geq Nx\big{)}$	$\displaystyle\leq\mathbb{E}\bigg{[}\exp\big{(}af^{N}_{T}-aNx\big{)}\bigg{]}$	(234)
	$\displaystyle\leq\big{\{}mM\big{(}1-2\Delta_{m}\bar{\sigma}a\big{)}^{-1/2}\big{\}}^{N}\exp(-aNx)$	(235)
	$\displaystyle=\big{\{}mM2^{1/2}\big{\}}^{N}\exp\big{(}-Nx/(4\Delta_{m})\big{)}$	(236)

∎

References

[1] Robert Adler and Jonathan Taylor. Random Fields and Geometry, volume 53. Springer, 2019.
[2] Ahmed El Alaoui, Andrea Montanari, and Mark Sellke. Optimization of mean-field spin glasses. Arxiv Preprint, 2020.
[3] G Ben Arous and A Guionnet. Large deviations for langevin spin glass dynamics. Probability Theory and Related Fields, 102, 1995.
[4] Gerard Ben Arous, Amir Dembo, and Alice Guionnet. Cugliandolo-kurchan equations for dynamics of spin-glasses. Probability Theory and Related Fields, 136:619–660, 2006.
[5] Gerard Ben Arous and Alice Guionnet. Symmetric langevin spin glass dynamics. The Annals of Probability, 25:1367–1422, 1997.
[6] Gerard Ben Arous, Song Mei, Andrea Montanari, and Mihai Nica. The landscape of the spiked tensor model. Communications on Pure and Applied Mathematics, 72:2282–2330, 11 2019.
[7] Nicolas Brunel. Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. Journal of Computational Neuroscience, 8:183–208, 2000.
[8] Nicolas Brunel and Xiao-Jing Wang. What determines the frequency of fast network oscillations with irregular neural discharges? i. synaptic dynamics and excitation-inhibition balance. Journal of Neurophysiology, 90:415–430, 2003.
[9] Amarjit Budhiraja and Paul Dupuis. Analysis and Approximation of Rare Events, volume 94. Springer, 2019.
[10] Tanguy Cabana and Jonathan D. Touboul. Large deviations for randomly connected neural networks: I. spatially extended systems. Advances in Applied Probability, 50:983–1004, 2018.
[11] Tanguy Cabana and Jonathan D. Touboul. Large deviations for randomly connected neural networks: Ii. state-dependent interactions. Advances in Applied Probability, 50:983–1004, 2018.
[12] B. Cessac. Linear response in neuronal networks: from neurons dynamics to collective response. 5 2019.
[13] Patrick Charbonneau, Enzo Marinari, Mark Mezard, Giorgio Parisi, Federico Ricci-Tersenghi, Gabriella Sicuro, and Francesco Zamponi, editors. Spin Glass Theory and Far Beyond. World Scientific, 2023.
[14] A. Crisanti and H. Sompolinsky. Path integral approach to random neural networks. Physical Review E, 98:1–20, 2018.
[15] Andrea Crisanti, Heinz Horner, and H.J. Sommers. The spherical p-spin interaction spin-glass model. Zeitschrift fur Physik B Condensed Matter, 92:257–271, 1993.
[16] L. F. Cugliandolo and J. Kurchan. Analytical solution of the off-equilibrium dynamics of a long-range spin-glass model. Physical Review Letters, 71:173–176, 1993.
[17] Amir Dembo and Eliran Subag. Dynamics for spherical spin glasses: Disorder dependent initial conditions. Journal of Statistical Physics, 181:465–514, 2020.
[18] Amir Dembo and Ofer Zeitouni. Large Deviations Techniques and Applications 2nd Edition. Springer, 1998.
[19] Diego Fasoli and Stefano Panzeri. Stationary-state statistics of a binary neural network model with quenched disorder. Entropy, pages 1–30, 2019.
[20] Olivier Faugeras and James MacLaurin. Asymptotic description of neural networks with correlated synaptic weights. Entropy, 17:4701–4743, 2015.
[21] Olivier Faugeras, Emilie Soret, and Etienne TanrÃ©. Asymptotic behaviour of a network of neurons with random linear interactions. Preprint HAL Id : hal-01986927, 2019.
[22] David Gamarnik. The overlap gap property: A topological barrier to optimizing over random structures. Proceedings of the National Academy of Scientists, 2021.
[23] M Grunwald. Sanov results for glauber spin-glass dynamics. Probability Theory and Related Fields, 106:187–232, 1996.
[24] A Guionnet. Averaged and quenched propagation of chaos for spin glass dynamics. Probability Theory and Related Fields, 109:183–215, 1997.
[25] Alice Guionnet and Boguslaw Zegarlinski. Decay to equilibrium in random spin systems on a lattice. Journal of Statistical Physics, 86:899–904, 1997.
[26] Moritz Helias and David Dahmen. Statistical Field Theory for Neural Networks. Springer, 2020.
[27] Ioannis Karatzas and Steven Shreve. Brownian motion and stochastic calculus 2nd edition, 1991.
[28] Itamar Daniel Landau and Haim Sompolinsky. Coherent chaos in a recurrent neural network with structured connectivity. PLoS Computational Biology, 14:1–27, 2018.
[29] George Lindgren, Holger Rootzen, and Maria Sandsten. Stationary Stochastic Processes for Scientists and Engineers. Chapman Hall, 2013.
[30] Eric Lucon. Quenched large deviations for interacting diffusions in random media. Journal of Statistical Physics, 166:1405–1440, 2017.
[31] Olivier Moynot and Manuel Samuelides. Large deviations and mean-field theory for asymmetric random recurrent neural networks. Probability Theory and Related Fields, 123:41–75, 5 2002.
[32] Gabriel Koch Ocker, Yu Hu, Michael A. Buice, Brent Doiron, Kresimir Josic, Robert Rosenbaum, and Eric Shea-Brown. From the statistics of connectivity to the statistics of spike times in neuronal networks. Current Opinion in Neurobiology, 46:109–119, 2017.
[33] Robert Rosenbaum and Brent Doiron. Balanced networks of spiking neurons with spatially dependent recurrent connections. Physical Review X, pages 1–9, 2014.
[34] Robert Rosenbaum, Matthew A. Smith, Adam Kohn, Jonathan E. Rubin, and Brent Doiron. The spatial structure of correlated neuronal variability. Nature Neuroscience, 20:107–114, 2017.
[35] Kai Segadlo, Bastian Epping, Alexander Van Meegen, David Dahmen, Michael KrÃ¤mer, and Moritz Helias. Unified field theoretical approach to deep and recurrent neuronal networks. Journal of Statistical Mechanics: Theory and Experiment, 2022, 10 2022.
[36] H. Sompolinsky, A. Crisanti, and H. J. Sommers. Chaos in random neural networks. Physical Review Letters, 61:259–262, 1988.

	$\displaystyle\sup_{p,q\in I_{M}\fatsemi s.t\in[0,T]}\big{\|}\mathfrak{W}^{\mu,pq}_{st}-\mathfrak{W}^{\nu,pq}_{st}\big{\|}$	$\displaystyle\leq C_{b}\lim_{\epsilon\to 0^{+}}\sup_{r\in I_{M}\fatsemi s\in[0,T]}\mathbb{E}^{\zeta_{\epsilon}}\big{[}\big{(}\lambda(z^{r}_{s})-\lambda(y^{r}_{s})\big{)}^{2}\big{]}$
	$\displaystyle\sup_{p,q\in I_{M}\fatsemi s.t\in[0,T]}\big{\|}\mathfrak{m}^{p}_{s}(\mu,\mathbf{g})-\mathfrak{m}^{p}_{s}(\nu,\mathbf{g})\big{\|}$	$\displaystyle\leq C_{b}\big{(}1+\\|\mathbf{g}\\|\big{)}\lim_{\epsilon\to 0^{+}}\sup_{r\in I_{M}\fatsemi s\in[0,T]}\mathbb{E}^{\zeta_{\epsilon}}\big{[}\big{(}\lambda(z^{r}_{s})-\lambda(y^{r}_{s})\big{)}^{2}\big{]}$