LQG Mean Field Games with a Major Agent: Nash Certainty Equivalence versus Probabilistic Approach

Mean field game (MFG) systems with major and minor agents were first introduced in [5] in an LQG setting, where there is a major agent (whose impact does not vanish in the limit of infinite population size) together with a population of minor agents (where each agent has individually asymptotically negligible effect). In the introduced setting the major agent’s state appears in both dynamics and the cost functional of each minor agent. Moreover, each agent is interacting with the average state of minor agents through couplings in the dynamics and cost functionals. As a result, the mean field for such systems is a progressively measurable stochastic process with respect to the filtration generated by the major agent’s Wiener process. In [5], the author uses the Nash certainty equivalence to establish the existence of Markovian closed-loop $\epsilon$-Nash equilibria and derive the individual agents’ explicit control laws which together yield an equilibrium. This methodology is extended in [8] for a general nonlinear case where the major agent’s state appears in nonlinear dynamics and cost functional of individual minor agents, and all agents are coupled with the empirical distribution of minor agents’ state. The best-response strategy of an agent in the limiting case is formulated as the solution to a set of coupled forward backward (FB) PDEs, i.e. a Hamilton Jacobi Bellman equation and a Fokker–Planck–Kolmogorov equation. Subsequently, it is shown that the set of best-response strategies yields a Markovian closed-loop $\epsilon$-Nash equilibrium for the system. This methodology which mainly uses the dynamic programming principle is known as the analytic approach in some literature.

A probabilistic approach to major-minor (MM) MFG systems by using the stochastic maximum principle is developed in [3], where the authors establish the existence of open-loop $\epsilon$-Nash equilibria for a general case as the solutions to a set of FBSDEs, and provide the explicit solutions for an LQG case. In [3, Section 6], it is discussed in detail that the obtained open-loop equilibrium is different from the Markovian closed-loop equilibrium derived in [5] for the LQG case. Following this work an alternative probabilistic formulation is proposed in [2], where the stochastic maximum principle is used and the search for Nash equilibria in the infinite-population limit is formulated as the search for fixed points in the space of best response control maps for the major and minor agents. Using this method, the authors retrieve the same set of FBSDEs as in [3] characterizing the open-loop equilibrium without explicitly solving it. This is while the paper does not present any comparison between the obtained Markovian closed-loop Nash equilibrium and that of the existing work [5]. Therefore, the paper is inconclusive about this important aspect.

The MFG master equation methodology, which encapsulates the MFG system in a backward nonlinear PDE in infinite dimension, is used in [7, 1] to characterize Nash equilibria for a general MM MFG system. Moreover, [1] shows that the solution of the finite-population MM MFG Nash system converges to the solution of the system of master equations as the number of minor agents tends to infinity.

The solutions to MM LQG MFG systems obtained using the above discussed methods are seemingly different . Consequently, in order to address the questions in the MFG community related to the consistency of the Nash certainty equivalence solutions [5] with the ones obtained via other methods in the literature, the following works have emerged. [4] uses a variational analysis to retrieve the Markovian solutions of [5], where no assumption is imposed on the mean field evolution a priori. Moreover, [6] establishes that the Nash certainty equivalence solutions [5] are equivalent to the Markovian closed-loop solutions obtained via the master equations [7] and asymptotic solvability. The current work serves as the last piece of the long-standing puzzle about MM LQG MFG systems. We demonstrate that the Markovian closed-loop Nash equilibria obtained through the Nash certainty equivalence [5] and the probabilistic approach [2] for the limiting MM LQG MFG systems are equivalent. (For the detailed analysis related to the derivation of consistency equations, best response strategies and $\epsilon$-Nash property we refer the reader to [5], [4] and [2].)

In this paper we first introduce finite-population MM LQG MFGs in Section 2. Next, we present the Nash certainty equivalence solutions ([4], [5]) in Section 4. We then present the Markovian closed-loop solutions obtained via the probabilistic approach ([2]) in Section 5. Finally, we show that the two solutions are equivalent in Section 6.

We consider a large population $N$ of minor agents, each denoted by $\mathcal{A}_{i},i\in{\mathfrak{N}}:=\{1,\dots,N\},\,N<\infty$, and a major agent denoted by $\mathcal{A}_{0}$. To capture the essence of the two approaches we consider a simple LQG case (for a general case with heterogeneous minor agents see [4, 6, 5]), where the major and minor agents’ states, respectively, satisfy

	$$\displaystyle dx_{t}^{0}=[A_{0}\,x_{t}^{0}+F_{0}\,x^{(N)}_{t}+B_{0}\,u_{t}^{0}]\,dt+\sigma_{0}\,dw^{0}_{t},$$		(1)
	$$\displaystyle dx_{t}^{i}=[A\,x_{t}^{i}+F\,x^{(N)}_{t}+Gx^{0}_{t}+B\,u_{t}^{i}]\,dt+\sigma\,dw^{i}_{t},$$		(2)

for $t\in{\mathfrak{T}}=[0,T]$, $i\in{\mathfrak{N}}$. Here $x^{i}_{t}\in\mathbb{R}^{n},\leavevmode\nobreak\ i\in{\mathfrak{N}}_{0}:=\{0,\dots,N\}$, are the states, $(u^{i}_{t})_{t\in{\mathfrak{T}}}\in\mathbb{R}^{m},\leavevmode\nobreak\ i\in{\mathfrak{N}}_{0}$, are the control inputs, $w=\{(w^{i}_{t})_{t\in{\mathfrak{T}}},w^{i}_{t}\in\mathbb{R}^{r},i\in{\mathfrak{N}}_{0}\}$ denotes $(N+1)$ independent standard Wiener processes. Moreover, $x^{(N)}_{t}:=\tfrac{1}{N}\sum_{i\in{\mathfrak{N}}}x^{i}_{t}$ denotes the average state of the minor agents. All matrices in (1) and (2) are constant and of appropriate dimension.

We denote $\|a\|_{B}^{2}\coloneqq a^{\intercal}Ba$, where $a$ and $B$ are matrices of appropriate dimension. We also denote $u^{-0}\coloneqq(u^{1},\dots,u^{N})$ and $u^{-i}\coloneqq(u^{0},\dots,u^{i-1},u^{i+1},\dots,u^{N})$. Then the cost functionals for a major agent and a minor agent $\mathcal{A}_{i}$, $i\in{\mathfrak{N}}$, are given by

	$$\displaystyle J_{0}^{N}(u^{0},u^{-0})=\tfrac{1}{2}\mathbb{E}\int_{0}^{T}\Big{\{}\|x^{0}_{t}-\Phi^{(N)}_{t}\|^{2}_{Q_{0}}+\|u^{0}_{t}\|_{R_{0}}^{2}\Big{\}}dt,\allowdisplaybreaks$$		(3)
	$$\displaystyle J_{i}^{N}(u^{i},u^{-i})=\tfrac{1}{2}\mathbb{E}\int_{0}^{T}\Big{\{}\|x^{i}_{t}-\Psi^{(N)}_{t}\|_{Q}^{2}+\|u^{i}_{t}\|_{R}^{2}\Big{\}}\,dt,\allowdisplaybreaks$$		(4)
	$$\displaystyle\Phi^{(N)}_{t}:=H_{0}\,x^{(N)}_{t}+\eta_{0},\quad\Psi^{(N)}_{t}\leavevmode\nobreak\ :=\leavevmode\nobreak\ H\,x^{0}_{t}+{\widehat{H}}\,x^{(N)}_{t}+\eta.$$		(5)

The dynamics and cost functional of the major agent $\mathcal{A}_{0}$ and a generic minor agent $\mathcal{A}_{i}$ in the infinite-population case are given by

	$$\displaystyle dx_{t}^{0}=[A_{0}\,x_{t}^{0}+F_{0}\,\bar{x}_{t}+B_{0}\,u_{t}^{0}]\,dt+\sigma_{0}\,dw^{0}_{t},\allowdisplaybreaks$$		(6)
	$$\displaystyle dx_{t}^{i}=[A\,x_{t}^{i}+F\,\bar{x}_{t}+Gx^{0}_{t}+B\,u_{t}^{i}]\,dt+\sigma\,dw^{i}_{t},\allowdisplaybreaks$$		(7)
	$$\displaystyle J_{0}^{N}(u^{0})=\tfrac{1}{2}\mathbb{E}\int_{0}^{T}\Big{\{}\|x^{0}_{t}-H_{0}\bar{x}_{t}+\eta_{0}\|^{2}_{Q_{0}}+\|u^{0}_{t}\|_{R_{0}}^{2}\Big{\}}dt,\allowdisplaybreaks$$		(8)
	$$\displaystyle J_{i}^{N}(u^{i},u^{0})=\tfrac{1}{2}\mathbb{E}\int_{0}^{T}\Big{\{}\|x^{i}_{t}-Hx^{0}_{t}-{\widehat{H}}\bar{x}_{t}-\eta\|_{Q}^{2}+\|u^{i}_{t}\|_{R}^{2}\Big{\}}\,dt,$$		(9)

where the mean field is defined as $\bar{x}_{t}:=\lim_{N\rightarrow\infty}\frac{1}{N}\sum_{j\in{\mathfrak{N}}}x^{j}_{t}$, if the limit exists. It is equivalently defined as the expected value $\bar{x}_{t}=\mathbb{E}[x^{i}_{t}|\mathcal{F}^{0}_{t}]$ of the state of a generic minor agent $\mathcal{A}_{i}$ given the information set $\mathcal{F}^{0}_{t}$ defined below.

Information Sets. We define (i) the major agent’s information set $\mathcal{F}^{0}:=(\mathcal{F}^{0}_{t})_{t\in{\mathfrak{T}}}$ as the filtration generated by $(w^{0}_{t})_{t\in{\mathfrak{T}}}$, and (ii) a generic minor agent $\mathcal{A}_{i}$’s information set $\mathcal{F}^{i}\coloneqq(\mathcal{F}_{t}^{i})_{t\in{\mathfrak{T}}}$ as the filtration generated by $(w^{i}_{t},w^{0}_{t})_{t\in{\mathfrak{T}}}$.

In the Nash certainty equivalence approach, first an a priori dynamics for the mean field is derived. Then the idea is to Markovianize (i) the major agent’s limiting system by extending its state with the mean field, and (ii) a generic minor agent’s limiting system by extending its state with the major agent’s state and the mean field. This state extension leads to a set of decoupled classical optimal control problems for individual agents, which are linked with each other through the major agent’s state and the mean field. Given the individual information sets, each agent can solve its own stochastic optimal control problem to obtain a best-response strategy. Subsequently, a Nash equilibrium is defined as the set of the best-response Markovian closed-loop strategies of individual agents such that they collectively generate the same mean field that was used in the first step to obtain the best response strategies. This yields a set of consistency equations, the fixed-point solution of which characterizes the Nash equilibrium. ([4] uses a variational analysis and obtains the same Nash equilibrium without assuming an a priori mean field evolution.)

Mean Field Evolution. According to [5, 6], if a generic minor agent adopts a Markovian linear closed-loop strategy, $\bar{x}_{t}$ satisfies

$$d\bar{x}_{t}=\left(\bar{A}\,\bar{x}_{t}+\bar{G}\,x^{0}_{t}+\bar{m}(t)\right)dt,$$

(10)

where $\bar{A},\,\bar{G}\in\mathbb{R}^{n\times n}$ and $\bar{m}\in\mathbb{R}^{n}$ are functions of the fixed-point solutions to the consistency equations (18)-(26). Now we present the agents’ Markovianized systems.

Major Agent. From (6) and (10) the major agent’s extended state $X^{0}_{t}=[x^{0\intercal}_{t}\,\,\bar{x}_{t}^{\intercal}]^{\intercal}$ satisfies

	$$\displaystyle dX^{0}_{t}=\left(\mathbb{A}_{0}X^{0}_{t}+\mathbb{B}_{0}u^{0}_{t}+\mathbb{M}_{0}\right)dt+\Sigma_{0}dw^{0}_{t},$$		(11a)
	$$\displaystyle\mathbb{A}_{0}=\begin{bmatrix}A_{0}&F_{0}\\ \bar{G}&\bar{A}\end{bmatrix},\mathbb{M}_{0}=\begin{bmatrix}0\\ \bar{m}\end{bmatrix},\mathbb{B}_{0}=\begin{bmatrix}B_{0}\\ 0\end{bmatrix},\Sigma_{0}=\begin{bmatrix}\sigma_{0}\\ 0\end{bmatrix}.$$		(11b)

The major agent’s cost functional in terms of $X^{0}_{t}$ is given by

	$$\displaystyle J_{0}^{\infty}(u^{0})=\tfrac{1}{2}\mathbb{E}\int_{0}^{T}\Big{\{}\|X_{s}^{0}\|^{2}_{\mathbb{Q}_{0}}+\|u^{0}_{s}\|_{R_{0}}^{2}-2(X_{s}^{0})^{\intercal}\bar{\eta}_{0}\Big{\}}ds,\allowdisplaybreaks$$		(12a)
	$$\displaystyle\mathbb{Q}_{0}=\|\left[{\mathbb{I}}_{n},-H_{0}\right]\|^{2}_{Q_{0}},\quad\bar{\eta}_{0}=\left[{\mathbb{I}}_{n},-H_{0}\right]^{\intercal}Q_{0}\eta_{0}.$$		(12b)

According to [4, Thm 5] (the finite-horizon version of [5, Thm 10] for general MM LQG MFG systems), the best response strategy for the major agent is given by

	$$\displaystyle u^{0,*}_{t}=-R_{0}^{-1}\mathbb{B}_{0}^{\intercal}\big{(}\Pi_{0}(t)X_{t}^{0,*}+s_{0}(t)\big{)},\allowdisplaybreaks$$		(13a)
	$$\displaystyle-\dot{\Pi}_{0}=\Pi_{0}\mathbb{A}_{0}+\mathbb{A}_{0}^{\intercal}\Pi_{0}-\Pi_{0}\mathbb{B}_{0}R_{0}^{-1}\mathbb{B}_{0}^{\intercal}\Pi_{0}+\mathbb{Q}_{0},\allowdisplaybreaks$$		(13b)
	$$\displaystyle-\dot{s}_{0}=\left[\mathbb{A}_{0}^{\intercal}-\Pi_{0}\,\mathbb{B}_{0}\,R_{0}^{-1}\,\mathbb{B}_{0}^{\intercal}\right]s_{0}+\Pi_{0}\mathbb{M}_{0}-\bar{\eta}_{0},$$		(13c)
	$$\displaystyle\Pi_{0}(T)=0,\leavevmode\nobreak\ s_{0}(T)=0.$$		(13d)

Minor Agent. From (6)-(7) and (10), for a generic minor agent $\mathcal{A}_{i}$, the extended state $X^{i}_{t}=[x^{i\intercal}_{t},x^{0\intercal}_{t},\bar{x}_{t}^{\intercal}]^{\intercal}$ is governed by the dynamcis

	$$\displaystyle dX^{i}_{t}=\left(\mathbb{A}X^{i}_{t}+\mathbb{B}\,u^{i}_{t}+\mathbb{M}(t)\right)dt+\Sigma\,dW^{i}_{t},\allowdisplaybreaks$$		(14a)
	$$\displaystyle\mathbb{A}=\begin{bmatrix}A&[G\,\,\,F]\\ 0&\mathbb{A}_{0}-\mathbb{B}_{0}R_{0}^{-1}\mathbb{B}_{0}^{\intercal}\Pi_{0}\end{bmatrix},\leavevmode\nobreak\ \mathbb{B}=\left[\begin{array}[]{c}B\\ 0\end{array}\right],\leavevmode\nobreak\ W^{i}_{t}=\begin{bmatrix}w^{i}_{t}\\ w^{0}_{t}\end{bmatrix},\allowdisplaybreaks$$		(14d)
	$$\displaystyle\mathbb{M}=\begin{bmatrix}0\\ \mathbb{M}_{0}-\mathbb{B}_{0}R_{0}^{-1}\mathbb{B}_{0}^{\intercal}s_{0}\end{bmatrix},\,\,\Sigma=\begin{bmatrix}\sigma&0\\ 0&\Sigma_{0}\end{bmatrix}.$$		(14e)

The cost functional for $\mathcal{A}_{i},\,i\in{\mathfrak{N}}$, in terms of $X^{i}_{t}$ is given by

	$$\displaystyle J_{i}^{\infty}(u^{i})=\tfrac{1}{2}\mathbb{E}\int_{0}^{T}\Big{\{}\|X_{s}^{i}\|_{\mathbb{Q}}^{2}+\|u_{s}^{i}\|^{2}_{R}-2(X_{s}^{i})^{\intercal}\,\bar{\eta}\Big{\}}ds,\allowdisplaybreaks$$		(15a)
	$$\displaystyle\mathbb{Q}=\|[{\mathbb{I}}_{n},-H,-{\widehat{H}}]\|^{2}_{Q},\quad\bar{\eta}=[{\mathbb{I}}_{n},-H,-{\widehat{H}}]^{\intercal}Q\eta.$$		(15b)

According to [4, Thm 5] and [5, Thm 10], the best response strategy for a generic minor agent $\mathcal{A}_{i}$ is given by

	$$\displaystyle u^{i,*}_{t}=-R^{-1}\mathbb{B}^{\intercal}\big{(}\Pi(t)\,X_{t}^{i,*}+s(t)\big{)},\allowdisplaybreaks$$		(16a)
	$$\displaystyle-\dot{\Pi}=\Pi\mathbb{A}+\mathbb{A}^{\intercal}\Pi-\Pi\mathbb{B}R^{-1}\mathbb{B}^{\intercal}\Pi+\mathbb{Q},\quad\Pi(T)=0,\allowdisplaybreaks$$		(16b)
	$$\displaystyle-\dot{s}=[\mathbb{A}^{\intercal}-\Pi\mathbb{B}R^{-1}\mathbb{B}^{\intercal}]s+\Pi\mathbb{M}-\bar{\eta},\quad s(T)=0.$$		(16c)

Mean Field Consistency Equations. We first define

$$\Pi_{k}=\begin{bmatrix}\Pi_{11}&\bar{{\Pi}}_{12}\\ \bar{\Pi}_{21}&\bar{\Pi}_{22}\end{bmatrix},\quad\bar{{\Pi}}_{12}=\begin{bmatrix}{\Pi}_{12}&{\Pi}_{13}\end{bmatrix},\quad s=\begin{bmatrix}s_{1}\\ \bar{s}_{2}\end{bmatrix},$$

(17)

where $\Pi_{11},\Pi_{12},\Pi_{13}\in{\mathds{R}}^{n\times n},\,\bar{\Pi}_{22}\in{\mathds{R}}^{2n\times 2n},\,\bar{\Pi}_{12}\in{\mathds{R}}^{n\times 2n}$, $\bar{\Pi}_{21}\in{\mathds{R}}^{2n\times n},\,s_{1}\in{\mathds{R}}^{n},\,\bar{s}_{2}\in{\mathds{R}}^{2n}$. Then according to the Nash certainty equivalence approach [5], the consistency equations are obtained by effectively equating (10) with the mean field equation resulting from the collective action of the mass of minor agents. Subsequently the consistency equations determining $\bar{A}$, $\bar{G}$, $\bar{m}$ are given by

		$\displaystyle-\dot{\Pi}_{0}=\Pi_{0}\mathbb{A}_{0}+\mathbb{A}_{0}^{\intercal}\Pi_{0}-\Pi_{0}\mathbb{B}_{0}R_{0}^{-1}\mathbb{B}_{0}^{\intercal}\Pi_{0}+\mathbb{Q}_{0},$			(18)
		$\displaystyle-\dot{\Pi}=\Pi\mathbb{A}+\mathbb{A}^{\intercal}\Pi-\Pi\mathbb{B}R^{-1}\mathbb{B}^{\intercal}\Pi+\mathbb{Q},$			(19)
		$\displaystyle\bar{A}=A-BR^{-1}B^{\intercal}\Pi_{11}+F-BR^{-1}B^{\intercal}\Pi_{13},$			(20)
		$\displaystyle\bar{G}=G-BR^{-1}B^{\intercal}\Pi_{12},$			(21)
		$\displaystyle\Pi_{0}(T)=0,\quad\Pi(T)=0,$			(22)

		$\displaystyle-\dot{s}_{0}=[\mathbb{A}_{0}^{\intercal}-\Pi_{0}\mathbb{B}_{0}R_{0}^{-1}\mathbb{B}_{0}^{\intercal}]s_{0}+\Pi_{0}\mathbb{M}_{0}-\bar{\eta}_{0},$			(23)
		$\displaystyle-\dot{s}=[\mathbb{A}^{\intercal}-\Pi\mathbb{B}R^{-1}\mathbb{B}^{\intercal}]s+\Pi\mathbb{M}-\bar{\eta},$			(24)
		$\displaystyle\bar{m}=-BR^{-1}\mathbb{B}^{\intercal}s,$			(25)
		$\displaystyle s_{0}(T)=0,\quad s(T)=0.$			(26)

Proof. Given that $\mathbb{B}^{\intercal}\Pi=B^{\intercal}\big{[}\Pi_{11}\,\,\bar{\Pi}_{12}\big{]}$, the optimal control (16) of the minor agent $\mathcal{A}_{i}$ is given by

$${u}^{i,\ast}_{t}=-R^{-1}B^{\intercal}\Big{(}\Pi_{11}x^{i}_{t}+\bar{\Pi}_{12}{\big{[}{x^{0}_{t}}^{\intercal}\leavevmode\nobreak\ \leavevmode\nobreak\ {\bar{x}_{t}}^{\intercal}\big{]}}^{\intercal}+s_{1}\Big{)}.$$

(37)

Hence only the first block row of $\Pi$ and $s$ appear in a generic minor agent’s optimal control and the other blocks are irrelevant. Therefore we use (19) and (24) to derive the equations that $\Pi_{11}$, $\bar{\Pi}_{12}$ and $s_{1}$ satisfy. To this end, we first treat the terms in (19) one by one. Block multiplications for the first and the second terms on the right hand side of (19) yield

	$$\displaystyle\Pi\mathbb{A}=\begin{bmatrix}\Pi_{11}A&\,\,\,E_{1}^{\intercal}\\ \bar{\Pi}_{21}A&\,\,\,E_{2}^{\intercal}\end{bmatrix},\vspace{0.2cm}\quad\mathbb{A}^{\intercal}\Pi=\begin{bmatrix}A^{\intercal}\Pi_{11}&\hskip 5.69046ptA^{\intercal}\bar{\Pi}_{12}\\ E_{1}&E_{2}\end{bmatrix},$$		(38)
	$$\displaystyle E_{1}=[G\,\,F]^{\intercal}{\Pi}_{11}+(\mathbb{A}_{0}-\mathbb{B}_{0}R_{0}^{-1}\mathbb{B}_{0}^{\intercal}\Pi_{0})^{\intercal}\bar{\Pi}_{21},\allowdisplaybreaks$$		(39)
	$$\displaystyle E_{2}=[G\,\,F]^{\intercal}\bar{\Pi}_{12}+(\mathbb{A}_{0}-\mathbb{B}_{0}R_{0}^{-1}\mathbb{B}_{0}^{\intercal}\Pi_{0})^{\intercal}\bar{\Pi}_{22}.$$		(40)

For the third and forth terms in (19) we have

	$$\displaystyle\Pi\mathbb{B}R^{-1}\mathbb{B}^{\intercal}\Pi=\begin{bmatrix}\Pi_{11}BR^{-1}B^{\intercal}\Pi_{11}&\Pi_{11}BR^{-1}B^{\intercal}\bar{\Pi}_{12}\\ 0&0\end{bmatrix},{}\allowdisplaybreaks$$
	$$\displaystyle\mathbb{Q}=\begin{bmatrix}Q&\hskip 5.69046pt-Q[H\,\,{\widehat{H}}]\\ -[H\,\,{\widehat{H}}]^{\intercal}Q&\,\,\,[H\,\,{\widehat{H}}]^{\intercal}Q[H\,\,{\widehat{H}}]\end{bmatrix}.$$		(41)

From (19) and (38)-(41), and through block by block correspondence, we obtain the ODEs that $\Pi_{11}$ and $\bar{\Pi}_{12}$ satisfy as in (36) and (28), respectively.

Similarly, block multiplications for the terms in (24) result in

	$$\displaystyle\Pi\mathbb{B}R^{-1}\mathbb{B}^{\intercal}s=\begin{bmatrix}\Pi_{11}BR^{-1}B^{\intercal}s_{1}\\ 0\end{bmatrix},\quad\bar{\eta}=\begin{bmatrix}Q\eta\\ -[H\,\,{\widehat{H}}]^{\intercal}Q\eta\end{bmatrix},{}\allowdisplaybreaks$$
	$$\displaystyle\Pi\mathbb{M}=\begin{bmatrix}\bar{\Pi}_{12}(\mathbb{M}_{0}-\mathbb{B}_{0}R_{0}^{-1}\mathbb{B}_{0}^{\intercal}s_{0})\\ \bar{\Pi}_{22}(\mathbb{M}_{0}-\mathbb{B}_{0}R_{0}^{-1}\mathbb{B}_{0}^{\intercal}s_{0})\end{bmatrix}.\leavevmode\nobreak\$$		(42)

From (24) and (42), $s_{1}$ satisfies (33). $\hfill\square$

In [2], the search for Nash equilibria for MM MFGs is formulated as the search for fixed points in the space of best response control maps for the major and minor agents in the infinite-population limit. In this section, we present the approach of [2] for obtaining a Markovian closed-loop equilibrium for MM LQG MFG systems.

To be self-contained, in Table 1 we match the notations used in the current work and in [2] for presenting the model parameters and the processes. Otherwise, the notations in the two works are the same.

In [2] Markovian linear closed-loop control actions as in Assumption 3 are considered for the major agent and a representative minor agent, denoted, respectively, by $\alpha^{0}_{t}$ and $\alpha^{i}_{t}$. The mean field dynamics is then obtained by forming the closed-loop system for the representative agent $\mathcal{A}_{i}$ using $\alpha^{i}_{t}$ and taking the conditional expectation $\mathbb{E}[x^{i}_{t}|\mathcal{F}^{0}_{t}]$ of its state $x^{i}_{t}$ given the information set $\mathcal{F}^{0}_{t}$. Subsequently, to obtain the solutions to the major agent’s problem, its state is extended with the mean field in the same manner as in [5]. Then using the stochastic maximum principle for the extended system, the major agent’s optimal control action is obtained as

$$\alpha^{0,\ast}_{t}=-(2R_{0})^{-1}\big{[}0\,\,B_{0}^{\intercal}\big{]}\mathbb{Y}_{t},$$

(43)

and a set of McKean-Vlasov FBSDEs is derived which solves for the major agent’s extended state and the decoupling field (adjoint process) $\mathbb{Y}_{t}$. To solve the FBSDEs, an ansatz is adopted for $\mathbb{Y}_{t}$ as in

$$\mathbb{Y}_{t}=K_{t}{\big{[}{\bar{x}_{t}}^{\intercal}\leavevmode\nobreak\ \leavevmode\nobreak\ {x^{0}_{t}}^{\intercal}\big{]}}^{\intercal}+k_{t},$$

(44)

where $K_{t},k_{t},$ are deterministic matrices of appropriate dimension. Then a set of ODEs that $K_{t}$ and $k_{t}$ satisfy is derived. Subsequently the notion of a deviating minor agent is introduced as an extra virtual minor agent who deviates from the strategy of its peers and aims to optimize in response to the major agent and the rest of minor agents. However, when dealing with the optimal control problem for the deviating minor agent, the major agent’s state and the mean field are considered as exogenous stochastic coefficients, which are determined offline by solving a set of SDEs, i.e. the major agent’s closed-loop extended system. Then using the stochastic maximum principle, the deviating minor agent’s optimal control is obtained as in

$$\alpha^{i,\ast}_{t}=(2R)^{-1}B^{\intercal}Y^{i}_{t},$$

(45)

and a set of FBSDEs with random coefficients (not of McKean-Vlasov type), that the minor agent’s state $x^{i}_{t}$ and decoupling field (adjoint process) $Y^{i}_{t}$ satisfy, are derived. To solve the FBSDEs, an ansatz is considered for $Y_{t}^{i}$ as in

$$Y^{i}_{t}=\mathbb{S}_{t}{\big{[}{\bar{x}_{t}}^{\intercal}\leavevmode\nobreak\ \leavevmode\nobreak\ {x^{0}_{t}}^{\intercal}\big{]}}^{\intercal}+S_{t}x^{i}_{t}+\bar{s}_{t},$$

(46)

where $\mathbb{S}_{t}$, $S_{t}$, and $\bar{s}_{t}$ are matrices of appropriate dimension. The mean field equation resulting from a minor agent using $\alpha^{i,\ast}_{t}$ must match the one obtained using $\alpha^{i}_{t}$ to solve for the exogenous stochastic coefficients $[{\bar{x}_{t}}^{\intercal}\leavevmode\nobreak\ {x^{0}_{t}}^{\intercal}]^{\intercal}$ in the minor agent’s optimal control problem. Subsequently the consistency equations whose fixed-point solutions determine $K_{t},k_{t}$ and $\mathbb{S}_{t},S_{t},\bar{s}_{t}$, are given by ([2, eq. (31)-(32)])

		$\displaystyle-\dot{K}_{t}=K_{t}[\mathbb{L}_{t}-\bar{\mathbb{B}}(2R)^{-1}B^{\intercal}\mathbb{S}_{t}]-K_{t}\bar{\mathbb{B}}_{0}(2R_{0})^{-1}\bar{\mathbb{B}}^{\intercal}K_{t}{}$
		$\displaystyle{\hskip 28.45274pt}+[\mathbb{L}_{t}-\bar{\mathbb{B}}(2R)^{-1}B^{\intercal}\mathbb{S}_{t}]^{\intercal}K_{t}+2\mathbb{F}_{0},$			(47)
		$\displaystyle-\dot{\mathbb{S}}_{t}=\mathbb{S}_{t}[\mathbb{L}_{t}-\bar{\mathbb{B}}(2R)^{-1}B^{\intercal}\mathbb{S}_{t}]-\mathbb{S}_{t}\bar{\mathbb{B}}_{0}(2R_{0})^{-1}\bar{\mathbb{B}}_{0}^{\intercal}K_{t}{}$
		$\displaystyle{\hskip 28.45274pt}+[L^{\intercal}-S_{t}B(2R)^{-1}B^{\intercal}]\mathbb{S}_{t}{}$
		$\displaystyle{\hskip 28.45274pt}+[S_{t}F-2QH_{1}\,\,\,S_{t}G-2QH],$			(48)
		$\displaystyle K_{T}=0,\,\,\mathbb{S}_{T}=0,$			(49)

		$\displaystyle-\dot{k}_{t}=[\mathbb{L}_{t}-\bar{\mathbb{B}}(2R)^{-1}B^{\intercal}\mathbb{S}_{t}]k_{t}-K_{t}\bar{\mathbb{B}}_{0}(2R_{0})^{-1}\bar{\mathbb{B}}_{0}^{\intercal}k_{t}{}$
		$\displaystyle\hskip 28.45274pt-K_{t}\bar{\mathbb{B}}(2R)^{-1}B^{\intercal}\bar{s}_{t}+2f_{0},$			(50)
		$\displaystyle-\dot{\bar{s}}_{t}=[L^{\intercal}-S_{t}B(2R)^{-1}B^{\intercal}]\bar{s}_{t}-\mathbb{S}_{t}\bar{\mathbb{B}}_{0}(2R_{0})^{-1}\bar{\mathbb{B}}_{0}^{\intercal}k_{t}{}$
		$\displaystyle\hskip 28.45274pt-\mathbb{S}_{t}\bar{\mathbb{B}}_{0}(2R)^{-1}B^{\intercal}\bar{s}_{t}-2Q\eta,$			(51)
		$\displaystyle k_{T}=0,\,\,\bar{s}_{T}=0,$			(52)

where

	$$\displaystyle-\dot{S}_{t}=S_{t}L+L^{\intercal}S_{t}-S_{t}B(2R)^{-1}B^{\intercal}S_{t}+2Q,\quad S_{T}=0,$$		(53)
	$$\displaystyle\mathbb{L}=\begin{bmatrix}L+F-B(2R)^{-1}B^{\intercal}S_{t}&G\\ F_{0}&L_{0}\end{bmatrix},\,\,\,\bar{\mathbb{B}}=\begin{bmatrix}B\\ 0\end{bmatrix},\,\,\,\bar{\mathbb{B}}_{0}=\begin{bmatrix}0\\ B_{0}\end{bmatrix},\,\,{}\allowdisplaybreaks$$
	$$\displaystyle\mathbb{F}_{0}=\begin{bmatrix}H_{0}^{\intercal}Q_{0}H_{0}&-H_{0}^{\intercal}Q_{0}\\ -Q_{0}H_{0}&Q_{0}\end{bmatrix},\,\,\leavevmode\nobreak\ f_{0}=\begin{bmatrix}H_{0}^{\intercal}Q_{0}\eta_{0}\\ -Q_{0}\eta_{0}\end{bmatrix}.$$		(54)

We start with the following theorem.

Proof. By inspection, the Markovian linear optimal control laws $\{u^{0,\ast},u^{i,\ast},i\in{\mathfrak{N}}\}$ obtained through the Nash certainty equivalence (see (13a), (37)) have the same structure as the ones $\{\alpha^{0,\ast},\alpha^{i,\ast},i\in{\mathfrak{N}}\}$ obtained through the probabilistic approach (see (43)-(46)). It remains to show the equivalency of the sets of consistency equations, the fixed-point solutions of which yield the coefficients in the above control laws. More specifically, we show that the reduced consistency equations (27)-(36) obtained via the Nash certainty equivalence are the same as the consistency equations (47)-(53) obtained via the probabilistic approach. To this end, we first define a block elementary operator which operates on a matrix to produce the desired interchanged block rows, as in

$${\mathfrak{I}}=\begin{bmatrix}0&{\mathbb{I}}\\ {\mathbb{I}}&0\end{bmatrix},$$

(55)

where the identity matrices $\mathbb{I}$ are of appropriate dimension. Then we correspond the processes in (27)-(36) with those in (47)-(53) as shown in Table 2.

Using Tables 1-2, we can retrieve (53) from (36) or vice versa. Now we correspond the terms in (27)-(28) to those in (47)-(48). First we use Table 1 to replace the system parameters in (27) with those considered in [2] (i.e. replace $Q_{0},R_{0},$ respectively, with $2Q_{0},2R_{0},$ in (27)). Then as per Table 2 to obtain $K_{t}$ we multiply both sides of (27) from right by ${\mathfrak{I}}$ and from left by ${\mathfrak{I}}^{\intercal}$ as in

$$-{\mathfrak{I}}^{\intercal}\dot{\Pi}_{0}{\mathfrak{I}}={\mathfrak{I}}^{\intercal}(\Pi_{0}\mathbb{A}_{0}+\mathbb{A}_{0}^{\intercal}\Pi_{0}-\Pi_{0}\mathbb{B}_{0}(2R_{0})^{-1}\mathbb{B}_{0}^{\intercal}\Pi_{0}+2\mathbb{Q}_{0}){\mathfrak{I}},$$

which by inspection is the same equation as (47), particularly

	$$\displaystyle{\mathfrak{I}}^{\intercal}\Pi_{0}\mathbb{A}_{0}{\mathfrak{I}}=K_{t}[\mathbb{L}_{t}-\bar{\mathbb{B}}(2R)^{-1}B^{\intercal}\mathbb{S}_{t}],\quad 2{\mathfrak{I}}^{\intercal}\mathbb{Q}_{0}{\mathfrak{I}}=2\mathbb{F}_{0},{}\allowdisplaybreaks$$
	$$\displaystyle{\mathfrak{I}}^{\intercal}\Pi_{0}\mathbb{B}_{0}(2R_{0})^{-1}\mathbb{B}_{0}^{\intercal}\Pi_{0}{\mathfrak{I}}=K_{t}\bar{\mathbb{B}}_{0}(2R_{0})^{-1}\bar{\mathbb{B}}^{\intercal}K_{t}.$$		(56)

Next we use Tables 1-2 to match (28) and (48). We first replace $R_{0},R,Q,\hat{H}$, respectively with $2R_{0},2R,2Q,H_{1}$ in (28), and then right multiply both sides by ${\mathfrak{I}}$ which gives us equation (48). Now we show that (50) can be retrieved from (32). From Tables 1-2, we first replace $R_{0},Q_{0}$ with $2R_{0},2Q_{0}$ in (32), and then left multiply its both sides by ${\mathfrak{I}}$ as in

$$-{\mathfrak{I}}\dot{s}_{0}={\mathfrak{I}}([\mathbb{A}_{0}^{\intercal}-\Pi_{0}\mathbb{B}_{0}(2R_{0})^{-1}\mathbb{B}_{0}^{\intercal}]s_{0}+\Pi_{0}\mathbb{M}_{0}-2\bar{\eta}_{0}),$$

(57)

where $-2{\mathfrak{I}}\bar{\eta}_{0}=2f_{0}$ and

$${\mathfrak{I}}\Pi_{0}\mathbb{M}_{0}=-K_{t}\begin{bmatrix}B(2R)^{-1}B^{\intercal}\bar{s}_{t}\\ 0\end{bmatrix}=-K_{t}\bar{\mathbb{B}}(2R)^{-1}B^{\intercal}\bar{s}_{t}.\\$$

Finally, we replace $R_{0},R,Q,$ respectively, with $2R_{0},2R,2Q$, in (33), which yields (51). This matches (33) and (51). $\hfill\square$