Second order local minimal-time Mean Field Games

Introduced around 2006 by Jean-Michel Lasry and Pierre-Louis Lions [21, 22, 23] and at the same time by Peter Caines, Minyi Huang, and Roland Malhamé [14, 15, 16], the theory of Mean Field Games (MFGs, for short) describes the interaction of a continuum of players, assumed to be rational, indistinguishable, and negligible, when each one tries to solve a dynamical control problem influenced only by the average behavior of the other players (through a mean-field type interaction, using the physicists’ terminology). The Nash equilibrium in these continuous games is described by a system of PDEs: a Hamilton–Jacobi–Bellman equation for the value function of the control problem of each player, where the distribution (density) of the players appears, coupled with a continuity equation describing the evolution of such a density, where the velocity field is the optimal one in order to solve the control problem, and is therefore related to the gradient of the value function. This system is typically forward-backward in nature: the density evolves forward in time starting from a given initial datum, and the value function backward in time, according to Bellman’s dynamical programming principle, and its final value at a given time horizon $T$ is usually known.

The literature about MFG theory is quickly growing and many references are available. The 6-year course given by P.-L. Lions at Collège de France, for which video-recording is available in French [26], explains well the birth of the theory, but the reader can also refer to the lecture notes by P. Cardaliaguet [6], based on the same course.

In most of the MFG models studied so far the agents consider a fixed time interval $[0,T]$ and optimize a trajectory $x:[0,T]\to\Omega$ (where $\Omega\subset\mathbb{R}^{d}$ is the state space) trying to minimize a cost of the form $\int_{0}^{T}L(t,x(t),x^{\prime}(t),\rho_{t})\operatorname{d\!}t+\Psi(x(T),\rho_{T}),$ where $\rho_{t}$ denotes the distribution of players at time $t$. The function $L$ is typically increasing in $\lvert x^{\prime}\rvert$ and, in some sense, in $\rho$. This means that high velocities are costly, and passing through areas where the population is strongly concentrated is also costly. Some MFGs, called MFGs of congestion (see, for instance, [1]), consider costs which include a product of the form $\rho_{t}(x(t))^{\alpha}\lvert x^{\prime}(t)\rvert^{\beta}$ (for some exponents $\alpha,\beta>0$), which means that high velocities are costly, and that they are even more costly in the presence of high concentrations. These models present harder mathematical difficulties compared to those where the cost is decomposed into $L(t,x(t),x^{\prime}(t))+g(t,x(t),\rho_{t})$. Indeed, in many cases the latter MFG admits a variational formulation: equilibria can be found by minimizing a global energy among all possible evolutions $(\rho_{t})_{t}$ (hence, they are potential games). This allows to prove the existence of the equilibrium via semicontinuity methods, and we refer to [4] and [30] for a detailed discussion of this branch of MFG theory.

When the MFG has no variational interpretation, then the existence of a solution is usually obtained via fixed-point theorems, but these theorems require much more regularity. Roughly speaking, given an evolution $\rho$ one computes the corresponding value function $\varphi$ as a solution to a Hamilton–Jacobi–Bellman equation and, given $\varphi$, one computes a new density evolution $\tilde{\rho}$ by following an evolution equation. We need existence, uniqueness, and stability results for these equations in order to find a fixed point $\tilde{\rho}=\rho$. This usually requires regularity of the velocity field $-\nabla\varphi$, which is difficult to prove, and can be essentially only obtained in two different frameworks: either the dependence of the cost functions on the distribution $\rho$ is highly regularizing (which usually means that it is non-local, and passes through averaged quantities such as convolutions $\int\eta(x-y)\operatorname{d\!}\rho(y)$), or diffusion of the agents is taken into account, transforming the optimal control problem into a stochastic one. In this latter case, agents minimize $\mathbb{E}[\int_{0}^{T}L(t,X_{t},\alpha(t),\rho_{t})\operatorname{d\!}t+\Psi(X_{T},\rho_{T})]$ where the process $X$ follows $\operatorname{d\!}X_{t}=\alpha_{t}\operatorname{d\!}t+\operatorname{d\!}B_{t}$ and $(B_{t})_{t\geq 0}$ represents a standard Brownian motion.

In [27], the second and third authors of the present paper introduced a different class of models, called minimal-time MFGs. The main difference is that instead of considering a cost for the players penalizing both the velocity and the density, and minimizing the integral of such a cost on a fixed time interval $[0,T]$, the dynamics is subject to a constraint where the maximal velocity of the agents cannot exceed a quantity depending on the density $\rho_{t}$, and the goal of each agent is to arrive to a given target as soon as possible. In the typical situation, the target of the agents is the boundary $\partial\Omega$ of the domain where the evolution occurs. This can model, for instance, an evacuation phenomenon in crowd motion. The system that one obtains is the following

(1.1)

$$\left\{\begin{aligned} &\partial_{t}\rho-\nabla\cdot\left(\rho k[\rho]\frac{\nabla\varphi}{\lvert\nabla\varphi\rvert}\right)=0,&\quad&\text{ in }(0,T)\times\Omega,\\ &-\partial_{t}\varphi+k[\rho]\lvert\nabla\varphi\rvert-1=0,&&\text{ in }(0,T)\times\Omega,\\ &\rho(0,x)=\rho_{0}(x),&&\text{ in }\Omega,\\ &\varphi(t,x)=0,&&\text{ on }(0,T)\times\partial\Omega,\end{aligned}\right.$$

where the function $k[\rho_{t}](x)$ denotes the maximal speed that agents can have at point $x$ at time $t$, i.e., the dynamics is constrained to satisfy $\lvert x^{\prime}(t)\rvert\leq k[\rho_{t}](x(t))$. Ideally, one would like to choose $k$ to be a non-increasing function of the density itself, such as $k[\rho](x)=(1-\rho(x))_{+}$. This choice is what is done in the well-known Hughes’ model for crowd motion [17, 18]. Indeed, this model is very similar to Hughes’, which also considers agents who aim at leaving in minimal time a bounded domain under a congestion-dependent constraint on their speeds.

The main difference between the model in [27] (from which the present paper stems) and Hughes’ is that, in the latter, at each time, an agent moves in the optimal direction to the boundary assuming that the distribution of agents remains constant, whereas in [27] and here agents take into account the future evolution of the distribution of agents in the computation of their optimal trajectories. This accounts for the time derivative in the Hamilton–Jacobi–Bellman equation from (1.1), which is the main difference between (1.1) and the equations describing the motion of agents in Hughes’ model and stands for the anticipation of future behavior of other agents.

Another crucial (and disappointing) similarity between the above MFG system and Hughes’ model is the fact that general mathematical results do not exist in the case $k[\rho]=(1-\rho)_{+}$ and more generally in the local case (except few results in the Hughes case in 1D). Indeed, the lack of regularity makes the model too hard to study, and the MFG case is not variational.

In some sense the closest MFG model to this one is the one with multiplicative costs in [1] (MFG with congestion). Indeed, an $L^{\infty}$ constraint $\lvert x^{\prime}\rvert\leq k[\rho]$ can be seen as a limit as $m\to\infty$ of an integral penalization

$$\int\left\lvert\frac{\lvert x^{\prime}(t)\rvert}{k[\rho_{t}](x(t))}\right\rvert^{m}\operatorname{d\!}t.$$

Note that the boundaries of the time interval have been omitted on purpose from the above integral, since the model in [1] is set on a fixed time horizon but this is not part of our setting. For MFG with congestion, [1] presents not only existence but also uniqueness results, under the assumption that the exponents appearing in the running cost satisfy a certain inequality. Unfortunately this inequality is never satisfied in the limit $m=+\infty$ as above, and it is not surprising that in our work we are not able to establish uniqueness results for our MFG system. Additional results for MFG with congestion were presented, for instance, in [12], under a smallness assumption on the time horizon, but this assumption cannot be made here, as the model is exactly meant to consider the case where a time horizon is not fixed.

Because of these difficulties, [27] studied the case of a non-local dependence of $k$ w.r.t. $\rho$ (say, $k[\rho](x)=\kappa(\int\eta(x-y)\operatorname{d\!}\rho(y))$, for a non-increasing function $\kappa$ and a positive convolution kernel $\eta$), and proved existence of an equilibrium, characterized it as a solution of a non-local MFG system, and analyzed some examples, including numerical simulations. Instead, in the present paper we want to study the local case with diffusion.

This means that we will consider a local dependence $k[\rho](x):=\kappa(\rho(x))$, and each agent solves a stochastic control problem

$$\inf\left\{\mathbb{E}[\tau]\,:\,X(\tau)\in\partial\Omega,X(0)=x_{0},\,\operatorname{d\!}X_{t}=\alpha_{t}\operatorname{d\!}t+\sqrt{2\nu}\operatorname{d\!}B_{t},\,\lvert\alpha_{t}\rvert\leq\kappa(\rho(t,X_{t}))\right\},$$

where $(B_{t})_{t\geq 0}$ denotes a standard Brownian motion and the Brownian motions for all players are assumed to be mutually independent. Defining the corresponding value function $\varphi$, from classical results on stochastic optimal control (see [10, Chapter IV]), under suitable assumptions, the optimal control is given in feedback form by

$$\alpha_{t}=-\kappa(\rho(t,X_{t}))\frac{\nabla\varphi(t,X_{t})}{\left\lvert\nabla\varphi(t,X_{t})\right\rvert},$$

(a definition which has to be carefully adapted to the case $\nabla\varphi=0$); moreover, the value function solves the Hamilton–Jacobi–Bellman equation

$$-\partial_{t}\varphi(t,x)-\nu\Delta\varphi(t,x)+K(t,x)\left\lvert\nabla\varphi(t,x)\right\rvert-1=0,\quad(t,x)\in[0,T)\times\Omega,$$

for $K=\kappa(\rho)$. Hence, we know the drift of the optimal stochastic processes followed by each agent, and this allows to write the Fokker–Planck equation solved by the law of this process. Putting together all this information, we obtain the following MFG system

(1.2)

$$\left\{\begin{aligned} &\partial_{t}\rho-\nu\Delta\rho-\nabla\cdot\left(\rho\kappa(\rho)\frac{\nabla\varphi}{\lvert\nabla\varphi\rvert}\right)=0,&\quad&\text{ in }\mathbb{R}_{+}\times\Omega,\\ &-\partial_{t}\varphi-\nu\Delta\varphi+\kappa(\rho)\lvert\nabla\varphi\rvert-1=0,&&\text{ in }\mathbb{R}_{+}\times\Omega,\\ &\begin{aligned} \rho(0,x)&=\rho_{0}(x),\\ \rho(t,x)&=0,\quad\varphi(t,x)=0,\end{aligned}&&\begin{aligned} &\text{ in }\Omega,\\ &\text{ on }\mathbb{R}_{+}\times\partial\Omega,\end{aligned}\end{aligned}\right.$$

where $\Omega\subset\mathbb{R}^{d}$ is an open and bounded set, whose boundary will be supposed to be of class $C^{2}$ in this paper, $\nu>0$ is a fixed constant, $\kappa:\mathbb{R}\to(0,+\infty)$, and $\rho_{0}\geq 0$ is the initial density. The Dirichlet condition on $\varphi$ comes as usual from the fact that, for agents who are already on the boundary, the remaining time to reach it is zero, and the Dirichlet condition on $\rho$ comes from the fact that we stop the evolution of a particle as soon as it touches the boundary (absorbing boundary conditions).

A crucial difference with the previous paper [27] concerns the time horizon. If we suppose that $\kappa$ is bounded from below in the model without diffusion, it is not difficult to see that all agents will have left the domain after a common finite time, so that the final value of $\varphi$ is not really relevant, and the problem can be studied on a finite interval $[0,T]$. This is not the case when there is diffusion, as a density following a Fokker–Planck equation with a bounded drift cannot fully vanish in finite time. As a consequence, the model should be studied on the unbounded interval $[0,\infty)$. For every time $t<\infty$ there is still mass everywhere, but this mass decreases to $0$ as $t\to+\infty$, which suggests that the value function $\varphi$ should converge to a function, that we call $\Psi$, which is the value function for the corresponding control problem with no mass, i.e. when $\kappa=\kappa(0)$. Since in this control problem $\kappa$ is independent of time, $\Psi$ is a function of $x$ only and solves a stationary Hamilton–Jacobi–Bellman equation which takes the form of an elliptic PDE

$$-\nu\Delta\Psi+\kappa(0)\lvert\nabla\Psi\rvert-1=0$$

with Dirichlet boundary conditions on $\partial\Omega$. It is then reasonable to investigate whether solutions of the above system satisfy further $\rho_{t}\to 0$ and $\varphi_{t}\to\Psi$ as $t\to+\infty$.

In order to study the above system, we will first study an artificial finite-horizon setting, where we stop the game at time $T$, choose a penalization $\psi:\Omega\to\mathbb{R}_{+}$ with $\psi=0$ on $\partial\Omega$, and look at the stochastic optimal control problem

$$\inf\Big{\{}\mathbb{E}[\min\{\tau,T\}+\psi(X_{\min\{\tau,T\}})]\,:\\ \,X(\tau)\in\partial\Omega,X(0)=x_{0},\,\operatorname{d\!}X_{t}=\alpha_{t}\operatorname{d\!}t+\sqrt{2\nu}\operatorname{d\!}B_{t},\,\lvert\alpha_{t}\rvert\leq\kappa(\rho(t,X_{t}))\Big{\}}.$$

This gives rise to the MFG system

(1.3)

$$\left\{\begin{aligned} &\partial_{t}\rho-\nu\Delta\rho-\nabla\cdot\left(\rho\kappa(\rho)\frac{\nabla\varphi}{\lvert\nabla\varphi\rvert}\right)=0,&\quad&\text{ in }(0,T)\times\Omega,\\ &-\partial_{t}\varphi-\nu\Delta\varphi+\kappa(\rho)\lvert\nabla\varphi\rvert-1=0,&&\text{ in }(0,T)\times\Omega,\\ &\begin{aligned} \rho(0,x)&=\rho_{0}(x),\quad&\varphi(T,x)&=\psi(x),\\ \rho(t,x)&=0,&\varphi(t,x)&=0,\end{aligned}&&\begin{aligned} &\text{ in }\Omega,\\ &\text{ on }(0,T)\times\partial\Omega,\end{aligned}\end{aligned}\right.$$

which corresponds to (1.2) with the unbounded time interval $\mathbb{R}_{+}$ replaced by $(0,T)$ and the additional final condition $\varphi(T,x)=\psi(x)$. We will prove the existence of a solution of the system for finite $T$ (note that for this system, as well as for its infinite-horizon counterpart, we are not able to prove uniqueness), and then consider the limit as $T\to\infty$. In order to guarantee suitable bounds, we just need to choose a sequence of final data $\psi_{T}$, possibly depending on $T$, which is uniformly bounded. We will then get at the limit a solution of the limit system which automatically satisfies $\rho_{t}\to 0$ (in the sense of uniform convergence) and $\varphi_{t}\to\Psi$ (this convergence being both uniform and strong in $H^{1}_{0}$).

The paper is organized as follows. After this introduction, Section 2 presents the tools that we need to study the two separate equations appearing in System (1.3) on a finite horizon, which come from the classical theory of parabolic equations. Section 3 is devoted to the existence of solutions of (1.3). After providing a precise definition of solution of (1.3) taking care of the case $\nabla\varphi=0$, we use the estimates of Section 2 to prove existence via a fixed-point argument based on Kakutani’s theorem. Section 4 concerns the limit $T\to\infty$. In this section, some estimates of Section 2 need to be made more precise, in order to see how constants depend on the time horizon $T$. In this way we are able to prove existence of a limit of the solutions of (1.3) as the time horizon $T$ tends to $+\infty$ and that this limit solves the limit system (1.2). Then we consider the asymptotic behavior of a solution $(\rho,\varphi)$ of (1.2) as $t\to+\infty$, proving first $\rho_{t}\to 0$ in $L^{1}$ and, thanks to a parabolic regularization argument, also in $L^{\infty}$. To prove convergence in $L^{1}$, which is true for general Fokker–Planck systems under very mild assumptions, we exploit the MFG nature of the system, i.e. the coupling between the two equations, which also provides exponential decrease. We then consider the limit in time of $\varphi$, and prove that any bounded solution of this equation, once we know $\kappa(t,x)\to\kappa(0)$, can only converge as $t\to+\infty$ to the stationary function $\Psi$. This convergence is a priori very weak, but we are able to improve it into $L^{\infty}\cap H^{1}_{0}$, and to prove that the uniform convergences of both $\rho$ and $\varphi$ occur exponentially fast. The paper is then completed by an appendix, which details some global $L^{\infty}$ estimates for a large class of parabolic equations, including the estimates that we use to prove uniform convergence in time of $\rho_{t}$ and $\varphi_{t}$ to $0$ and $\Psi,$ respectively. These estimates are not surprising and not difficult to prove, using standard Moser iterations, but are not easy to find in the literature under the sole assumption of boundedness of the drift term in the divergence. The computations and the results are essentially the same as in the appendix of [7], but the boundary conditions are different.

This section presents some preliminary results on Fokker–Planck and Hamilton–Jacobi–Bellman equations which are useful for the analysis of the Mean Field Game systems (1.2) and (1.3). We recall that, in the whole paper, $\Omega$ denotes an open and bounded set whose boundary $\partial\Omega$ is assumed to be $C^{2}$. Even though some of the results presented in this preliminary section also hold without the smoothness assumption on $\partial\Omega$ (such as existence and uniqueness results for both Fokker–Planck and Hamilton–Jacobi–Bellman equations in Propositions 2.2 and 2.5), this assumption is first used to obtain higher regularity of solutions of Hamilton–Jacobi–Bellman equations in Proposition 2.5 and is required for almost all of the subsequent results, including in particular our main results in Sections 3 and 4, as a consequence of the need of higher regularity of $\varphi$.

We now consider the MFG system with a finite time horizon (1.3). One of the difficulties in the analysis of (1.3) is that the velocity field in the continuity equation depends on $\frac{\nabla\varphi}{\lvert\nabla\varphi\rvert}$, which is defined only when $\nabla\varphi\neq 0$. In order to handle this difficulty, we make use of the following definition of weak solution.

The main result of this section is the following.

The proof of Theorem 3.3 relies on a fixed-point argument on the velocity field $V$ of the Fokker–Planck equation in (1.3). Before turning to the proof, we need some continuity results on solutions of (2.1) with respect to the velocity field $V$ and on solutions of (2.4) with respect to the function $K$, which we state and prove now.