Mean Field Game GAN

Given a generator network $G_{\theta}$ and a discriminator network $D_{\omega}$, the original GAN objective is to find an mapping that maps a random noise to an expected distribution, it is written as a minimax problem

The loss function is derived from the cross-entropy between the real and generated distributions. Since we never know the ground truth distribution, when compared with other generative modeling approaches, GAN has lots of computational advantages. In GAN framework, it is easy to do sampling without requiring Markov chain, we utilize expectations which is also easy to compute to construct differentiable loss function. Finally, the framework naturally combine the advantages of neural networks, avoiding curse of dimensionality and is extended to various GANs with different objective functions. The disadvantage of original GAN is that the training process is delicate and unstable due to the reasons theoretically investigated in [2].

Wasserstein distance provides a better and more natural measure to evaluate the distance between two distributions[29]. The objective function of Wasserstein GAN is constructed on the basis of Kantorovich-Rubinstein duality form [28] for distributions $\rho_{1}$ and $\rho_{2}$

Given duality form, one would like to rewrite the duality form in a GAN framework by setting generator $G_{\theta}$, discriminator $D_{\omega}$ and then minimizing the Wasserstein distance between target distribution and generated distribution

Though WGAN has made a great success in various applications, it brings a well-known Lipschitz constraint problem, namely, we need to make our discriminator satisfies $\|\nabla D_{\omega}\leq 1\|$. Lipschitz constraint in WGAN has drawn a lot of attention of researchers, several methods such as weight clipping[3], adding gradient penalty as regularizer[13], applying weight normalization[27] and spectral normalization[24]. These methods enable convenience training process, however, besides these approximation-based method, we seek for a new formulation to avoid Lipschitz constraint in WGAN.

A variety of other GANs have been proposed in recent years. These GANs enlarge the application areas and provide great insights of designing generators and discriminators to facilitate the training process. One line of the work is focused on changing objective function such as f-GAN[25], Conditional-GAN[23], LS-GAN[22] and Cycle-GAN[30]. Another line is focused on developing stable structure of generator and discriminator, including DCGAN[26], Style-GAN[31] and Big-GAN[6]. Though these GANs has made a great success, there is no "best" GAN[20].

Hamilton-Jacobi equation in density space (HJD) plays an important role in optimal control problems[4, 7, 14]. HJD determines the global information of the system[10, 11] and describes the time evolution of the optimal value in density space. In applications, HJD has been shown very effective at modeling population differential games, also known as MFGs which study strategic dynamical interactions in large populations by extending finite players’ differential games[16, 12]. Within this context, the agents reach a Nash equilibrium at the optimum, where they can no longer unilaterally improve their objectives. The solutions to these problems are obtained by solving the following partial differential equations (PDEs)[9]

where $\rho(x,t)$ represents the population density of individual $x$ at time $t$. $\Phi(x,t)$ represents the velocity of population. We also have each player’s potential energy $f$ and terminal cost $g$. The Hamiltonian $H$ is defined as

where $L$ can be freely chosen.

Furthermore, a game is called a potential game when there exists a differentiable potential energy $F$ and terminal cost $G$, such that the MFG can be modeled as a optimal control problem in the density space[9]

where the infimum is taken among all vector fields $v(x,t)$ and densities $\rho(x,t)$ subject to the continuity equation

Our method is derived from this formulation, the details will be introduced in the next section.

To better understand the connections between MFGs and WGAN, we also would like to have a review on optimal transport(OT). OT rises as a popular topic in recent years, OT-based theories and formulations have been widely used in fluid mechanics[1], control[8], GANs[3] as well as PDE[21]. We study OT problem because it provides an optimal map $T$ to pushforward one probability distribution $\mu$ to another distribution $\nu$, with minimum cost $c$. One formulation is Monge’s optimal transport problem[28]

However, the solution of Monge problem may not exist and even if the solution exists, it may not be unique. Another formulation called Kantorovich’s optimal transport problem[5] reads

Notably, the fluid dynamic formulation of OT[5] reads

where $v(x,t)$ is the velocity vector. We solve this problem by applying Lagrange multiplier $\Phi(x,t)$ and optimality conditions, finally we have its dual form

We also refer readers to another related dynamical formulation of OT [17].

Let’s rewrite 6 and 7 with defining flux function $m(x,t)=\rho(x,t)v(x,t)$, then we have

with

We solve above primal-dual problem by applying Lagrange multiplier $\Phi(x,s)$

After integration by parts and reordering it leads to

where

Now let $L(x,v)=\frac{\|v\|^{p}}{p}$ where $\|\cdot\|$ can be any norm in Euclidean space. For example, choose $\|\cdot\|=\|\cdot\|_{2}$, namely, $\|v\|=\|v\|_{2}=\sqrt{v_{1}^{2}+v_{2}^{2}+...+v_{n}^{2}}$. For arbitrary norm $s$, we have the minimizer of 16 and corresponding $H$ as

Furthermore we have

Now we are able to plug $H(\nabla\Phi)$ back into 15, let $T=1$ and make $F$ and $G$ as 0, then

Next, we extend 21 to a GAN framework. We let $\rho(z,t)$ be the generator that pushforward Gaussian noise distribution to target distribution. Specially, we set $\rho_{1}$ as the distribution of our ground truth data. We treat $\Phi(x,t)$ as discriminator thus it leads to our objective function

We solve this minimax problem by algorithm 1. Notably, if we extend 22 to a general case with a general Hamiltonian $H$, we revise 22 as

Now we are going to derive the same objective function from the perspective of OT. Base on the formulation 11, we directly apply Lagrange multiplier $\rho(x,s)$ on the constraint to reformulate it as a saddle problem

We extend 24 to a GAN framework: we let $\rho(z,s)$ be the generator and $\Phi(x,s)$ as the discriminator thus finally we achieve the same problem

which has the same structure with 22, thus 22 can also be seen as minimizing Wasserstein-2 distance. A similar formulation is introduced in our recent work [19], where we consider two known distributions and compute their Wasserstein geodesic.

Syn-1: In this case we test our algorithm on 2D case, the target distribution is a Gaussian $\mathcal{N}(\mu,\Sigma)$, where $\mu$=(5,5) and $\Sigma$= [[1,0],[0,1]], here we choose $q=10,p=1.1$. The generated distributions are shown in Figure 1 (a) and (b).

Syn-2: Here and in all following tests we set $q=2,p=2$, in this case we still generate a 2D Gaussian distribution, with the same mean and covariance matrix we used in Syn-1. The generated distributions are shown in Figure 1 (c) and (d).

Syn-3: In this case we generate a 10D Gaussian as our target distribution. Some of 2D projections of the true distributions and generated distributions are shown in Figure 2.

MNIST: We first test our algorithm on MNIST data set. The generated handwritten digits are shown in Figure 3. We believe more carefully designed neural networks such as CNN with extra time input dimension will improve the quality of generated pictures.