Discriminator Contrastive Divergence: Semi-Amortized Generative Modeling by Exploring Energy of the Discriminator

$f$-GAN [27] is based on the variational estimation of $f$-divergence [1] with only samples from two distributions available:

In $f$-GAN [28], the discriminator $D_{\phi}$ is actually the function $T$ parameterized with neural networks. Theorem. 1 indicates the density ratio estimation view of $f$-GAN’s discriminator, as illustrated in [37]. More specifically, the discriminator$D_{\phi}$ in $f$-GAN is optimized to estimate a statistic related to the density ratio between $p_{\text{data}}$ and $p_{G_{\theta}}$, i.e. $\frac{p_{\text{data}}}{p_{G_{\theta}}}$, and the $\frac{p_{\text{data}}}{p_{G_{\theta}}}$ can be acquired easily with $D_{\phi}$. For example, in the original GANs [10], the corresponding $f$ in $f$-GAN literature is $f(x)=x\log x-(x+1)\log(x+1)+2\log 2$. Assuming the discriminator is trained to be optimal, the output is $D_{\phi}(x)=\frac{p_{\text{data}}}{p_{\text{data}}+p_{G_{\theta}}}$, and we can get the density ratio $\frac{p_{\text{data}}}{p_{G_{\theta}}}=\frac{D_{\phi}(x)}{1-D_{\phi}(x)}$. However, it should be noticed that the discriminator is hard to reach the optimality. In practice, without loss of generality, the density ratio implied by a sub-optimal discriminator can be seen as the density ratio between an implicitly defined distribution $p^{*}$ and the generated distribution $p_{G_{\theta}}$. It has been studied both theoretically and empirically in the context of GANs [2, 15, 17], with the same inductive bias, that learning a discriminative model is more accessible than a generative model. Based on the above fact, the rejection-sampling based methods are proposed to use the estimated density ratio, e.g., $\frac{D_{\phi}(x)}{1-D_{\phi}(x)}$ in original GANs, to conduct rejection sampling[3] or Metropolis-Hastings sampling[36] based on generated distribution $p_{G_{\theta}}$. These methods radically modify the generated distribution $p_{G_{\theta}}$ to $p^{*}$, the improvement in empirical performance as shown in [3, 36] demonstrates that we can construct a superior distribution $p^{*}$ to prescribe the empirical distribution $p_{\text{data}}$ by involving the remaining information in discriminator.

Different from $f$-GANs, the objective of WGANs is derived from the Integral Probability Metric, and the discriminator can not naturally be derived as an estimated density ratio. Before leveraging the remaining information in the discriminator, the property of the discriminator in WGANs needs to be investigated first. We introduce the primal problem implied by WGANs objective as follows:

Let $\pi$ denote the joint probability for transportation between $P$ and $Q$, which satisfies the marginality conditions,

The primal form first-order Wasserstein distance $W_{1}$ is defined as:

the objective function of the discriminator in Wasserstein GANs is the Kantorovich-Rubinstein duality of Eq. 7, and the optimal discriminator has the following property[13]:

Theorem. 2 states that for each sample $x$ in the generated distribution $p_{G_{\theta}}$, the gradient on the $x$ directly points to a sample $y$ in the $p_{\text{data}}$, where the $(x,y)$ pairs are consistent with the optimal transport plan $\pi^{*}$. All the linear interpolations $x_{t}$ between $x$ and $y$ satisfy that $\nabla_{x_{k}}D_{\phi}^{*}\left(x_{t}\right)=\frac{y-x}{\|y-x\|}$. It should also be noted that similar results can also be drawn in some variants of WGANs, whose loss functions may have a slight difference with standard WGAN [41]. For example, the SNGAN uses the hinge loss during the optimization of the discriminator, i.e., $r(\cdot)$ and $g(\cdot)$ in Eq. 1 is selected as $\max(0,-1-u)$ for stabilizing the training procedure. We provide a detailed discussion on several surrogate objectives in Appendix. E.

The above property of discriminator in WGANs can be interpreted as that given a sample $x$ from generated distribution $p_{G_{\theta}}$ we can obtain a corresponding $y$ in data distribution $p_{\text{data}}$ by directly conducting gradient decent with the optimal discriminator $D_{\phi}^{*}$:

It seems to be a simple and appealing solution to improve $p_{G_{\theta}}$ with the guidance of discriminator $D_{\phi}$. However, the following issues exist:

1) there is no theoretical indication on how to set $w_{x}$ for each sample $x$ in generated distribution. We noticed that a concurrent work [33] introduce a search process called Discriminator Optimal Transport(DOT) by finding the corresponding $y^{*}$ through the following:

However, it should be noticed that Eq. 9 has a non-unique solution. As indicated by Theorem 2, all points on the connection between $x$ and $y$ are valid solutions. We further extend the fact into the following theorem:

Theorem 3 provides a theoretical justification for the poor empirical performance of conducting DOT in the sample space, as shown in their paper.

2) Another problem lies in that samples distributed outside the generated distribution ($p_{G_{\theta}}$) are never explored during training, which results in much adversarial noise during the gradient-based search process, especially when the sample space is high dimensional such as real-world images.

To fix the issues mentioned above in leveraging the information of discriminator in Wasserstein GANs, we propose viewing the discriminator as an energy function. With the discriminator as an energy function, the stationary distribution is unique, and Langevin dynamics can approximately conduct sampling from the stationary distribution. Due to the monotonic property of MCMC, there will not be issues like setting $w_{x}$ in Eq. 8. Besides, the second issue can also be easily solved by fine-tuning the energy spaces with contrastive divergence. In addition to the benefits illustrated above, if the discriminator is an energy function, the samples from the corresponding energy-based model can be obtained through Langevin dynamics by using the gradients of the discriminator which takes advantage of the property of discriminator as shown in Theorem 2. With all the facts as mentioned above, there is strong motivation to explore further and bridge the gap between discriminator in WGAN and the energy-based model.

Baselines. We evaluated the following models as our baselines: we take PixelCNN [38], PixelIQN [29], and MoLM [30] as representatives of other types of generative models. For the energy-based model, we compared the proposed method with EBM [8] and NCSN [32]. For GAN models, we take WGAN-GP [13], Spectral Normalization GAN (SNGAN) [25], and Progressiv eGAN [19] for comparison. We also take the aforementioned DRS [3], DOT [33] and MH-GAN [36] into consideration. The choices of EBM and GANs are due to their close relation to our proposed method, as analyzed in Section 4. We omit other previous GAN methods since as a representative of a state-of-the-art GAN model, SNGAN and Progressive GAN has been shown to rival or outperform several former methods such as the original GAN [10], the energy-based generative adversarial network [40], and the original WGAN with weight clipping [2].

Evaluation Metrics. For evaluation, we concentrate on comparing the quality of generated images since it is well known that GAN models cannot perform reliable likelihood estimations [35]. We choose to compare the Inception Scores [31] and Frechet Inception Distances (FID) [15] reached during training iterations, both computed from 50K samples. A high image quality corresponds to high Inception and low FID scores. Specifically, the intuition of IS is that high-quality images should lead to high confidence in classification, while FID aims to measure the computer-vision-specific similarity of generated images to real ones through Frechet distance.

Data. We use CIFAR-10 [22] and STL-10 [5], which are all standard datasets widely used in generative literature. STL-10 consists of unlabeled real-world color images, while CIFAR-10 is provided with class labels, which enables us to conduct conditional generation tasks. For STL-10, we also shrink the images into $32\times 32$ as in previous works. The pixel values of all images are rescaled into $[-1,1]$.

Network Architecture. For all experiment settings, we follow Spectral Normalization GAN (SNGAN) [25] and adopt the same Residual Network (ResNet) [14] structures and hyperparameters, which presently is the state-of-the-art implementation of WGAN. Details can be found in Appendix. D. We take their open-source code and pre-trained model as the base model for the experiments on CIFAR-10. For STL-10, since there is no pre-trained model available to reproduce the results, we train the SNGAN from scratch and take it as the base model.

For quantitative evaluation, we report the inception score [31] and FID [15] scores on CIFAR-10 in Tab. 1 and STL-10 in Tab. 2. As shown in the Tab. 1, in pixel space, by introducing the proposed DCD algorithm, we achieve a significant improvement of inception score over the SNGAN. The reported inception score is even higher than most values achieved by class-conditional generative models. Our FID score of $21.67$ on CIFAR-10 is competitive with other top generative models. When the DCD is conducted in the latent space, we further achieve a $9.11$ inception score and a $16.24$ FID, which is a new state-of-the-art performance of IS. When combined with label information to perform conditional generation, we further improve the FID to $15.05$, which is comparable with current state-of-the-art large-scale trained models [4]. Some visualization of generated examples can be found in Fig 3, which demonstrates that the Markov chain is able to generate more realistic samples, suggesting that the MCMC process is meaningful and effective. Tab. 2 shows the performance on STL-10, which demonstrates that as a generalized method, DCD is not over-fitted to the specific data CIFAR-10. More experiment details and the generated samples of STL-10 can be found in Appendix. F.