Roundtrip: A Deep Generative Neural Density Estimator

The key idea of Roundtrip is to approximate the target distribution as a convolution of a Gaussian with a distribution induced on a manifold by transforming a base distribution where the transformation is learned by joint training of two GAN models (Figure 1). Density estimation is an offline algorithm which is typically conducted after the two GAN models are well trained in Roundtrip. Next, we will first introduce our framework on how to model densities with deep generative networks before providing details on training strategy and model architecture.

Consider two random variables $\textbf{z}\in\mathbb{R}^{m}$ and $\textbf{x}\in\mathbb{R}^{n}$ where z has a known density $p(\textbf{z})$ (e.g., standard Gaussian) and x is distributed according to a target density $p(\textbf{x})$ that we intend to estimate based on $i.i.d$ observations from it. We introduced two functions $G(\cdot)$ and $H(\cdot)$ for learning an forward and backward mapping relationship between the two distributions. These two functions are learned by two neural networks (Figure 1). The model architecture is similar to CycleGAN (Zhu et al. 2017) while we intend to exploit it for a new task of density estimation. To do this, we denote $G(\textbf{z})=\tilde{\textbf{x}}$ and $H(\textbf{x})=\tilde{\textbf{z}}$ and assume that the forward mapping error follows a Gaussian distribution

Typically, we set $m<n$, which means that $\tilde{\textbf{x}}$ takes values in a manifold of $\mathbb{R}^{n}$ with intrinsic dimension $m$. Basically, this roundtrip model utilizes $G(\cdot)$ to produce a manifold and then approximate the target density as a mixture of Gaussians where the mixing density is the induced density $p(\tilde{\textbf{x}})$ on the manifold. In what follows, we will set $p(\textbf{z})$ to be a standard Gaussian $p(\textbf{z})=(\frac{1}{\sqrt{2\pi}})^{m}e^{-\frac{\left\|\textbf{z}\right\|_{2}^{2}}{2}}$. Based on the model assumption, $p(\textbf{x}|\textbf{z})=(\frac{1}{\sqrt{2\pi}\sigma})^{n}e^{-\frac{\left\|\textbf{x}-G(\textbf{z})\right\|_{2}^{2}}{2\sigma^{2}}}$. Then, the target density can be expressed as

where $v(\textbf{x},\textbf{z})=\left\|\textbf{z}\right\|_{2}^{2}+\sigma^{-2}\left\|\textbf{x}-G(\textbf{z})\right\|_{2}^{2}$. The density estimation problem has been transformed to computing the integral in equation (3). We will postpone model training details to section 2.3-2.5. Assuming that $G(\cdot)$ and $H(\cdot)$ have already been well learned, we now discuss how to evaluate integral in (3) by either importance sampling or Laplace approximation.

The Roundtrip model consists a pair of two GAN models. For the forward GAN mapping, $G$ aims at generating samples $\{\tilde{\textbf{x}}_{i}\}_{i=1}^{N}$ that are similar to observation data $\{\textbf{x}_{i}\}_{i=1}^{N}$ while the discriminator $D_{x}$ tries to discern observation data (positive) from generated samples (negative). The backward mapping function $H$ and the discriminator $D_{z}$ aims to transform the data distribution to approximate the base distribution in latent space. Discriminators can be considered as binary classifiers where a input data point will be asserted to be positive (1) or negative (0). The objective loss functions of the above four neural networks ($G$,$H$,$D_{z}$, and $D_{x}$) in the training process can be represented as the following

where z and x are sampled from base density $p(\textbf{z})$ and data density $p(\textbf{x})$, respectively. In practice, sampling x from data density $p(\textbf{x})$ can be regarded as a procedure of randomly sampling from $i.i.d$ obervations data with replacement. Minimizing the loss of a generator (e.g., $\mathcal{L}_{GAN}(G)$) and the corresponding discriminator (e.g., $\mathcal{L}_{GAN}(D_{x})$) are somehow contradictory as the two networks ($G$ and $D_{x}$) compete with each other during the training process. Note that the least square loss functions we used in equation (12) has been detailedly discussed in LSGAN (Mao et al. 2017).

During the training, we also aim to minimize the roundtrip loss which is defined as $\rho(\textbf{z},H(G(\textbf{z})))$ and $\rho(\textbf{x},G(H(\textbf{x})))$ where z and x are sampled from the base density $p(\textbf{z})$ and the data density $p(\textbf{x})$. The principle is to minimize the distance when a data point goes through a roundtrip transformation between two data domains. If $m$¡$n$, this will ensure that $\textbf{x}\to H(\textbf{x})\to G(H(\textbf{x}))$ will stay close the projection of x to the manifold induced by $G$, and $\textbf{z}\to G(\textbf{z})\to H(G(\textbf{z}))$ will stay close to z. In practice, we used $\textit{l}_{2}$ loss for both $\rho(\textbf{z},H(G(\textbf{z})))$ and $\rho(\textbf{x},G(H(\textbf{x})))$ as minimizing $\textit{l}_{2}$ loss implies the data is drawn from a Gaussian distribution (Mathieu, Couprie, and LeCun 2015), which exactly matches our model assumption. We denoted the roundtrip loss as

where $\alpha$ and $\beta$ are two constant coefficients. The idea of roundtrip loss which exploits transitivity for regularizing structured data can also be found in previous works (Zhu et al. 2017; Yi et al. 2017).

Combining the adversarial training loss and roundtrip loss together, we can get the full training loss for generator networks and discriminator networks as $\mathcal{L}(G,H)=\mathcal{L}_{GAN}(G)+\mathcal{L}_{GAN}(H)+\mathcal{L}_{RT}(G,H)$ and $\mathcal{L}(D_{x},D_{z})=\mathcal{L}_{GAN}(D_{x})+\mathcal{L}_{GAN}(D_{x})$, respectively. To achieve joint training of the two GAN models, we iteratively updated the parameters in the two generative models ($G$ and $H$) and the two discriminative models ($D_{z}$ and $D_{x}$), respectively. Thus, the overall iterative optimization problem in Roundtrip can be represented as

After an iterative model training process, the learned networks $G^{*}$ and $H^{*}$ will then be used as $G(\cdot)$ and $H(\cdot)$ functions in the density estimation procedure. Note that traditional GAN-based models lack a robust stop criteria for model training. However, the training of Roundtrip can be easily evaluated by monitoring the average log likelihood of the validation set. We stop training Roundtrip when there is no further improvement of the average log likelihood on the validation set (see training details in Section 3.1).

The model architecture for Roundtrip is highly flexible. In most cases, when it is utilized for density estimation tasks with vector-valued data, we used fully-connected networks for both generative networks and discriminative networks. Specifically, the $G$ network contains 10 fully-connected layers and each layer has 512 hidden nodes while the $H$ network contains 10 fully-connected layers and each layer has 256 hidden nodes. The $D_{x}$ network contains 4 fully-connected layers and each layer has 256 hidden nodes while the $D_{z}$ network contains 2 fully-connected layers and each layer has 128 hidden nodes. The leaky-Relu activation function is deployed as a non-linear transformation in each hidden layer.

We also extended Roundtrip for estimating the density of tensor-valued data (e.g., images) by introducing a one-hot encoded class label y as an additional input to both $G$ and $D_{x}$ networks in a conditional GAN (CGAN) manner (Mirza and Osindero 2014). y will be combined in the hidden representations in $G$ and $D_{x}$ networks by concatenation. Compared to vector-valued data, tensor-valued data such as images will be flattened to and reshaped from vector-valued data when taken as input and output to all networks in Roundtrip, respectively. Similar to the model architecture in DCGAN (Radford, Metz, and Chintala 2015), we used transposed convolutional layers for upsampling images from latent space for $G$ network. Besides, we used traditional convolutional neural networks for $H$, $D_{x}$ while $D_{z}$ still adopts a fully-connected network architecture. Note that Batch normalization (Ioffe and Szegedy 2015) is applied after each convolutional layer or transposed convolutional layer (detailed hyperparameters were provided in Appendix E).

We test the performance of Roundtrip model in a series of experiments, including simulation studies and real data studies. In these experiments, we compared Roundtrip to the widely used Gaussian kernel density estimator as well as several neural density estimators, including MADE (Germain et al. 2015), Real NVP (Dinh, Sohl-Dickstein, and Bengio 2016) and MAF (Papamakarios, Pavlakou, and Murray 2017). In the outlier detection experiment, we additionally compared to two commonly used outlier detection methods: One-class SVM (Schölkopf et al. 2001) and Isolation Forest (Liu, Ting, and Zhou 2008). Note that the default setting of Roundtrip model was based on the importance sampling strategy. Results of Roundtrip density estimator based on Laplace approximation are reported in Appendix C.

The neural networks in Roundtrip model were implemented with TensorFlow (Abadi et al. 2016). In all experiments, we set $\alpha$=10 and $\beta$=10 in equation (13). For the parameter $\sigma$ in our model assumption, we first pretrained the Roundtrip model for 20 epoches and selected from $\{0.01,0.05,0.1,0.2,0.4,0.5\}$ of which the value maximizes the average likelihood on validation test. Sample size $N$ in importance sampling is set to 40,000. An Adam optimizer (Kingma and Ba 2014) with a learning rate of 0.0002 was used for backpropagation and updating model parameters. We stopping model training when there is no improvement on the average log-likelihood in the validation set in 10 consecutive epochs (early stopping).

We took Gaussian kernel density estimator (KDE) as a baseline where the bandwidth is selected by Silverman’s ”rule of thumb” (Silverman 1986) or Scott’s rule (Scott 1992). We choose the one with better results to present. The three alternative neural density estimators (MADE, Real NVP, and MAF) were implemented from https://github.com/gpapamak/maf. In outlier detection tasks, we implemented One-class SVM and Isolation Forest using scikit-learn library (Pedregosa et al. 2011), where the default parameters were used. To ensure fair model comparison, both simulation and real data were randomly split into a 90% training set and a 10% test set. For neural density estimators including Roundtrip, 10% of the training set was kept as a validation set. The image datasets with training and test set were directly provided which require no further data split.

For simulation datasets with two dimensions, we directly visualized both true density and estimated density on a 2D bounded region. For simulation datasets with higher dimensions where the true density can be calculated, we evaluate different density estimators by calculating the Spearman (rank) correlation between true density and estimated density based on the test set. For real data where the ground truth density is not available, the average estimated density (natural log-likelihood) on the test set will be considered as a measurement.

In the application of outlier detection, we measure performance by calculating the precision at $k$, which is defined as the proportion of correct results in the top $k$ ranks. We set $k$ to the number of outliers in the test set.

We first designed three 2D simulation datasets to test the performance of different neural density estimators where the truth density can be calculated.

(a) Independent Gaussian mixture.
$x_{i}\sim\frac{1}{3}(N(-1,0.5^{2})+N(0,0.5^{2})+N(1,0.5^{2}))$, $i$=1,2.

(b) 8-octagon Gaussian mixture.
$\textbf{x}\sim\frac{1}{8}\sum_{i=1}^{8}N(\bm{\mu}_{i},\mathbf{\Sigma}_{i})$ where $\bm{\mu}_{i}=(3\cos\frac{\pi i}{4},3\sin\frac{\pi i}{4})$ and $\mathbf{\Sigma}_{i}=\bigl{(}\begin{smallmatrix}\cos^{2}\frac{\pi i}{4}+0.16^{2}\sin^{2}\frac{\pi i}{4}&(1-0.16^{2})\sin\frac{\pi i}{4}\cos\frac{\pi i}{4}\\ (1-0.16^{2})\sin\frac{\pi i}{4}\cos\frac{\pi i}{4}&\sin^{2}\frac{\pi i}{4}+0.16^{2}\cos^{2}\frac{\pi i}{4}\end{smallmatrix}\bigr{)}$, $i$=1,…,8.

(c) Involute.
$x_{1}\sim N(r\sin(2r),0.4^{2})$, $x_{2}\sim N(r\cos(2r),0.4^{2})$ where $r\sim U(0,2\pi)$.

20000 $i.i.d$ points were sampled from each of the above true data distribution. After model training, we directly estimated the density in a 2D bounded region (100$\times$100 grid) with different methods (Figure 2). For the independent Gaussian mixture in case (a), Roundtrip clearly separates the independent components in the Gaussian mixture while other neural density estimators either failed (MADE) or contain obvious trajectory between different components (Real NVP and MAF). Roundtrip can capture a better density distribution even for the highly non-linear structure in case (c). Then we took the case (a) for a further study by increasing the dimension up to 10 (containing $3^{10}$ modes). The performance of kernel density estimator (KDE) will decrease dramatically when dimension increases. Roundtrip still achieves a Spearman correlation of 0.829 at dimension 10, compared to 0.669 of Real NVP, 0.595 of MAF, and 0.14 of KDE (See Appendix C).

Finally, we applied Roundtrip model to an outlier detection task, where a data point with extremely low density value is regarded as likely to be an outlier. We tested this method on three outlier detection datasets (Shuttle, Mammography, and ForestCover) from ODDS database (http://odds.cs.stonybrook.edu/). Each dataset is split into training, validation and test set (details of data description can be found in Appendix D). Besides the neural density estimators, we also introduced two baselines One-class SVM (Schölkopf et al. 2001) and Isolation Forest (Liu, Ting, and Zhou 2008). The results shown in Table 2 were based on the average precision of three independent running of each algorithm. Roundtrip achieves the best or comparable results in different outlier detection tasks. Especially in the last dataset ForestCover, in which the outlier percentage is only 0.9%, Roundtrip still achieves a precision of 17.7% while the precision of other neural density estimators is less than 6%.