DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition

Given a pair of NIR-VIS data $I_{N}$ and $I_{V}$ with the same identity, the dual variational generator learns disentangled joint distribution in the latent space. Specifically, a face recognition network pre-trained on MS-Celeb-1M [36] is adopted as the feature extractor $F$. The compact embedding features extracted by $F$ are thought to be only identity related [52]. Meanwhile, considering that $F$ is better at extracting the features of VIS images than those of NIR images, we use $f=F(I_{V})$ as the identity representation of both $I_{N}$ and $I_{V}$. Then, the encoders $E_{N}$ and $E_{V}$ are leveraged to learn domain-specific attribute distributions $z_{N}=q_{\phi_{N}}(z_{N}|I_{N})$ and $z_{V}=q_{\phi_{V}}(z_{V}|I_{V})$, respectively. $\phi_{N}$ and $\phi_{V}$ are the parameters of the encoders. According to the reparameterization trick [15], $z_{N}=u_{N}+\sigma_{N}\odot\epsilon$ and $z_{V}=u_{V}+\sigma_{V}\odot\epsilon$, where $u$ denotes mean and $\sigma$ denotes standard deviation. $\odot$ denotes Hadamard product and $\epsilon$ is a noise sampled from a multi-variate standard Gaussian distribution. Subsequently, in order to ensure that $z_{N}$ and $z_{V}$ are merely attribute related, we impose an angular orthogonal loss between the attribute and the identity representations. Finally, the disentangled identity and attribute representations constitute the joint distribution of paired NIR-VIS data, and then are fed to the decoder $G$ to reconstruct the inputs $I_{N}$ and $I_{V}$.

A total of four loss functions are involved in the above process, including an angular orthogonal loss, a distribution learning loss, a pairwise identity preserving loss, and an adversarial loss. These losses are introduced below.

Angular Orthogonal Loss. The angular orthogonal loss is imposed between $z_{N}$ and $f$ and between $z_{V}$ and $f$. For the $L_{2}$ normalized $z_{V}$ and $f$, their cosine similarity is formulated as:

where $\langle\cdot,\cdot\rangle$ denotes inner product. When the two representations are orthogonal, Eq. (3) is equal to zero. Minimizing the absolute value of Eq. (3) will force $z_{V}$ and $f$ to be orthogonal, and thus they are disentangled. Ultimately, the angular orthogonal loss that considers both normalized $z_{N}$ and $z_{V}$ is defined as:

In the following parts, $z_{N}$, $z_{V}$, and the output of $F(\cdot)$, e.g. $f=F(I_{V})$, all denote the normalized ones.

Distribution Learning Loss. Inspired by VAEs [15], the posterior distributions $q_{\phi_{N}}(z_{N}|I_{N})$ and $q_{\phi_{V}}(z_{V}|I_{V})$ are constrained by a Kullback-Leibler divergence:

where both the priors $p(z_{N})$ and $p(z_{V})$ are multi-variate standard Gaussian distributions. After obtaining the identity representation $f$ as well as the domain-specific attribute representations $z_{N}$ and $z_{V}$, the decoder network is required to reconstruct the inputs $I_{N}$ and $I_{V}$:

In practice, $\mathcal{L}_{\text{rec}}$ is implemented as:

where $\hat{I}_{N}=G(f,z_{N})$ and $\hat{I}_{V}=G(f,z_{V})$.

In general, according to [15], the distribution learning loss is a combination of $\mathcal{L}_{\text{kl}}$ and $\mathcal{L}_{\text{rec}}$:

Pairwise Identity Preserving Loss. In order to preserve the identity of the generated data, previous conditional generation based methods usually adopt an identity preserving loss [10, 53, 14]. A pre-trained face recognition network is used to extract the embedding features of the generated data and those of the real target data respectively, and then the two features are forced to be as close as possible. However, since there are neither intra-class nor inter-class constraints, it is challenging to guarantee that the generated images belong to specific classes consistent with the targets.

As discussed in Section 1, different from the previous methods, we focus on the identity consistency of the generated paired images, rather than whom the generated images belong to. Therefore, we propose a pairwise identity preserving loss that constrains the feature distance between $\hat{f}_{N}=F(\hat{I}_{N})$ and $\hat{f}_{V}=F(\hat{I}_{V})$:

Minimizing Eq. (9) will increase the cosine similarity between $\hat{f}_{N}$ and $\hat{f}_{V}$. Besides, in order to stabilize the training process, we also constrain the feature distance between $\hat{f}_{N}$ and $f$ and that between $\hat{f}_{V}$ and $f$:

The pairwise identity preserving loss that considers both $\mathcal{L}_{\text{ip-pair}}$ and $\mathcal{L}_{\text{ip-rec}}$ is formulated as:

Adversarial Loss. Same as [54], an extra adversarial loss $\mathcal{L}_{\text{adv}}$ [13] is also introduced to increase the sharpness of the generated data, where the dual variational generator and a discriminator are optimized alternately. We first fix the discriminator and train the generator with the adversarial loss to confuse the discriminator:

where $D$ denotes the discriminator. Then, we fix the generator and train the discriminator to distinguish the generated data from the real data:

Overall Loss. The overall loss for the training with paired heterogeneous data is a weighted sum of the above distribution learning loss $\mathcal{L}_{\text{dis}}$, the angular orthogonal loss $\mathcal{L}_{\text{ort}}$, the pairwise identity preserving loss $\mathcal{L}_{\text{ip}}$, and the adversarial loss $\mathcal{L}_{\text{adv}}$:

where $\lambda_{1}$, $\lambda_{2}$, and $\lambda_{3}$ are trade-off parameters. Among them, $\lambda_{3}$ is fixed to 0.1 according to [54].

The number of the paired heterogeneous training data is limited, which may affect the identity diversity of the generated images. Therefore, we introduce abundant identity information of large-scale VIS data into the generated images. For the acquisition of the identity information, a straightforward approach is to use a pre-trained face recognition network to extract identities from the large-scale VIS data, as shown in the lower right corner of Fig. 2 (b). However, in this situation, if we desire to generate large-scale new paired data at the testing phase, we must have the same number of VIS data with different identities. This will make our framework a conditional generative model, leading to the diversity problem as mentioned in Section 1. In order to overcome this obstacle, inspired by [55], we introduce an identity sampler to replace the recognition network. Specifically, we first adopt the recognition network to extract the embedding features of MS-Celeb-1M [36], and then leverage these embedding features to train a VAE model [15]. After the training, the decoder of VAE is used as the identity sampler, which can map the points in a standard Gaussian noise to identity representations, as shown in Fig. 2 (b). Equipped with the identity sampler, our framework becomes an unconditional generative model. The required identity representations can be flexibly sampled from noises. A detailed discussion about the face recognition network and the identity sampler is provided in Section 4.3.4.

Since these sampled identity representations do not have corresponding ground truth paired heterogeneous images, we propose to train the generator in an unpaired way. To begin with, we sample an identity representation $\tilde{f}$ via the identity sampler $F_{s}$. Then, $\tilde{f}$ as well as the attribute representations $z_{N}$ and $z_{V}$ in Eq. (4) are fed into the decoder $G$. Finally, a pair of new heterogeneous images, i.e. $\tilde{I}_{N}=G(\tilde{f},z_{N})$ and $\tilde{I}_{V}=G(\tilde{f},z_{V})$, that does not belong to the heterogeneous database is generated. We constrain $\tilde{I}_{N}$ and $\tilde{I}_{V}$ from the aspects of appearance and semantic. For appearance, we introduce a small-weight reconstruction loss [52] to force the appearance of the generated images to be consistent with that of the inputs:

where $\eta$ is set to $0.1$ according to [52]. For semantic, same as the aforementioned pairwise identity preserving manner in Eq. (11), we constrain $\tilde{f}_{N}=F(\tilde{I}_{N})$ and $\tilde{f}_{V}=F(\tilde{I}_{V})$ via:

Consequently, the overall loss for the training with unpaired VIS data is a weighted sum of $\mathcal{L}_{\text{rec}}^{\text{u}}$, $\mathcal{L}_{\text{ip}}^{\text{u}}$, and an extra adversarial loss $\mathcal{L}_{\text{adv}}^{\text{u}}$ (replacing $\hat{I}_{N}$/$\hat{I}_{V}$ in Eq. (12) and Eq. (13) with $\tilde{I}_{N}$/$\tilde{I}_{V}$):

where $\lambda_{2}$ and $\lambda_{3}$ are the same as those in Eq. (14). Algorithm 1 shows the training process with both paired heterogeneous data and unpaired VIS data.

As discussed in Section 3, the generated paired heterogeneous images are required to have the same identity. In order to verify this property, we propose to measure MS (Mean Similarity) between the paired images. Specifically, we utilize the pre-trained LightCNN to extract the $L_{2}$ normalized embedding features of the paired images, and then calculate their cosine similarity. The MS value is a mean of the similarities of 50K randomly generated paired heterogeneous images. As reported in Table III, the MS value of DVG-Face is 0.53, while that of the real database is 0.31. It is observed that the former is even larger than the latter, demonstrating that our method does in fact guarantee the identity consistency property. In addition, we can see that the preliminary version DVG also has this property due to the usage of the same pairwise identity preserving manner. By contrast, the MS value of CoGAN [42] is only 0.23 that is significantly smaller than that of the real database. This reveals that it is difficult for the weight-sharing manner of CoGAN to ensure the identity consistency property.

Identity diversity plays a crucial role in our method. The more diverse the generated data, the more valuable it will be for the HFR network. Inspired by the instance discrimination criterion [19, 20] that regards each image as an independent class, we propose a MIS (Mean Instance Similarity) metric to measure the identity diversity property. Intuitively, if the generated images have abundant identity diversity, the images obtained from different samplings can be considered to have different identities.

Specifically, by randomly sampling twice from noises, two paired images ($\tilde{I}_{N}^{1}$, $\tilde{I}_{V}^{1}$) and ($\tilde{I}_{N}^{2}$, $\tilde{I}_{V}^{2}$) are generated. Then, we calculate the cosine similarities of the embedding features (extracted by LightCNN) between ($\tilde{I}_{V}^{1}$, $\tilde{I}_{V}^{2}$) and between ($\tilde{I}_{N}^{1}$, $\tilde{I}_{V}^{2}$), yielding IS${}_{\text{VIS-VIS}}$ (Instance Similarity) and IS${}_{\text{NIR-VIS}}$, respectively. After repeating the above sampling and calculation process 50K times, we average these IS${}_{\text{VIS-VIS}}$ and IS${}_{\text{NIR-VIS}}$ values to obtain MIS${}_{\text{VIS-VIS}}$ and MIS${}_{\text{NIR-VIS}}$ values, respectively. As listed in Table III, both CoGAN and DVG get a much higher MIS${}_{\text{VIS-VIS}}$ value than DVG-Face, suggesting the presence of massive similar VIS images in the sampled data of CoGAN and DVG. We also observe that CoGAN gets a low MIS${}_{\text{NIR-VIS}}$ value due to the identity inconsistency between the generated paired heterogeneous images. The lowest MIS${}_{\text{VIS-VIS}}$ and MIS${}_{\text{NIR-VIS}}$ values of DVG-Face prove that it can really generate more diverse data. In addition, when using the generated data to train the HFR network via the contrastive loss in Eq. (19), it is observed that both CoGAN and DVG obtain a much worse recognition performance than DVG-Face. This is because that the validity of the contrastive loss is based on the identity consistency and the identity diversity properties, neither of which is simultaneously satisfied by CoGAN and DVG.

The distribution of the generated data and that of the real data should be consistent, otherwise it will be difficult to boost the recognition performance with the generated data. FID (Fr$\acute{\text{e}}$chet Inception Distance) [64] is employed to measure the distance between these two distributions in the feature space. In addition, considering the advantage of the face recognition network in extracting facial features, we use LightCNN rather than the conventional Inception model as the feature extractor. Specifically, we first generate 50K paired NIR-VIS data, and then calculate the FID value between the generated NIR data and the real NIR data, as well as the FID value between the generated VIS data and the real VIS data. Subsequently, these two FID values are averaged to get the final FID value. Table III suggests that both DVG and DVG-Face obtain a much lower FID value than CoGAN, proving that our method can really learn the distribution of the real data.

Furthermore, Fig. 4 shows the visualization comparisons of CoGAN, DVG, and DVG-Face. We can see that the generated results of CoGAN have many artifacts. It seems difficult for CoGAN to generate good results under such a large domain discrepancy case. Besides, it is observed that the generated paired images of CoGAN are not particularly similar, which is consistent with the MS value in Table III. By contrast, both DVG and DVG-Face generate photo-realistic results, further demonstrating the distribution consistency property of our method.

In the end, it is obvious that the real training data are crucial for the dual variational generator to learn domain distributions. This motivates us to further investigate the effect of the number of the real data on domain learning. Specifically, apart from the whole training set of Tufts Face that has 1,940 images belonging to 50 identities, we also train the generator on 5 subsets that include 5, 10, 20, 30, and 40 identities with 198, 378, 734, 1,130, and 1,499 images, respectively. Subsequently, we measure the domain distances between the generated data of the above settings and the real data of the whole training set via FID. The lower the FID value, the closer the domain of the generated data is to that of the real data. The experimental results in Fig. 5 (left) suggest that as the number of the trained identities increases, the FID value decreases gradually before 30 identities and then tends to be stable. Thus, 30 identities containing 1,130 real training images seem enough for the generator to learn the real domain.

As stated in Section 3.1.2, the identity representation can be obtained from the pre-trained face recognition network (LightCNN [61]) or from the identity sampler. Correspondingly, there are three potential identity sampling approaches: (1) Using LightCNN in both the training and the testing stages of the dual generation. In this case, DVG-Face becomes a conditional generative model. At the testing stage, if we desire to generate large-scale paired data, we must have the same number of VIS data with different identities to provide the required identity representations. Therefore, the sampling diversity is limited by the size of the available VIS data. (2) Using LightCNN in the training stage and the identity sampler in the testing stage. By this means, we no longer need large-scale available VIS data in the testing stage. The required rich identity representations can be easily sampled from noises. However, the identity representations generated by the identity sampler conform to the Gaussian distribution, while those of LightCNN do not satisfy the Gaussian distribution hypothesis. Thus, there may be a distribution gap between the two types of identity representations. (3) Using the identity sampler in both the training and the testing stages. In this way, it is flexible to obtain massive identity representations in the both stages.

The performances of the above three approaches are shown in Table IV. We observe that the second approach obtains the worst results because of the aforementioned distribution gap. In addition, the third approach gets better results than the first approach, even though the identity sampler is trained on the extracted identity representations of LightCNN. This indicates that the identity sampler may produce new diverse identity representations based on the property of VAEs.

We also investigate the effect of the number of the generated data on the HFR performance. In particular, we use 0, 10K, 30K, 50K, 80K, 100K, 130K, 150K, 180K, and 200K generated paired heterogeneous data to train the HFR network, respectively. Fig. 5 (right) depicts the results under different numbers of generated data on the Tufts Face database. It is clear that from 0 to 100K, the recognition performance is improved with increasing the number of the generated data. Compared with only using the real data (0 generated data), VR@FAR=1$\%$ is significantly improved by 26.1$\%$ after adding 100K generated paired data. These phenomena demonstrate that the generated data really contain abundant useful information that is helpful for recognition. Besides, as the number of the generated paired data exceeds 100K, no obvious improvement is observed, indicating that the generated data have been saturated.

Generally, the large-scale VIS database, i.e. MS-Celeb-1M [36], is only used to pre-train the HFR network [17, 18]. The intuition behind this is that training the large-scale VIS data together with the small-scale heterogeneous data will lead to severe domain biases that may degrade the performance of HFR. In order to verify this point, four experiments are elaborately designed: (A) directly trains LightCNN on the large-scale VIS database MS-Celeb-1M. (B) fine-tunes the pre-trained LightCNN on the heterogeneous database, and (C) fine-tunes on both the heterogeneous database and MS-Celeb-1M. (D) is our method that fine-tunes on both the heterogeneous database and the generated data.

Table V shows that the results of (C) are merely slightly better than those of (A), and are much worse than those of (B), revealing that the domain biases actually affect the performance of HFR. Moreover, (D) outperforms the others by a large margin. This demonstrates the effectiveness of our method that introduces the identity representations of MS-Celeb-1M to enrich the identity diversity of the generated data. In this case, the domain biases are mitigated because the generated data belong to the same domain of the real heterogeneous database.

For Dual Generation. The angular orthogonal loss $\mathcal{L}_{\text{ort}}$ in Eq. (4) and the pairwise identity preserving loss $\mathcal{L}_{\text{ip}}$ in Eq. (11) (including $\mathcal{L}_{\text{ip}}^{\text{u}}$ in Eq. (16)) are studied to analyze their respective roles. Both quantitative and qualitative results are presented for a comprehensive comparison.

Fig. 6 depicts the qualitative comparisons between our method and its two variants. Without $\mathcal{L}_{\text{ort}}$, the generated images are almost unchanged when the identity representation is varied. This means that we cannot enrich the identity diversity of the generated data via injecting more identity information. When omitting $\mathcal{L}_{\text{ip}}$, the generated paired images look inconsistent in identities, suggesting the effectiveness of $\mathcal{L}_{\text{ip}}$. In addition, Table VI displays the quantitative comparison results. It is observed that the recognition performance drops significantly if one of the losses is discarded, which further proves the importance of each loss. For example, VR@FAR=0.1$\%$ decreases by 19.8$\%$ when $\mathcal{L}_{\text{ort}}$ is omitted.

For HFR. We design four experiments to study the usage of the generated data: (I) only uses the real data to train the HFR network. (II) discards the real data and merely uses the generated data. (III) leverages both the real data and the generated data, and the latter is utilized via the pairwise distance loss as our preliminary version DVG. (IV) is our method that uses the real data and the generated data with the softmax loss and the contrastive loss, respectively.

Table VII shows that (II) gets the worst results. However, compared with the backbone LightCNN pre-trained on MS-Celeb-1M (Rank-1 = 29.4$\%$, listed in Table V), (II) still gains an improvement of 7.2$\%$ in terms of the Rank-1 accuracy, suggesting the value of the generated data. Compared with (I), the Rank-1 accuracy is boosted at least by 15.8$\%$ after introducing the generated data, including both (III) and (IV). This further demonstrates the importance of the generated data. In addition, we find that compared with our preliminary version DVG (Rank-1 = 56.1$\%$, listed in Table XIII), (III) improves the Rank-1 accuracy by 14.2$\%$. Since the difference between the two methods lies in the generated data, we attribute the improvement to the richer identity diversity of the generated data of DVG-Face, as discussed in Section 4.3.2. (IV) gains better results than (III), which verifies the effectiveness of the introduced contrastive loss. It optimizes both the intra-class and the inter-class distances, facilitating to learn domain-invariant and discriminative embedding features. By contrast, the pairwise distance loss of DVG can only assist in reducing domain discrepancy. Note that the effectiveness of the contrastive loss is based on the identity consistency and the identity diversity properties of the generated data, as stated in Section 4.3.2.

Table VIII lists the results of sensitivity studies of the parameters $\lambda_{1}$ and $\lambda_{2}$ in Eq. (14), $\eta$ in Eq. (15), and $\alpha$ in Eq. (20). The parameters $\lambda_{1}$, $\lambda_{2}$, and $\alpha$ are set to balance the magnitude of the corresponding loss functions. $\eta$ is set according to [52]. It is observed that our method is not sensitive to these trade-off parameters in a large range. For instance, when $\lambda_{1}$ is varied from 10 to 100, the verification rate only changes from 67.4$\%$ to 68.5$\%$. Besides, Table VIII suggests that as the parameter becomes larger gradually, the recognition performance first increases and then decreases. The optimal setting is $\lambda_{1}$=50, $\lambda_{2}$=0.5, $\eta$=0.1, and $\alpha$=1e-3. More sensitivity studies of the parameters $\lambda_{1}$, $\lambda_{2}$, $\eta$, and $\alpha$ on the BUAA-Vis-Nir, the MultiPIE, and the NJU-ID databases are presented in Tables 4, 5, and 6 of the supplementary material, respectively. Finally, we further explore the setting of the margin parameter $m$ in Eq. (19). Table IX reveals that the best margin value is 0.5.

Finally, it is worth noting that our method is computationally efficient. When using one TITAN Xp GPU, it takes about 20 minutes to train the generator on the Tufts Face database. In the inference stage of the generator, it merely takes about 4.7 ms to generate a pair of heterogeneous face images. Furthermore, training the HFR network with 100K generated data and about 2K real data costs about 5 hours.

In order to verify the effectiveness of DVG-Face, we compare it with 16 state-of-the-art methods, including 6 traditional methods and 10 deep learning based methods. The traditional methods include H2(LBP3) [65], DSIFT+PCA+LDA [66], CDFL [67], Gabor+RBM [68], Recon.+UDP [69], and CEFD [25]. The deep learning based methods consist of IDNet [70], HFR-CNN [26], Hallucination [33], DLFace [71], TRIVET [72], W-CNN [3], PACH [14], RCN [17], MC-CNN [73], and DVR [27].

Table X shows the results of the Rank-1 accuracy and verification rates of the above methods. We can see that most of the deep learning based methods outperform the traditional methods. Nevertheless, the deep learning based method HFR-CNN performs worse than the traditional method Gabor+RBM. This indicates that it is challenging for the deep learning based methods to train on the small-scale heterogeneous data. The methods, such as W-CNN, MC-CNN, and DVR, which are devoted to tackling the over-fitting caused by the limited training data, get higher performances. Moreover, our method obtains the best results than all the competitors. In particular, regarding the Rank-1 accuracy and VR@FAR=0.1$\%$, DVG-Face exceeds the conditional generation based method PACH by 1.0$\%$ and 1.6$\%$ respectively, suggesting the superiority of our unconditional generation approach. Compared with the state-of-the-art method DVR, we improve VR@FAR=0.01$\%$ from 98.6$\%$ to 99.2$\%$. More importantly, the ROC curves in Fig. 8 (a) show that DVG-Face improves the harder indicator True Positive Rate (TPR)@False Positive Rate (FPR)=10^-5 by 3.0$\%$ over DVR. In addition, compared with the preliminary version DVG [21], DVG-Face further improves TPR@FPR=10^-5 from 97.8$\%$ to 98.6$\%$. The improvement highlights the importance of the abundant identity diversity and the consequent contrastive loss. The generated paired NIR-VIS images of DVG-Face are exemplified in Fig. 7 (a).

A total of 5 traditional methods and 4 deep learning based methods are compared on the BUAA-VisNir database. The former includes MPL3 [57], KCSR [74], KPS [22], KDSR [75], and H2(LBP3) [65], and the latter includes TRIVET [72], W-CNN [3], PACH [14], and DVR [27].

Here as well, we observe from Table XI that the deep learning based methods are always superior to the traditional methods. Moreover, the improvements of DVG-Face over the conditional generation based method PACH are significant. VR@FAR=0.1$\%$ is increased from 93.5$\%$ to 99.1$\%$. In addition, although the state-of-the-art DVR has achieved very high results, DVG-Face still exceeds it by 2.2$\%$ in terms of VR@FAR=0.1$\%$. The ROC curves in Fig. 8 (b) further demonstrate the advantages of our method. Regarding TPR@FPR=10^-5, DVG-Face leads DVR by 5.4$\%$ and the preliminary version DVG by 5.1$\%$. The generated paired images of DVG-Face on the BUAA-VisNir database are presented in Fig. 7 (b).

Considering the crucial role of the Sketch-Photo recognition in criminal investigation, we further evaluate our method on the IIIT-D Sketch Viewed database. The compared methods include 5 traditional methods: Original WLD [76], SIFT [77], EUCLBP [78], LFDA [24], MCWLD [4], as well as 4 deep learning based methods: LightCNN [61], CDL [18], RCN[17], and MC-CNN [73].

Table XII suggests that on such a Sketch-Photo database, the overall performance discrepancy between the traditional methods and the deep learning based methods is not particularly large. The Rank-1 accuracy of MCWLD is even better than that of LightCNN, and is comparable with that of CDL. In addition, our method exhibits superior performances over all the competitors. The best Rank-1 accuracy is improved from 90.34$\%$ [17] to 97.21$\%$. Fig. 7 (c) depicts the dual generation results of Sketch-Photo.

As mentioned in Section 4.3.6, the face recognition network pre-trained on MS-Celeb-1M merely obtains a Rank-1 accuracy of 29.4$\%$ on the Tufts Face database, indicating the large domain discrepancy between the thermal data and the VIS data. At the same time, it is observed that after fine-tuning on the training set of Tufts Face, the Rank-1 accuracy only increases to 54.5$\%$ that is still unsatisfactory. This phenomenon is caused by the limited number of the Thermal-VIS training data. Obviously, data augmentation is a straightforward approach to alleviate this problem. Both the preliminary version DVG and the current DVG-Face aim to generate more data to facilitate the training of the HFR network. The comparisons of DVG, DVG-Face, and the baseline LightCNN are reported in Table XIII. We can see that compared with DVG, DVG-Face boosts the Rank-1 accuracy from 56.1$\%$ to 75.7$\%$, and VR@FAR=0.1$\%$ from 17.1$\%$ to 36.5$\%$. The dual generation results on Tufts Face are shown in Fig. 7 (d).