LADA: Look-Ahead Data Acquisition via Augmentation for Active Learning

This paper trains a classifier network, $f_{\theta}$, with dataset $\mathscr{X}$ while our scenario is differentiated by assuming $\mathscr{X}=\mathscr{X}_{U}\cup\mathscr{X}_{L}$ and $|\mathscr{X}_{U}|\gg|\mathscr{X}_{L}|$. Here, $\mathscr{X}_{U}$ is a set of unlabeled data instances, and $\mathscr{X}_{L}$ is a labeled dataset. Given these notations, a data augmentation function, $f_{aug}(x;\tau)\colon\mathscr{X}\to V(\mathscr{X})$, transforms a data, $x\in\mathscr{X}$, into a modified data, $\tilde{x}\in V(\mathscr{X})$; where $\tau$ is a parameter describing the policy of transformation, and $V(\mathscr{X})$ is the vicinity set of $\mathscr{X}$ (Chapelle et al. 2001). On the other hand, a data acquisition function, $f_{acq}(x;f_{\theta})\colon\mathscr{X}_{U}\to\mathbb{R}$, calculates a score of each data instance, $x\in\mathscr{X}_{U}$, based on the current classifier, $f_{\theta}$; and $f_{acq}$ represents the instance selection strategy in the learning procedure of $f_{\theta}$ with the instance, $x\in\mathscr{X}_{U}$.

In the conventional data augmentations, $\tau$ in $f_{aug}(x;\tau)$ indicates the predefined degree of rotating, flipping, cropping, etc. $\tau$ is manually chosen by the modeler to describe the vicinity of each data instance.

Another approach of modeling $\tau$ is utilizing the feedback from the current classifier network, $f_{\theta}$. Spatial Transformer Network (STN) is a transformer to generate a virtual example by training $\tau$ to minimize the cross-entropy (CE) loss of the transformed data (Jaderberg et al. 2015):

where $y$ is the ground-truth label of the data instance, $x$.

Recently, Mixup-based data augmentations generate a virtual data instance from the vicinity of a pair of data instances. In Mixup, $\tau$ becomes the mixing policy of two data instances, $x_{i}$ and $x_{j}$ (Zhang et al. 2017):

where the labels are also mixed by the proportion $\lambda$. While Eq.(2) corresponds to the input feature mixture, Manifold Mixup mixes the hidden feature maps from the middle of neural networks to learn smoother decision boundary at multiple levels of representations (Verma et al. 2018). Whereas $\tau$ is a fixed value without any learning process, AdaMixup learns $\tau$ by adopting a discriminator, $\varphi^{ada}$ (Guo, Mao, and Zhang 2019):

We focus on the pool-based active learning with uncertainty score (Settles 2009). Given this scope of active learning, the data acquisition function measures the utility score of the unlabeled data instances, i.e. $x^{*}=\operatorname*{argmax}_{x}f_{acq}(x;f_{\theta}).$

The traditional acquisition functions measure the predictive entropy, $f_{acq}^{Ent}(x;f_{\theta})=\mathbb{H}[y|x;f_{\theta}]$ (Shannon 1948); or the variation ratio, $f_{acq}^{Var}(x;f_{\theta})=1-max_{y}\text{P}(y|x;f_{\theta})$ (Freeman 1965). The recent acquisition function calculates the hypothetical disagreement by $f_{\theta}$ on a data instance, $f_{acq}^{BALD}(x;f_{\theta})=\mathbb{H}[y|x;f_{\theta}]-\mathbb{E}_{\text{P}(\theta|D_{train})}[\mathbb{H}[y|x;f_{\theta}]]$ (Houlsby et al. 2011).

Besides the classifier network, $f_{\theta}$, additional modules are applied to measure the acquisition score. To find the most dissimilar instance in $\mathscr{X}_{U}$ compared to $\mathscr{X}_{L}$, a discriminator, $\varphi^{VAAL}$, is introduced to estimate the probability of belonging to $\mathscr{X}_{U}$ (Sinha, Ebrahimi, and Darrell 2019):

To diversely select uncertain data instances, the gradient embedding from the pseudo label, $\hat{y}$, is used in k-MEANS++ seeding algorithm (Ash et al. 2020):

There are a few prior works in integrating active learning and data augmentation effectively. Bayesian Generative Active Deep Learning (BGADL) integrates acquisition and augmentation by selecting data instances via $f_{acq}$, then augmenting the selected instances via $f_{aug}$, which is VAE, afterward (Tran et al. 2019). However, BGADL limits the vicinity to preserve the labels, and BGADL demands on large labeled instances to train generative models. More importantly, BGADL does not consider the potential gain of data augmentation in the process of acquisition.

Since we look ahead the acquisition score of augmented data instances, it is natural to integrate the functionalities of acquisitions and augmentations. This paper proposes a Look-Ahead Data Acquisition via augmentation, or LADA framework. Figure 2(a) depicts the LADA framework, which consists of the data augmentation component and the acquisition component. The goal of LADA is enhancing the informativeness of both 1) real-world data instance, which is unlabeled at current, but will be labeled by the oracle in the future; and 2) virtual data instance, which will be generated from the unlabeled data instances that are selected. This goal is achieved by looking ahead of their acquisition scores before actual selections for the oracle annotations.

Specifically, LADA trains the data augmentation function, $f_{aug}(x;\tau)$, to maximize the acquisition score of the transformed data instance of $x_{U}$ before the oracle annotations. Eq.(6) specifies the learning objectives of the augmentation policy via the feedback from acquisition.

With the optimal $\tau^{*}$ corresponding to $x_{U}$, $f_{acq}$ calculates the acquisition score of $x_{U}$ (see Eq.(7)), and the score also considers the utility of the augmented instance, $f_{aug}(x_{U};\tau^{*})$:

Whereas the proposed LADA framework is a generalized framework that can adopt the various types of acquisition and augmentation functions, this section mainly adopts Mixup for $f_{aug}$, i.e. $f_{aug}^{Mixup}$ and Max Entropy for $f_{acq}$, i.e. $f_{acq}^{Ent}$. To begin with, we introduce an integrated single function to substitute the composition of functions as $f_{integ}=f_{acq}\circ f_{aug}(x_{U})=f_{acq}(f_{aug}(x_{U};\tau);f_{\theta})$ for generality.

As we introduce LADA with $f_{integ}$ to look ahead the acquisition score of the virtual data instances, $f_{integ}$ can be a simple composition of well-known acquisition functions and augmentation functions where the policy of augmentation is fixed. However, this does not enhance the informativeness of the virtual data instances. Hence, we propose the integration where the policy of data augmentation is optimized to maximize the acquisition score, within a single function. Here, we introduce InfoMixup as a learnable data augmentation.

We assume that we start the $j^{th}$ active learning iteration with already acquired labeled dataset $\mathscr{X}_{L}^{j}$. With the allowed budget per acquisition as $k$, we acquire top-$\frac{k}{2}$ pairs, i.e. $\mathscr{X}_{S}$ among the subsets, $\mathscr{X}^{\prime}_{S}\subset\mathscr{X}_{U}\times\mathscr{X}_{U}$, with $|\mathscr{X}^{\prime}_{S}|=\frac{k}{2}$.

At this moment, oracle annotates the true labels on $\mathscr{X}_{S}$. Also, we have a virtual instance dataset, $\mathscr{X}_{M}$, generated by InfoMixup with the optimal mixing policy, $\tau^{*}$:

where $\lambda_{i}\sim\text{Beta}(\tau_{i}^{*},\tau_{i}^{*})$. Here, $\tau^{*}$ is dynamically inferred by the neural network of $\pi_{\phi}$ per each pair.

Up to this phase, our training dataset becomes $\mathscr{X}_{L}^{j+1}=\mathscr{X}_{L}^{j}\cup\mathscr{X}_{S}$ and $\mathscr{X}_{M}$. Our proposed algorithm, described in Algorithm 1, utilizes $\mathscr{X}_{M}$ for this active learning iteration only, with various $\lambda_{i}$s sampled at each training epoch. The classifier network’s parameter, $\theta$, is learned via the gradient of the cross-entropy loss,

where $y_{i}$ is the corresponding ground-truth label annotated from the oracle for Eq.(18); or the mixed label according to the mixing policy for Eq.(19).

Since we propose the integrated framework of acquisition function and data augmentation to look ahead the informativeness of the data, we can use various acquisition functions and data augmentations in our LADA framework. For example, we may substitute the Max Entropy, which is the feedback of the acquisition function to the data augmentation in InfoMixup, with another simple feedback, Var Ratio. Also, if we apply the VAAL acquisition function, LADA with VAAL trains the generator network, $\pi_{\phi}$, to maximize the discriminator’s indication on the unlabeled dataset, $\text{P}(x\in\mathscr{X}_{U};\varphi^{VAAL})$, for the generated instances.

Similarly, we may substitute the data augmentation of InfoMixup with Spatially Transform Networks (STN) (Jaderberg et al. 2015), a.k.a. InfoSTN. STN may be trained with a subset of unlabeled data as input to maximize their predictive entropy when propagated to the current classifier network. The score to pick the most informative data is formulated as $f_{acq}^{STN}(x_{i};f_{\theta})=\mathbb{H}[\hat{y}^{STN}_{i}|x^{STN}_{i}]$, where $x^{STN}_{i}$ is the spatially transformed output of the data $x_{i}$, and $\hat{y}^{STN}_{i}$ is the corresponding prediction by the current classifier network. We provide more details in Appendix C.

This section denotes the proposed framework as LADA, and we specify the instantiated data augmentation and acquisition by its subscript, i.e. the proposed InfoMixup as $\mathrm{LADA_{EntMix}}$ which adopts Max Entropy as data acquisition and Mixup as data augmentation to select and generate informative samples. If we change the entropy measure to the Var Ratio or the discriminator logits of VAAL, it results in the subscript of VarMix or VaalMix, respectively. Also, if we change the augmentation policy to the STN network, the subscript becomes EntSTN.

We compare our models to 1) Coreset (Sener and Savarese 2018); 2) BADGE (Ash et al. 2020); and 3) VAAL (Sinha, Ebrahimi, and Darrell 2019) as the baselines for active learning. We also include some data augmented active learning: 1) BGADL, 2) Manifold Mixup; and 3) AdaMixup. Here, BGADL is an integrated data augmentation and acquisition method, but it should be noted that BGADL has no learning mechanism in the augmentation from the feedback of acquisition. We also add ablated baselines to see the effect of learning $\tau$, so we introduce the fixed $\tau$ case as $\mathrm{LADA^{fixed}}$. The classifier network, $f_{\theta}$, adopts Resnet-18 (He et al. 2016), and the policy generator network, $\pi_{\phi}$, consists of a much smaller neural network. Appendix A.2 provides more details on the networks and their training.

We experiment the above-mentioned models with three benchmark datasets: FashionMNIST (Fashion) (Xiao, Rasul, and Vollgraf 2017), SVHN (Netzer et al. 2011), and CIFAR-10 (Krizhevsky, Hinton et al. 2009). Throughout our experiments, we repeat the experiments for five times to validate the statistical significance, and the maximum acquisition iteration is limited to 100. More details about the treatment on each dataset are in Appendix A.1.

We evaluate the models under the pool-based active learning scenario. We assume that the model has 20 training instances, which are randomly chosen and balanced. As the active learning iteration progresses, we acquire 10 additional training instances at each iteration, and we use the same amount of oracle queries for all models, which results in selecting top-5 pairs when adopting Mixup as data augmentation in the LADA framework.

Table 3 shows the average test accuracy, and the accuracy of each replication represents the best accuracy over the acquisition iterations. Since we introduce the generalizable framework, Table 3 separates the performances by the instantiated acquisition functions. The group of baselines does not have any learning mechanism on the acquisition metric, and this group has the worst performances. We suggest three acquisition functions to be adopted by our LADA framework, which are 1) the predictive entropy by the classifier, 2) the discriminator logits in VAAL, and 3) the classifier variation ratio. Given that VAAL uses a discriminator and a generator, the VAAL-based model has more parameters to optimize in terms of complexity, which provides an advantage in a complex dataset, such as CIFAR-10.

When we examine the general performance gains across datasets, we find the best performers as $\mathrm{LADA_{VarMix}}$ in Fashion; and $\mathrm{LADA_{VaalMix}}$ in SVHN and CIFAR-10. In terms of the data augmentation, Mixup-based augmentation outperforms STN augmentation. As the dataset becomes complex, the greater gain of performance is achieved by $\mathrm{LADA_{EntSTN}}$ in SVHN or CIFAR-10, compared to Fashion. In all combinations of baselines and datasets, the integrations of augmentation and acquisition, a.k.a. LADA variations, show the best performance in most cases. In terms of the ablation study, the learning of the data augmentation policy, $\tau$, is meaningful because the 10 learning case of LADA is better than the fixed case in 12 variations of LADA. Figure 3 shows the convergence speed to the best test accuracy by each model. As the dataset becomes complex, the performance gain by LADA becomes apparent.

Additionally, we compare the integrated framework to the pipelined approach. Max Entropy does not have an augmentation part, so it becomes the simplest model. Then, Ent w.Manifold Mixup adds the Manifold Mixup augmentation, but it does not have a learning process on the mixing policy. Finally, Ent w.AdaMixup has a learning process on the mixing policy, but the learning is separated from the acquisition. These pipelined approaches show lower performances than the integration cases of LADA.

Finally, as LADA is a generalizable framework to work with the various acquisition and augmentation functions, Figure 4(a) and Figure 4(b) show the ablation study on the instantiated LADA with the VAAL acquisition function and the $STN$ augmentation function, respectively. The figures confirm the effects of both integration and learnable augmentation policy with feedback from the acquisition.

Besides the quantitative comparison, we need reasoning on the behavior of LADA. Therefore, we selected $\mathrm{LADA_{EntMix}}$ to contrast to the pipelined approach. We investigate on 1) achieving the informative data instances by acquisition, 2) the validity of the optimal $\tau^{*}$ in the augmentation learned from the policy generator network $\pi_{\phi}$, and 3) examining the coverage of the explored space.

To check the informativeness of data instances, Figure 5 shows the different acquisition process between Max Entropy and $\mathrm{LADA_{EntMix}}$. Max Entropy selects a data instance with the highest predictive entropy value. Compared to Max Entropy, $\mathrm{LADA_{EntMix}}$ selects a pair of two data instances with the highest value of the aggregated predictive entropy, which is the summation of the predictive entropy from two data instances and one InfoMixup instance. By mixing two unlabeled data instances with the corresponding optimal mixing policy $\tau^{*}$, the virtual data instance, generated along with the vicinal space, results in a high entropy value, which can be higher than the selected instance by Max Entropy. The virtual data instance helps the current classifier model to clarify the decision boundary between two classes along the interpolation line of two mixed real instances.

To confirm the validity of the optimal $\tau^{*}$, we compare three cases of 1) the inferred $\tau$ ($\mathrm{LADA_{EntMix}}$); 2) the fixed $\tau$ ($\mathrm{LADA^{fixed}_{EntMix}}$); and 3) the pipelined model’s $\tau$ (Ent w.Manifold Mixup). Figure 6(a) shows the entropy of the virtual data instances over the acquisition process. As expected, the optimal $\tau^{*}$ learned in $\mathrm{LADA_{EntMix}}$ produces the highest entropy over the acquisition process, but it should be noted that the differentiation becomes significant after some acquisition iterations, which comes from the requirement of training the classifier. Figure 6(b) shows the distribution of entropy of virtual instances, with the median value of each interval as $x$-axis. This also shows that the optimal $\tau^{*}$ has the highest density beyond the interval of the median 2.2.

To examine the coverage of the explored latent space, Figure 7 illustrates the latent space of the acquired data instances and the augmented data instances. Ent w.AdaMixup has a potential capability of interpolating distantly paired data instances, but its learned $\tau$ limits a sample of $\lambda$ to be placed near either one of the paired instances because of the aversion on the manifold intrusion. Therefore, in the experiments, Ent w.AdaMixup ends up exploring the space near the acquired instances. The generated virtual data instances by $\mathrm{LADA_{EntMix}}$ show further exploration than Ent w.AdaMixup. The latent space makes the linear interpolation of $\mathrm{LADA_{EntMix}}$ to be curved by the manifold, but it keeps the interpolation line of the curved manifold. The extent of the interpolation is broader than AdaMixup because the optimal $\tau^{*}$ is guided by the entropy maximization, which is adversarial in a sense. This adversarial approach is different from the aversion of the manifold intrusion because the latter is more conservative to the currently learned parameter.