Understanding and Improving Early Stopping for Learning with Noisy Labels

When trained with noisy labels, if clean labels are of majority within each noisy class, deep networks tend to first fit clean labels during an early learning stage before eventually memorizing the wrong labels, which can be explained by the memorization effect. Many current methods utilize this property to counteract the influence of noisy labels by stopping the optimization at an early learning phase. Specifically, a deep classifier can be obtained by optimizing the following objective function with a relatively small epoch number $T$:

where $f(\cdot;\Theta)$ is a deep classifier with model parameters $\Theta$ and $\mathcal{L}$ is the cross-entropy loss. When trained with noisy data, early learning regularization (ELR) [17] reveals that, for the most commonly used cross-entropy loss, the gradient is well correlated with the correct direction at the early learning phase. Therefore, with a properly defined small epoch number $T$, the classifier can have higher accuracy than at initialization. While, if we continue to optimize the deep model after $T$ epochs, the classifier will be able to memorize more noise labels. Therefore, it is critical to select a proper epoch number $T$ to utilize the memorization effect and alleviate the influence of noisy labels.

Current methods typically select the epoch number $T$ by considering the network as a whole. However, as Figure 1 makes clear, the impact of noisy labels on different DNN layers are different, which implies that the traditional early stopping trick, which optimizes the whole network all at once, may make different DNN layers to be antagonistically affected by each other, thus degrading the final model performance.

To this end, we propose to separate a DNN into different parts and progressively train layers in different parts with different training epochs. Specifically, assume that the whole network $f(\cdot;\Theta)$ can be constituted with $L$ DNN parts

where $f_{l}(\cdot;\Theta_{l})$ is the $l$-th DNN part and $\bm{z}_{l}$ is the corresponding output. The output of the last part $\bm{z}_{L}$ is the prediction. The network $f(\cdot;\Theta)$ can also be represented as $f(\cdot;\Theta_{1},...\Theta_{L})$. To counteract the impact of noisy labels, We initially optimize the parameter $\Theta_{1}$ for the first part by training the whole network for $T_{1}$ epochs with the following objective

Then, we keep the obtained parameter $\Theta_{1}^{*}$ fixed, reinitialize and progressively learn the $l$-th ($l=2,\dots,L$) DNN part with the parameters for preceding DNN parts fixed. The training procedure is conducted with $T_{l}$ epochs by optimizing the following objective

We gradually optimize the $(l+1)$-th DNN part with the obtained parameter $\Theta_{l}^{*}$ fixed, the optimization is continued until all the parameters have been optimized. As elaborated above, latter DNN parts are more sensitive to noisy labels than their former counterparts. Therefore, for the above initializing optimization in Eq. (3) and the following $L-1$ steps of optimization in Eq. (4), we gradually reduce the training epochs (i.e. $T_{1}\geq T_{2}\geq\dots\geq T_{L}$) to better exploit the memorization effect. After optimization, we can obtain the final network as $f(\cdot,\Theta)=f(\cdot;\Theta_{1}^{*},\dots,\Theta_{L}^{*})$. Since this model is obtained by progressively exploiting the early stopping strategies for different DNN parts, we term the proposed method as progressive early stopping (PES).

To explicitly verify the effectiveness of the proposed PES method, we conduct several pilot experiments, which compare the traditional early stopping and PES with label noise from different types and different levels. The results are illustrated in Figure 2, from which we can see that, compared with models trained with traditional early stopping, models trained with PES can achieve superior classification accuracy with smaller variations in all the cases. Current state-of-the-art methods [16] usually adopt models with the traditional early stopping as base models to distill confident examples and then utilize semi-supervised learning techniques by considering confident examples as labeled data and other noisy examples as unlabeled data to further improve the results. The final performance still heavily relies on the base model trained with noisy labels. By improving the performance of the base model, our method combined with semi-supervised learning techniques is able to establish new state-of-the-art results. In the following subsections, we will elaborate on how to utilize PES to distill confident examples and further combine it with semi-supervised learning techniques.

Based on the deep network optimized with progressive early stopping, we can select confident examples to facilitate the model training. Here, confident examples refer to examples that have high probabilities with clean labels. In this paper, we treat examples whose predictions are consistent with given labels as confident examples. In addition, to make the results more robust, we generate two different augmentations for any given input and use the average prediction to decide its predicted label. Formally, we can obtain the confident example set $\mathcal{D}_{l}$ as

where $\text{Augment}(\cdot)$ indicates normal data augmentation operation including horizontal random flip and random crops, and $f^{k}(x;\Theta)$ is the predicted probability of $\bm{x}$ belonging to class $k$. Note that $\text{Augment}(\cdot)$ is a stochastic transformation, so the two terms in Eq (5) are not identical. The average prediction of augmented examples provides a more stable prediction and is found empirically to improve performance. After obtaining the confident example set, one can easily train a classifier by considering confident examples as clean data. However, since the number of confident examples for different classes can vary greatly, directly training the model with the obtained confident example set may introduce a severe class imbalance problem. To this end, we adopt a weighted classification loss

where $w_{i}$ is the corresponding class weight. Assuming that $\sigma_{k}=|\{(\bm{x}_{i},\tilde{y}_{i})|\tilde{y}_{i}=k,(\bm{x}_{i},\tilde{y}_{i})\in{\mathcal{D}_{l}}\}|$ denotes the cardinality of the confident example set belonging to the $k$-th class. Then, we can set $w_{i}={\sigma_{i}}/{(\sum_{j=1}^{K}\sigma_{j})}$ to indicate the corresponding class importance.

Training with only confident examples neglects the rest data and may suffer from insufficient training examples. To tackle this problem, we further resort to semi-supervised learning techniques by considering confident examples as labeled data and other noisy examples as unlabeled data. Specifically, the labeled data set and unlabeled data set can be obtained as

where the labeled data set $\mathcal{D}_{l}$ is the same as that in Eq (6), and $\mathcal{D}_{u}$ is the rest unlabeled data set. Similar to [16], we adopt MixMatch [4] as the semi-supervised learning framework to train the final classification models. For more details about semi-supervised learning, we refer to  [4]. The whole learning algorithm is summarized in Algorithm 1.

Datasets: We evaluate our method on two synthetic datasets, CIFAR-10 and CIFAR-100 [13] with different levels of symmetric, pairflip, and instance-dependent label noise (abbreviated as instance label noise) and a real-world dataset Clothing-1M [35]. Both CIFAR-10 and CIFAR-100 contain 50k training images and 10k test images of size $32\times 32$. Following previous works [10, 33, 17, 31], symmetric noise is generated by uniformly flipping labels for a percentage of the training dataset to all possible labels. Pairflip noise flips noisy labels into their adjacent class. And, instance noise is generated by image features. More details about the synthetic label noise are given in the supplementary material. For the flipping rate, it can include [10, 33] or ex-include [16, 17] true labels. We use the flipping rate including correct labels in Table 3 to compare with results in [16], and use without correct labels in the rest of the experiments. Clothing-1M [35] is a large-scale dataset with real-world noisy labels, whose images are clawed from the online shopping websites, and labels are generated based on surrounding texts. It contains 1 million training images, and 15k validation images, and 10k test images with clean labels.

Baselines: Semi-supervised learning may strongly boost the performance, we separately compare our method with approaches with or without semi-supervised learning. For the comparison with baselines with semi-supervised learning, we combine our proposed method with MixMatch used in [16] as indicated in Subsection 2.3. (1) Approaches without semi-supervised learning: Co-teaching [10], Forward [26], Joint Optim [27], T-revision [33], DMI [37], and CDR [31]. (2) Methods with semi-supervised learning: M-correction [1], DivideMix [16], and ELR+ [17]. We also adopt standard training with cross-entropy (CE) and MixUp [41] as baselines to show improvements.

Network structure and optimization: Our method is implemented by PyTorch v1.6. Baseline methods are implemented based on public codes with hyper-parameters set according to the original papers. For DivideMix and ELR+, we evaluate the test accuracy with the first network. To better demonstrate the robustness of our algorithm, we keep the hyper-parameters fixed for different types of label noise. More technique details are given in the supplementary material.

For experiments without semi-supervised learning, we follow [33], and use ResNet-18 [11] for CIFAR-10 and ResNet-34 for CIFAR-100. We split networks into three parts, the layers above block 4 as part 1, block 4 of ResNet as part 2, and the final layer as part 3. $T_{1}$ is defined as 25 for CIFAR-10 and 30 for CIFAR-100, $T_{2}$ as 7, and $T_{3}$ as 5. The network is trained for 200 epochs and SGD with $0.9$ momentum is used. The initial learning rate is set to $0.1$ and decayed with a factor of $10$ at the 100th and 150th epoch respectively, and a weight decay is set to $10^{-4}$. For $T_{2}$ and $T_{3}$, we employ an Adam optimizer with a learning rate of $10^{-4}$.

For experiments with semi-supervised learning, we follow the setting of [16] with PreAct Resnet-18. We set the final layer as part 2, the rest as part 1. $T_{1}$ is defined as 20 for CIFAR-10 and 35 for CIFAR-100, and $T_{2}$ as 5. The network is trained for $300$ epochs. For optimization, we use a single cycle of cosine annealing [20], and the learning rate begins from $2\times 10^{-2}$ and ends at $2\times 10^{-4}$, with a weight decay of $5\times 10^{-4}$. An Adam optimizer is adopted with a learning rate of $10^{-4}$ for $T_{2}$. For hyper-parameters from MixMatch, we set them according to the original paper [4].

For Clothing-1M [35], we follow the previous work [27], and employ a ResNet-50 [11] pre-trained on ImageNet [14]. We set the final layer as part 2, the rest as part 1. $T_{1}$ and $T_{2}$ are defined as 20 and 7 respectively. The network is trained with CE loss for 50 epochs and SGD is used with $0.9$ momentum and a weight decay of $10^{-3}$. The learning rate is $5\times 10^{-3}$ and decayed by a factor of $10$ at the 20th and 30th epoch respectively. We employ an Adam optimizer with a learning rate of $5\times 10^{-6}$ for $T_{2}$.

In Figure 2, we can observe that with the PES trick, the performance of classifiers is generally improved compared with that the traditional early stopping trick. In this section, we further carefully analyze the quality of extracted labels by examining them from three aspects, i.e., test accuracy, label precision, and label recall. Here, label precision indicates the ratio of the number of extracted confident examples with correct labels in the total confident example set, and label recall represents the ratio of the number of confident examples with correct labels among the total correctly labeled examples. Specifically, we train a neural network on CIFAR-10 with different kinds and levels of label noise for $25$ epochs respectively and report the performance for each case before and after the proposed PES is applied.

Results in Table 1 clearly show that, compared with the traditional early stopping, PES can help to obtain higher accuracies, precisions, and recalls for most cases. For instance-dependent label noise, PES can achieve higher recall values with comparable label precision values. Note that models with high recall values can help to collect more confident examples, which is critical for learning with confident examples and semi-supervised learning. Therefore, by enhancing the performance of the initial model, PES can help to improve the final classification performance in all cases, which is also verified by the experiments in Section 3.3.

Synthetic datasets. We first verify the effectiveness of our proposed method without semi-supervised learning techniques on two synthetic datasets: CIFAR-10 and CIFAR-100. For both of these two datasets, we leave 10% of data with noisy labels as noisy validation set. Results are presented in Table 2, which shows that our proposed method can consistently outperform all other baselines across various settings by a large margin.

Table 3 and Table 4 present the mean accuracy and standard deviation for our method and all baselines on CIFAR-10 and CIFAR-100, respectively. From the results, we can get that the proposed method can outperform all baselines in all cases. For pairflip label noise, the advantages of our proposed method become more apparent, and it significantly outperforms state-of-the-art methods by over 8% on both CIFAR-10 and CIFAR-100. These empirical results support our proposal that PES can improve the quality of selected confident examples, which helps improve performance and reduce the variance of the final classifier.

Real-world dataset. We evaluate the performance of the proposed method on a real-world dataset with Clothing-1M [35] and select methods such as CE, Forward, Joint-Optim, DMI, and T-revision, which use a single network, and also methods such as DivideMix and ELR+, which adopt an ensemble model with two different networks, as baselines. We also report the results for the proposed PES with a single network as ours and the results for PES, which ensembles two networks, as ours*. The overall results are reported in Table 5, from which we can observe that the proposed PES with a single network can outperform all baselines using a single network. And with an ensemble model, which contains two different networks, our method can outperform all the adopted baselines. These results clearly demonstrate that, by improving the performance of the initial classification network, our method is more flexible to handle such real-world noise problems.

In this section, we investigate the hyper-parameter sensitivity for the training iteration number $T_{2}$ and $T_{3}$, respectively. We firstly analyze the training epoch number for the second DNN part by varying $T_{2}$ from the range of $[0,10]$. The results are illustrated in Figure 3a, from which can find that, with the increasing of $T_{2}$, the performance of PES first increase and then decrease in all the cases except for 45% Pairflip noise on the CIFAR-10 dataset. While the model achieves the best performance with $T_{2}$ as $7$ for all types of noisy labels. Then we fix $T_{2}$ as 7, and analyze the impact of the third DNN part by varying $T_{3}$ from the range of $[0,10]$. The results are shown in Figure 3b. Although the performance variance for different $T_{3}$ is smaller than that for $T_{2}$, we can still observe that the best performance can be obtained when $T_{3}$ is set as $5$. More importantly, from these two figures, we can get that both $T_{2}$ and $T_{3}$ are robust to the different types of noisy labels.

In this section, we compare the training time of our method and other state-of-the-art baselines. All the experiments are conducted on a server with a single Nvidia V100 GPU. The training times for all the methods are reported in Table 6, from which we can get that our algorithm with cross-entropy loss achieves the fastest speed across all baselines, only about 1 hour. Our method combining with MixMatch [4] is also fast, only a little more than half of the training time of DivideMix. The time of ELR+ [17] shows superior, but ELR+ trains the network with fewer epochs, with 200 epochs compared with ours for 300 epochs.