Learning from Noisy Labels with Coarse-to-Fine Sample Credibility Modeling

Formally, for multi-class classification with noisy label problem, let $\mathcal{D}=\{(x_{i},y_{i})\}_{i=1}^{N}$ denote the training data, where $x_{i}$ is data sample and $y_{i}\in\{0,1\}^{C}$ is the one-hot label over $C$ classes. $f(x_{i};\theta)$ denotes sample feature extracted by DNN model. With the loss $\mathcal{L}(x,y)$ from the DNN model, clean set $\mathcal{X}_{c}$, and noisy set $\mathcal{X}_{u}$ are separated via the widely used low-loss criterion [10, 45],

where $\tau$ is the threshold, determined by a dynamic memory rate $R(t)\in[0,1]$. Which is set for DNN to gradually distinguish $(1-R(t))$ data with highest loss value as noisy samples while keeping other samples as the clean set. The loss value simply serves as credibility of each sample in a coarse level. However, as illustrated in Fig. 1, this simple separation criterion can not strictly eliminate noisy samples. Hence we choose to handle them respectively, where $\mathcal{X}_{c}$ is exploited to update DNN parameters via fine-grained sample credibility guided loss adjustment (Sec. 3.2), and $\mathcal{X}_{u}$ is leveraged via another label learning scheme (Sec. 3.3).

Sequential credibility analysis. Previous works prefer to assess the data reliability purely based on its statistical property on a single point of time (e.g. the loss value in current epoch) during training process, i.e. they regard the credibility $w(x,y)$ of the $i$-th data sample $(x,y)$ proportional to the joint distribution or likelihood of its data-label pair,

However, as shown in Fig. 1, the training curve of normal and noisy samples usually yield different statistic information, where noisy samples usually have relatively larger loss values and poorer prediction consistency compared with clean ones, therefore the historical record of data training is also informative enough to help distinguish noisy and clean data.

This observation inspires us to estimate the data credibility in a sequential manner. To be specific, we define a sequence with length $n$ as:

Eq (3) illustrates a sliding window covering the feature snapshot of data from previous $n$ epochs to current time point, where $\theta_{t}$ denotes the model parameters at the $t$-th epoch, and we model the data credibility with the likelihood and consistency of these historical sequences,

The Eq (4) can be decoupled into two items, where $C(\mathbf{L}^{n}_{t},y)$ measures the stability of training sequence given its label $y$, while $\log{P\left(\mathbf{L}^{n}_{t}|\mathbf{f}_{t-n},y\right)}$ denotes log-likelihood of sequence generated from the $(t-n)$-th data observation of neural network training process. To estimate the sequential log-likelihood, we further assume that the observation in sequence $\mathbf{L}^{n}_{t}$ conforms to a certain markov property as:

The assumption in Eq (5) is reasonable since in most iterative learning algorithm like SGD, the data feature distribution is only decided by the last observation and its label. With this assumption, we can further derive the likelihood as:

With Eq (6), we can represent the sequential likelihood as the summation of the conditional likelihood of data observation at each adjacent epochs within a sliding window of length $n$. In the implementation, we can apply a normalized mixture model like GMM [16] or BMM [1] as estimator to estimate the conditional probability $P\left(\mathbf{f}_{t-i}|\mathbf{f}_{t-i-1},y\right)$ in Eq (6) via modeling the sample-wise loss value distribution. Meanwhile, with the conditional probability estimation, the stability measurement $C(\mathbf{L}^{n}_{t},y)$ is further designed as a modulator to suppress loss on training sequence with intense fluctuation,

Adaptively loss adjustment. The sequential likelihood $\bar{P}\left(\mathbf{L}^{n}_{t}\right)$ and stability measurement $C(\mathbf{L}^{n}_{t},y)$ reflects how confident of the sample being clean. With the estimated credibility we reweight loss to update DNN as:

Where $\mathcal{L}$ is the objective function. $w(x,{y})$ is the sample credibility and it modulates the contribution of each sample through gradient descending algorithm. Note that Eq (9) is only applied on clean set $\mathcal{X}_{c}$, in this way, the negative impact of hard noisy samples within $\mathcal{X}_{c}$ can be mitigated.

Objective function. Inspired by the design of symmetric cross entropy (SCE) function [44], a symmetric JS-divergence function with a co-regularization term is leveraged in CREMA as:

Where $h(x)$ is the softmax function, $f_{1}(x)$ and $f_{2}(x)$ are features extracted by two models. The reason we choose JS-divergence [27] instead of cross entropy (CE) as loss function is that CE tends to over-fit noisy samples as these samples contribute relatively large gradient values during training. While JS-divergence mitigates this problem via using predictions of the current model as supervising signals as well. Since for noisy samples, DNN predictions are usually more reliable than its label. Following previous work [37], a prior label distribution term and a negative entropy term are included to regularize training and further alleviate the over-fitting problem.

Following the divide-and-conquer idea, we attempt to leverage the separated noisy samples $\mathcal{X}_{u}$ as well. Some hard clean samples are blended with the separated noisy ones. Thus instead of discarding these wrongly labeled data as in most sample detection methods [10, 45], we resort to label correction approaches [37, 48] to exploit them with gradually corrected labels and further boost performance.

Specifically, labels of $\mathcal{X}_{u}$ are treated as extra parameters and updated through back-propagation to optimize a certain objective, this means both the network parameters and labels are updated simultaneously during the training process, where original one-hot labels ${{y}}$ will turn into a soft label distribution $\tilde{{y}}=h(y)$ after updating. Formally, $\tilde{{y}}$ is updated as $\tilde{{y}}\leftarrow\tilde{{y}}-\lambda(\partial\mathcal{L}_{l}/\partial\tilde{{y}})$. Where $\lambda$ is learning rate and $\mathcal{L}_{l}$ is the objective to supervise the label correction process as $\mathcal{L}_{l}=D_{\mathrm{JS}}(h(f_{1}(x;\theta)||\tilde{{y}})+D_{\mathrm{JS}}(h(f_{2}(x;\theta)||\tilde{{y}})$.

Empirical insight. In our experiments. we find that global label learning strategy (i.e., correcting labels of all training data) suffers from correction error in clean data. This can be observed from Fig. 3 (a), large gradient value will also be imposed on lots of correctly-labeled samples. Consequently, labels for these clean samples are unnecessarily updated. Compared with global label correction manners, we choose to only update the separated noisy samples $\mathcal{X}_{u}$ (mostly noisy). As shown in Fig. 3 (b), the proposed selective label correction strategy focuses more on learning noisy labels. The number of correctly-labeled samples with large gradient value is way less than a global correction scheme. Indicating that the selective label correction scheme can mitigate the problem of correction error. Experiments in Sec 4.3 also quantitatively verify the effectiveness of the selective label learning strategy over the global label learning manner.

Putting this all together, Algorithm 1 delineates the proposed CREMA in detail. In a nutshell, CREMA is built on a divide-and-conquer framework. Firstly, clean set $\mathcal{X}_{c}$ and noisy set $\mathcal{X}_{u}$ are separated based on the low-loss criterion [10]. For $\mathcal{X}_{c}$, we compute the likelihood of historical credibility sequence, which helps to adaptively modulate the loss term of each training sample. As for $\mathcal{X}_{u}$, a selective label correction scheme is leveraged to update label distribution and model parameters simultaneously. After each training epoch, memory sequence $\mathbf{L}^{n}_{1,t}$ and $\mathbf{L}^{n}_{2,t}$ are updated with the feature snapshot of the most current epoch.

Datasets. To validate the effectiveness of the proposed method, we experimentally investigate on four synthetic noisy datasets, i.e., IMDB [25], MNIST, CIFAR-10, CIFAR-100 [15] and two real-world label noise datasets, i.e., Clothing1M [46], and Animal10N [36]. IMDB [25] is a collection of highly polarized movie reviews (positive/negative). It consists of 25,000 training samples and 25,000 samples for testing. The task is formalized as a binary classification task to decide the polarity of sentiment for a review. MNIST consists of 70,000 images of size $28\times 28$ for 10 classes, in which 60,000 images for training and the left 10,000 images for testing. Both CIFAR-10 and CIFAR100 contain 50,000 training images and 10,000 testing images of size $32\times 32\times 3$. Differently, the former has 10 classes, while CIFAR-100 has 100 classes. For the Clothing1M, it is a large-scale real-world noisy dataset which is collected from multiple online shopping websites. It contains 1 million training images and clean training subsets (47K for training, 14K for validation and 10K for test) with 14 classes. The noise rate for this dataset is around 38.5%. Animal-10N contains 55,000 human-labeled online images for 10 confusing animals. It includes approximately 8% noisy-labeled samples. Following the settings in previous works [36, 54], 50,000 images are exploited as a training set while the left for testing.

Implementation Details. For the three synthetic image classification noisy datasets, MNIST, CIFAR-10 and CIFAR-100, we follow the setting in previous works [10, 49, 45], experiments with three kinds of noise types are considered, i.e., symmetric noise (uniformly random), asymmetric noise, and pairflip noise. For the IMDB text classification dataset, we tokenize each sentence and the word embeddings have dimension 10,000. Symmetric noises with different noise level are tested. Specifically, symmetric noise is generated by replacing labels in each class with labels of other classes uniformly. Asymmetric noise simulates fine-grained classification (for example, lynx and cat in Animal-10N [36] with noisy labels, where labels are corrupted to a set of similar classes. Pairflip noise is generated by flipping each class to its adjacent class. Varying noise rates $\tau$ are conducted to fully evaluate the proposed method, where for symmetric label noise, we set $\tau\in\{20\%,50\%,80\%\}$ on image datasets, $\tau\in\{20\%,40\%\}$ on text dataset, $\tau=40\%$ for asymmetric noise and $\tau\in\{40\%,45\%\}$ for pairflip label noise. For real-world noisy Clothing1M dataset, following  [48, 54], we do not use the 50K clean data, and a randomly sampled pseudo-balanced subset includes about 260K images is leveraged as training data.

For the network structure, a 2-layer bi-directional LSTM network is adopted for IMDB. It is of 128 embedding size and 128 hidden size. A 9-layer CNN with Leaky-ReLU activation function [10] is used for MNIST, CIFAR-10, and CIFAR-100, while ResNet-50 is adopted for Clothing1M and Animal-10N datasets. The batch size is set as 64 for all the datasets. For fair comparisons, we train our model for 200 epochs in total and choose the average test accuracy of last 10 epochs as the final result in three image synthetic noisy datasets. For IMDB dataset, we set total training epochs as 100 and also test the accuracy of last ten epochs. Total training epochs for Clothing1M and Animal-10N are 80 and 150 respectively. Additionally, all the methods are implemented in PyTorch and run on NVIDIA Tesla V100 GPUs. Moreover, we use Adam optimizer for all the experiments and set the initial learning rate as 0.001, then it is degraded by a factor of $5$ every $30$ epochs for Clothing1M and $50$ epochs for Animal-10N. The two classifiers in our methods are two networks with the same structure but different initialization parameters. Following [10], $R(t)$ is linearly decreased along with training until reach a lower bound value $\sigma$, for Clothing1M and Animal-10N datasets, we empirically set lower bound $\sigma$ as 0.8 and 0.92 respectively.

Results on synthetic noisy datasets. Table 1, Table 2, Table 3 and Table 6 show the detailed results of the proposed CREMA and other methods in multiple synthetic noisy cases on four widely used datasets, i.e., MNIST, CIFAR-10, CIFAR-100 IMDB. Specifically, four state-of-the-art LNL methods that are highly related to our work are chosen for comparison: PENCIL [48], Co-teaching [10], Co-teaching+ [49], JoCoR [45]. Standard DNN training with cross entropy is also included as baseline. All the results of these methods are reproduced with their public code and suggested hyper-parameters for fair comparison. From these tables, we can observe that most of these methods show better performance than Standard in the most natural Symmetry-20% case, except for PENCIL on IMDB dataset, which verifies their robustness. Among them, JoCoR performs much better over other compared methods. However, when it comes to Pairflip-40% and Pairflip-45% noisy cases on image datasets, their performance drops significantly. On the contrary, the proposed CREMA can achieve consistent improvements over other methods on four benchmarks across various noise settings. In the Pairflip-40% and Pairflip-45% cases, the proposed method outperforms other baselines by a large margin. Specifically, CREMA can achieve 16.44% and 25.26% improvement in accuracy over JoCoR on CIFAR-10. When dealing with extremely noisy scenario, e.g. Symmetry-80%, CREMA can also perform generally better than other compared methods.The result demonstrates the superiority and generality of the proposed robust learning method across various types and levels of label noise on multimedia (i.e., image and text) datasets.

Results on real-world noisy datasets. Experiments on real-world noisy datasets Clothing1M [46], Animal-10N [36] are also conducted to verify the effectiveness of the proposed method. The baseline methods are chosen from recently proposed LNL methods. Specifically, several loss adjustment methods, including GCE [55], IMAE [42], SCE [44], DM [43], F-correction [29], M-correction [1], Masking [9], ActiveBias [4], label correction methods, including Joint-Optim [37], PENCIL [48], Self-Learning [11], PLC [54], ProSelfLC [41] noisy sample detection approaches, including Co-teaching [10], JoCoR [45], C2D [57], DivideMix [16] and a hybrid method SELFIE [36] are compared with the proposed method. Table 6 and Table 6 show results on two real-world noisy datasets respectively. On the large-scale Clothing1M dataset, CREMA outperforms all compared methods. Note that CREMA follows the standard DNN training procedure, and is similar to other co-training methods [10, 49, 45] in terms of training time since the time cost for sample credibility modeling is negligible compared with DNN update. It is worth noting that the proposed method outperforms these co-teaching methods by a large margin. Indicating that the discarded samples by co-teaching methods are actually valuable, and CREMA well utilized all training samples. The best test accuracy is achieved by CREMA among the compared methods in Animal-10N as well. The results indicate that the proposed method can work well on high noise level (i.e., Clothing1M) and fine-grained (i.e., Animal-10N) real-world noisy datasets.

$\bullet$ Component Analysis. CREMA contains several important components, including selective label learning strategy, sequential likelihood $\log{P\left(\mathbf{L}^{n}_{t}|\mathbf{f}_{t-n},y\right)}$ and stability measurement $C(\mathbf{L}^{n}_{t},y)$. To verify the effectiveness of each component, we conduct experiments on large-scale noisy dataset Clothing1M. The baseline method is built upon a simple co-teaching framework [45] combined with global label correction schemes (as in [48]), without the credibility guided loss adjustment strategy. The results are shown in Table 8, we can see that, conform to the observation in Fig. 3, the proposed selective label learning strategy achieves better results compared with the global correction counterpart. The sequential likelihood and stability measurement further boost the model performance with 0.75% and 0.53% accuracy gain, this indicates that the proposed sequential sample credibility modeling can effectively combat hard noisy samples mixed with clean ones. With all the three key components above, CREMA can achieve 74.53% test accuracy on Clothing1M.

$\bullet$ Length of sequence $n$. We also conduct experiments to investigate how the length of sequence $n$ affects the performance. Fig 4(a) shows results on Clothing1M with various values (in {1, 2, 3, 4, 5, 6}) of $n$. It can be observed that increasing the length of sequence helps achieve higher accuracy at first but turn poor after hitting the peak value. Intuitively, when no temporal information is provided when $n=1$, CREMA can not utilize consistency metric to identify hard noisy samples that blended with clean ones, thus leading to an inferior result. When $n$ is larger than 4, we also notice that performance degrades, this is probably due to unreliable model inside the very long sequence can harm sample credibility modeling and reversely hinder the final result.

$\bullet$ Effect of different estimators. The probabilistic model plays the role of estimating the conditional probability $P\left(\mathbf{f}_{t}|\mathbf{f}_{t-1},y\right)$ in Eq (6). We compare two different estimators, Gaussian Mixture Model (GMM) [31] and Beta Mixture Model (BMM) [1] on Clothing1M. Table 8 shows the results. We can see that GMM obtains a relatively higher test accuracy, but BMM can also achieve good results (74.09%) as well. This indicates that the choice of normalized mixture model is not sensitive to the final result.

$\bullet$ Reliability of the estimated sample credibility. Sample credibility plays the role of dynamical weight to modulate the loss term of each training sample, as in Eq (9). In Fig. 4 we visualize the empirical PDF of the learned credibility weight of all training samples between (b) the sequential estimation manner within CREMA and (a) its non-sequential counterpart (i.e., $n=1$) under two different noisy settings. It can be observed that the overall credibility weight of training samples is distinguishable for clean and noisy data in (b). Specifically, clean samples possess larger weight, thus contributing more gradients during the process of DNN training. Noisy samples are assigned with a relatively small weight to alleviate their negative impact. However, the non-sequential weights yield significantly more samples that cannot be correctly-separated via a fixed threshold. Indicating that the proposed fine-grained sequential credibility estimation is more effective for reliable sample weight modeling.