Online Class Incremental Learning on Stochastic Blurry Task Boundary
via Mask and Visual Prompt Tuning

Class Incremental Learning (CIL) [12] assumes that each task contains distinct classes not overlapping with another and that the class observed once in a task never appears again in subsequent tasks. CIL is categorized into the (1) regularization-based method, (2) replay-based method, (3) parameter isolation method, and (4) prompt-based method. The regularization-based method uses previous knowledge for regularizing the network while training new tasks [23, 17, 4, 27, 42]. The replay-based method stores a few samples from the old task and replays them in the new task to mitigate catastrophic forgetting [34, 38, 2, 6, 29]. The parameter isolation method expands the network or consists of sub-networks in a single network for each task [37, 43, 1, 36, 44]. The prompt-based method proposed in natural language processing (NLP) for transfer learning attaches a set of learnable parameters, named prompt, to the frozen pre-trained model [40, 39, 15].

Blurry Continual Learning [33, 3] assumes no new classes appear after the first task even though classes overlap across the tasks. A blurry setup has some requirements. First of all, each task is streamed sequentially. Second, the major class of each task is different. Last, a model can leverage only a small portion of data from the previous task. A blurry setup seems realistic. However, a blurry scenario has a shortage to apply real-world scenarios in that observing new classes is commonplace in a real-world scenario. i-Blurry [18] proposes a more realistic setting that considers a blurry scenario with a class-incremental setting. However, the i-Blurry scenario also has a limitation of properly reflecting the real-world scenario due to: (1) the same number of new classes appearing in every task, and (2) new classes and blurry classes having the same proportion in every task. This is why we propose a stochastic incremental blurry scenario, which focuses on the stochastic property critical to real-world scenarios.

Class imbalance, known as the long-tail problem, is that the classes are not represented equally in the classification task. Class imbalance is common in the real-world and it can cause inaccurate prediction performance in classification problems. In continual learning, the replay-based method suffers severe catastrophic forgetting due to the inequality between stored old samples and streamed new samples [42]. To address this problem, existing methods consider gradient information to get the knowledge of prior tasks during training [29, 2], episodic memory management to enhance model performance by sampling effective samples [5, 28, 41], and calibration of the bias [42]. However, in a blurry scenario, incoming samples per class are different and this causes a class imbalance problem. Class imbalance in a task leads to bias for disjoint classes and major classes which exacerbates the training of the minority classes.

In a real-world scenario, the quantity of input data and tasks tends to change a stochastic manner. To simulate this, we propose the Stochastic incremental-Blurry (Si-Blurry) scenario. From [18], we divide the classes into two categories using disjoint class ratio: disjoint classes and blurry classes. As shown in Figure 2, we randomly assign each blurry class and disjoint class to each blurry task ($T^{B}$) and disjoint task ($T^{D}$) by the disjoint class ratio. In blurry tasks, we gather the sample of blurry sample ratio and randomly distribute it to each task. This makes the classes on each task overlap, which blur explicit task boundary. Each task with a stochastic blurry task boundary $T^{B+D}$ consists of $T^{B}$ and $T^{D}$. Figure 1(b) shows an example of the distribution of Si-Blurry. Because the Si-Blurry task is stochastic, the batches get more diverse and imbalanced. As a result, there are lack of explicit task boundaries and the data imbalance, which pose significant challenges to formal continual learning methods. We define two problems that are exacerbated on Si-Blurry in the following subsections.

First, intra-task forgetting can be interpreted as inter-batch forgetting. This problem also presents in joint training but is largely addressed by randomizing the sample order in each batch. However, in online learning scenarios, each sample is only presented once, making it impossible to apply this strategy. Additionally, the stochastic nature of Si-Blurry creates more diverse batches, thereby intensifying the issue of intra-task forgetting. Intra-task forgetting can severely limit the ability of the model to learn and generalize well in the face of evolving and dynamic data distributions.

Inter-task forgetting, which refers to the phenomenon of losing previously learned knowledge of a task due to the changes in class distribution, is a major challenge in online continual learning. Si-Blurry, a scenario proposed to simulate the complexity of real-world data in a stochastic manner, does not have clear task boundaries like those in conventional continuous learning, making it difficult to handle this problem. The stochastic change in class distribution in Si-Blurry exacerbates this problem. Effective strategies need to be devised to enable the model to learn the knowledge continually from data with varying class distributions, without losing previously acquired knowledge, and any catastrophic forgetting.

Minor-class ignorance and overfitting to major classes cause the class imbalance problem. The issue of minor-class ignorance arises when the number of samples in a batch varies and this issue causes an imbalanced weighted loss. The loss imbalance leads to the negligence of the minority of samples in a batch, resulting in their poor representation while training the model. Overfitting to major classes is, conversely, a phenomenon where a class with a large number of samples deteriorates the model performance by acquiring unnecessary knowledge relating to generalization performance.

These are particularly important issues in the context of Si-Blurry, where no explicit task boundaries exist, and continual learning requires a model to capture the knowledge from a wide range of samples. Finding a solution to this problem is essential for ensuring that the model remains effective in recognizing and classifying all samples, regardless of the number of samples per class, and that it continues to learn and improve over time.

The Si-Blurry scenario considers learning a model with only a few samples because it cannot access the whole current training data. When given the accessible data $\mathcal{B}=\left\{\mathbf{x_{i}},y_{i}\right\}_{i=1}^{N}$, where $\mathbf{x_{i}}\in\mathcal{X}$, $y_{i}\in\mathcal{Y}$, we reshape the samples to a flattened patch shape $\mathbb{R}^{L\times(S^{2}\times C)}$ to feed into the pre-trained model $f:\mathbb{R}^{L\times(\mathrm{S}^{2}\times\mathrm{C})}\to\mathbb{R}^{L\times\mathrm{D}}$ which is frozen, where $L$, $S$, $C$, and $D$ represent the token length, patch size, channel, and embedding dimension, respectively. A linear classifier $W\in\mathbb{R}^{D\times\left|\mathcal{Y}\right|}$ is trainable.

Previous studies demonstrate the effectiveness of using knowledge from the pre-trained model and tuning the small size of parameters [40, 39] for continual learning. To this end, we adopt the prompt tuning method [22] for online continual learning. Similarly to DualPrompt [39], we utilize a pre-trained Vision Transformer (ViT) as a feature extractor for the query. We match the query with the key to apply the contrastive visual prompt tuning loss and select the prompt.

Existing prompt methods assume the explicit task boundary and require information on the task boundary for training, which is not feasible in Si-Blurry, where no explicit task boundary exists. The cross-entropy loss is highly effective in optimizing classification models, but it requires a proper comparison target to acquire sufficient knowledge. Unlike traditional joint training, the absence of comparison constantly leads to forgetting in online continual learning. To address this problem causing intra- and inter-task forgettings, we propose a new instance-wise logit masking technique.

To complement the prompt-based continual learning approach and further enhance the performance of the model, we introduce a learnable mask paired with prompts that helps the model to learn more intra-relevant and easier learning goals. Since the key-value mechanism is used to select each prompt, where a feature extracted by the pre-trained model serves as a query, each prompt can be responsible for a certain region of the feature space in which classes have similar extracted features. As illustrated in Figure 3, we apply the mask to logit using an element-wise product and train the mask, and then calculate cross-entropy loss which makes the mask divide the tasks into easier classification tasks. The logit masking provides the model with a scaled gradient during back-propagation, which protects the knowledge that has been sufficiently trained and encourages the learning of classes to be learned.

The logit mask assumes that each key enables the prompts to learn similar knowledge. We empirically find that the existing prompt-based method [40] converges to a single point, which renders the query selection mechanism inaccurate and meaningless. Moreover, the keys are updated continuously, which causes forgetting. To overcome these challenges and leverage the benefits of prompt-based continual learning, we propose a novel loss function, called Contrastive Visual Prompt Tuning Loss. We formulate this loss term as follows:

where $\delta$ is cosine distance, $P$ denotes the size of the prompt pool, $\textbf{q}_{n}\in\mathbb{R}^{D}$ indicates the query feature, $\textbf{k}_{n}\in\mathbb{R}^{D}$ denotes the key of $n^{th}$ prompt, and $\mathcal{C}_{n}$ means the count of selection of $n^{th}$ prompt. In $\mathcal{L}_{CVPT}$, $(\mathcal{C}_{p}+1)$ plays a role of the temperature to control the softness. If $\mathcal{C}_{n}$ is large, the effect of loss to key lessens. As illustrated in Figure 3, the $\mathcal{L}_{CVPT}$ increases the distances between keys. Also, as the prompt learns, the keys become heavier to ensure consistency in key selection. The instance-wise logit masking coupled with $\mathcal{L}_{CVPT}$ can prevent inter-task and intra-task forgetting by ensuring that each prompt divides its responsible region and preserves the knowledge.

Since the blurry setup has a task that comprises of imbalanced classes, it is challenging for the model to extract the knowledge of all the observed classes in the blurry setup. Due to the stochastic nature of Si-Blurry, we cannot guarantee a minimum number of samples for minor classes. To mitigate the aforementioned minor class ignorance, we propose a Gradient Similarity-based Focal loss $\mathcal{L}_{GSF}$ (GSF loss). It focuses on the loss from ignored samples leveraging ignore scores $\mathrm{Score}^{ign}$. The ignore score $\mathrm{Score}^{ign}_{i}$ denotes how much a sample $\mathbf{x_{i}}$ is ignored by other samples during training. We use cosine distance in between a gradient vector from each sample $\nabla W_{y_{i}}(f(\mathbf{x_{i}}))\in\mathbb{R}^{D}$ and the averaged gradient vector $\nabla W_{y_{i}}(f(\mathcal{B}))\in\mathbb{R}^{D}$ from accessible data $\mathcal{B}$ to yield an ignore score for a sample $\textbf{x}_{i}$. We formulate ignore score and GSF loss as:

where $(\textbf{x},y)\in\mathcal{B}$ is a training sample, $\mathcal{L}_{CE}$ is a cross-entropy loss, and $W_{y_{i}}$ is the weights of corresponding label, respectively. Eq. 4.4 represents ignore score for a sample $\textbf{x}_{i}$. When the $\mathrm{Score}^{ign}$ has a high value, it implies the model is hard to extract the knowledge from the sample, whereas the low $\mathrm{Score}^{ign}$ means vice versa. Leveraging $\mathrm{Score}^{ign}$, we can emphasize the loss from the ignored sample and capture more knowledge of minor classes than before as illustrated in Figure 3. Eq. 3 represents GSF loss which considers the amount of ignorance. In [24], the focal loss dynamically scales cross-entropy loss considering confidence in the correct class. Our proposed GSF loss also dynamically scales cross-entropy loss. However, our loss scales the cross-entropy loss considering ignore score. Ignore score is the degree of ignorance that is conceptually different from confidence. GSF loss can mitigate the class ignorance problem which minor classes highly suffered, and enables balanced class learning.

In online learning, the model cannot access all the data of the current task but access a few samples. In our novel Si-Blurry scenario, each task has a class imbalance problem. When the model access training data, there are no or few samples of minor classes. It makes the model overfit to major classes and newly streamed disjoint classes. To mitigate the overfitting problem caused by the class imbalance, we propose Adaptive Feature Scaling (AFS) which expands or contracts the feature vector considering the marginal benefit score $\mathrm{Score}^{MB}$. Using the $\mathrm{Score}^{MB}$, the model can learn new knowledge from the accessible data while preserving the knowledge from inaccessible prior data.

As prior works [9, 25, 26] suggest, the similarity between the feature vector and the weights from the last fully connected layer relates to the prediction when the model is trained by cross-entropy loss with softmax function. We propose a marginal benefit score $\mathrm{Score}^{MB}$ that represents how similar the feature vector is with the weights of the corresponding label. Leveraging this, we estimate the marginal benefit from the given instance and adjust the model updates by the given instance. We can calculate $\mathrm{Score}^{MB}$ as follows:

where m is a margin, and $\mathbf{h_{i}}$ is a scaled feature vector by $\mathrm{Score}^{MB}_{i}$. When the $\mathrm{Score}^{MB}$ has a high value, it implies the given sample has a large marginal benefit. The $\mathrm{Score}^{MB}$ reduces the feature vector to increase the expected loss value. Enlarged expected loss makes the model learn enough knowledge from the given sample. In contrast, when the $\mathrm{Score}^{MB}$ has a low value, it implies the given sample has a little marginal benefit for the model. In this case, a feature vector is expanded to decrease the expected loss value. The model is less trained due to the curtailed expected loss. We estimate the marginal benefit that can be extracted from a sample and scale up and down the feature vector considering the marginal benefit.

To overcome the class imbalance problem, we propose two components that seem similar: gradient similarity-based focal loss (GSF) and adaptive feature scaling (AFS). Although GSF and AFS look similar, their main roles are different. GSF emphasizes learning minor classes to tackle the class imbalance problem in the task. AFS regularizes learning major classes to address the overfitting problem.

Finally, we train our model in an end-to-end manner. The total loss for our method is defined as:

where $\mathcal{L}_{\mathrm{CE}}$ is cross entropy loss with instance-wise logit masking. We use hyperparameter $\alpha$ for the balanced training and $\gamma$ at the gradient similarity-based focal loss to scale the ignore score.