Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks Using an Incompetent Teacher

We aim to remove the information about the requested data-points by using two teachers (competent and incompetent) and one student. The student is initialized with knowledge about the complete data i.e., the parameters of the fully trained model. The idea is to selectively remove the information about the forget samples from this model. At the same time, the information pertaining to the retain set should not to be disturbed. Thus, the unlearning objective is to remove the information about $D_{f}$ while retaining the information about $D_{r}$. We achieve this by using a pair of (competent/smart ($T_{s}$) and incompetent/dumb ($T_{d}$)) teachers to manipulate the student ($S$) as depicted in Figure 1. The bad knowledge about $D_{f}$ from the incompetent teacher $T_{d}$ is passed on to the student which helps the student to forget $D_{f}$ samples. Such an approach consequently induces random knowledge about the forget set in the student instead of completely making their prediction accuracy zero. This serves as a protection against the risk of information exposure about the samples to forget. The bad (random) inputs from $T_{d}$ may invariably corrupt some of the information about the retain set $D_{r}$ in the student. Therefore, we selectively borrow correct knowledge related to $D_{r}$ from the competent teacher $T_{s}$ as well. In this manner, both the incompetent and competent teachers help the student forget and retain the corresponding information, respectively.

For a student $S$, incompetent/dumb teacher $T_{d}$, and competent/smart teacher $T_{s}$, we define the KL-Divergence between $T_{d}$ and $S$ in Eq. 1.

$$\mathcal{KL}(T_{d}(x)||S(x))=\sum_{i}{t_{d}^{(i)}log(t_{d}^{(i)}/s^{(i)})}$$

(1)

where $i$ corresponds to the data class. Similarly, the KL-Divergence between the fully trained competent teacher $T_{s}$ and student $S$ is given in Eq. 2.

$$\mathcal{KL}(T_{s}(x)||S(x))=\sum_{i}{t_{s}^{(i)}log(t_{s}^{(i)}/s^{(i)})}$$

(2)

The unlearning objective can be formulated as in Eq. 3.

$$L(x,l_{u})=(1-l_{u})*\mathcal{KL}(T_{s}(x)||S(x))+l_{u}*(\mathcal{KL}(T_{d}(x)||S(x)))$$

(3)

where $l_{u}$ is the unlearning label and $x$ is a data sample. The data samples used by the proposed unlearning method consists of all the samples from $D_{f}$ and a small subset of samples of $D_{r}$. The student is then trained to optimize the loss function $L$ for all these samples. The intuition behind optimizing over $L$ is that we selectively transfer bad knowledge about forget data $D_{f}$ from $T_{d}$ by minimizing KL-Divergence between $S$ and $T_{d}$ and the accurate knowledge corresponding to $D_{r}$ is fed from $T_{s}$ by minimizing KL-Divergence between $S$ and $T_{s}$. The student learns to mimic $T_{d}$ for $D_{f}$, thus removing information exclusively pertaining to those samples while retaining all the generic information which can be obtained by other samples of same class.

The effectiveness of an unlearning method is evaluated employing several metrics in the literature. Some frequently used metrics are ‘accuracy on forget set and retain set’ (Golatkar, Achille, and Soatto 2020a; Tarun et al. 2021; Golatkar et al. 2021; Chundawat et al. 2023), relearn time (Tarun et al. 2021), membership inference attacks (Golatkar et al. 2021; Graves, Nagisetty, and Ganesh 2021), activation distance (Golatkar, Achille, and Soatto 2020a; Golatkar et al. 2021), Anamnesis Index (Chundawat et al. 2023), and layer-wise distance (Tarun et al. 2021). Excluding the forget and retain set accuracy, all of the remaining metrics in the literature require a retrained model i.e., training a model from scratch without using the forget set. These metrics can only be interpreted with reference to such a retrained model. Such dependency on the retrained model for unlearning evaluation would lead to higher time and computational costs. Simply measuring the performance on $D_{f}$ and $D_{r}$ does not reveal whether the information is actually removed from the network weights. Thus it is not a comprehensive measure of unlearning.

We propose a novel ‘Zero Retrain Forgetting Metric’ (ZRF) to enable evaluation of unlearning methods free from dependence on the retrained model. It measures the randomness in the model’s prediction by comparing them with the incompetent teacher $T_{d}$. We calculate the Jensen–Shannon (JS) divergence(Lin 1991) between an unlearned model $M$ and the incompetent teacher $T_{d}$ as below.

$$\mathcal{JS}(M(x),T_{d}(x))=0.5*\mathcal{KL}(M(x)\|m)+0.5*\mathcal{KL}(T_{d}(x)\|m)$$

(4)

where $m=\frac{M(x)+T_{d}(x)}{2}$. The ZRF metric is defined as

$$\mathcal{ZRF}=1-\frac{1}{n_{f}}\sum_{i=0}^{n_{f}}\mathcal{JS}(M(x_{i}),T_{d}(x_{i}))$$

(5)

where $x_{i}$ is $i^{th}$ sample from $D_{f}$ with a total of $n_{f}$ samples. The ZRF compares the output distribution for the forget set in the unlearned model with the output of a randomly initialized model, which is our incompetent teacher in most of the cases. The ZRF score lies between 0 and 1. The score will be close to 1 if the model behaviour is completely random for the forget samples and it will be close to 0 if the model shows some specific pattern.

What is an ideal ZRF score? Suppose there is a class aeroplanes that contains images of Boeing aircraft along with other aircraft models in the training set. If we unlearn Boeing aircraft, we don’t expect the model to now classify them as animals, vegetables or any other totally unrelated class. We still expect most of these unlearned images to be classified as aeroplanes. This comes from the intuition that the model must have been designed and trained with generalization in mind. An unlearning method that makes the performance much worse than the generalization error for aeroplanes is not actually unlearning. It is just teaching the model to be consistently incorrect when it sees a Boeing aeroplane. The ZRF score will be 0 when the model almost always classifies a Boeing aircraft as an animal or some other totally different class. The ZRF will be 1 if the model always classifies all classes with same random probability for Boeing aircraft. Both of these ($\sim 0$ or $\sim 1$) are not the desirable outcomes. We expect the unlearned model to have a generalization performance similar to that of a model trained without the Boeing aircraft. It will have some random predicted logits since the Boeing aircraft class was not overfitted during training.

An ideal value of ZRF score depends on the model, dataset and the forget set. Ideally, the optimal ZRF value is what a model trained without the forget set would have. But in practical scenarios we do not have access to the retrained model. So, a good proxy for the ideal ZRF value could be the ZRF value obtained on a test set. The test set by definition is a set about which the model has never learned anything specifically. It is equivalent to saying, a set that the model has unlearned perfectly.

Machine unlearning of a specific class or cohort often leads to a decrease in accuracy or performance on forget set. Although, it is an expected result when the forget set is orthogonal to the retain set i.e., there are no samples in the retain set similar to the ones in the forget set. But this may not be true when retain set contains data points similar to the forget set samples. The accuracy may drop slightly, may not drop at all, or even increase in some cases. An unlearning method should bring the forget set performance closer to the gold (retrained) model instead of simply reducing it. If the performance of the unlearned model deviates a lot from the gold model, it could lead to Streisand effect. This effect refers to unexpected behaviour of the model on forget samples which may leak information about that data. The leak could be in the form of being consistently & maximally incorrect about only the forget samples, signalling that a deliberate effort was made to forget a selected set of samples. The aim should be to avoid this in order to ensure that the information about the forget set has been properly erased. For example, as mentioned earlier in the aeroplanes example, when method unlearns Boeing aircraft, if it is maximally wrong whenever it sees any Boeing aircraft image and classifies it as sea, animals, mushroom, etc., it will be suspicious. Other unseen aeroplanes will not be classified incorrectly so consistently. That means it has not really erased the information of Boeing aircraft. That information still exists which the model uses to be deliberately incorrect about the forget set.

CIFARSuper20. The CIFAR100 is made up of 20 super classes i.e., there are different variants for each of these 20 classes. We merge all classes of the CIFAR100 into their super classes and convert it into a 20 class set named CIFARSuper20. Each class in CIFARSuper20 have 5 sub-classes, which are actual classes of CIFAR100. We conduct experiments on CIFARSuper20 by forgetting one sub-class from each super class. This setup makes unlearning more difficult than a regular scenario as we need to unlearn a sample/class without damaging the information of another sample/class that looks quite similar to it (for example, forget baby from people super class consisting of baby, boy, girl, man, woman).

We present unlearning results on ResNet18, ResNet34, and Vision Transformer. We use pretrained models to train/fine-tune for 5 epochs using Adam optimizer with a batch size of 256. The learning rate is 0.001 for final layer and 0.0001 for pretrained weight layers. A learning plateau with patience of 3 and reduce factor 0.5 is used. We conduct multiple runs (5 times) of our algorithm which didn’t show any significant variation in performance (refer Table 1). Therefore, we report the results of single run for all the models in this paper.

We unlearn various sub-classes from a super-class. We use 30% of retain data and a single epoch of unlearning for all models. The learning rate of 0.0001 is used for unlearning. Table 1 shows unlearning results on ResNet18 and Vision Transformer (Dosovitskiy et al. 2021). The evaluation is performed on all the metrics discussed earlier. It can be observed in Table 1 that performance of our method is very close to that of the retrained model. There is very low probability of membership inference attack on our unlearned model. The accuracy of our method on the forget and retain set when forgetting rocket images from vehicles is almost same as the retrained models. The membership inference attack probability on the model for samples from rocket class drops to 0.002 from 0.982 after unlearning. The JS-Divergence between predictions of the retrained model and our model is 0.04 in the forget set. This implies the output distribution of unlearned model is very close to the retrained model. The ZRF score of our model becomes 0.99 from 0.87 after unlearning, thus indicating effective forgetting. Furthermore, Table 4 shows the unlearning results in ResNet34 with different types of teachers.

Epileptic Seizure Detection. The dataset consists of the status of seizure in medical patients. There are a total of 178 predictor variables and 5 classes. A 3-layer DNN is trained for classification. The model is trained for 50 epochs using Adam Optimizer with learning rate of 0.01 and plateau with patience 10 and reduce factor 0.1. We unlearn 50 and 100 randomly selected data points. The results are presented in Figure 2(a) and Figure 2(b). We observe that the proposed method performance is close to the retrained model. The accuracy on the forget set indicates that we have indeed effectively unlearned the forget set as the forget accuracy is reduced from around 100% to a generalized performance. For example, in case of forgetting 100 samples, the accuracy on the forget set drops from 90% to 74% in our method which is close the 70% accuracy of the retrained model.

Human Activity Recognition. This is a task of classifying the activity of a person using the readings collected from smartphone sensors that an individual is carrying with her. The observation were taken from 30 different persons. The dataset contains 6 different types of activities which can be classified using time-series data with sensors giving 9 readings at each time-step. An LSTM Model with 2 dense layers after each LSTM is trained to predict the activity. The model is trained for 50 epochs using Adam Optimizer with learning rate of 0.01 and plateau with patience 10 and reduce factor 0.1. Table 2 contains the results of forgetting person 1 and person 3. Detailed results and effects of various parameters on unlearning are present in the supplementary material.

Comparison with Amnesiac learning (Graves, Nagisetty, and Ganesh 2021). We compare our result with Amnesiac learning which fine-tunes the model with random labels on forget samples. Table 2 shows the comparison between both the methods. We compare the activation distance and accuracy on the forget and retain set. A lower activation distance indicates closeness to the retrained model. This subsequently indicates better unlearning and an accuracy closer to the retrained model is desired on forget and retain set. The activation distance for our method is very low compared to Amnesiac method in most of the cases (refer Table 2). Amnesiac method causes too much damage in the forget set of epileptic seizure and human activity recognition dataset, indicating Streisand effect. The accuracy in epileptic seizure dataset (forget set of 50 samples) is 98% for the original model, 70% for retrained model, 74% for our method, and 30% for amnesiac method. The Amnesiac method damages the performance on forget set by a huge margin. It reduces the forget set accuracy to 30% which otherwise should be close to 70%. It should also be noted that activation distance from retrained model is 0.47 for our method and 0.77 for Amnesiac method. Amnesiac method is causing undesired effects and the generated model is very different from the retrained model. Our method, besides being more effective and robust, requires access to only a subset of retain data. We use only 30% of retain data to obtain the results in our method. Our method is $\sim 2\times$ faster than Amnesiac method, more effective even when limited data is available for use.

We also demonstrate full-class (single and multiple classes) unlearning capability of our method. We show results on CIFAR10 and CIFAR100 with ResNet18, AllCNN, and MobileNetv2 models. Class-level unlearning results are compared with an existing method with configuration as in (Tarun et al. 2021). The model update in our method is performed for 1 epoch using 30% of the retain data. The learning rate at the time of unlearning is set to 0.001. Table 3 gives a performance comparison between the proposed and the existing methods. The accuracy on the retain set in CIFAR10 single-class forgetting is 71.06% for UNSIR, 78.32% for the retrained model, and 78.46% for our method. The results are quite similar in all three methods. The accuracy on forget set is zero in the retrained model and UNSIR but our method retains some accuracy on the forget set. This is because the method learns from a randomly initialized teacher which does random predictions and predicts each class with  10% probability and forget model learns the same. This in turn leads to better protection against the risk of privacy exposure.

Our method does not place any constraints on the architecture of the incompetent teacher. Preferably the architecture should be kept same as the student for proper transfer of information. But it can be replaced with smaller models without significantly affecting the results. As the teacher is initialized with random weights, such behaviour can be obtained by a significantly smaller model, or even hard coded algorithms to generate random predictions. A cheaper teacher can make the unlearning process faster without compromising in the quality of unlearning. We replace the incompetent teacher with (i) a small randomly initialized Neural Network, and (ii) a random prediction generator. The random prediction generator first assigns equal probability to all classes and then a adds Gaussian noise to the predictions. The performance with these teachers is shown in Table 4. With ResNet34 as teacher, the performance on $D_{f}$ (forget class: Rocket) is 1% while using the same model as teacher, 15% while using ResNet18, and 5% while using a random predictor as a teacher. Similarly, the performance on retain set while using ResNet34 as teacher is 85.8%, 85.86% while using ResNet18, and 86.04% while using a random predictor as a teacher. There is a negligible change in performance when we use simpler models as teachers. Thus, it can be used to reduce the computational costs without much loss in the performance.

A partially trained (PT) model on a subset of retain data can be used as an incompetent teacher in the proposed framework. Similarly, smaller models trained on a small subset of the retain data can also serve as an incompetent teacher. We show the results of using PT models as incompetent teachers to induce forgetting. Similarly, we further investigate the effectiveness of PT teacher on CIFARSuper20 in Table 5. The teacher is trained for 1 epoch on 50% of the data. The accuracy on forget set Rocket is 3% for retrained model, 2% for our method with PT teacher and 2% for RI teacher. The accuracy on retain set for Rocket is 85.79% for retrained model, 85.07% for our method with PT teacher and 85.05% for our method with RI teacher. Besides, the JS-Divergence between retrained model & RI model based unlearning is 0.04 and 0.02 in case of PT teacher model based unlearning. This shows that in addition to accuracy improvement, PT teacher based unlearning may also give output distribution more similar to the retrained model.

We compare the run-time comparison of the retrained model, the existing methods, and the proposed methods. The random weights based setup is $\sim 70\times$ faster than retraining and more than $2\times$ faster than Amnesiac learning (Graves, Nagisetty, and Ganesh 2021). The method is faster when cheaper unlearning teachers are used. The proxy model based setup is about $20\times$ faster than retraining method. The ideal trade-off between efficiency and performance can be obtained by using smaller models partially trained on retain data but they come with the expense of additional training. This (partial training) further comes with a trade-off between computational cost and closeness of the model to the retrained model. The right amount of partial training should be decided. We observed that the cheap randomly initialized models are more efficient and generally perform well in most cases.