Protecting Multiple Types of Privacy Simultaneously
in EEG-based Brain-Computer Interfaces

Given an EEG dataset $\mathcal{D}=\{(\mathbf{x}_{i},y_{i},\mathcal{S}_{i})\}_{i=1}^{N}$, where $\mathbf{x}_{i}\in\mathcal{X}\subset\mathbb{R}^{c\times t}$ is the $i$-th EEG trial with $c$ channels and $t$ time domain samples, $y_{i}\in\mathcal{Y}=\{1,...,K\}$ the task label (e.g., target or non-target in event related potential classification), $\mathcal{S}_{i}=\{p_{i,m}\}_{m=1}^{M}$ the set of privacy label that can be inferred from the EEG trial $\mathbf{x}_{i}$, and $p_{i,m}\in\mathcal{P}_{m}=\{1,...,P_{m}\}$ a specific privacy label (e.g., male and female in gender).

Typically, a Task-Classifier $F$ can be trained on $\mathcal{D}$ to learn the mapping from the EEG input space to the task label space, i.e., $F:\mathcal{X}\rightarrow\mathcal{Y}$. However, EEG data not only record task-related information but may also contain rich personal privacy. So, a Privacy-Classifier $G_{m}$ can be constructed from EEG data to mine the private information, i.e., $G_{m}:\mathcal{X}\rightarrow\mathcal{P}_{m}$ maps the EEG input space into a specific privacy space.

This paper aims to generate perturbations for the original EEG data to protect multiple types of private information simultaneously, while preserving the task-related information to ensure the normal usage. Specifically, we generate a perturbed EEG dataset $\mathcal{D}^{\prime}=\{(\mathbf{x}^{\prime}_{i},y_{i},\mathcal{S}_{i})\}_{i=1}^{N}$, where $\mathbf{x}^{\prime}=\mathbf{x}+\boldsymbol{\delta}$ and $\boldsymbol{\delta}$ is a deliberately designed perturbation. For any privacy $\mathcal{P}_{m}$, it is difficult to train a Privacy-Classifier $G_{m}$ from the perturbed dataset, making the private information in EEG data unlearnable. However, this perturbed dataset can still be used to train a good Task-Classifier $F$ as the original unperturbed dataset $\mathcal{D}$.

To prevent Privacy-Classifiers from learning private information in the EEG data, we design perturbations highly correlated with the private information, which can mislead the Privacy-Classifier to learn the perturbation pattern rather than the true privacy pattern. Additionally, due to the distribution differences between the privacy feature space and the task feature space, the perturbations generated for privacy protection may have little impact on the primary BCI task, which will be verified in Sections III-G and III-H.

Specifically, we first train a Privacy-Classifier $G^{\prime}_{m}$ for each privacy type $\mathcal{P}_{m}$ to evaluate the influence of perturbations on the privacy-related information, which may be different from $G_{m}$. The loss function for $G^{\prime}_{m}$ is

where $\boldsymbol{\theta}_{G^{\prime}_{m}}$ is the parameter set of $G^{\prime}_{m}$, and $\ell_{\mathrm{CE}}$ the cross-entropy loss.

Then, for each privacy $\mathcal{P}_{m}$, we generate a class-wise perturbation $\boldsymbol{\delta}_{m}=[\boldsymbol{\delta}_{m,1},...,\boldsymbol{\delta}_{m,P_{m}}]$ by minimizing following loss function:

where $\boldsymbol{\delta}_{m,p_{m}}$ is the perturbation for class $p_{m}$ of privacy type $\mathcal{P}_{m}$, and $\alpha$ is a trade-off parameter. The first term enhances the correlation between the perturbation $\boldsymbol{\delta}_{m,p_{m}}$ and the privacy class $p_{m}$, and the second term constrains the perturbation amplitude.

After generating perturbation for each privacy type, we can add them to each EEG trial $\mathbf{x}$ in $\mathcal{D}$ to protect multiple different types of privacy simultaneously:

Finally, the EEG dataset with privacy protection is $\mathcal{D}^{\prime}=\{(\mathbf{x}^{\prime}_{i},y_{i},\mathcal{S}_{i})\}_{i=1}^{N}$. Since the perturbation for each privacy type is very small, the aggregated perturbation is still small enough to have little impact on the primary task performance.

Algorithm 1 gives the pseudo-code of privacy-protected EEG dataset generation.

The publicly available EEG dataset introduced by Lee et al. [16] was used in our experiments. It was collected from 54 healthy subjects, among which 16 were experienced BCI users. Each subject performed a sequence of three tasks: a 36-symbol event-related potential (ERP)  [17] task, a binary-class motor imagery (MI) [6] task, and a four-target steady-state visually evoked potential (SSVEP) [18] task. During each task, 62-channel EEG data were recorded with a sampling rate of 1,000Hz. The entire experimental procedure was repeated twice, so the dataset includes two sessions. The details of each task are as follows:

1. 1.

   ERP: In the ERP task, the subjects were instructed to focus on the target symbol which was flashed randomly to elicit a P300 response. The goal was to classify whether the target symbol flashed or not. The collected EEG data for the ERP task contained 4,140 trials.

2. 2.

   MI: In the MI task, the subjects performed the imagery task of grasping with the corresponding hand when the left or right arrow cue appeared. The collected EEG data for the MI task contained 200 trials with balanced left and right hand imagery tasks.

3. 3.

   SSVEP: In the SSVEP task, there were four blocks with different flickering frequencies presented in four positions on a monitor, and the subjects were asked to gaze at the target block. The goal was to classify which block the subject gazed at. The collected EEG data for the SSVEP task contained 200 trials, with 50 trials per block.

For preprocessing, we downsampled EEG trials to 128Hz and applied a [1,40]/[4,40]/[4,64]Hz band-pass filter for ERP/MI/SSVEP, respectively. Next, we extracted EEG trials between [0,2]s after each task stimuli and standardized the EEG trials separately for each subject, task, and session using $z$-score normalization. To balance the data volume for each task, we downsampled each session of the ERP task to 200 trials.

Before the experiments, the subjects filled out a questionnaire to record their personal information and to check their physical and mental condition, which are all personal privacies. The subjects’ identity, gender and BCI-experience (whether they had prior BCI experimental experience or not) from the questionnaire were used in our experiments. We first validated that these three types of private information can be inferred from the original EEG data, and then designed perturbations to protect them.

We used the following three convolutional neural networks (CNN) as Privacy-Classifiers:

1. 1.

   EEGNet [19]: EEGNet is a compact CNN architecture tailored for EEG-based BCIs. It comprises two convolutional blocks, utilizing depthwise and separable convolutions instead of traditional convolutions, to reduce the number of model parameters.

2. 2.

   DeepCNN [20]: DeepCNN is composed of four convolutional blocks. The initial block is tailored for processing EEG data, while the subsequent three blocks follow a standard convolutional design.

3. 3.

   ShallowCNN [20]: ShallowCNN is a more straightforward variant of DeepCNN, drawing from filter bank common spatial patterns. It features a single convolutional block with a larger kernel and employs a different pooling strategy compared to DeepCNN

Given class-imbalance in gender and BCI-experience classification, balanced classification accuracy (BCA) was used to evaluate the Privacy-Classifier’s performance:

where $P_{m}$ is the number of classes for privacy type $\mathcal{P}_{m}$, $N_{p_{m}}$ the number of test samples in class $p_{m}$, and $\mathbb{I}(\cdot)$ an indicator function.

We performed leave-one-session-out cross-validation, i.e., trials from three tasks in one session were mixed as the training set, and the other session was used as the test set. The average BCA from two sessions were computed. The entire cross-validation process was repeated 5 times, and the average results are reported.

The original EEG signals contain rich personal private information. To demonstrate that, Privacy-Classifiers were trained on the unperturbed EEG dataset to classifier user identity, gender, and BCI-experience. The test performance of Privacy-Classifiers are shown in the ‘Unperturbed EEG’ panel of Table I. Regardless of which CNN model (EEGNet, DeepCNN, or ShallowCNN) was used as the Privacy-Classifier, the BCAs for the three types of private information were much higher than random guess. These results confirmed that rich personal private information can be inferred from EEG data across different BCI tasks, highlighting the necessity of privacy protection in EEG-based BCIs.

To safeguard the private information contained in EEG data, we converted the original EEG data into privacy-protected EEG data, making it difficult for machine learning models to learn the private information.

Specifically, we generated privacy-protected EEG data as described in Algorithm 1. The test performance of Privacy-Classifiers trained on privacy-protected EEG data are shown in the ‘Perturbed EEG’ panel of Table I. For each privacy type (identity, gender, and BCI-experience) classification, the BCAs of the Privacy-Classifiers trained on the perturbed EEG data were significantly lower than their counterparts on the original EEG data. Especially, for gender and BCI-experience classification, the BCAs after perturbations were close to random guess, demonstrating that little private information can be learned from the privacy-protected EEG data.

Notice that although the privacy-protected EEG data were generated by EEGNet, they remained effective for DeepCNN and ShallowCNN, indicating good generalization of privacy-protected EEG data.

In addition to privacy protection, the perturbations should also minimize the impact on the primary BCI tasks, i.e., the performance of Task-Classifiers trained on privacy-protected EEG data should be similar to their counterparts on the original unperturbed EEG data.

Table II shows the test performance of the three CNN models and a traditional model [i.e., xDAWN [21] filtering and Logistics Regression (LR) classification for ERP, common spatial pattern (CSP) [22] filtering and LR classification for MI, and canonical correlation analysis (CCA) [23] for SSVEP] as Task-Classifier trained on the privacy-protected EEG data and the original unperturbed EEG data. The BCAs after applying perturbations were very close to their counterparts on the original unperturbed EEG, suggesting that privacy-protected EEG had almost no negative impact on the performance of primary BCI tasks.

We further show that the characteristics of the EEG data before and after privacy protection are very similar from various perspectives, ensuring the effectiveness of the primary BCI tasks.

Fig. 1 shows the original unperturbed EEG trials and their perturbed counterparts from ERP, MI and SSVEP tasks. For clarity, only three EEG channels (F4, Cz, and F3) are shown. We can observe that the privacy-protected EEG trials and the original unperturbed EEG trials are almost identical for all three BCI tasks.

Fig. 2 shows the average Cz channel spectrograms of the original unperturbed EEG trials and the privacy-protected EEG trials for the target class on the ERP task, the imagination of the right hand on the MI task, and the target with 12Hz flickering frequency on the SSVEP task. The perturbed spectrograms were almost identical to the unperturbed counterparts, suggesting that the perturbations hardly affect the spectrogram of the EEG trials.

Fig. 3 shows the average topoplots of the original unperturbed EEG trials and the privacy-protected EEG trials for the target class on the ERP task, the imagination of the right hand on the MI task, and the target with 12Hz flickering frequency on the SSVEP task. One can hardly observe any differences between the unperturbed and perturbed topoplots regardless of the BCI task, indicating that perturbations hardly change the spatial information of the EEG trials.

Fig. 4 shows how the training and test BCAs of the Privacy-Classifiers and the Task-Classifiers change with the number of training epochs on the original unperturbed EEG data and their perturbed counterparts, respectively. Observe that:

1. 1.

   The training BCAs of the Privacy-Classifiers on the original unperturbed EEG data and the privacy-protected EEG data were close after convergence; however, the test BCAs of the Privacy-Classifiers trained on the privacy-protected EEG data were significantly lower than those trained on their unperturbed counterparts, suggesting that the private information in the EEG data had been successfully concealed.

2. 2.

   The BCA curves for both training and test of the Task-Classifiers on the original unperturbed EEG data and their perturbed counterparts nearly indistinguishable, indicating once again that the perturbations had little impact on the primary BCI tasks.