Analysis of EEG frequency bands for Envisioned Speech Recognition

The emergence of ubiquitous electronic devices has increased the requirement for natural ways of interaction with electronic devices. This has in turn led to significant improvement in human computer interfaces (HCI) such as those driven by speech and gesture [1]. However, there are certain situations in which the use of such mediums of HCI are limited. This may be due to speech disabilities, conditions such as locked-in syndrome or situations in which privacy is of utmost priority to the user. In such cases, imagined speech recognition systems based on non-invasive Electroencephalography could be used as an exciting alternative. Through years, various Brain Computer Interfaces (BCIs) such as those on the principle of text entry or imagination of motor movements have been proposed. Most of these however are focused on a user who is required to focus on a particular image or letter for a significant duration of time in order to select it or imagine a particular movement for providing commands [2]. A BCI framework based on recognition of imagined speech can provide a user with a more natural way for interaction with electronic devices [3]. Imagined speech Recognition here may be defined as the automated recognition of a given object, word or a letter from brain signals of the user. There are various techniques to measure brain signals ranging from invasive methods such as surgically implanted electrodes to the non-invasive EEG. In this work, we focus on non-invasive brain signals acquired using EEG technique. Despite of the low spatial resolution of EEG, it has been widely popular owing to it’s simplistic nature and little to no discomfort to the user [4].

The recognition of imagined speech using EEG has been attempted at various levels such as word, syllable, vowel imagination [4, 5, 6]. In their work, Gonzalez-Castaneda et al. [7] proposed an imagined word classification system using EEG comprising of $5$ words. They first applied a Common Average Reference method to improve the Signal to Noise ratio and then extracted features using the Discrete Wavelet Transform. In [3], classified $2$ words by using Mel Frequency Cepstral Coefficients and a k-Nearest Neighbor classifier. The majority of current envisioned speech recognition systems are heavily reliant on extracting meaningful handcrafted features, which requires in-depth domain expertise [8]. With advancements in the field of deep learning, researchers switched attention towards automatic extraction of features from raw EEG signals. The features extracted from the signals are such that they contain discriminatory information about the classification task. Zhang et al. [8] developed a combination of cascade and parallel Convolutional Recurrent Neural Network (CRNN) architecture for intention recognition using EEG signals. The EEG signals were processed by using 2D-CNN by constructing a 2-dimensional mesh as per the spatial information utilizing the electrode distribution. The features thereby extracted were fed to a LSTM for classification. For a $5$-class classification task, the authors reported an improvement of around $18\%$ over features extracted from frequency information.

In this work, we study the contribution of different EEG frequency bands ($\delta,\theta,\alpha,\beta,\gamma$) and signals acquired from different lobes of the brain (frontal, temporal, occipital and parietal) towards imagined speech recognition using EEG signals. The signals are first bandpass filtered according to different frequency bands as (a) $\delta:<4Hz$, (b) $\theta:4-7Hz$, (c) $\alpha:8-15Hz$, (d) $\beta:16-31Hz$, and (e) $\gamma:>32Hz$. These bandpass filtered signals are fed to a 1DCNN-LSTM based architecture. The purpose of the 1D-CNN block is to extract spatial features from the signals while the LSTM block is used in order to extract the temporal information. We also select specific lobes of the brain and signals from only the selected lobes are fed to the classifier and the classification performance is obtained. Based on our observation that the frequency bands $\theta$ and $\alpha$ have least contribution towards the classification of envisioned speech, we filter out these bands by using a bandreject filter to feed the the classifier. We show that such an approach improves the classification accuracy compared to using all the entire raw EEG signal. We demonstrate the efficacy of this approach on a publicly available envisioned speech dataset [9] consisting of $3$ tasks - digit, character and images. We achieve classification accuracy of $85.93\%$, $87.27\%$ and $87.51\%$ for the three tasks respectively.

We use a publicly available envisioned speech dataset containing recordings from 23 participants aged between 15-40 years [9]. EEG was recorded using Emotiv EPOC+ [10] wireless neuro headset. The headset consists of $14$ channels namely AF3, AF4, F3, F4, F7, F8, FC5, FC6, T7, T8, P7, P8, O1 and O2. The electrodes were placed at the scalp of the subject according to International 10-20 system of electrode placement as depicted in Figure 6. In addition to the aforementioned electrodes, two reference electrodes CMS and DRL were placed above the ears. The signals were recorded at a sampling frequency of $2048$ Hz and then down-sampled to $128$ Hz.

As depicted in Figure 2, the subject is displayed an object on the screen. Then, subject is instructed to imagine the viewed object for $10$ seconds while keeping the eyes closed and in resting state. Before displaying the next image to the participant, a gap of $20$ seconds is maintained in order to ensure that the subject returns to the state of rest before displaying the next item. In this manner, three categories of prompts have been shown to the participants and the corresponding EEG has been recorded.

The first category comprises of digits $0-9$, second comprises of $10$ uppercase English alphabets - A, C, F, H, J, M, P, S, T, Y. The last category consists of items used in daily life - Apple, Car, Dog, Gold, Mobile, Rose, Scooter, Tiger, Wallet and Watch. In total, $230(23*10)$ EEG recordings per category (comprising of $10$ classes) have been collected.

Table I lists the performance of EEG signals obtained from different lobes of the brain and bandpass filtered for different EEG frequency bands. In terms of different frequency bands, it can be seen that the low frequency $\delta$ band contains the most information followed by the $\gamma$ and $\beta$ bands. The performance of $\theta$ and $\alpha$ bands are significantly low compared to the other frequency bands. Thus, when the signal is bandpass filtered and the $\theta$ and $\alpha$ frequency bands are removed, we see a boost in classification performance. This is very well in line with the literature of different frequency bands and their normal occurrence with respect to the task at hand. Furthermore, it can also be seen that the frontal lobe contains most information and therefore the accuracy of signals when only the frontal lobe is considered is significantly high. In Tables II and III, the results of experiments performed on the character and image tasks of the dataset are presented. The trend observed in these two tasks are also similar to that observed for the digit classification task.

Due to the rapid advancement in low-cost EEG measurement systems, there has been a proportionate increase in real-world Brain Computer Interface applications. The recognition of imagined speech is an important task for people suffering from voice disorders or those with conditions such as locked-in syndrome. Moreover, it is also of importance for people with high privacy needs such as in public places where speech is not an ideal medium of communication. In spite of the tremendous possible use cases, there has been scarce work in this field compared to other motor tasks such as grasp and lift, movement etc. Since speech is a natural way of communication among people and also for interaction with other electronic devices, it is expected that a system based on decoding imagined speech should be of extreme importance. Most work surrounding the recognition of imagined speech has been done by using traditional signal processing based approaches such as extracting features like common spatial patterns (CSP), auto-regression coefficients, low-level descriptors such as energy, entropy etc. along with machine learning classifiers such as SVM, Random Forest k-NN, and Logistic Regression. In more recent works, there has been some focus on leveraging the advancement in Deep Learning by using architectures such as CNN, LSTM, GRU etc. for learning the complexities in the EEG signals.

In this work, we aim to explore the role of signals from different lobes of the brain (frontal, occipital, parietal and temporal) along with different EEG frequency bands ($\delta,\theta,\alpha,\beta,\gamma$) for imagined speech recognition. We use parallel bandpass filters to filter the EEG signal into corresponding frequency bands and feed it to a 1DCNN-LSTM based deep learning architecture for imagined speech and object recognition. The 1DCNN block extracts spatial features from the raw EEG signals and the LSTM block models the temporal information over the extracted features. We observe that the frequency bands $\theta$ and $\alpha$ do not significantly contribute towards imagined speech recognition and therefore, we suggest bandreject filter for preprocessing the EEG signals before using them for the classification of imagined speech. Such an approach has been found to boost the classification performance on comparison with feeding directly the raw EEG signals to the 1DCNN-LSTM architecture. Furthermore, we also observe that the frontal lobe is the most significant contributor compared to other lobes of the brain. Both the observations are well in line with the literature on the functions of the frequency bands and the lobes. We perform experiments on a publicly available imagined speech dataset and achieve classification accuracies of $85.93\%$, $87.27\%$ and $87.51\%$ for the task of classifying digits, characters and images respectively.