Light Convolutional Neural Network with Feature Genuinization
for Detection of Synthetic Speech Attacks

In the recent years, automatic speaker verification (ASV) systems are deployed in different real-world applications [1, 2, 3]. These systems are exposed to spoofing attacks for unauthorized access, hence detection of such attacks attracts much attention [4, 5]. Various spoofing attacks are broadly classified into replay, impersonation, voice conversion (VC) and text-to-speech synthesis (TTS) attacks [6]. The latest progress in VC and TTS systems can produce perceptually natural sounding speech, which poises a threat to fool the ASV systems [7, 8, 9].

The research on spoofing countermeasures grew in the last decade since the inception of ASVspoof¹¹1http://www.asvspoof.org/ challenge series. The challenge provided a platform to the researchers across different domains to explore fake speech detection using a common benchmarked corpus [10, 11]. Its recent edition ASVspoof 2019 is devoted to detection of both synthetic and replay speech with two subtasks [12]. The logical access track focuses on detection of synthetic speech created using state-of-the-art VC and TTS systems, which is the focus of this paper.

The explorations on spoofing attack detection cover two directions from the perspective of a detection task. The spoofing countermeasures either focus on novel front-end features or effective classifiers. Some of the former studies focused on the importance of robust features such as cochlear filter cepstral coefficient and instantaneous frequency (CFCCIF) [13], linear frequency cepstral coefficients (LFCC), subband spectral flux coefficients and spectral centroid frequency coefficients [14]. Later, the long-term constant-Q transform (CQT) based constant-Q cepstral coefficients (CQCC) proved to be one of the strong front-ends for synthetic speech detection [15]. The recent explorations with features derived from CQT are also found to effective for spoof detection [16, 17, 18].

With the advent of deep learning methods, robust classifiers are investigated for detection of spoofing attacks. Some of these include end-to-end systems with light convolutional neural networks (LCNN) [19, 20], squeeze excitation and residual networks [21, 22]. The end-to-end systems have much difference with the works that focus on novel features. The former are data driven deep learning methods, while the latter emphasize on hand-crafted feature, which require prior knowledge. Further, we note that the same neural network based system can perform differently for a range of features [19]. Therefore, a robust spoofing countermeasure is required to have a strong feature extractor that captures the discriminative artifacts along with an effective classifier.

The synthetic speech attacks can be created with a wide range of TTS and VC algorithms [6]. In general, spoofing countermeasures do not handle synthetic speech from unseen sources because of lack of generalization ability [23]. We note that genuine examples have a comparatively lower variance than synthetic speech. We believe that the consistent characteristics of genuine speech set genuine speech apart from a variety of different synthetic speech. A recent study using temporal domain information shows that spoofing detection can be improved by modifying the probability mass function of spoofed speech close to that of the genuine speech [24]. This process is termed as genuinization, which is found to be effective when applied to both train and test examples for synthetic speech detection.

In a similar direction, we hypothesize that, if we are able to derive a model that fits well the distribution of the genuine speech, such a model will take genuine speech as the input and generate genuine speech as the output following the same distribution of the genuine speech. However, when the model takes spoof speech as input, it will generate very different output, that amplifies the difference to genuine speech. With this hypothesis, we propose to derive a model using the genuine speech features with convolutional neural network (CNN) that is referred to as genuinization transformer. Further, the process is referred to as feature genuinization as a given feature representation is projected on a domain learned using the genuine features. The genuinization transformer is then used together with an LCNN system for detection of synthetic speech attacks.

The rest of the paper is organized as follows. Section 2 introduces the details of proposed feature genuinization. Section 3 describes the feature genuinization based LCNN system for detection of spoofing attacks. The experiments and their results with discussion are reported in Section 4 and Section 5, respectively. Finally, the paper is concluded in Section 6.

We aim to learn a transformer that does not change the characteristics of genuine speech features, whereas it projects spoof speech to a different output, maximizing the difference between genuine and spoof speech. Figure 1 shows the block diagram of the proposed feature genuinization process. It can be observed that there are two stages of the process. The first stage basically focuses on training a feature genuinization transformer using the characteristics features derived from only genuine speech. During the second stage, this trained feature genuinization transformer is used to convert any given features that enhances the discrimination of genuine and spoof speech.

The CNN based architectures have shown their effectiveness in the field of anti-spoofing research [25]. In this regard, we use CNN for training the genuinization transformer as shown in Figure 1. The detailed architecture of the CNN used in this framework can be seen from Figure 2. It can be observed that the functionality of the proposed genuinization transformer is similar to that of an autoencoder. However, the output of genuinization transformer is considered as the final transformer result. In addition, we apply a full convolutional layer and therefore, there is no fully connected layers in the transformer. This can thereby force the network to focus on the temporal correlation between the input signal and the whole stratification process. Further, it reduces the number of training parameters, which significantly results in less training period.

A study in [26] shows that it is a good practice to use strided convolution rather than pooling to downsample as it allows the network to learn its own pooling function. Therefore, we use this method during the training of genuinization transformer. In addition, batchNorm2d and leaky rectified linear unit (ReLU) activation function are used in the training because they can promote healthy gradient flow, which is critical for the learning process.

The architecture of proposed genuinization transformer shown in Figure 2 consists of two functionalities: encoding and decoding. During the encoding phase, the input signal is compressed through a number of strided convolutional layers, and then the convolution result is obtained by leaky ReLU. In the decoding phase, the encoding process is reversed by deconvolution, and then by ReLU. In this way, the transformer works as an autoencoder that learns the characteristics of genuine speech [27]. As a result of this, it amplifies the discrimination of genuine and spoof speech in the transformed domain.

Once the genuinization transformer is trained, it can be used to transform any given genuine/spoofed features to a transformed domain that is learned using the only genuine feature characteristics. This novel way of transforming the feature is referred as feature genuinization as mentioned earlier. Next, we discuss about the LCNN system using the feature generalization for detection of spoofing attacks.

Various deep learning systems have shown their effectiveness for spoofing attack detection [19, 21, 28, 29, 22]. Therefore, we plan to use the proposed feature genuinization with a deep learning system. The LCNN is one of the strongest systems that has proven to be useful for its compactness and efficacy for anti-spoofing [30, 19]. In this work, we use LCNN based system with the transformed features obtained using genuinization transformer.

Figure 3 shows the block diagram of the proposed feature genuinization based LCNN system. We consider log power spectrum (LPS) of a given speech as the input feature to the genuinization transformer. It transforms the given input LPS to a genuinized feature, which is an input to the LCNN. During training, the training data and their corresponding label information is fed to the LCNN system. Once the training is completed, the detection result for a given input to the system can be obtained to identify the spoofing attacks.

We used Max-Feature-Map (MFM) activation function instead of commonly used ReLU function for the LCNN system similar to that in [19]. The main advantage of MFM is that it can learn compact features instead of sparse high-dimensional ones like ReLU. Further, MFM resorts to max function to suppress the activations of a small number of neurons so that MFM based CNN models are light and robust. Therefore, these are applied to reduce the dimensionality of the output and obtain more discriminative feature maps.

In this section, we discuss the database and experimental setup for the studies.

The proposed system is a pipeline with a feature genuinization followed by LCNN. We compare the proposed system with LCNN baseline without feature genuinization. Further, we also consider the two baseline spoofing countermeasures of ASVspoof 2019 challenge. They are based on CQCC and LFCC features with Gaussian mixture model (GMM) classifier [31, 12].

Table 2 shows the results of proposed feature genuinization based LCNN system, that we refer as FG-LCNN, on ASVspoof 2019 logical access corpus and its comparison to the baseline systems. We observe that introducing feature genuinization module in the baseline LCNN system improves the detection of spoofing attacks. While the results on the development set are close, the improvement from the proposed system is evident from the results on the evaluation set, which contains more challenging spoofing attacks of unseen nature. This confirms our hypothesis to use a feature genuinization model exploiting the characteristics of genuine speech. Further, we find that the performance of the proposed system is much better than the two ASVspoof 2019 challenge baselines.

We further perform a justification experiment for validation of our proposed method. The idea behind feature genuinization process is based on the assumption that the genuine speech examples are considered to be less varied than the synthetic speech attacks created using a wide range of methods. We perform a contrast experiment, where we learn a transformation model using CNN by only considering the spoofed speech features. We refer this process as feature spoofing and the model as spoofing transformer, similar to the case of our proposed method. This spoofing transformer is then used to transform any given feature of genuine or spoofed speech to another domain, which is then used in the LCNN system pipeline, that we call FS-LCNN. The rest of the experimental setup remains the same to that our proposed method.

Table 3 shows the performance comparison of the FS-LCNN contrast system with our proposed FG-LCNN and the baseline LCNN system. We consider the results of evaluation set for the comparison as the results of development set can show very accurate detection of synthetic speech attacks. We find that the FS-LCNN contrast system does not perform better than our proposed FG-LCNN system, but rather degrades from the baseline LCNN system. This further strengthens our proposed idea of using feature genuinization process with LCNN system for detection of spoofing attacks.

We are now interested in comparing the proposed system to various single system based results available of ASVspoof 2019 logical access corpus. In this regard, we consider some of the well performing front-end as well as back-ends that have shown their effectiveness for spoofing attack detection in ASVspoof 2019 challenge. Some of the those front-ends are zero time windowing cepstral coefficients (ZTWCC), single frequency cepstral coefficients (SFCC) and instantaneous frequency cepstral coefficients (IFCC) that are implemented with GMM based classifier [34]. Further, deep learning based classifiers such as deep neural network (DNN), ResNet and LCNN are used for detection of spoofing attacks using front-ends like mel frequency cepstral coefficient (MFCC), constant-Q statistics-plus-principal information coefficients (CQSPIC), CQCC, LFCC, LPS of discrete Fourier transform (DFT) and fast Fourier transform (FFT) in ASVspoof 2019 challenge [35, 36, 19, 37]. We report the respective system results from their published works for the comparison on the evaluation set of ASVspoof 2019 logical access corpus.

Table 4 reports the performance comparison of the proposed FG-LCNN system to some of the single systems reported in ASVspoof 2019 challenge discussed above. It is observed that the LCNN based systems represent the best performing single system, that justifies its use as the baseline LCNN in this work. Further, the effectiveness of proposed feature genuinization is evident on using it with the LCNN system, which outperforms other reported single systems in terms of EER on ASVspoof 2019 logical access corpus.

This work proposes a novel feature genuinization based LCNN system for detection of synthetic speech attacks. The characteristics of genuine speech are exploited to learn a model using CNN. It transforms a genuine feature distribution more close to that of the genuine speech, whereas leads to a different output for features of spoof speech, thereby maximizing their difference. The transformed features are then used with an LCNN system. The studies conducted on ASVspoof 2019 logical access corpus show the effectiveness of the feature genuinization based LCNN system for detecting synthetic speech attacks. The comparison of the proposed system to various state-of-the-art spoofing countermeasures showcases it as one of the strong single anti-spoofing system. The future work will focus on extending the studies to replay attack detection.

This research work is partially supported by Programmatic Grant No. A1687b0033 from the Singapore Government’s Research, Innovation and Enterprise 2020 plan (Advanced Manufacturing and Engineering domain), Human-Robot Interaction Phase 1 (Grant No. 192 25 00054) by the National Research Foundation, Prime Minister’s Office, Singapore under the National Robotics Programme. This work is also part of a collaboration with Kriston AI Lab, China in 2019.