Defense against Adversarial Attacks on Hybrid Speech Recognition using
Joint Adversarial Fine-tuning with Denoiser

In today’s world, voice-based smart assistants are ubiquitous–be it using phones, dedicated home assistants like Alexa, Google Home, Apple Pod, or as call center agents [1]. A core technology behind these assistants is automatic speech recognition (ASR) whose goal is to transcribe speech to text. Recent work [2, 3, 4] has shown that ASR systems are vulnerable to adversarial inputs, which contain specially crafted mostly human inaudible noise. Considering the threat of these adversarial attacks, it becomes of foremost importance to propose defensive countermeasures to protect ASRs. These countermeasures broadly fall into three categories–pre-processing, stochastic, and adversarial training. Pre-processing defenses intend to remove the adversarial noise from the signal before before passing it into the machine learning system [5, 6, 7]. Stochastic defenses introduce randomness into the model. Thus the model used to craft the adversarial sample differs slightly (but stochastically) from the model used to evaluate the sample, reducing attack effectiveness. Randomized smoothing is the most common stochastic defense [8]. Finally, adversarial training tries to make the model inherently robust by training on dynamically generated adversarial examples [9, 10]. The major contributions of this work are highlighted below:

• •

  We evaluate the robustness of a strong baseline K2 Conformer hybrid ASR model against white-box attacks, i.e. when the adversary is aware of parameters of the model. To the best of our knowledge, this is the first work that fully utilizes a differentiable hybrid ASR model to study adversarial robustness.

• •

  We propose four defenses–a pre-processing time-domain denoiser, adversarial fine-tuning and two variants of joint adversarial training of a pre-processing denoiser with ASR model.

• •

  We evaluate these defenses against strong adaptive white-box attacks i.e. when the adversary is aware of parameters of defense model along with those of ASR.

The rest of the paper is as follows. In Section 2, we introduce adversarial attacks on ASR. In Section 3 , we describe defenses; followed by experiments and results in Section 4 and 5 respectively.

An ASR system can be considered as a function $\hat{\mathbf{y}}=f(\mathbf{x},\theta)$, which predicts a sequence of words $\hat{\mathbf{y}}$ given a audio waveform $\mathbf{x}$. $f$ is an statistical model described by a set of parameters $\theta$. ASR systems are known to be vulnerable to adversarial attacks [2]. The attacker adds a small perturbation to the benign signal to alter the prediction of the system. Depending on the goals of the attacker, we find different attack modalities. When an adversarial example fools the ASR into predicting a particular target phrase that an adversary desires, it is called a targeted adversarial attack. Untargeted attacks, by comparison, simply induce transcription errors and are not of as much concern in the ASR context [3]. Table 1 explains the concept of targeted attack via an example. Suppose we have a speech signal $\mathbf{x}$. Without any attack, a human will transcribe $\mathbf{x}$ as actual = This is the human ASR output. Now using $\mathbf{x}$ as input to the ASR model, the output transcription by the ASR is denoted as benign = Thus is the real ASR output. Suppose, the adversary wants the ASR model to predict the target phrase denoted by target = Transfer $1000 from my account. To achieve this goal, he/she crafts an adversarial example $\mathbf{x^{\prime}}$ such that when $\mathbf{x^{\prime}}$ is given as input to the ASR, the model produces the output transcription adversarial = Transfer a sand from my account. One can find Word Error Rate (WER) between the pairs of the transcriptions as shown in Table 2. Let WER between reference ($ref$) and hypothesis ($hyp$) be denoted by $WER(ref,hyp)$. $Benign=WER(actual,benign)$ denotes the Benign ground truth WER, which measures the performance of the ASR system in non-attack conditions. A good ASR system, and hence defense, should have Benign WERs as low as possible, indicated by $\downarrow$ in the column Defense success. $GT=WER(actual,adversarial)$ is called the adversarial ground truth WER. GT is a performance indicator of how much attacker succeeded in denial-of-service i.e. introducing untargeted spelling errors. An higher value indicates as successful untargeted attack while a low value is characteristic of a robust ASR. $TGT=WER(target,adversarial)$ denotes the adversarial target WER and is performance indicator whether the adversary was successful in getting the ASR predict the chosen target phrase. While the adversary wants TGT WER to as low as possible (meaning, target phrase to be recognised perfectly), an ideal defense will make it be as high as possible. This is indicated by $\uparrow$ in the column Defense success.
Attack Algorithms: An adversarial example is computed as $\mathbf{x^{\prime}}=\mathbf{x}+\mathbf{\delta}$ where is $\mathbf{x}$ is a benign signal and $\mathbf{\delta}$ is a small adversarial perturbation. Many attack algorithms in the literature [4] propose different ways to compute $\mathbf{\delta}$. In this work, we consider FGSM [11] and Projected Gradient Descept (PGD) attacks [9]. For targeted attacks, PGD optimizes delta by gradient descent iterations that minimizes minimizing the ASR loss $L$ between the target phrase selected by the attacker $\mathbf{y}^{\mathrm{target}}$ and the adversarial transcript predicted by ASR. Thus for iteration $i+1$,

$\displaystyle\mathbf{\delta_{i+1}}=\mathrm{clip}_{\varepsilon}(\mathbf{\delta}_{i}-\alpha\mathrm{sign}(\nabla L(f(\mathbf{x},\theta),\mathbf{y}^{\mathrm{target}}))\;.$

(1)

Throughout this paper, PGD-i indicates the number of iterations used for PGD attack (eg: PGD-7 means 7 iterations). At every iteration, $\mathrm{clip}$ function (projection) ensures that $\lVert\mathbf{\delta}\rVert_{\infty}\leq\varepsilon$, keeping the attack imperceptible. We choose the learning rate $\alpha$ at every iteration is one fifth of the max-norm. FGSM is single iteration version of PGD with step $\alpha=\varepsilon$. While attacking any system (with or without defense), we assume that it fully white-box and adaptive, meaning the adversary knows not only the speech recognition model parameters but also the defense. This is the worst-case scenario to evaluate robustness, a commonly expected norm for evaluation of defenses as it exposes the systems weakest links [12].

Dataset, Baseline, and Adversarial Attacks Our experimental setup was based on LibriSpeech dataset [15]. We used a Hybrid DNN-HMM ASR model implemented on the K2 framework¹¹1https://k2-fsa.github.io/k2/index.html using Pytorch [16]. A Conformer [17] network was used to compute frame-level posteriors, which were used as input to the K2 WFST decoder. The Conformer consisted of 12-layers with dimension=256, heads=4, and feed-forward dimension=2048; and used 80 log-Mel-filterbank features. The whole pipeline is end-to-end differentiable to be able to compute adversarial examples in time domain. This model was trained on the full 960 hours LibriSpeech corpus for 20 epochs, and the parameters from the last 5 epochs were averaged to get the final model. We denote this model as the undefended baseline. To evaluate the robustness of the model, we applied targeted FGSM and PGD attacks with different strengths, i.e., max. $L_{\infty}$ norm levels: {0.0001, 0.001, 0.01, 0.1, 0.2}. For PGD, we evaluated attacks with 7 and 500 iterations (PGD-7 and PGD-500). The target phrases used to craft the attacks were taken from the LibriSpeech training set. For each utterance, we chose a target phrase with length similar to the benign transcript. We evaluated the attacks on the first 100 examples from LibriSpeech test-clean. We chose to work on this reduced set because of three reasons. First, the computational cost for PGD iterations is too high for the full set, so in the literature, it is common to experiment on a limited number of utterances [2]. Second, for the baseline, the performance of the reduced set did not statistically change w.r.t. the full set (see Table 3). Lastly, this setup is the same as in previous work [18] and in DARPA-GARD evaluations. Another important thing to point-out is that we always evaluated using white-box adaptive attacks. That means that the adversary knows the defenses and their parameters and it is able to back-propagate through them to create the adversarial examples. Many works in the adversarial literature do not consider adaptive attacks based their defense in obfuscating the gradients of the system as evidenced in [19].
Attacks Dataset: We create a dataset by generating offline PGD adversarial samples against LibriSpeech train sets using $L_{2}=\{0.2,0.5,1.5,1.9\}$ and $L_{\infty}=\{0.001,0.01,0.1\}$ threat models with number of iterations $\{10,20,50,100,200\}$ sampled with more bias towards high norm and high iteration attacks. Generating offline adversarial examples has the advantage that we can use computationally expensive PGD attacks with large number of iterations without letting a model run for months on online attacks. The generated attacks were used to train the denoiser as described in Section 1. Then, the pre-trained denoiser could be finetuned in tandem with ASR model to increase its robustness.
Denoiser: After experimenting with a few denoiser architectures, we choose TasNet [20], a time-domain model for source separation and speech enhancement. It is an all-convolutional 1-D Convolutional Neural Network (CNN), which consists of encoder, separator, and decoder. The encoder and decoder are single convolutional layers, while the separator stage consists of multiple CNNs called stacks. The stacks output are combined to produce a mask which is applied to the encodings and passed to the decoder stage. We used 128-dim encodings obtained with kernel-size=16 and stride=8. The separator used one stack with 16 layers with kernel dilations increasing with a factor of 2 [20]. We trained the denoiser on our dataset of offline attacks using the corresponding benign example as clean target.
ADV-FINETUNE-ASR: We fine-tuned the baseline ASR model on on-the-fly PGD-7 attacks. The $L_{\infty}$ for this attacks were randomly sampled from a log uniform distribution [0.0001,0.02]. The learning rate was 10x lower than the one used in the training phase.
ADV-FINETUNE-JOINT: Instead of adversarially fine-tuning just the ASR model, we jointly fine-tuned the tandem denoiser+ASR. We evaluated another variant where attacks were generated on the tandem denoiser+ASR but we only updated the denoiser weights while the ASR model is frozen. We call this defense as ADV-FINETUNE-JOINT-ASRfrozen.

We evaluate the robustness of baseline ASR and all proposed defenses against FGSM and PGD-7 attacks (Table  3) and PGD-500 attacks (Table  4). For FGSM and PGD-7, the WER w.r.t. the target phrase was always greater than 90%. In other words, the adversary is not able to make the ASR to recognize the malicious target phrase. Hence we omitted TGT WER in Table 3 and included only WER w.r.t ground truth phrase (GT WER). We analyze the undefended baseline for full and reduced Librispeech test-clean set. We observe that the results for both sets are similar. Therefore, henceforward, all models were evaluated on the reduced set to alleviate the large cost of generating attacks (and other reasons mentioned in Section 4). Next, we evaluated the different defenses. We observe that the denoiser defense trained on offline attacks performed better than randomized smoothing for most $L_{\infty}$ values and on par with the adversarially trained ASR. Both defenses jointly adv. fine-tuning denoiser and ASR performed significantly better than the offline denoise and the adv. fine-tuned ASR. ADV-FINETUNE-JOINT yielded the largest robustness against FGSM attacks with mean absolute decrease of 19.3% GT WER w.r.t. the baseline. ADV-FINETUNE-JOINT-ASRfrozen was the best for PGD-7 with a mean absolute decrease in GT WER of 22.3% w.r.t. the baseline. We can observe that for both attacks, the best defense kept the system robust up to $L_{\infty}\leq 0.01$.

When increasing the number of PGD iterations to 500 (Table 4), the WER w.r.t. the target phrase (TGT) decreases, meaning that the attacker starts being sucessful in making the system to recognitize a particular malicious phrase. However, the obtain an usable TGT WER of 16%, it needs to increase $L_{\infty}$ to 0.1, which is a very perceptible attack. Here, the goal of the defense is increasing TGT WER while reducing GT WER. Again, the best system by far was ADV-FINETUNE-JOINT-ASRfrozen. The mean absolute decrease GT WER was 45.08% GT WER and increase in TGT WER was 68.05%. Although there was a slight increase (1.17%) in the benign WER with reference to the baseline model, the gain adversarial robustness overshadowed it. The proposed method significantly outperformed the baseline defenses in the literature, i.e., randomized smothing and ASR adversarial training. Unfortunately, for large $L_{\infty}=0.1$, the defenses could not reduce GT WER much. However, note that these are very perceptible attacks and even using Gaussian noise (non adversarial) of that level would significantly damage the ASR system.

We evaluated the robustness of K2 Hybrid ASR model alongwith four defenses–pre-processing time-domain denoiser defense, adversarial fine-tuning of ASR model and two variants of joint adversarial training of pre-processing denoiser and ASR model. We evaluated these defenses against strong adaptive white-box attacks i.e. when the adversary is aware of parameters of defense model along with those of ASR. To understand the big picture of the defenses, we convert the Table  3 to boxplot shown in Figure 1. Please note that we exclude FGSM-$L_{\infty}=0.2$ and PGD-$L_{\infty}=0.2$) as they are perceptible. The dotted blue line indicates the clean/benign performance of the individual systems. The best defense should have WER distribution concentrated around the blue line. The results show that ADV-FINETUNE-JOINT is the best defense, followed closely by ADV-FINETUNE-JOINT-ASRfrozen. On the other hand, for PGD-500 attacks, ADV-FINETUNE-JOINT-ASRfrozen performs the best, consistently yielding low GT WER and high TGT WER. This is at the cost of slight degradation in benign WER ($<$1.2%), however this is expected for adversarial defenses. In the future, we would like to bridge this gap furthur.