Adversarial example devastation and detection on speech recognition system by adding random noise

Given a speech example $x$, a perturbation $\delta$ can be constructed that is almost imperceptible to human hearing, but $x+\delta$ can be recognized as any desired text. This is an end-to-end white-box attack, assuming the attacker can obtain the structure and parameters of the identification system. The attack mode is to send the speech directly to the ASR system, and it cannot attack in the air.

Given an original example $x$ and target text $t$, the optimization object is

where $c$ weighs whether to make the adversarial example closer to the original example or to make it easier to attack successfully, $l(\cdot,\cdot)$ is the loss function, and $dB_{x}(\delta)\leq\tau$ ensures that the perturbation is not too large. To calculate the loss function requires a definite alignment $\pi$. The attack algorithm has two steps. An initial adversarial example is generated by CTC loss, which defines the current $\pi$. Fixing the current $\pi$, an adversarial example with less perturbation is generated.

DeepSpeech is an end-to-end text-dependent ASR system based on a DNN [29] that has higher recognition performance than traditional methods, with an excellent effect in noisy environments. Experimental results [14] have shown that adversarial examples generated by the OPT method can enable DeepSpeech to output a specified text content with a 100% attack success rate. The average disturbance size of the generated adversarial example is $-31dB$. The longer the length of specified text the more difficult it is to generate. The perturbation of the generation will also increase; on average, each extra character will increase the perturbation by $0.1dB$. If the text of the original example is longer, the adversarial example will be less difficult to generate.

This method uses a genetic algorithm based on a free gradient to generate adversarial examples on command-dependent ASR systems. Shown as Algorithm 1, it uses normal speech and a target command as input, and creates a group of candidate adversarial examples by adding random noise to the subset examples in a given speech segment. To minimize the impact of noise on human hearing perception, it is only added to the least significant bit (LSB) of the speech. The deterministic score of each population member is calculated according to the predicted score of the target label. Through the application of selection, crossover, and mutation, next-generation adversarial examples are generated from the current generation. Members of the population with higher scores are more likely to be part of the next generation. Crossover is to mix pairs of population members to generate a new example and add it to the new population. Mutation adds random noise to the offspring with minimal probability before passing it on to the next generation. The process is repeated until a preset value is reached or the attack is successful.

The command-dependent ASR system SpeechCommand [32] recognizes a command label from speech, and is essentially a multi-classification network. Experimental results have shown that its success rate of attack can reach 87%.

Given the speech signal $x$, attackers use the adversarial example generation algorithm to generate a local gradient in the input layer of the network structure, which is consistent with the size of the input signal $x$. We add perturbation $\delta^{*}$ to $x$ to generate adversarial example

Random noise $\hat{\delta}$ is added to the adversarial example, whose size is consistent with Gaussian noise, to generate adversarial example $\hat{x}^{*}$. Mixed noise $\hat{\delta}^{*}$ is the sum of the addition of disturbance $\delta^{*}$ and Gaussian noise $\hat{\delta}$,

where the average $\mu$ and standard deviation $\sigma$ can represent the intensity of noise added to the speech signal. Finally, the modified adversarial example can be equivalent to the addition of mixed noise $\hat{\delta}^{*}$ to the original speech,

Because there is a certain direction when adding adversarial example perturbation, which is equivalent to the addition of purposeful disturbance to make the example close to the target class, these small perturbations play a great role in the discrimination of the model. Even if the input data change slightly, the final calculated value is closer to the distribution of the target class after a series of calculations. When the intensity of of $\delta^{*}$ is similar to that of $\hat{\delta}$, or the strength is greater, the superimposed noise $\hat{\delta}^{*}$ will lose the particularity of $\delta^{*}$ and become ordinary noise, which will affect the purpose of adversarial perturbation, i.e., the adversarial example $\hat{x}^{*}$ will not be adversarial, and the devastation strategy will work.

After adding random noise $\hat{\delta}$ to the normal speech signal $x$, we can obtain the modified normal speech, i.e., some useless values are added to the normal speech signal. If the intensity of noise is slight, the speech sounds like the original. Even if the modified normal speech is put into the classifier, the result will not be greatly changed. When the intensity of $\hat{\delta}$ is small, the result is similar to normal. In other words, the addition of small random noise has little effect on the recognition of normal examples, and the noise lacks a direction. As shown in Fig. 1, adding random noise to the normal example, the result will not be changed, but the adversarial example will lose the effect of the attack. Because the adversarial perturbation is devastated by the noise, the model discriminating the speech is affected by the input signal. Adversarial examples that carry a purposeful value can make a model misclassify the signal.

Using a normal example of TIMIT, and generating an adversarial example by OPT algorithm, random and Gaussian noise are added to the normal and adversarial examples. Fig. 2 shows the spectrogram of the example. Table 1 shows the recognition results of examples in the DeepSpeech system. It can be concluded that the addition of noise to the normal examples does not change the recognition results. However, the recognition result of the adversarial example changes greatly, and is close to that of the normal example. This shows that the slight random noise will not affect the recognition result of the normal example, but it can invalidate the adversarial example and become closer to the normal example. Therefore, we can use random noise to devastate the adversarial example without seriously affecting the normal example. According to these experimental results, we can further propose the detection method of adversarial examples.

The detection algorithm is motivated by the devastation algorithm. Given a speech sample $x$ (normal or adversarial), we only need to add noise before inputting it to the ASR system. For a normal example, because the low-intensity random noise will not affect the content of the speech, the recognition result of the ASR system will not change much. For an adversarial example, because the perturbation is added to the current speech, the added random noise will devastate the particularity of the perturbation, and the recognition result of the ASR system will be totally different from the original. Therefore, according to the recognition results before and after adding noise, we can determine whether an example is adversarial.

The detection strategy determines whether unknown speech is an adversarial example. Because the recognition result of such an example is more likely to be affected by random noise, we can use the change rate (CR) of the recognition result after adding random noise to detect an unknown example. According to the recognition results in Table 1, the variation of the adversarial example is very large relative to a normal sample. As shown in Fig. 3, we can detect the adversarial example following the process shown as Algorithm 2.

According to Algorithm 2, whether an unknown example is adversarial can be detected on the basis of not seriously devastating the example. In terms of the previous change in the recognition rate of the example after adding noise, the change beyond a certain threshold can indicate an adversarial example. From our experiments, it can be concluded that random noise will not have much impact on a normal example.

Our experimental data included text- and command-dependent speech databases. The text-based speech database was made up of TIMIT and LibriSpeech, both with a sampling rate of 16 kHz and a bit depth of 16 bits. TIMIT is an acoustics-phoneme continuous speech corpus built by Texas Instruments, Massachusetts Institute of Technology, and SRI International. The database includes 630 speakers from different parts of the United States, 70% male, and mostly adult and white. Each participant spoke 10 sentences, and a total of 6,300 examples were obtained, all manually tagged at the phoneme level. LibriSpeech [30] is a corpus of about 1000 hours of English pronunciation from audiobooks from the LibriVox project. In our experiments, we downloaded a test-clean dataset and used the first 100 examples.

The SpeechCommand Dataset (SpeechCommands) from Google contains 105829 speech files, each consisting of 35 words. The sampling frequency is 16 kHz, the bit depth is 16 bits, the duration is near 1 second, and the format is WAV. The dataset includes 2618 participants who were asked to say 35 words, each participating only once. They had 1.5 seconds to read out each word, at one-second intervals. Examples with no sound content or whose speech content was different from the words were deleted. Each segment was checked manually to delete speech content that was different from the given words. Recordings are in OGG format, and files smaller than 5 KB were deleted. Speech signals were converted to WAV format normalized to [-1.0, 1.0], and files whose average values of the normalized speech signal were less than 0.004 were deleted.

We used the GA and OPT attack methods (section 3.2) to generate adversarial examples. For the text-dependent ASR system, we chose classical DeepSpeech[29] as our target system, and OPT to generate adversarial examples to attack the system. TIMIT, LibriSpeech, and CommandVoice were the speech databases. For the command-dependent ASR system, SpeechCommand was the target, and the GA-based method was chosen to generate adversarial examples. For added noise, $\mu$ had the range (10, 30, 50, 70, 100, 200, 500), and $\sigma$ was in the range (10, 30, 50, 70, 100, 200, 500).

Defense methods on adversarial examples were evaluated differently for text- and command-dependent ASR systems. The defense method of text-dependent adversarial examples is illustrated from two aspects: (1) the effect on adversarial examples was to be as large as possible, and we used the similarity of recognition results after adding noise $SR_{adv}$; and (2) the impact on normal examples was to be as small as possible, and we again measured it by the similarity of recognition results of normal examples after adding noise $SR_{benign}$. Similarity is the matching ratio between the initial recognition result of a sample after adding noise. We calculated

where $x_{benign}$ is a normal example, $x_{adv}$ is an adversarial example, $y$ is the real text, $D(\cdot,\cdot)$ is the distance function, and the editing distance proposed by Levenshtein et al. [31] represented the input transformation function (e.g., downsampling, quantization, local smoothing, compression).

The defense effect of command-dependent ASR system adversarial examples is illustrated from two aspects: (1) the impact on the adversarial example should be large, as measured by the change of the average attack success rate $ASV_{avg}$ after adding noise; and (2) the impact on a normal example should be small, as measured by the change of the recognition accuracy $ACC$ after adding noise.

In the experiment, we calculated

where $X^{*}$ is the adversarial example set generated from original example set $X$, $n^{*}$ is the number of examples in $x^{*}$, $x^{*}$ is a single example in $X^{*}$, and $y^{*}$ is a label for attacking $x^{*}$. A lower $ASR_{avg}$ on adversarial examples indicates better performance of the method.

The recognition accuracy is the ratio of the number of correctly recognized speech examples to the total,

where dataset $X$ has $n$ examples, and $x$ is an example whose real label is $y_{0}$. The closer ACC is before and after adding noise, the smaller the influence of the defense method, and the better the defense performance.

We show the effect of devastation on text-dependent adversarial examples, and detection results on command-dependent adversarial examples.