Universal Fourier Attack for Time Series

We primarily focus on attacking classifier models. For each of our datasets, we finetune a ResNet18 [8] that has been pretrained on ImageNet [4]. The spectrogram obtained as described above is the input to the classifier. During training, we add Gaussian noise to the signal in the time domain before it is converted to a spectrogram, and then apply random time and frequency masking to the spectrogram.

We use the adversarial success rate (ASR) as our primary evaluation metric. The ASR of an attack is defined as the percentage of originally correct model predictions that the attack successfully changes the prediction of. Unlike the error rate, the ASR only counts inputs where the attack changes the model prediction and does not give the attack credit for inputs the model was originally wrong on. An ASR close to one indicates a highly effective attack. More formally, for a model $f$, attack $v$, and dataset $\mathcal{D}=\{(x_{i},y_{i})\}_{i=0}^{n}$ with inputs $x_{i}$ and labels $y_{i}$:

We find the ASR as a function of the signal-to-noise ratio (SNR). As in [18], the SNR is $10\text{log}_{10}\frac{P_{x}}{P_{v}}$ where $P_{x}$ is the power of the unperturbed input, $\frac{1}{T}\sum_{i}^{T}x_{i}^{2}$, and $P_{v}$ is the power of the attack, $\frac{1}{T}\sum_{i}^{T}v_{i}^{2}$. The SNR is large when the attack is small and presumably less perceptible.

Additionally, we compare our models against two simple, baseline adversarial attacks: Fast Gradient Sign Method (FGSM) [6] and Universal Adversarial Perturbation (UAP) [12]. We emphasize that unlike our attack and the UAP attack, the FGSM attack is not a universal attack, and rather a separate attack is generated for each input to the model.

In Fig. 4, we test the time invariance of the attack with the SNR fixed at 10 dB. We repeatedly play the attack on a cycle and shift the start time of the speech recording or URE data as depicted in Fig. 1. The plot also includes an ablation study, where we removed the time invariance transformation from the attack training by fixing the random training time shift to $t=0$ in line 4 of Algorithm 1.

Both our white box (FFT-WB) and black box (FFT-BB) attack have a relatively constant ASR on the Speech Commands dataset, ranging between about 30-37% and 15-20%, respectively. In contrast, the white box ablation attack has a high ASR of 35% with a time shift of zero at evaluation time, but for all evaluation time shifts greater than zero the ASR drops below 15%. The black box ablation attack is also sensitive to the evaluation time shift, dropping from 13% to below 8% for nonzero evaluation time shifts. The results with the Corona Duff dataset are similar as both the white box and black box versions of our attack have an ASR between 70% and 73% at all evaluation time shifts, and the ablation attack exhibits a decreased ASR when the evaluation time shift is nonzero.

To evaluate the robustness of the attack to filtering, we propose a set-up where a defender receives an input, which could be benign or adversarial. The defender filters the received input and then evaluates it on a model that has been trained on filtered benign data. Since our datasets are composed of mostly low frequency information, we use low-pass filtering. If the received input is adversarial and the attack is composed of high frequencies, the filtering should remove most of the attack before the model makes its prediction, thus reducing the effectiveness of the attack. If the received input is benign, the model prediction should be accurate because the model has been trained on filtered benign data.

More formally, let $f_{k}$ denote a classifier which has been trained on data low-pass filtered with cutoff frequency $k$. Then, $f_{k}(\textit{lowpass}_{k}(x))$ is the classifier prediction on benign input $x$ and $f_{k}(\textit{lowpass}_{k}(x+\alpha v))$ is the classifier prediction on adversarial input $x+\alpha v$. We evaluate the ASR of our attack and our baseline attacks on $f_{k}$ for a range of frequencies. Note that the attacks are the same attacks as in the previous sections. These were learned on an unfiltered model with unfiltered data and were not trained on the $f_{k}$ they are evaluated on.

Fig. 5 depicts the results. For reference, we also include a version of our attack trained without either $\mathcal{L}_{\text{spectrum}}$ loss term so that the frequency spectrum of this ablation attack is not constrained to match the frequency spectrum of the benign data. We adjust the SNR of each attack so that the ASR of each attack is comparable when the cutoff frequency is half the sampling rate.

On the Speech Commands dataset with low pass filtering the ASR of our attack is greater than or equal to 32% for all frequencies tested. In contrast, for all other attacks, as lower cutoff frequencies are used, the effectiveness of the attack is reduced from an ASR of 40% at a cutoff frequency of 8 kHz to below 20% for cutoff frequencies of 500 Hz and below. As depicted in Fig. 2, the UAP and FGSM attacks have high frequency components which low-pass filtering removes.

Similarly, on the Corona Duff dataset our attack has an approximately constant ASR, averaging 82% across all cutoff frequencies tested. In contrast, the other attacks demonstrate a significant reduction in ASR with filtering from a 70% ASR at 192 kHz to below a 40% ASR for the UAP and FSGM attacks. Thus, by restraining our attack to only use the low frequencies present in the original data, we have designed an attack that is much more robust to this filtering test than any of the baseline attacks tested.

Besides filtering, to defend speech recognition systems against white box adversarial attacks, several works have proposed a tranform-and-compare pipeline to detect adversarial examples. Given an input, which could be benign or adversarial, this pipeline compares model predictions on the input and a transformed version of the input. Several different transformation functions have been proposed including the addition of random noise [13, 5], audio compression [14, 19], quantization [9], down-up sampling [9], and filtering [10]. If the distance between the model predictions on the transformed and original input is higher than a threshold, the input is flagged as adversarial because adversarial examples are generally less robust against perturbations than benign examples are.

In the speech recognition system defense pipeline, the character error rate is typically used to measure the distance between model predictions. However, because our models are generic classifiers rather than speech-to-text models, we use the $L_{2}$ distance between the model outputs of the original input and transformed input. We calculate this distance for 800 benign inputs and 800 adversarial inputs. From the resulting distributions, a threshold can then be determined to use for flagging adversarial examples.

Using MP3 compression as the transformation function on Speech Commands data, we plot the distribution of distances between the original and transformed benign inputs and the distribution of distances between the original and transformed adversarial inputs for our attack (FFT) and the FGSM and UAP baselines in Fig. 6. For the FGSM and UAP attacks, the distances for adversarial data are generally larger than for benign data, making adversarial examples generated with these attacks more easily identifiable. For our attack, we find much more overlap in the distribution of benign distances and the distribution of adversarial distances, demonstrating that our attack is much more difficult to flag using this defense method.

Table  I reports the area under the curve (AUC) for all transformations tested, averaged over 5 runs of attack training. Full details of each transformation function, as well as the full AUC plots, are in Appendix  A-B. On both datasets, for all transformations tested, our attack has an AUC score close to $0.50$, averaging an AUC of $0.57$ across all transformations on Speech Commands and $0.56$ on Corona Duff. For comparison, the AUC scores of the baseline attacks are much higher with average AUC scores of $0.87$ and $0.81$ for the UAP and FGSM attack on Speech Commands and $0.83$ and $0.72$ for the UAP and FGSM attack on Corona Duff. This indicates that our attack is more difficult to identify using the tranform-and-compare defense pipeline.

The universal attack trained on the Speech Commands classifier also transfers to a speech-to-text model. We test the learned attack on a pretrained, off-the-shelf Deep Speech [7] model. In Table II we report the accuracy (ACC), the Character Error Rate (CER) and ASR of our attack with a fixed SNR of 5 dB. The CER is defined as $(S_{c}+D_{c}+I_{c})/N_{c}$ where $S_{c},D_{c},$ and $I_{c}$ refer to the number of character substitutions, deletions, and insertions, respectively.

While our attack as proposed achieves metrics only slighty better than just adding random Gaussian noise to the benign data, if we only use ${L}_{\text{spectrum}_{1}}$ in training and forego the second training phase with ${L}_{\text{spectrum}_{2}}$, the attack is much more effective and transfers very well.