Adversarial Attacks and Robust Defenses in Speaker Embedding based Zero-Shot Text-to-Speech System
^††thanks: * Corresponding Author: Ming Li

Adversarial examples refer to instances with imperceptible perturbations that are deliberately introduced. These perturbations are obtained by solving an optimization problem and lead a well-trained model to make incorrect predictions. This study utilizes PGD [17] and Adam [25] optimizer-based methods to generate adversarial examples. Both methods are gradient-based white-box attacks. We aim to attack the speaker encoder of a speaker embedding based Zero-Shot TTS system by adding small perturbations to the input speech, causing the output speaker embeddings to change, which leads the TTS system to generate speech that mimics the target speaker.

The core idea of the PGD method is to iteratively update the input samples based on the gradient sign of the loss function while constraining the perturbation magnitude. Specifically, given input speeches $x$, target labels $y^{\prime}$ and a well-trained speaker encoder model $f(\cdot)$. The predicted labels $\hat{y}$ can be obtained by computing the index of the maximum cosine similarity between the speaker embeddings of the source speeches and the spoofed ones with adversarial examples, which are extracted by the speaker encoder. The adversarial perturbations $\delta$ can be generated by:

where $sign(\cdot)$ represents the sign of the gradient, $\alpha$ and $\epsilon$ control the magnitude of each update and the maximum allowable perturbation, respectively.

Compared to PGD, Adam is an adaptive learning rate optimization algorithm. In each iteration, Adam not only utilizes the current gradient information but also incorporates momentum from previous iterations to update the perturbation:

where $m_{t}$ and $v_{t}$ are the second-moment and second-moment estimates of the gradient, respectively. $lr$ is the learning rate, $\xi$ is the numerical stability constant, $\beta_{1}$ and $\beta_{2}$ are the decay rates for the first and second-moment estimates, respectively.

Adversarial training is one of the most widely used and effective methods for defending against adversarial attacks, as it enhances the model’s robustness by incorporating adversarial examples into the training process. In this work, we also employ adversarial training. For each speech sample within a batch $\{(x_{i},y_{i})\}_{i=1}^{b}$, we randomly assign a target speaker label $y^{\prime}$ that differs from the source speaker and then apply adversarial attack methods to generate adversarial examples $\{(\hat{x}_{i},y^{\prime}_{i})\}_{i=1}^{b}$. Subsequently, these adversarial examples are labeled with the source speaker’s label and are used alongside the source speech $\{(\hat{x}_{i},y_{i})\cup(x_{i},y_{i})\}_{i=1}^{b}$ to fine-tune the well-trained speaker encoder model. Finally, the fine-tuned speaker encoder model is used to retrain the zero-shot TTS system.

Diffusion-based adversarial purification is an emerging defense technique against adversarial attacks, which utilizes diffusion models to remove adversarial perturbations from input data, thereby restoring clean speech for effective defense. As a plug-and-play module, diffusion models effectively circumvent the issues of domain shifts and secondary training associated with adversarial training. Furthermore, they do not require training on predefined adversarial examples, which endows them with solid generalization capabilities and allows them to address a wide range of attack methods.

A diffusion model normally consists of a forward diffusion process and a reverse sampling process. The forward diffusion process gradually adds Gaussian noise to the input speech until the distribution of the noisy speech converges to a standard Gaussian distribution:

where $x_{0}$ is the clean speech, $x_{t}$ represents the noisy speech at time step $t$, hyperparameter $\bar{\alpha}_{t}$ controls the noise level.

The reverse sampling process takes the standard Gaussian noise as input and gradually denoises the noisy speech to recover clean speech. The reverse process is approximated by learning a model $p_{\theta}(x_{t-1}\mid x_{t})$:

where $\mu_{\theta}(x_{t},t)$ and $\Sigma_{\theta}(x_{t},t)$ are represent the predicted mean and covariance for time step $t$, respectively.

The optimization objective is to minimize the speech reconstruction error, which is achieved using a Mean Squared Error (MSE) loss function:

where $\hat{x_{0}}$ represents the speech obtained by denoising the noisy speech $x_{t}$, $N$ is the number of samples in speech $x_{0}$.

Additionally, considering that adversarial purification might affect clean audio, we introduce a binary classifier before the diffusion module to distinguish between audio samples with adversarial perturbations and those that are clean.

We utilize the ResNet34 [26] architecture as the speaker encoder model and the VITS [3] structure as the zero-shot TTS component. For ResNet34, the residual block channels are set to {64,128,256,512}, and the output feature maps are aggregated with a global statistics pooling layer that calculates each feature map’s means and standard deviations. The acoustic features are 80-dimensional log Mel-filterbank energies with a frame length of 25ms and a hop size of 10ms.

The speaker encoder model is pretrained on the VoxCeleb2 [27] development set and tested on the VoxCeleb1-O [28] test set. We adopt the on-the-fly data augmentation [29] to add additive background noise or convolutional reverberation noise for the time-domain waveform. The MUSAN [30] and RIR Noise [31] datasets are used as noise sources and room impulse response functions, respectively. The input utterances are truncated to 2 seconds. We employ the ArcFace [32] classifier, with the margin and scale parameters set as 0.2 and 32, respectively. Network parameters are updated using an SGD optimizer with an initial learning rate 0.1. The learning rate is decayed by a factor of 0.1 every ten epochs until 1e-5.

We use the clean subsets of the train and development sets from LibriTTS [33] to train the zero-shot TTS component. The speaker embeddings for the utterances are obtained from the well-trained speaker encoder. The TTS network parameters are updated using the AdamW [34] optimizer with a learning rate of 2e-4. The batch size is 32, and the total epoch is 40.

The adversarial attack methods are described in II-A. For the PGD method, the perturbation limit $\epsilon$ is set to the 5% of the maximum amplitude of each audio sample, with 20 interations and the step size $\alpha$ is decreases from 4e-3 to 4e-4 with cosine delay. For the Adam optimizer-based method, $\epsilon$ is set to the 5% of the maximum amplitude of each audio sample, with 50 iterations and the learning rate $lr$ that decays from 1e-3 to 1e-5 with cosine delay.

In adversarial training, the batch-wise random targeted attack strategy is employed, where a batch of speech samples is selected, and each sample within the batch is randomly assigned a target sample with different speaker identity. Adversarial examples are then generated using the adversarial attack method and combined with the original speeches to fine-tune the speaker encoder model. No data augmentation is applied. The model is optimized using an Adam optimizer with a cosine decay learning rate schedule, starting at 1e-3 and decaying to 1e-5. The batch size is 256, and the number of epochs is 3.

DiffWave [35], a representative diffusion model in the waveform domain, is used as our defensive purification model. We use the same settings as those in [24] for diffusion parameters. The VoxCeleb2 development set is employed for model training, with input utterances truncated to 2 seconds. The learning rate is set to 2e-4, and the batch size is 16.

We introduce a ResNet18-based binary classifier before the diffusion model to prevent the diffusion model from damaging normal speech. The ArcFace (m=0.2, s=32) classifier is introduced to identification. A subset of the VoxCeleb2 development set is selected, from which 256,000 adversarial samples are generated using adversarial attack methods and split 9:1 for training and testing. The model is updated using the Adam optimizer with a cosine decay learning rate schedule, starting at 1e-3 and decaying to 1e-5. The batch size is 256, and the total number of epochs is 10.

Both attack methods exhibited substantial effectiveness, nearly achieving a 100% attack success rate. After applying the attacks, the cosine similarity between the adversarial samples and the target speech was measured at 0.934 and 0.959, respectively. In contrast, the cosine similarity between the adversarial samples and the source speech decreased to 0.134 and 0.074. Quantitatively, the PGD attack method demonstrated greater effectiveness compared to the Adam-based attack method.

After incorporating adversarial samples into the training process, the model’s robustness improved, with defense success rates rising from 0.47% and 0% to 90.35% and 95.98%, respectively. However, the introduction of adversarial samples, as cross-domain data, caused a degree of performance degradation on normal data. The impact of this degradation was more pronounced with stronger attack methods, as evidenced by the EER on the Vox1-O, which increased from 0.957% to 2.35% and 4.626%. Additionally, it is noteworthy that models trained with adversarial samples generated by the weaker Adam-based attack exhibited significantly reduced defense performance against stronger PGD attacks. In contrast, models trained with PGD-generated adversarial samples retained strong defense capabilities against Adam-based attacks, achieving a high defense success rate of 98.44%.

Diffusion-based Adversarial Purification is indeed a promising emerging technology. We observe that after adversarial purification, the attack success rates of adversarial samples decreased from 99.53% and 100% to 0.39% and 2.34%, respectively. However, the defense success rates improved by only 91.41% and 83.98%. This is because the perturbations removed by the diffusion module cannot perfectly match the added perturbations. Consequently, the denoised speaker embeddings may resemble those of other speakers rather than the source speaker’s. Moreover, controlling the denoising strength in diffusion-based Adversarial Purification is crucial. As shown in Fig.2, with an increase in diffusion steps, although the attack success rate and similarity with the target speaker decrease, the defense success rate and similarity with the source speaker also eventually decrease after reaching a certain inflection point.

Additionally, it is important to note that the diffusion module can also introduce some damage to normal speech. As shown in Fig. 2, the similarity between the normal speech and the source speaker rapidly declines with increasing diffusion steps. This degradation can further affect the quality of TTS synthesis. Therefore, we introduced a ResNet18-based discriminator in front of the diffusion model. Experimental results show that, after training on both positive and negative samples, this discriminator achieved a 100% recognition rate for adversarial samples of the attack types it was trained on. We will evaluate on unseen adversarial methods in the future.

We also explored the impact of different methods on the quality of zero-shot TTS synthesis. By extracting the speaker embeddings from adversarial samples and synthesizing speech using the text ”Good morning, good afternoon, and good evening!”, we evaluated the results by calculating the cosine similarity between the speaker embeddings of the synthesized speech and those of the source and target speakers. We observed that adversarial training defenses resulted in adversarial samples maintaining a high similarity to the source speaker, but with a relatively high similarity to the target speaker as well. In contrast, adversarial purification methods significantly reduced the similarity to the target speaker but also degraded a substantial portion of the source speaker’s information.