Detecting Textual Adversarial Examples through
Randomized Substitution and Vote

Let $\mathcal{X}$ be the set of input texts and $\mathcal{Y}=\{y_{1},y_{2},\cdots,y_{N}\}$ be the set of corresponding labels. Given an input text $x=\langle w_{1},w_{2},\cdots,w_{n}\rangle\in\mathcal{X}$ with the ground-truth label $y_{true}\in\mathcal{Y}$, a well-trained natural language model $f:\mathcal{X}\to\mathcal{Y}$ predicts its label $\bar{y}=y_{true}$ with high probability, using the maximum posterior probability:

The attacker perturbs the natural text $x$ slightly to generate a textual adversarial example $x^{adv}$ which misleads the model prediction:

In this work, we focus on detecting the synonym substitution based adversarial examples generated by substituting a few words, in which each word $w_{i}\in x$ is substituted with one of its synonym $\hat{w}_{i}$, so as to maintain the semantic consistency while attacking.

For the task of adversarial example detection, given an input text $x$, the detector $D$ should determine whether $x$ is benign sample or adversarial example. A stronger detector would also contain a corrector $E$, which could restore the correct label for the input adversarial example:

The optimization process of synonym substitution based textual adversarial attacks could be regarded as searching a specific sequence for word replacement, termed replacement sequence, in which the words mutually influence each other and contribute together to mislead the target classifier. Intuitively, we postulate that we could eliminate the perturbation if we could successfully break the mutual interaction of the words in the replacement sequence.

To validate our hypothesis, we first generate adversarial examples using the Textfooler attack [Jin et al., 2020] on $1,000$ randomly sampled texts from the AG’s News test set [Zhang et al., 2015] that are correctly classified by Word-CNN [Kim, 2014]. To break the interaction among words in the replacement sequence, we first randomly mask words in the benign samples or adversarial examples as unknown (“UNK”) token, denoted as UNK Benign or UNK Adversarial, with different rates and feed these processed texts to the model. As shown in Figure 1, the classification accuracy is still over 90% when we mask 40% words in the benign text, showing the stability and robustness of model to such randomized mask. In contrast, the classification accuracy on adversarial examples increases remarkably when we mask more words and we could recover 50% adversarial examples when masking 40% words. This validates our hypothesis that we could eliminate the adversarial perturbation if we successfully break the mutual interaction of words in the replacement sequence.

However, since we randomly substitute the meaningful words with meaningless “UNK” token, the semantic meaning of the text would be destroyed significantly when masking a certain number of words, leading to poor performance on benign samples and limited improvement on adversarial examples. Inspired by the existing synonym substitution based attacks, we attempt to randomly substitute the words with its synonyms to break the interaction of words without destroying the original semantic meaning. As shown in Figure 1, we find that randomly substituting words with its synonyms in the benign samples or adversarial examples, denoted as SYN Benign or SYN Adversarial, could consistently and significantly improve the robust accuracy against adversarial examples at the same time maintaining the high accuracy on benign samples under various substitution rates, which further validates our hypothesis.

Based on the above observation, we propose a novel textual adversarial example detection method called RS&V, which randomly substitutes the words in the input text with their synonyms to effectively detect the adversarial examples and restore the correct label for the input adversarial example.

Motivated by the observation that randomly substituting the words with their synonyms could eliminate the adversarial perturbation remarkably while maintaining the high clean accuracy, we propose a novel adversarial example detection method called Randomized Substitution and Vote (RS&V), to detect textual adversarial examples. As shown in Figure 2, given an input text $x$, RS&V contains two main stages, i.e. randomized substitution and vote & detection, for detecting whether $x$ is an adversarial example and restoring its correct label if it is adversarial.

Randomized Substitution. Given an input text $x=\langle w_{1},w_{2},\cdots w_{n}\rangle$, we first construct the synonym set $\mathcal{S}(w_{i})=\{\hat{w}_{i}^{1},\hat{w}_{i}^{2},\cdots,\hat{w}_{i}^{k}\}$ for each word $w_{i}\in x$ by combining the synonyms from WordNet [Miller, 1992] and neighbor words within a given Euclidean distance in the GloVe embedding space post-processed by counter-fitting [Mrkšić et al., 2016]. Then we randomly sample $n\cdot p$ words {$w_{i}\in x$} and substitute each selected word with its arbitrary synonym $\hat{w}_{i}^{j}\in\mathcal{S}(w_{i})$ to break the interaction among words in the replacement sequence for adversarial examples. Moreover, we find that substituting some high-frequency words (e.g., “a”, “the”, “hello”, etc.) is not only useless to mitigate the adversarial effect, but also has negative impact on classifying benign samples. Inspired by the Term Frequency–Inverse Document Frequency (TF-IDF) technique, we set the words with the top $s$ frequency among the training set as stopwords, where $s$ is a hyper-parameter, and ignore these words for randomized sampling. Such randomized substitution would repeat for $k$ times to generate $k$ different converted texts $\{x_{1},x_{2},\cdots,x_{k}\}$.

Vote & Detection. Although we have shown that randomized substitution could remarkably mitigate the adversarial effect with an acceptable decay on the clean accuracy, we seek to further improve the recovery ability of randomized substitution and decrease the negative impact on benign samples. To eliminate the variance introduced by randomized substitution and stabilize the detection, we feed $k$ various converted texts $\{x_{1},x_{2},\cdots,x_{k}\}$ and accumulate the logits output on these texts to vote for the final label $\operatorname*{argmax}\sum_{i=1}^{k}f(x_{i})$. Note that the method is different from the naive approach depicted in Section 3.2. Our RS&V would detect the input text $x$ as an adversarial example if the voted RS&V label is inconsistent with the prediction label $\operatorname*{argmax}f(x)$ and output the RS&V label as the prediction result for $x$.

In summary, as depicted in Figure 2, the proposed RS&V first generates multiple samples by randomly substituting words in text with synonyms and accumulates the logits of these samples to vote the final label. If the voted label is not consistent with the prediction label for the input text, RS&V would treat the input text as adversarial example and output the voted label. The overall algorithm of RS&V is summarized in Algorithm 1.

Datasets and Models. We adopt three widely investigated benchmark datasets, i.e. IMDB [Maas et al., 2011], AG’s News, Yahoo! Answers [Zhang et al., 2015], including sentiment analysis and news or topic classification. Details about the datasets are summarized in Table 1. To validate the generalization to model architectures of our RS&V, we adopt several popular deep learning models that exhibit state-of-the-art performance on text classification tasks, including Word-CNN [Kim, 2014] and two pre-trained language models, BERT [Devlin et al., 2019] and RoBERTa [Liu et al., 2019].

Evaluation Setup. We consider static attack evaluation (SAE) and targeted attack evaluation (TAE) [Si et al., 2021]. In SAE setting, the attacker does not know the detection module and generates adversarial examples on the original model. In TAE setting, the attacker could access and attack the detection module simultaneously when attacking target model. We evaluate the detection performance against various attacks, i.e. GA [Alzantot et al., 2018], PWWS [Ren et al., 2019], PSO [Zang et al., 2020], Textfooler [Jin et al., 2020] and HLA [Maheshwary et al., 2021], using F1 score and detection accuracy, i.e. classification accuracy on samples recovered by detection modules.

Baselines. We compare our method with normally trained models and two recently proposed detection methods designed for synonym substitution based attacks:

• •

  DISP [Zhou et al., 2019] trains a perturbation discriminator to identify the perturbed tokens and an embedding estimator to reconstruct the original text.

• •

  FGWS [Mozes et al., 2021] replaces the low-frequency words with their most frequent synonyms in the dictionary to eliminate the adversarial perturbation.

Hyperparameters. For our RS&V, the number of votes is set to 25, the substitution rate $p$ is set to $80\%$ on IMDB and $60\%$ on AG’s News as well as Yahoo! Answers based on the text length, and the stopword selection portion $s$ is set to $2\%$. We mix up the synonyms from WordNet [Miller, 1992] and embedding space as synonyms set, where the embedding space synonyms were calculated by ranking the Euclidean distance between words based on GloVe vectors processed by counter-fitting [Mrkšić et al., 2016]. We select at most 6 words on embedding space to keep the diversity of the synonyms set, and the portion $s$ for stopwords is set to 5 for IMDB and 2 for AG’s News and Yahoo base on the frequency drops, For the baselines, we use the same way in the original paper to tune hyper-parameters on the validation set.

To validate the effectiveness of RS&V, we evaluate the detection accuracy, the classification accuracy on the recovered samples by the detection methods and F1 score against adversarial attacks on three datasets, and do comparison with DISP and FGWS. Due to the high computational cost of generating adversarial examples for the attack methods, we randomly select 1,000 samples on each dataset and adopt the attacks to generate adversarial examples for each model. The results are summarized in Table 2.

We could observe that RS&V generally exhibits superior detection performance against various adversarial attacks with high classification accuracy on benign samples. Specifically, all the normally trained models achieve the lowest classification accuracy on three datasets against five attacks. Compared with two detection baselines, RS&V consistently achieve much better detection accuracy and F1 score on three datasets for various models against five adversarial attacks, while maintaining the high classification accuracy on benign samples. For instance, on Yahoo! Answers dataset for RoBERTa model, RS&V exhibits even better (1.3% higher) classification accuracy on benign samples than normally trained model and averagely outperforms DISP and FGWS with a clear margin of $16.1\%$ and $3.0\%$, respectively. This convincingly validates the high effectiveness of RS&V.

To further gain insight on the performance improvement of RS&V, we present a randomly sampled adversarial example for Word-CNN from AG’s News generated by Textfooler and the recovered examples for three detection methods in Table 3. As we can see, none of DISP, FGWS, and RS&V could really recover the benign sample as we expect, but all of them could lead to correct classification result. The results also show the fragility of textual adversarial example and might inspire more powerful detection and defense methods by pre-processing the input text without extra training or modification on the architecture in the future.

The SAE setting, where the attacker does not know there is a detection module, is the only case considered by the existing textual adversarial example detection methods. To further validate the effectiveness of RS&V, we consider a more rigorous setting of TAE, in which the attacker could access and attack the detection module to generate adversarial examples. Specifically, we generate adversarial examples on AG’s News for Word-CNN in the TAE setting. Due to the high computational cost of DISP in inferring the extra models for the attacks, here we do not consider DISP as the detection baseline.

As shown in Table 4, FGWS exhibits little effectiveness against various attacks under the rigorous setting when compared with the normally trained model. One possible reason for the poor effectiveness of FGWS would be that the attacker could easily bypass the low-frequency words and generate adversarial examples by just using the high-frequency words. In contrast, RS&V could maintain the high effectiveness against various attacks. We postulate that the uncertainty of RS&V at each iteration makes it harder for the attacker to find good local minima to craft adversarial examples, leading to high robustness of RS&V. This further verifies the high effectiveness and superiority of RS&V.

We also conduct a series of ablation experiments to investigate the impact of hyper-parameters in RS&V, namely the substitution rate $p$ in the randomized substitution, the number of votes $k$ during the vote process, and the stopword selection portion $s$. All the experiments are conducted on AG’s News for Word-CNN against Textfooler using 1,000 randomly sampled texts that are correctly classified. To eliminate the variance of randomness, we repeat the experiments five times and report the average detection accuracy. The default values are $p=60\%$, $k=25$ and $s=2\%$.

On the effectiveness of substitution rate. In Figure 3(a), we study the impact of substitution rate $p$ on the detection accuracy. When $p=0$, RS&V degenerates to the normally trained model without the detection module, which exhibits high classification accuracy on benign samples but low detection accuracy on adversarial examples. With the increment on the value of $p$, more words in the input text would be substituted, but the accuracy on the benign sample is stable with slight decay, indicating the model’s high robustness against such randomized synonym substitution. In contrast, the detection accuracy on adversarial examples increases significantly when we increase the value of $p$ till $p=60\%$, which is intuitive that replacing more words in the input text with synonyms would be more likely to eliminate the adversarial perturbation. Thus, we set $p=60\%$ in the main experiments.

On the effectiveness of number of votes. As shown in Figure 3(b), when the number of votes $k=1$, there would be only a single input text and the vote cannot make any difference, hence, RS&V exhibits the lowest accuracy on either benign samples or adversarial examples. With the increment on the value of $k$, the accuracy on both benign samples and adversarial examples increases significantly, especially for the detection accuracy. With a large value of $k\geq 10$, RS&V maintains high and stable performance, supporting that our vote could eliminate the variance of randomized substitution and stabilize the detection process. In the main experiments, we simply select a large value of $k=25$ on all datasets.

On the effectiveness of stopword selection portion. We continue to investigate the impact of the stopword selection portion $s$ on the detection accuracy. As illustrated in Figure 3(c), the classification accuracy on benign samples is generally stable for various values of $s$, as our vote strategy has mitigated the variance introduced by randomness, leading to high and stable performance. The detection accuracy, however, increases when $s<2\%$ and decreases significantly when more words in the input text cannot be substituted and thus could not eliminate the adversarial perturbation effectively. In the main experiments, we set $s=2\%$ for better performance.

In summary, the substitution rate $p$, number of votes $k$ and stopword selection portion $s$ significantly influence the performance when they are small ($p\leq 60\%$, $k\leq 10$ or $s\leq 2\%$), while the performance becomes stable for larger $p$ and $k$ but decreases for larger $s$. In our experiments, we adopt $p=60\%$, $k=25$ and $s=2\%$ for high and stable performance.