Towards a Novel Perspective on Adversarial Examples Driven by Frequency

The notion of adversarial examples is initially introduced by Szegedy et al. Goodfellow et al. (2014). They leverage gradient information to ingeniously engineer minor modifications in images. These alterations, generally imperceptible to the human eye, are sufficient to mislead classification algorithms into incorrect predictions. According to knowledge of the target model, adversarial attacks can be classified as white-box attacks or black-box attacks. In white-box attack scenarios, attackers have access to gradients of models, enabling them to generate highly precise perturbations Athalye et al. (2018); Carlini and Wagner (2017); Kurakin et al. (2018); Madry et al. (2017). Black-box attacks are broadly categorized into query-based and transfer-based types. Query-based attacks leverage feedback from model queries to craft adversarial examples Chen et al. (2020); Cheng et al. (2018); Tu et al. (2019); Guo et al. (2019); Andriushchenko et al. (2020). Brendel et al. Brendel et al. (2017) pioneered the decision-based black-box attack Cheng et al. Chen et al. (2017) introducing a zero-order optimization technique through gradient estimation. Conversely, transfer-based attacks involve creating adversarial examples on substitute models and transferring them to the target model Cheng et al. (2019); Huang and Zhang (2019); Huang et al. (2019); Shi et al. (2019). However, compared to query-based attacks, transfer-based attacks are often constrained by challenges in building effective substitute models and their relatively lower rates of attack success.

Numerous studies Tsuzuku and Sato (2019); Guo et al. (2018); Sharma et al. (2019); Lorenz et al. (2021); Wang et al. (2020a) have analyzed DNNs from a frequency domain perspective. Tsuzuku & Sato Tsuzuku and Sato (2019) pioneer the frequency framework by investigating the sensitivity of DNNs to different Fourier bases. Building on this, Guo et al. Guo et al. (2018) design pioneering adversarial attacks targeting low-frequency components. This approach is corroborated by Sharma et al. Sharma et al. (2019), who demonstrate the efficacy of such attacks against models fortified with adversarial defenses. Concurrently, efforts by Lorenz et al. Lorenz et al. (2021) and Wang et al. Wang et al. (2020a) have propelled forward the detection of adversarial examples, utilizing frequency domain strategies during model training.

Additionally, some studies Wang et al. (2020b, c) have demonstrated that DNNs are more vulnerable to high-frequency components, resulting in models with low robustness. It is also the principal rationale for preprocessing-based defence methods. Confusingly, these findings are contradictory to the idea of low-frequency adversarial attacksIlyas et al. (2019); Sharma et al. (2019). To explain the performance of adversarial examples in frequency domains, Maiya et al. Maiya et al. (2021) argue that adversarial examples are neither high-frequency nor low-frequency and are only relevant to the dataset. In addition, Abello et al. Abello et al. (2021) and Caro et al. Caro et al. (2020) argue that the frequency of images is an intrinsic characteristic of the robustness of models. Recent studies Qian et al. (2023) have shown that attacks targeting both low-frequency and high-frequency information can enhance attack performance, indicating that the relationship between learning behavior and different frequency information remains to be explored.

To examine the characteristics of adversarial perturbations, we first generate these disruptive inputs. FGSM Goodfellow et al. (2014) and PGD Madry et al. (2017) are prominent techniques for creating adversarial examples, commonly used to assess the robustness of deep learning models. Consequently, we employ these two methods for generating adversarial examples. To be specific, given some images $X=\{x_{1},x_{2},...,x_{n}\}$, whose corresponding labels $Y=\{y_{1},y_{2},...,y_{n}\}$. The adversarial example generation formula for FGSM is as Eq.1:

where, $\epsilon$ is a parameter that controls the magnitude of the perturbation. $sign(\nabla_{x}J(\theta,x,y))$ represents the sign of the gradient of the loss function with respect to the input image, which determines the direction of the perturbation. PGD generates an adversarial example by performing the iterative update with Eq.2:

where $\alpha^{t}$ is the step size at iteration $t$, $x^{t}_{adv}$ is adversarial example at iteration $t$ and $P_{x,\epsilon}(\cdot)$ clips the input to the $\epsilon-$ball of $x$. $sign(\nabla_{x}J(x_{t},y))$ is the gradient at iteration $t$.

Wavelet packet decomposition (WPD) employs multi-level data decomposition to extract detailed and approximate information from signals. This method iteratively decomposes all frequency components, enabling a comprehensive frequency stratification. This contrasts traditional wavelet analysis, which primarily decomposes only the low-frequency parts. Through WPD, an image is separated into a series of frequency bands, each capturing distinct components of the image. The low-frequency band typically encapsulates fundamental image characteristics, such as overall luminosity, color, and structural composition. In contrast, high-frequency bands contain secondary image information like edges and textures Prasad and Iyengar (2020).

We apply WPD for frequency decomposition. The wavelet packet $\{\mu_{n}(t)\}_{n\in Z}$ comprises a set of functions that include the scaling function $\mu_{0}(t)$ and the wavelet function $\mu_{1}(t)$, as depicted in Eq.3:

where $p_{k}$ is low-pass filter, $q_{k}$ is high-pass filter. The decomposition process of WPD can be expressed as Eq.4:

where $f^{2n}_{jk}$ and $f^{2n+1}_{jk}$ are wavelet packet coefficients obtained by the decomposition of ${f^{n}_{j+l,l}}$. So we decompose $X$, $X_{adv}$ into frequency domains by Eq.4, and we get different wavelet packet coefficients $F=\{f_{1},f_{2},...,f_{n}\}$ and $F_{adv}=\{{f_{1}^{\prime}},{f_{2}^{\prime}},...,{f_{n}^{\prime}}\}$ respectively. The wavelet packet coefficient determines the frequency band, so we think of them as the frequency band.

For ease of description, high-frequency components are denoted as $d$, and low-frequency components as $a$. WPD follows a binary decomposition process, resulting in a tree structure as illustrated in Figure 1. After $n$ decompositions, $2^{n}$ frequency bands are obtained. These decomposed frequency bands can be reassembled using Eq.5:

We employ WPD to dissect both normal images and their corresponding adversarial examples into a series of frequency bands. This approach allows us to compute the cosine similarity between adversarial and clean samples within each band, thereby analyzing the spread of adversarial perturbations across different frequency segments. For an image $X$ and its adversarial counterpart $X_{adv}$, we apply WPD to decompose $X$ into frequency bands $F=\{f_{1},f_{2},...,f_{n}\}$ and $X_{adv}$ into $F_{adv}=\{f_{1}^{\prime},f_{2}^{\prime},...,f_{n}^{\prime}\}$. Subsequently, we calculate their cosine similarity $cos(\theta)$ as per Eq.6.

The experimental results are depicted in Figure 2, demonstrating that lower similarity corresponds to greater perturbations. Initially, after the first decomposition ($1^{st}$ layer), adversarial perturbations predominantly appear in the high-frequency component $\{d\}$, aligning with previous research that identifies adversarial perturbations as high-frequency components Wang et al. (2020b, c). Interestingly, after the second decomposition ($2^{nd}$ layer), these perturbations concentrate more in the $\{da\}$ band, with further decrease in similarity observed in the $\{daa\}$ band after the third decomposition ($3^{rd}$ layer). The $\{dad\}$ band which decomposes from the low-frequency band ${ad}$ originating from the high band $\{d\}$, also exhibits reduced similarity. This pattern suggests that adversarial perturbations are prevalent in higher-frequency components within low-frequency bands. Furthermore, at the third decomposition level, the band with the third-lowest similarity is the highest frequency band $\{ddd\}$, indicating that adversarial perturbations, although inherently high-frequency, depend on the low-frequency information from their preceding levels. For more details, see Section 6.2.

As discussed in Section 3.3, our experimental findings indicate that adversarial perturbations predominantly appear in specific high-frequency bands originating from low-frequency bands. Particularly at the $3^{rd}$ layer, the $\{daa\}$ and $\{dad\}$ bands, decompose from $\{aa\}$ and $\{ad\}$ respectively, exhibit lower similarity. This indicates a higher concentration of adversarial perturbations in these bands. The lowest frequency band $\{aaa\}$ encompasses crucial image information like color, brightness, and structural luminance, contrasting with the other seven bands, which are mainly associated with edges and textures. This difference implies that the $\{aaa\}$ band holds more substantial information, as illustrated in Figure 1, where areas with lower information content are in black (value of 0). Consequently, we combine the low-frequency band $\{aaa\}$ and the high-frequency components of low-frequency bands $\{daa,dad\}$ as the main search area. The ablation experiments for these three frequency bands are detailed in Section 5.

Inspired by the work of Guo et al Guo et al. (2019), we propose a black-box adversarial attack. This approach includes orthogonal direction vector selection, iterative approach and perturbation limitations.

Orthogonal direction vector selection. To effectively reduce the number of searches, verifying that all vectors in frequency bands $W_{s}$ are orthonormal is essential. Decompose space $L^{2}(R)$ into wavelet subspace $W_{j}$, such as $L^{2}(R)=\oplus_{j\in Z}W_{j}$. We further orthogonal decompose the subspace $W_{j},j>1$ into $W_{j}=\oplus_{n\in Z}U^{n}_{j}$. According to wavelet theory, wavelet packet decompositions satisfy Eq.7. Therefore, after three times decomposition, we yield $\{aaa,daa,ada,dda,aad,dad,add,ddd\}$, which are orthogonal to each other. So the frequency band $Q\in W_{s}$ are mutually orthogonal. The $q$, which we choose from $Q$ without repeat, is also orthogonal, where $q$ is the orthogonal direction vector specified in this paper.

Iterative approach. Pick band $Q$ from $W_{s}$ according to the initialization probability $Pro$. Then, initialize a matrix $q$ of size $Q$ with the value $1$ for $n$ random points and $0$ for the rest. No point in $q$ will be selected twice. Therefore, each $q\in Q$ is an orthogonal vector we picked. We add perturbations to the frequency band $Q$ through $\alpha q$. Since $Q$ belongs to $W_{s}$ and $W_{s}$ belongs to $W$, our perturbations are added to $W$. We can reconstruct the perturbed $W$ to image $x^{\prime}$ by Eq.5. Input $x^{\prime}$ into the model for a query. These modifications may likely lower the $p=ph(y|x^{\prime})$. And we get the $p$ in the current search. Then, we update the probability $Pro$ based on the $p$ and choose another orthogonal vector $q$ for the subsequent iteration. To maintain the query efficiency, we ensure that no two directions cancel each other out.

Perturbation limitations. We can control perturbations by adjusting the step size $\epsilon$ and $n$ (parameter $n$ determines the generation of $q$). In each iteration, $n$ points are added, subtracted, or discarded (if the output probability is not reduced in either direction). Suppose that $\alpha\in\{-\epsilon,0,\epsilon\}$ represents the symbol of the search direction chosen at step $T$. Therefore, perturbations after $t$ steps in frequency domains satisfy Eq.8. Our analysis highlights a critical trade-off: (1) for query-limited situations, raising $\epsilon$ and $n$ can increase the attack success rate while decreasing the number of inquiries. (2) Reduce $\epsilon$ and $n$ to produce imperceptible perturbations in perturbation-limited situations.

Following the previous study Guo et al. (2019), we randomly pick 1000 images from validation sets of CIFAR-10 Krizhevsky et al. (2009), CIFAR-100 Krizhevsky et al. (2009) and ImageNet-1K Russakovsky et al. (2015), respectively, as test dataset. These samples are initially correctly classified to avoid artificially boosting the success rate. In CIFAR-10 and CIFAR-100, we are standardized training the models of ResNet-50, VGG16 and InceptionV3 following the steps outlined in He et al. (2016); Huang et al. (2017); Szegedy et al. (2016). On ImageNet-1K, we choose pre-trained models of ResNet-50, VGG16, InceptionV3 and DenseNet-121 from Pytorch Paszke et al. (2019).

We analyze the distribution of adversarial perturbations in the frequency domain using a combination of three datasets (ImageNet-1K, CIFAR-10, and CIFAR-100), four models (VGG16, ResNet-50, DenseNet-121, and InceptionV3), and two attacks (FGSM and PGD). By computing the cosine similarity between adversarial and pure examples, we assess the distribution of adversarial perturbations across various frequency bands, as detailed in Table 1. The results reveal a consistent trend across diverse datasets, models, and attack strategies: the cosine similarity between normal and adversarial samples is markedly lower within $\{da\}$, $\{daa\}$ and $\{dad\}$ frequency bands. This trend suggests that adversarial perturbations predominantly occur in high-frequency bands of low-frequency bands. Moreover, although perturbations are distinctly observed in certain frequency bands, the complete separation of adversarial perturbations remains elusive, both through the use of different wavelet functions and further decomposition of the third-layer frequency band. This suggests that current wavelet functions are inadequate for the separation of adversarial perturbations. Designing new wavelet functions specifically for the characteristics of adversarial perturbations might facilitate a more complete separation of these perturbations.

In this section, we evaluate our method on multiple datasets and models and also compare it with state-of-the-art approaches. All experiments are untargeted attacks. For performance evaluation and comparison, we employ the Attack Success Rate (ASR) and six other distinct metrics. These include the Average Number of Queries (ANQ), Median Number of Queries (MNQ), $L_{2}$ norm, $L_{\infty}$ norm, Structural Similarity Index (SSIM), and our proposed NDV.

Evaluations on Different Models. We evaluate our proposed attack method on DenseNet121, ResNet50 and InceptionV3 using CIFAR-10 and CIFAR-100 datasets. As shown in Table 2, the results demonstrate that our method can efficiently attack various models, with ASR exceeding 98%. In particular, for the InceptionV3 model, an average of only a few dozen queries is needed for a successful attack.

Comparisons with State-of-the-Art Methods. We compare our attack with state-of-the-art black-box attack algorithms, including Boundary Attack Brendel et al. (2017), GeoDA attack Rahmati et al. (2020), Square Attack Andriushchenko et al. (2020), $\mathcal{N}$ Attack Li et al. (2019), SimBA Guo et al. (2019). As shown in Table 3, our attack achieves a fine balance between attack success, imperceptibility, and query efficiency, marking it as a superior strategy for adversarial attacks in practical applications. Figure 5 illustrates the relationship between the number of queries and attack success rate, with results indicating that our method can achieve a high success rate with fewer queries.

Ablation Studies. We investigate the contributions of three selected frequency bands—denoted as A ($\{aaa\}$), B ($\{daa\}$), and C ($\{dad\}$)—in generating adversarial perturbations, with the results detailed in Table 4. Band A demonstrates the highest individual ASR, underscoring its significant role in adversarial attacks, aligning with the studies of Wang et al Wang et al. (2020c). Combinations AB and AC yield higher ASR than each band individually, suggesting that the integration of bands B and C enhances attack efficacy. Notably, the ABC combination achieves the highest ASR, outperforming all single and dual-band combinations, while other evaluated metrics remain relatively unchanged.

To enhance the comparative analysis between NDV and the $L_{2}$ norm, we select three sets of examples, as demonstrated in Figure 6. The first column displays the clean samples, while the second and third columns showcase adversarial examples created by our proposed method and the square attack method, respectively. The outcomes reveal that, despite nearly identical $L_{2}$ norms, the adversarial examples produced by the square attack exhibit more noticeable color variations and artifacts, thus making them more discernible to the human eye. In contrast, the changes in NDV values more accurately reflect the perceptual sensitivity of human observers.