Detecting Generated Images by Real Images Only

Existing generated image detection methods mainly rely on deep learning. They can be divided into image-artifact-detection-based and massive-training-based approaches. The artifact detection approach probes the traces left by the generative networks. Using these traces in the spatial or frequency domain, a trained classifier can discern the generated images. In contrast, by deploying massive training, the second group of approaches takes advantage of the strong learning ability of deep neural networks to search for the decision boundaries between real and generated images.

Typical classifiers like SVM are two- or multi-class classifiers which generalize to new data by maximizing a margin between existing training data from different classes [26] in some feature space. One-class classifiers decide a boundary using one class of data. Any data outside the boundary will be classified as a different class. There are two implementations to construct the decision boundary. The first, proposed by Schölkopf et al. [27], maximizes the distance between a hyperplane decided by support vectors and the coordinate origin. The second, by Tax and Duin, restricts training data to a hypersphere [28]. One-class classification problems commonly exist in novelty and outlier detection [29, 30, 31, 32]. Successful application of one-class classifiers depends on whether the feature space is well constructed. An improper feature space contains large voids between the class instances, resulting in model failure or producing many false classifications.

We consider a sequence $\mathcal{T}$ consisting of some real images $t_{1},t_{2},...,t_{i},...,t_{\ell}$, where $\ell\in\mathbb{N}$ is the number of images. Suppose a mapping $\Phi:\mathcal{T}\rightarrow\mathcal{S}$, where the dot product in the feature of $\Phi$ can be computed by a kernel $K$:

where $K$ can be a simple kernel such as a Radial Basis Function (RBF) kernel:

To create a dense subspace $\mathcal{S}$ that contains real images only, we can limit these real images within a hypersphere. Or, more conveniently, we can separate the real images using a hyperplane parameterized by vector normal $\bm{w}$ and offset $\rho$ from the origin by optimizing

to maximize the distance $\theta$ between the origin and this hyperplane. Then $\mathcal{S}$ will be the space which satisfies $\bm{w}\cdot\Phi(t_{i})\geq\rho$. In Eq. 3, the column vector $\bm{\eta}=[{\eta_{1}},{\eta_{2}},\ldots,{\eta_{\ell}}]$ and each $\eta_{i}$ is the slack variable corresponding to the $i$-th real image; $v\in(0,1)$ is a trade-off parameter between optimization terms. Finally, for an image $I\notin\mathcal{T}$, the decision function:

will decide whether the image $I$ is in $\mathcal{S}$ or not. For real images $t_{i}$ in the sequence $\mathcal{T}$, the above sign function will be positive. Using the kernel trick in Eq. 1, the sign function in Eq. 4 can also be denoted in the form of support vector expansion:

To determine whether real images are mapped to $\mathcal{S}$ via $\Phi$, we analyze the probability $P$ that a real image $t_{i}\in\mathcal{T}$ falls outside $\mathcal{S}$ after mapping. For any $\theta\in\mathbb{R}$, according to Theorem 7 in [33] (proved by Lemma 3.9 in [34]), all images $t_{1},t_{2},...t_{i},...,t_{\ell}\in\mathcal{T}$ are mapped to $\mathcal{S}$ with probability $1-\delta$ if $f(t_{i})\geq\theta+\gamma$ holds for any $t_{i}$ (given that there is a class $\mathcal{F}$ of functions which map $\mathcal{T}$ to $\mathbb{R}$, and $f\in\mathcal{F}$). The mapping methods are not limited to the one we introduced, and we use $f$ as some function in the set. The probability $P$ that the mapping of the real image $t_{i}$ falls outside $\mathcal{S}$ is

where $\mathcal{N}(\gamma,\mathcal{F},2\ell)=\text{sup}_{\mathcal{T}^{2\ell}}\mathcal{N}_{d}(\gamma,\mathcal{F})$, and $\mathcal{N}_{d}(\gamma,\mathcal{F})$ is the $\gamma-$covering number of function set $\mathcal{F}$ with the distance $d$ defined by

where $f,g\in\mathcal{F}$. From Eq. 6 we can see that if the real images can be mapped to $\mathcal{S}$, $P\{f(t_{i})<\theta-\gamma\}$ should be small. The number of real images $\ell$ in the sequence $\mathcal{T}$ and the choice of mapping function $f\in\mathcal{F}$, or more specifically $\Phi$ will affect the value of $P$.

We can also analyze the probability that a real image $I\notin\mathcal{T}$ is falsely mapped outside the constructed space $\mathcal{S}$. We fix $\theta\in\mathbb{R}$ and suppose that the range of the function set $\mathcal{F}$ is $[a,b]$. According to Theorem 9 in [33], with probability $1-\delta$ over the sequence $\mathcal{T}$, for all $\gamma>0$ and any $f\in\mathcal{F}$,

where

and $\mathcal{D}$ is defined as the sum of the distances from all real images $t_{i}\in\mathcal{T}$ falsely classified as generated images to the decision boundary:

Eq. 8 shows that the probability $P\{f(I)<\theta-\gamma\ \mathrm{and}\ I\notin\mathcal{T}\}$ is bounded by the term of the ratio of the logarithmic covering numbers which is at scale proportional to $\gamma$, the size $\ell$ of the real image sequence $\mathcal{T}$.

As shown in Fig. 3(a1)$\sim$(a3), different values of $\ell$ (the number of real images in $\mathcal{T}$) will affect the decision boundary and consequently the number of false classifications. Also, the choice of mapping function $f$ decides the term $\mathcal{N}(\gamma,\mathcal{F},2\ell)$. Fig. 3(b1)$\sim$(b3) show the classification results of using different $f$ while $\ell$ is fixed at 500. Therefore, for making the $\gamma-$covering $\mathcal{N}_{d}$ take a smaller value, $f$ should reflect the commonality of the real images so that it can map $\mathcal{T}$ to a dense subspace $\mathcal{S}$.

Previous research [4, 9] found that the upsampling operation in conditional generative models changes the noise characteristics of input real images. Moreover, noise transformation after initial sampling is also in the image generation process of unconditional models. Therefore, we will explore the commonality of real images from the noise characteristics.

To extract image noise $n$ from the image $I$, applying a denoising filter $\Xi(\cdot)$ is the most common practice:

where $(x,y)$ denotes coordinates. We adopted the denoising filters proposed in [35]. These filters keep image edges very effectively in practice, and obtain the noise relatively unaffected by image semantics. To better analyze the characteristics of the Filtered Noise Pattern (FNP) $n(x,y)$, we transform it by the Discrete Fourier Transform (DFT):

where $M\times N$ represents the image size. We then calculate the amplitude spectrum $A(u,v)$ of $\zeta(u,v)$:

where ${Rp}_{u,v}$ and ${Ip}_{u,v}$ denote the real and imaginary parts of $\zeta(u,v)$, respectively.

Since the Fourier frequency spectrum is symmetric, we can average the amplitude spectra $A(u,v)$ in either one dimension to show its characteristics. We took $A_{u}$ as an instance and calculated it by

We then averaged $A_{u}$ across all images in each image subset (the image subsets are detailed in the experiment section) and denoted it as $\bar{A}_{u}$. It can be seen in Fig. 4(b) that the $\bar{A}_{u}$ of generated images' FNP show different distributions and periodic patterns in some subsets. However, the $\bar{A}_{u}$ for the real images' FNP do not show periodic patterns and have similar distributions, as shown in Fig. 4(a).

To confirm this commonality of real images, considering the strong learning ability of deep neural networks, we used a denoising network to extract image noise. For convenience, the equations only show the processing of one color channel, but we process all three channels of each color image similarly. The calculation can be formulated as

where $\Psi(\cdot)$ is a denoising network, $\theta_{\Psi}$ represents the set of all parameters of $\Psi(\cdot)$, and $\Psi(I(x,y))$ denotes the clean output image. $L(x,y)$ is called the Learned Noise Pattern (LNP).

As shown in Fig. 4(c)(d), compared to FNP, LNP can better reflect the commonality of real images and their discrepancy from generated images. The $\bar{A}_{u}$ of generated images' LNP show periodic peaks whereas the $\bar{A}_{u}$ of real images' LNP do not. Moreover, the $\bar{A}_{u}$ of generated images' LNP from different models are distinct and show various characteristics. This explains why learning from one particular type of generated image causes a generalization problem. In contrast, the $\bar{A}_{u}$ of real images' LNP are much more similar. Therefore, mapping real images to the dense subspace constructed by noise features $f:\mathcal{T}\rightarrow\mathcal{S}$ is feasible and better reflects the commonality of real images. Consequently, the probability that a real image $t_{i}\in\mathcal{T}$ falls outside $\mathcal{S}:P\{f(t_{i})<\theta-\gamma\}$ can be made small.

The general form of the LNP is described in Eq. 15, and there are many choices for the denoising network $\Psi(\cdot)$. Early denoising networks, such as DNCNN [36], directly add Gaussian white noise (AWGN) to clean images to create noisy images and then use the clean/noisy pairs to train $\Psi(\cdot)$. However, in real scenarios, the noise may be non-Gaussian. To extract the noise in the real world, CycleISP [37] deploys networks to simulate the actual imaging pipeline of cameras by reconstructing RAW images from RGB images, adding noise to the RAW images, and then converting the RAW images to RGB images to get clean/noisy image pairs.

Inspired by the CycleISP [37] network, the LNP extraction in our proposed generated image detection method is shown in Fig. 5(b). Specifically, an image $I$ of size $M\times N$ will be processed as

where ${K_{3}}$ denotes a $3\times 3$ convolution. Many low-level applications including denoising [36] benefit from a residual learning framework [15], therefore we used the Recursive Residual Group (RRG) module to process the features. The RRG module is composed of two Dual Attention Blocks (DAB). Each DAB achieves calibration of the features by two types of channel attention and spatial attention, as shown in Fig. 5(c). Its dual attention structure suppresses less useful features and keeps informative ones, which is ideal for denoising. We used four consecutive RRG modules:

Finally, by a convolution operation ${M_{2}}={K_{3}}({M_{1}})$, the three-channel feature map ${M_{2}}$ can be obtained, and then the LNP $L(x,y)$ in Eq. 15 is calculated by

in our generated image detection method.

Some LNP examples obtained by Eq. 18 are shown in Fig. 6(b). We can see that the LNP in smooth regions of images from generative models exhibits grid artifacts, while the LNP of real images does not show that. Since a grid effect in the spatial domain will show periodicity in the frequency domain, we also transformed it by the DFT:

and calculated amplitude spectra $A(u,v)$ by Eq. 13.

Similar to $\bar{A}_{u}$, we also averaged $A(u,v)$ of all images in each image subset and denote it as $\bar{A}(u,v)$. Fig. 6(c) shows $\bar{A}(u,v)$ for each generated subset and real images from all real subsets. For the BigGAN [38] and HiSD [39] models using nearest-neighbor interpolation for upsampling, the period in the horizontal direction of $\bar{A}(u,v)$ is $M$/4, and the period in the vertical direction of $\bar{A}(u,v)$ is $N$/4. For StyleGAN [40] and StyleGAN2 [41] using bilinear interpolation for upsampling, the period in the horizontal direction of $\bar{A}(u,v)$ is $M$/8, and the period in the vertical direction of $\bar{A}(u,v)$ is $N$/8. For CycleGAN [42] and StarGAN [43], upsampling is performed using deconvolution, and the $\bar{A}(u,v)$ have strong vibrations with a period of $M$/4 in the horizontal direction of $\bar{A}(u,v)$ and a period of $N$/4 in the vertical direction. $\bar{A}(u,v)$ of all generated images' LNP show significant periodicity. In contrast, the $\bar{A}(u,v)$ of real images' LNP does not present periodicity. Therefore, we can calculate the amplitude spectrum $A(u,v)$ of LNP and then extract features from it to characterize the periodic signal.

To make the amplitude spectrum $A(u,v)$ of LNP more discriminative, we performed spectrum enhancement by increasing the gap between the high and low energy of the spectrum. Since the discriminative high peaks in Fig. 4 are obtained by spectrum row averaging (Eq. 14), amplitudes less than the average row value are not bright spots, and we set their value to 0. Then the differences of remaining non-zero amplitudes are enlarged by squaring. The enhancement is formulated as

where $A_{u}$ is calculated by Eq. 14 on the amplitude spectrum $A(u,v)$ of LNP.

Since the LNP amplitude spectra of generated images present periodicity, and the LNP amplitude spectra of real images are generally low in energy except at the center, we extract features based on this property. The extracted features can be sampled by

where $\delta(.)$ is a 2-D impulse function, $k$ is a sampling interval, and $m$ and $n$ are integers from $[0,\lfloor(M-1)/k\rfloor]$ and $[0,\lfloor(N-1)/k]\rfloor]$, respectively. The choice of $k$ will be discussed in the experiment. At this point, we can map real images $\mathcal{T}$ to a dense subspace $\mathcal{S}$ by $F(m,n)$ and finally optimize the one-class classifier.

We trained and tested the proposed detection method on publicly available datasets whenever possible. We used five datasets provided by [10] containing real images and GAN-generated images from conditional GANs (BigGAN [38], StarGAN [43], CycleGAN [42]), and unconditional GANs (StyleGAN [40], StyleGAN2 [41]) commonly used in generated image detection. To demonstrate the detection ability of our method on emerging generative models, we also included AE structure-based generative model HiSD [44], flow-based model Glow [16] whose network structure is reversible, Disco Diffusion [45] guided by CLIP [46], and the currently most advanced image generative diffusion models Stable Diffusion [25] and Latent Diffusion [47]. For the AE-based, Glow, and diffusion generative models, since public generated image datasets are not available, we used the officially published pre-trained models to produce generated image datasets without post-processing operations on the images. We denote the datasets using the model's name and a “-DB” suffix.

We randomly selected a certain number of real images from the real image subset provided by [10]. The experiment shows that $\mathcal{T}$ with just 800 real images are enough to construct a dense subspace $\mathcal{S}$ by $F(m,n)$ and optimize the one-class classifier, which will be discussed in Subsection 4.4.3. During training, the proposed generated image detection method did not see any generated images. All other generated image detection methods were implemented with their best performance. In the testing phase, we randomly selected 500 real and 500 generated images from each generative model dataset to infer their authenticity and evaluate our method and the other methods. All training was implemented on an NVIDIA GeForce GTX Titan X GPU and an Intel Xeon E5-2603 v4.

Generated image detection can be seen as a binary classification problem. Each image belongs to one of two classes, real or generated, and a decision must be made for each image. Since our ultimate goal is to discern the authenticity of an image, Accuracy ($ACC$) is a proper evaluation metric for this task.

Precision in a binary classification task is related to the selection of classification thresholds. Therefore, we use the Average Precision ($AP$), which calculates the precision for all possible classification thresholds and then takes the average.

$F1$ score takes into account the Precision and Recall of the classification method. The $F1$ score can be considered a harmonic average of the Precision and Recall, with its maximum value of 1 and minimum value of 0.

Existing generated image detection methods may achieve satisfactory detection results in the laboratory. In practice, a detection method may face ever-growing generative models. Re-training a deployed detection network with existing and new model-generated images is possible, but the cost is too high in practice. A feasible solution is to use new model-generated images to fine-tune deployed detection networks. We experimented with this scenario to study whether this learning paradigm works.

In our experiment, we used the generated image detection network proposed by [10], which uses ResNet50 as the backbone. 5000 images from each generative model were used for fine-tuning, and 1000 per model were used for testing. The network was first trained with 5000 images from StyleGAN-DB. Then, images from datasets of StyleGAN2, CycleGAN, BigGAN, StarGAN, HiSD, Glow, and Latent Diffusion Models were used to fine-tune the network sequentially. After the initial training and each fine-tuning, we tested the network's detection performance on every generative model. Results are in Table I.

We can see that as the new model-generated images are used to fine-tune the network, it tends to favor the newly seen generative model, while the detection accuracy of the previously seen generative models decreases, resulting in catastrophic forgetting [48]. This experimental result is also consistent with our viewpoint depicted in Fig. 2 that simply increasing the type of training set does not maintain or improve the test accuracy. Therefore, fine-tuning with new model-generated images does poorly and does not improve the model's generalization ability.