Fake-image detection with Robust Hashing
^*

Fake images are manually generated by using image editing tools such as Photoshop. Splicing, copy-move, and deletion are also carried out under the use of such a tool. Similarly, resizing, rotating, blurring, and changing the color of an image can be manually carried out.

In addition, recent rapid advances in deep image synthesis techniques such as GANs have automatically generated fake images. CycleGAN[10] and StarGAN[11] are typical image synthesis techniques with GANs. CycleGAN is a GAN that performs one-to-one transformations, e.g. changing apples to oranges, while StarGAN is a GAN that performs many-to-many transformations, such as changing a person’s facial expression or hair color (see Figs.1 and 3). Furthermore, fake videos created using deep learning are called Deepfake, and various tampering methods have emerged, such as those using autoencoders, Face2Face[12], FaceSwap[13], and so on.

Real-world fake images may include the influence of a number of manipulation techniques such as image compression, resizing, copy-move at the same time, even if fake-images are generated by using GANs. Therefore, we have to consider such conditions for detecting real-world fake images.

Image tampering has a longer history than that of deep learning. Fragile watermarking[14], detection of double JPEG compression with a statistical method[15][16], and use of PRNU (photo-response non-uniformity) patterns of each camera[17][18] have been proposed to detect such tampers. However, most of them do not suppose to detect fake images generated with GANs. Moreover, they cannot detect the difference between fake images and just manipulated ones such as resized images, which are not fake images in general.

With the development of deep learning, fake detection methods with deep leaning have been studied so far. The methods with deep learning do not employ a reference image or the features of a reference image to detect tamper ones. The methods also assume that images are generated by using a specific manipulation technique to detect unique features caused by the manipulation technique.

There are several detection methods with deep learning for detecting fake images generated with an image editing tool as Photoshop. Some of them focus on detecting the boundary between tampered regions and an original image[19][20][21]. Besides, a detection method[22] enables us to train a model without tamper images.

Most detection methods with deep learning have been proposed to detect fake images generated by using GANs. An image classifier trained only with ProGAN was shown to be effective in detecting images generated by other GAN models[23]. Various studies have focused on detecting checkerboard artifacts caused in both of two processes: forward propagation of upsampling layers and backpropagation of convolutional layers[24]. In this work, the spectrum of images is used as an input image in order to capture the checkerboard artifacts.

To detect fake videos called DeepFake, a number of detection methods have been investigated so far. Some methods attempt to detect failures in the generation of fake videos, in terms of poorly generated eyes and teeth[25], the frequency of blinking as a feature[26], and the correctness of facial landmarks[27] or head posture[28]. However, all of these methods have been pointed out to have problems in the robustness against the difference between training datasets and test data[1]. In addition, the conventional methods have not considered the robustness against the combination of various manipulations such as the combination of resizing and DeepFake.

Figure2 shows an overview of the proposed method. In the framework, robust hash value is computed from easy reference image by using a robust hash method, and stored in a database. Similar to reference images, a robust hash value is computed from a query one by using the same hash method. The hash value of the query is compared with those stored the database. Finally, the query image is judged whether it is real or fake in accordance with the distance between two hash values.

Various robust hashing methods have been proposed to retrieval similar images to a query one[29, 30]. In this paper, we apply the robust hashing method proposed by Li et al[29] for applying it to fake-image detection. This robust hashing enables us to robustly retrieve images, and has the following properties.

• •

  Resizing images to 128$\times$128 pixels prior to feature extraction.

• •

  Performing 5$\times$5-Gaussian low-pass filtering with a standard deviation of 1.

• •

  Using rich features extracted from spatial and chromatic characteristics.

• •

  Outputting a bit string with a length of 120 bits as a hash value.

In the method, the similarity is evaluated in accordance with the hamming distance between the hash string of a query image and that of each image in a database.

Let vectors $\boldsymbol{u}=\{u_{1},u_{2},\dots,u_{n}\}$ and $\boldsymbol{q}=\{q_{1},q_{2},\dots,q_{n}\}$, $u_{i},\ q_{i}\in\{0,1\}$ be the hash strings of reference image $U$ and query image $Q$, respectively. The hamming distance $d_{H}(\boldsymbol{u},\boldsymbol{q})$ between $U$ and $Q$ is given by:

where

To apply this similarity to fake-image detection, we introduce a threshold $d$ as follows.

where $\mathbb{U}$ is a set of reference images and $\mathbb{U}^{\prime}$ is the an of images generated with image manipulations from $\mathbb{U}$, which does not include fake images. According to eq. (5), $Q$ is judged whether it is a fake image or not.

In the experiment, four fake-image datasets: Image Manipulation Dataset[31], UADFV[26], CycleGAN[10], and StarGAN[11] were used. The details of datasets are shown in Table I (see Figs. 1 and 3). The datasets consist of pairs of a fake-image and the original one. JPEG compression with a quantization parameter of $Q_{J}=80$ was applied to all query images. $d=3$ was selected as threshold $d$ in accordance with the EER (Equal error rate) performance.

As one of the state-of-the-art fake detection methods, Wang’s method[23] was compared with the proposed one. Wang’s method was proposed for detecting images generated by using CNNs including various GAN models, where a classifier is trained by using ProGAN.

The performance of fake-image detection was evaluated by using AP (Average Precision) and Accuracy (fake), given by,

where $N_{Qf}$ is the number of fake query images, and $N_{tn}$ is the number of fake query ones that are correctly judged as fake images.

Table II shows experimental results under the use of the two detection methods. From the table, it is shown that the proposed method had a higher performance than Wang’s method in terms of both AP and Acc (fake). In addition, the performance of Wang’s method heavily decreased when using the image manipulation and UADFV datasets. The reason is that Wang’s method focuses on detecting fake images generated by using CNNs. The image manipulation dataset does not consist of images generated with GANs. In addition, although UADFV consists of images generated by using DeepFake, they have the influence of video compression.

JPEG compression with $Q_{J}=70$, resizing with a scale factor of 0.5, copy-move or splicing was applied to query images. Therefore, when query images were fake ones, the fake query ones included the effects of two manipulations at the same time.

Table III shows experimental results under the additional manipulation, where 50 fake images generated by using CycleGAN, in which horses were converted to zebras, were used (see Fig.1). The proposed method was confirmed to still maintain a high accuracy even under the additional manipulation. In contrast, Wang’s method suffered from the influence of the addition manipulation. In particular, for splicing and resizing, Wang’s method was affected by these operations. That is why the method assume that fake images are generated by using CNNs, to detect unique features caused by using CNNs. However, splicing and resizing don’t depend on CNNs, although CycleGAN includes CNNs.