Camera Trace Erasing

We view a captured image ($im$ for short) as two parts. One is camera trace ($trs$ for short) and the other is content signal ($sig$ for short), which can be formulated as

The goal of a camera trace erasing method $F(\cdot)$ is to achieve $F(im)=sig$. As detailed in Sec. 2, existing methods cannot provide a suitable definition to either the $sig$ part or the $trs$ part for a single image.

To address this issue, we propose to take multiple images into consideration at the same time. Firstly, we define that camera trace should be a distinguishable part in an image. In other words, a certain kind of camera trace should be different from the other one, when these two images are captured by different types of cameras. From this definition, there shall exist a certain kind of similarity between images, which increases along with the decrease of the distinguishable part (i.e., camera trace) in each image. Taking two images ($im_{1}$ and $im_{2}$) as an example, we denote $\phi(\cdot,\cdot)$ as the similarity between two images. With these notations, we have an inequation as

when a trace erasing method $F_{1}(\cdot)$ is better than $F_{2}(\cdot)$.

Moreover, we define that camera trace should be the only distinguishable part, regardless of the content signal. Ideally, when $sig_{n}=F(im_{n}),trs_{n}=im_{n}-F(im_{n}),n=1,2$, we have an equation as

We use the above formulation to define camera trace and motivate our proposed trace erasing method.

Let $F_{\Theta}(\cdot)$ denote a parametric method for camera trace erasing, where $\Theta$ is the trainable parameters. In this paper, we adopt the CNN structure proposed in [6] as an embodiment of $F_{\Theta}(\cdot)$. It is worth mentioning that, the focus of this paper is not the network design. $F_{\Theta}(\cdot)$ can be implemented by other network structures.

For the setup of network, we adopt the Siamese architecture proposed in [9]. Taking two images as an example, we illustrate the Siamese architecture in Fig. 3, where $F_{\Theta}(\cdot)$ is duplicated in two branches with shared parameters. Such a design can be easily generalized to multiple images, by adding more branches. Following the name of architecture, we term our proposed method as Siamese Trace Erasing (SiamTE). To train the network, we propose a hybrid loss as detailed below and shown in Fig. 3.

Embedded similarity loss $L_{es}$. Motivated by inequation (2), we propose the embedded similarity loss to guide the network training. Inspired by deep metric learning [27], we adopt a learned metric to calculate the similarity between images, by embedding images to a trace-related space with $E(\cdot)$. In this embedding space, we calculate the normalized Euclidean distance between features to obtain the similarity. The calculation process of embedded similarity loss is summarized in Algorithm 1.

Truncated fidelity loss $L_{tf}$. Generally, the intensity of camera trace is limited compared to content signal since it is a by-product in imaging process. Motivated by this prior knowledge, we propose truncated fidelity loss to restrict the manipulation of a camera trace method $F_{\Theta}(\cdot)$. Specifically, we calculate the Manhattan distance between $im$ and $F_{\Theta}(im)$ to measure the degree of manipulation. To preserve essential manipulation while avoiding potential over-manipulation, we truncate the distance values below a threshold $T$ to zero, which can be formulated as

Cross identity loss $L_{ci}$. Motivated by equation (3), we propose cross identity loss for better disentanglement between camera trace and content signal. Suppose we have $G$ types of cameras, images captured from different types of cameras are grouped as network input $im_{n},n=1,2,...,G$. Let $trs_{m}$ denotes a camera trace produced by the $m^{th}$ camera. The cross identity loss aims to maximize the probability of synthetic images $sig_{n}+trs_{m},n=1,2,...,G(n\neq m)$ to be identified as the ones captured by the $m^{th}$ camera device. In our implementation, we combine an estimated trace $im_{m}-F_{\Theta}(im_{m})$ with signals from the other devices $F_{\Theta}(im_{n}),n=1,2,...,G(n\neq m)$, to obtain these synthetic images. Then, we forward them into a trained classifier $C(\cdot)$ to obtain feedback. The calculation process of cross identity loss is summarized in Algorithm 2.

In Algorithms 1 and  2, we adopt a cyclic shift operator $S(\cdot,k)$ with a step size of $k$, in order to calculate the one-to-one operation between multiple images in a parallel computing friendly way. We illustrate the operation of cyclic shift in Fig. 4. For the image origin classifier $C(\cdot)$, we train a ResNet [18] on KCMI+ (a dataset detailed in Sec. 4.1). The weights of convolutions in ResNet are initialized by an ImageNet pretrained model [35]. After training, we utilize the stacked convolutions in this network as the embedding function $E(\cdot)$ for $L_{es}$. Lastly, we linearly combine the above three loss functions to form the hybrid loss as $\lambda_{1}L_{es}+\lambda_{2}L_{tf}+\lambda_{3}L_{ci}$, where $\lambda_{n}(n=1,2,3)$ denote the weighting factor.

KCMI. Kaggle Camera Model Identification (KCMI) is a dataset proposed by IEEE’s Signal Processing Society [43]. In KCMI, $2,750$ images are captured with $10$ types of cameras. We separate $550$ images from it to build a test set KCMI-550, with $55$ images for each camera. For training and validation, we first retrieve and download additional $2,800$ images from Flickr, which are captured with the same $10$ types of cameras. Then, we combine them with the rest $2,200$ images in KCMI to build KCMI+, with $500$ images for each camera.

VISION. It is a large-scale dataset for source identification [42], in which images are captured by $30$ types of cameras. Among these cameras, $28$ are different from those in KCMI. We adopt $1,500$ images from VISION to build a test set VISION-1500, with $50$ images for each camera.

Settings of training. We adopt KCMI+ to train SiamTE. Images from KCMI+ are randomly copped into patches with a size of $336\times 336$. We randomly gather $4$ patches as a group (i.e., $G=4$), in which patches are cropped from images captured by different types of cameras. $64$ groups are randomly gathered into a mini-batch for the stochastic gradient descent. We adopt Adam [24] for training with the momentum factor set as $0.9$. In our implementations, we set $\lambda_{1}$:$\lambda_{2}$:$\lambda_{3}$ as $3$:$1000$:$1$ or $3$:$500$:$1$ for the hybrid loss. For the hyper-parameters, we set the margin $M=0.5$ and the threshold $T=3$.

Classification. In this task, given an input image, the forensic methods predict a camera type as output. We adopt two kinds of CNN-based classifiers proposed in [29] for evaluation, named as ResNet50 and DenseNet201 according to respective network structures. Since the input size of classification networks ($224\times 224$) is much smaller than that of a full image, we randomly crop $4$ patches in an image as its representative, and choose the majority prediction among $4$ patches as the final output. Since the classification accuracy varies with different cropped patches, we repeat each experiment $10$ times and report the averaged results. KCMI-550 is adopted for evaluation.

Clustering. Besides the classification task conducted on KCMI-550, we perform clustering on VISION-1500 to evaluate the generalization ability of our proposed method. Specifically, for a SiamTE trained on KCMI+, most of camera types on VISION-1500 are unseen¹¹1The unseen camera type is the one not contained in the training set (i.e., unknown to methods), but we still know its origin. during training. Thus, clustering is a more challenging task. We adopt the stacked convolutional layers in ResNet50 and DenseNet201 [29] for feature extraction, and perform K-means clustering on extracted features [3]. Since VISION-1500 contains $30$ types of cameras, we set the number of clustering center $K=30,60,$ and $90$ for evaluation, respectively. In order to provide quantitative results for clustering performance, we define an accuracy for clustering. Specifically, in a group of clustered images, we assign the majority camera type to them as predictions and compare these predictions with known image origins. Therefore, such a clustering accuracy will decrease when images with different camera types are wrongly clustered together. We randomly crop $128$ patches in an image as its representative and repeat each experiment $10$ times to obtain the averaged results.

Verification. Furthermore, we conduct evaluation in the verification task. Given two input images, the forensic method in this task predicts if they are captured by the same type of camera. We adopt KCMI+ to build fingerprints for each type of camera, by averaging the extracted noise residuals from images captured by the same type of camera. We adopt a hand-crafted method to extract the noise residual [14]. Then, we adopt Peak-to-Correlation Energy (PCE) [13] to measure the correlation between an image from KCMI-550 and its corresponding camera fingerprint. The higher PCE means a forensic method can identify the image origin with higher confidence [22].

Metrics for image assessment. In addition to the specific metrics for each task, we provide an auxiliary metric to measure the manipulation degree of a trace erasing method, by calculating the Manhattan ($L_{1}$) distance between input and output of the method. Generally, a large $L_{1}$ distance indicates that the input image has been heavily manipulated, which usually causes a destruction to content signal. While a small $L_{1}$ distance may indicate that a method fails to perform a valid manipulation on the input. Moreover, we conduct objective quality assessments using the non-reference metric NIQE [33] (a smaller value denotes a higher quality).

We conduct a comprehensive investigation on various existing anti-forensic methods for camera trace erasing, including median filter (MF), Gaussian filter (GF), JPEG compression (CP), gradient-based adversarial method (AD) [16], blind image deblocking (DB) [50], internal similarity based real-world denoising (DN-I) [48], and learning-based real-world denoising (DN-E) [6]. The number after MF and GF (i.e., $3$ and $5$) denotes the kernel size of filter, the number after CP (i.e., $30$, $40$, and $50$) denotes the quality factor, and the number after AD (i.e., $1$ and $2$) denotes the scale factor of adversarial dithering.

As described in Sec. 4.2, we adopt the $L_{1}$ distance to measure the degree of image manipulation. Taking the results of CP in Table 1 as an example, a better anti-forensic performance is achieved with a larger degree of manipulation (e.g., CP30 vs. CP50). However, such a performance improvement comes at the cost of more severe signal destruction (reflected by a larger NIQE value). Thus, to measure the efficiency of a camera trace erasing method, we need to consider the degree of manipulation at the same time. An efficient method should achieve good anti-forensic performance with as little manipulation as possible.

According to the quantitative results listed in Tables 1, 2, 3 and visual results shown in Figs. 5 and 6, we analyze the performance of above mentioned anti-forensic methods. Median filter and JPEG compression are effective to erase camera trace, yet the processed images suffer from blurring and blocking artifacts. Gaussian filter is less effective since it cannot significantly degrade the performance of forensic methods even at a large degree of manipulation. The deblocking method [50] cannot provide a valid manipulation to erase camera trace, since it can only remove the part of blocking artifact in camera trace. The overall performance of the two real-world image denoising methods [48, 6] is relatively better than other baseline methods, yet still has a notable gap to ours.

The gradient-based adversarial method is specially designed for CNN-based classifiers [16]. It thus achieves a satisfactory anti-forensic performance on CNN-based classifiers, which is on par with our proposed method (as listed in Table 1). However, the generalization ability of this adversarial method is poor. On one hand, the adversarial dithering is less effective than ours in the verification task, since a classic forensic method using hand-crafted features rather than a CNN-based one is adopted in this task (as listed in Table 3). On the other hand, the calculation of gradient is dependent on labels. Therefore, the adversarial method cannot generalize to images with unseen camera types, which makes it not capable of processing images in the clustering task, thus not listed in Table 2.

Compared with the above baseline methods, our proposed SiamTE is more efficient for camera trace erasing. Specifically, at the similar (or lower) degree of manipulation, SiamTE significantly reduces the classification accuracy of ResNet50 from $99.80$% to $20.42$%, as listed in Table 1. In the clustering task, the performance of K-means clustering is halved by SiamTE, as listed in Table 2. In the verification task, SiamTE achieves $87.2$% decrease on the correlation between an image and its camera type, as listed in Table 3. From the visual comparisons conducted in Figs. 5 and 6, the perceptual quality of our results are satisfactory, which is also verified by the lower NIQE values listed in Tables 1 and 2. The comprehensive experiments demonstrate a clear advantage of SiamTE for camera trace erasing over existing anti-forensic methods.

Our proposed hybrid loss consists of three parts: (a) embedded similarity $L_{es}$, truncated fidelity $L_{tf}$, and cross identity $L_{ci}$. In this section, we provide ablation study to demonstrate the function of each part. As listed in Table 4, each part of the hybrid loss has a contribution to the overall performance. Without $L_{es}$, the anti-forensic performance in terms of classification accuracy is significantly weakened. Without $L_{tf}$, the degree of manipulation loses control, which results in heavy destruction of content signal, as reflected by the large NIQE and $L_{1}$ distance. Without $L_{ci}$, unfavorable artifacts are introduced to the visual results, which degrade the image quality and may reveal the adversarial process, as shown in Fig. 7.

In this section, we analyze the extracted camera trace separately. With a trained SiamTE, we extract camera trace from KCMI+ and KCMI-550, respectively. We then train a DenseNet201 classifier on KCMI+ to identify origin from the extracted camera trace instead of the image itself. On KCMI-550, we achieve an accuracy of $93.21\pm 0.31$% with camera trace, which is close to that of $99.87\pm 0.12$% with original images. It verifies that the extracted camera trace contains most of the camera-distinguishable information from original images. We visualize the extracted camera trace in Fig. 8, which demonstrates that camera trace varies with different types of cameras. In comparison, patches in a single image have similar camera trace, as shown in Fig. 9.