Robustness of Practical Perceptual Hashing Algorithms to Hash-Evasion and Hash-Inversion Attacks

This study focuses exclusively on blackbox adversarial attacks, rather than whitebox attacks, to reflect the practical scenario where PHA implementations are proprietary. In many cases, the details of PHA architectures and model weights are not publicly available due to their commercial nature. For example, Microsoft enforces non-disclosure agreements for PhotoDNA users, providing only an emulation of the algorithm. Similarly, although [22] released weights for the first-generation NeuralHash, Apple has since updated the algorithm, and the details of the latest version remain confidential.

Consider the original image ${\bf I}_{0}$ with the hash value ${\bf h}_{0}={\cal H}({\bf I}_{0})$. The objective of the client adversary is to generate a very similar image ${\bf I}_{\rm adv}$ whose new hash ${\bf h}_{\rm adv}={\cal H}({\bf I}_{\rm adv})$ is sufficiently different from ${\bf h}_{0}$ in untargeted attacks or sufficiently similar to some target hash ${\bf h}_{\rm tar}$ in targeted attacks. We use normalized Hamming distance $D_{h}({\bf h}_{0},{\bf h}_{\rm adv})$ to evaluate the similarity of two hashes, and use per-pixel normalized $L_{2}$ norm $L_{2}({\bf I}_{0},{\bf I}_{\rm adv})$ to evaluate the similarity of two images.

Because the output of PHAs is hash bits, not logits or class decisions, we propose a joint soft-label hard-label attack (JSHA) algorithm that can drastically improve blackbox attack query efficiency. JSHA consists of two steps. The first step is to apply a soft-label blackbox attack to generate an initial adversarial image ${\bf I}_{\rm init}$ so that its hash ${\bf h}_{\rm init}={\cal H}({\bf I}_{\rm init})$ is far away from ${\bf h}_{0}$ for untargeted attacks or close to ${\bf h}_{\rm tar}$ for targeted attacks. This step is formulated as the optimization

The optimization is conducted iteratively. In each iteration, the adversary queries the PHAs with its adversarial input ${\bf I}_{0}+\delta$ to get the hash, and uses the hash to optimize the loss. After reaching the query number threshold or the hash distance threshold $D_{0}$, this first step stops with output ${\bf I}_{\rm init}={\bf I}_{0}+\delta$.

The second step applies a hard-label blackbox attack to reduce the image distortion while maintaining the hash distance obtained in the first step. Specifically, this step solves the following optimization

Starting from ${\bf I}_{\rm adv}={\bf I}_{\rm init}$, the adversary queries the PHAs to get the hash and uses the queried hash to optimize (2) until the query number threshold or the threshold of $L_{2}({\bf I}_{0},{\bf I}_{\rm adv})$ is reached.

One of major advantages of this algorithm is high query efficiency. The first step is an unconstrained optimization and a large learning rate can be applied to get an initial adversarial sample quickly. In the second step, since we start from ${\bf I}_{\rm init}$ which is only a noisy version of ${\bf I}_{0}$, the convergence is fast.

GAN (generative adversarial network) was trained in [12] to invert CelebA images. However, the adversary may need more data-efficient hash-inversion attack algorithms because the training data may not be abundant. For this, we has developed new and more data-efficient generative models to reconstruct the original input images from the hashes. We designed a decoder-style deep model, similar to the Fast Style Transfer [23], to convert the $N$-bit hashes to $(64,64)$ images. This model is mainly a combination of residual blocks depicted in Fig. 1(b), with fractionally-strided convolutions/deconvolutions for learned upsampling. The entire architecture is described in Fig. 1(a), where the left network is for the MNIST and CelebA dataset while the right network is used for the STL-10 dataset.

For a given training dataset, we first compute the hashes of each image offline and store the image-hash pairs. We then train the model outlined above to reconstruct the images as closely as possible by minimizing the mean-square-error (MSE) loss between the model output and the image from which the hash was generated.

The adversaries rely on gradient estimation to solve the optimization problems. It is well known that hash bits of PHAs change slightly and randomly with respect to slight changes in input images. Based on existing work on random perturbation defense [24][25], we have the following proposition:

Proposition 1. Gradient estimation based on queried hash is not reliable due to the random variation of hash bits.

An outline of the proof is in the Appendix A.1. The random variation of hash bits makes the adversaries’ optimization struggle to converge, which will be demonstrated in the next section. Furthermore, considering this observation, we propose a defense method where the PHAs randomly invert $q$-portion of the hash bits to guarantee sufficient hash randomness, where $0\leq q\leq 1$.

We use the attack success rate for assessing the robustness against blackbox adversarial hash-evasion attacks. Given a set of $M$ original images ${\bf I}_{0}^{m}$ with their corresponding hashes ${\bf h}_{0}^{m}$, the adversary generates adversarial images ${\bf I}_{\rm adv}^{m}$ with hashes ${\bf h}_{\rm adv}^{m}$. The attack success rate is defined as

where $\mathbb{I}(\cdot)$ is the indicator function, $p\in[0,1]$ is the pre-set hash distance threshold and $\theta$ is the pre-set image distortion threshold. For the operator “$\gtrless$”, $>$ is used for untargeted attacks, $<$ is for targeted attacks. Small or zero ASR($p$) means a PHA is robust to an attack.

For the thresholds, we set $\theta=0.1$. On the other hand, we should apply large enough $p$ to evaluate untargeted attacks because ${\bf h}_{\rm adv}$ should be different from ${\bf h}_{0}$. It was found in [11] that most pairs of distinct images had $D_{h}({\bf h}_{0},{\bf h}_{1})>0.3$. Therefore, we primarily use $p=0.3$ in (3) to determine the success of untargeted attacks. In contrast, we should use small enough $p$ for targeted attacks to make sure ${\bf h}_{\rm adv}$ is sufficiently close to ${\bf h}_{\rm tar}$. Prior studies [11] indicate that many normal image edits would change an image’s hashes to the level of $D_{h}({\bf h}_{0},{\bf h}_{1})<0.1$. Therefore, we mainly use $p=0.1$ in (3) for assessing the success of targeted attacks. This important parameter $p$ was unfortunately neglected by most existing adversarial robustness studies.

For hash-inversion attacks, we use the $L_{2}$ distortion, the SSIM (Structural Similarity Index Measure) [26], and the LPIPS (Learned Perceptual Image Patch Similarity) [27] between the reconstructed image and the true image to evaluate the performance. No ASR or query budget is involved.

In implementing our proposed JSHA algorithm, we modified three different soft-label attack algorithms: SimBA [28], NES [29], and ZOsignSGD [30], to use in Step One, and modified the hard-label attack algorithm HSJA [31] to use in Step Two. For instance, “SimBA+HSJA” means our JSHA with a modified SimBA as Step One and a modified HSJA as Step Two. Modifications included removing/changing constraints and adjusting hyper-parameters, etc. We compared the proposed JSHA with the direct application of the original SimBA, NES, and ZOsignSGD algorithms. We also compared it with existing works [17] and [19].

The experiments were conducted on 100 randomly selected images from the ImageNet validation dataset. Step One was executed for a total of 3000 queries unless the early stopping criterion was met. Step Two was run for an additional 2000 queries unless meeting the early stopping criterion. The main challenge for us was the CPU-based non-parallel PHA executables, which limited the number of images or queries we could process. In comparison, [19] only utilized 30 images in experiments.

For targeted hash-evasion attacks, experiment results are presented in Table 1 and Appendix A.3. With a focus on ASR(0.1), we observed that all targeted attacks failed, yielding ASR$(0.1)=0$. To our knowledge, no literature has ever reported non-zero ASR$(0.1)$, suggesting that all the PHAs are robust against blackbox targeted hash-evasion attacks.

For untargeted attacks, experiment results are presented in Table 2 and Appendix A.4. With a focus on ASR(0.3), we see that PDQ was robust against all adversarial attacks, with a maximum ASR$(0.3)$ of 1% only. It’s worth noting that although [17] achieved a much higher attack success rate against PDQ, it required significantly more queries. The results also demonstrated that our defense method was effective because it drastically reduced ASR(0.3) and made the PHAs robust.

Experiments were conducted over three datasets: MNIST, CelebA, and STL-10. For each hashing algorithm and dataset, we trained the inversion network depicted in Fig. 1 for 50 epochs, with a batch size of 64 and an initial learning rate of 0.005. We utilized the MSE loss function with the AdamW optimizer and the cosine annealing learning rate scheduler. While MNIST and CelebA models were relatively easy to train, we encountered substantial challenges when training STL-10 models. We had to degrade STL-10 images to black-and-white in order to get meaningful reconstructed images.

Some sample images are shown in Fig. 2 whereas major experiment results are left to Appendix A.5 to save space. In general, hash-inversion failed over STL-10 while some level of success could be seen in MNIST and CelebA over PhotoDNA hash only. The reason is that the MNIST and CelebA images have similar or regular formation. The reconstructed images tend to converge to such common formation when the hash is long enough. However, even in this case, the reconstructed CelebA images could not be used to recognize the original face.

The effectiveness of random hash perturbation defense can be clearly seen in Fig. 2(b). Even the slightest perturbation $q=0.1$ made the reconstructed image quite different from the original image.