PointDP: Diffusion-driven Purification against Adversarial Attacks on 3D Point Cloud Recognition

In this section, we briefly review the background of diffusion models in 3D vision tasks. As mentioned in § 1, diffusion models involve the forward and reverse processes.

Given a clean point cloud sampled from the unknown data distribution ${\bm{x}}(0)\sim q({\bm{x}})$, the forward process of the diffusion model leverages a fixed Markov chain to gradually add Gaussian noise to the clean point cloud ${\bm{x}}_{0}$ over a pre-defined $T$ time steps, resulting in a number of noisy point clouds ${\bm{x}}(1)$, ${\bm{x}}(2)$, …, ${\bm{x}}(T)$. Mathematically, the forward process is defined as

where $\beta(t)$ is a scheduling function of the added Gaussian noise (e.g., $\beta(t)=\beta_{\min}+(\beta_{\max}-\beta_{\min})t$).

The reverse process, in contrast, is trained to recover the diffused point cloud in an iterative manner. 3D Point clouds have less semantics than 2D images due to the lack of texture information. Therefore, point cloud diffusion models leverage a separate encoder $e$ to as a latent feature $z_{\bm{x}}=e({\bm{x}})$ as a condition to help recover the clean point cloud.

where $\mu_{\bm{\theta}}$ denotes the approximated mean value parameterized by a neural network, and ${\bm{z}}=e({\bm{x}}_{a})$. The training objective is to learn the variational bound of the negative log-likelihood luo2021diffusion . In practice, we jointly train the encoder $e$ with $\mu_{\bm{\theta}}$. Essentially, the sampling process is similar to the DDPM model dhariwal2021diffusion :

where $\alpha(t)=\prod_{s=1}^{t}\alpha(s)$. Point cloud diffusion models have recently achieved SOTA performance on generating and autoencoding 3D point clouds, which provides us with opportunities for adversarial point cloud purification.

Figure 1 illustrates the pipeline of PointDP. Nie et al. nie2022diffusion have shown that diffusion-driven purification is able to remove the adversarial effect for 2D images. As mentioned in § 2.1, conditional diffusion models were proposed in the 3D point cloud space. Specifically, we use the design in luo2021diffusion as the base model for the purification process in our study. Note that we do not aiming at designing new point cloud diffusion models, but instead propose a novel purification pipeline along with rigorous evaluation as our main contributions.

Let ${\bm{x}}_{a}$ be an adversarial example w.r.t. the pristine classifier $f$, we initialize the input of the forward diffusion process as ${\bm{x}}_{a}$, i.e., ${\bm{x}}(t=0)={\bm{x}}_{a}$. The forward diffusion process can be solved by Equation 4 from $t=0$ to $t=t^{*}$

where $\alpha(t)=e^{-\int_{0}^{t}\beta(s)ds}$, ${\bm{\epsilon}}\sim\mathcal{N}(0,\mathbf{I})$. We leverage Equation 3 to recover the clean point clouds. Equivalently, the reverse can be also solved by the SDE solver in nie2022diffusion , noted as: $\mathbf{sdeint}$:

where the six inputs are initial value, drift coefficient, diffusion coefficient, Wiener process, initial time, and end time nie2022diffusion .

where score function $s_{\bm{\theta}}$ is derived from $s_{\bm{\theta}}({\bm{x}},t)=-\frac{1}{\sqrt{1-\alpha(t)}}{\bm{\epsilon}}_{\bm{\theta}}({\bm{x}}(t),t,{\bm{z}})$ ho2020denoising .

Besides, the hyper-parameter $t*$ and $T$ trades off the denoising performance and efficiency. We empirically choose $t^{*}=10$ and $T=200$ in our study, which has shown satisfactory results in our evaluation (§ 4).

PointDP is a pre-processing module that purifies the adversarial perturbations. athalye2018obfuscated have shown that input transformation-based methods can be broken by specifically designed attacks. Therefore, it is essential to model the adaptive attacks on PointDP to demonstrate its lower-bound adversarial robustness. We thus formulate two types of adaptive attacks on PointDP.

Attack on Latent Feature. As PointDP utilizes conditional diffusion models for adversarial purification, the latent feature $z$ is a good candidate for adversaries to launch attacks. Concretely, adversaries can set the goal to maximize some distance metric $\mathcal{D}$ between the latent feature of the optimized adversarial examples and the oracle latent feature of clean inputs ${\bm{z}}_{\text{oracle}}$. Without loss of generality, the adaptive attacks can be formulated as:

where ${\bm{x}}_{s}$ denotes the adversarial examples from the $s$-th step, $\mathrm{Proj}$ is the function to project the adversarial examples to the pre-defined space $\mathcal{S}$, and $\alpha$ is the attack step size. We choose two distance metrics in our study, where the first one is the KL divergence goldberger2003efficient and the other is the the $\ell_{1}$ norm distance. In our evaluation (§ 4), we report the lowest accuracy achieved under attacks with two distance metrics.

Attack Using BPDA. We follow nie2022diffusion to formulate the adaptive attack as an augmented SDE process. We re-state the attack formulation as below. For the SDE in Equation 5, the augmented SDE that computes the gradient $\frac{\partial\mathcal{L}}{\partial{{\bm{x}}}(t^{*})}$ of backward propagating through it is given by:

where $\frac{\partial\mathcal{L}}{\partial\hat{{\bm{x}}}(0)}$ is the gradient of the objective $\mathcal{L}$ w.r.t. the output $\hat{{\bm{x}}}(0)$ of the SDE in Equatrion 5), and

where $\mathbf{1}_{d}$ and $\mathbf{0}_{d}$ denote the $d$-dimensional vectors of all ones and all zeros, respectively. Nie et al. nie2022diffusion have demonstrated that such approximation align well with the true gradient value. Therefore, we leverage this adaptive attack formulation for our evaluation.

2D computer vision has achieved stellar progress on architectural designs of convolutional neural networks he2016deep , followed by vision transformers dosovitskiy2020image . However, there is currently no consensus on the architecture of 3D perception models since there is no standard data format for 3D perception sun2022benchmarking . As raw data from both 3D scanners and triangular meshes can be efficiently transformed into point clouds, they are becoming the most often utilized data format in 3D perception. 3D networks at the early stage use dense voxel grids for perception wang2015voting ; maturana2015voxnet ; DeepSlidingShapes ; tchapmi_segcloud_3dv17 , which discretize a point cloud to voxel cells for classification, segmentation, and object detection. PointNet pioneered to leverage global pooling help achieve memory-efficient permutation invariance in an end-to-end manner. PointNet++ qi2017pointnet++ and DGCNN wang2019dynamic followed up to add sophisticated local clustering operations to advance the performance. Sparse tensors are the other direction in 3D network designs SubmanifoldSparseConvNet ; choy20194d to use 3D convolutions to improve 3D perception performance. PointCNN and RSCNN reformed the classic pyramid CNN to improve the local feature generation li2018pointcnn ; liu2019relation . PointConv and KPConv designed new convolution operation for point cloud learning wu2019pointconv ; thomas2019kpconv . PointTransformer and PCT advanced self-attention blocks in the 3D space and achieved good performance zhao2021point ; guo2020pct . Various novel local clustering operations xiang2021walk ; ma2022rethinking also show enhancements on the clean performance. In this work, we focus on PointNet, PointNet++, DGCNN, PCT, CurveNet, and PointMLP as our evaluation backbones since they are representative and widely used and achieve state-of-the-art results in point cloud recognition mn40 .

Adversarial attacks have become the main obstacle that hinder deep learning models from real-world deployments, especially in safety-critical applications eykholt2018robust ; sun2020lidar ; cao2019adversarial ; zhang2021emp ; Zhang_2022_CVPR . There are a lot of adversarial attacks proposed in the 2D space to break the various vision models carlini2017towards ; xiao2018generating ; yang2020patchattack ; xie2017adversarial ; huang2019universal ; huang2020universal ; xiao2018spatially ; sun2021certified . To fill this gap between standard and robust accuracies, many mitigation solutions have been studied and presented to improve the robustness against adversarial attacks yang2019me ; xu2017feature ; bafna2018thwarting ; papernot2016distillation ; meng2017magnet ; zhang2019towards ; xiao2018characterizing ; zhang2020robust ; xiao2019advit and 3D domains dong2020self ; zhou2019dup ; sun2020adversarial . However, most of them including adding randomization liu2019extending ; dhillon2018stochastic ; dong2020self , model distillation papernot2016distillation , adversarial detection meng2017magnet , and input transformation yang2019me ; xu2017feature ; papernot2017extending ; bafna2018thwarting ; zhou2019dup have been compromised by adaptive attacks sun2020adversarial ; tramer2020adaptive ; athalye2018obfuscated . Adversarial training (AT) madry2017towards ; goodfellow2014explaining ; wong2020fast ; shafahi2019adversarial , in contrast, delivered a more longstanding mitigation strategy xie2020smooth ; Xie2020Intriguing ; zhang2019theoretically . However, the robust accuracy achieved by AT is still not satisfactory enough to be used in practice. Most recently, Nie et al. proposed DiffPure nie2022diffusion that leverages diffusion models to defend against adversarial attacks, and following-up studies to extend it to certified defenses carlini2022certified .

Adversarial attacks and defenses also extend to 3D point clouds. Xiang et al. xiao2018generating first demonstrated that point cloud recognition models are vulnerable to adversarial attacks. They also introduced different threat models like point shifting and point adding attacks. Wen et al. wen2019geometry enhanced the loss function in C&W attack to achieve attacks with smaller perturbations and Hamdi et al. hamdi2020advpc presented transferable black-box attacks on point cloud recognition. Zhou et al. zhou2019dup and Donget al. dong2020self proposed to purify the adversarial point clouds by input transformation and adversarial detection. However, these methods have been successfully by sun2020adversarial through adaptive attacks. Moreover, Liu et al. liu2019extending made a preliminary investigation on extending countermeasures in the 2D space to defend against simple attacks like FGSM goodfellow2014explaining on point cloud data. Sun et al. sun2021adversarially conducted a more thorough study on the application of self-supervised learning in adversarial training for 3D point clodu recognition. Besides adversarial training, advanced purification methods IF-Defense wu2020if and LPC li2022robust were proposed to transform the adversarial examples to the clean manifold. In this work, we present PointDP, that utilizes 3D diffusion models to purify adversarial point clouds. We also demonstrate that standard adversarial training suffer from strong black-box attacks and SOTA purification methods (i.e., IF-Defense and LPC) are vulnerable to PGD-styled adversaries (§ 4.3).

Datasets and Network Architectures. We conduct all the experiments on the widely used ModelNet40 point cloud classification benchmark wu20153d , consisting of 12,311 CAD models from 40 artificial object categories. We adopt the official split with 9,843 samples for training and 2,468 for testing. We also uniformly sample 1024 points from the surface of each object and normalize them into an edge-length-2 cube, following most of the prior arts qi2017pointnet . As mentioned before, there are various backbones for 3D point cloud recognition in the literature. To demonstrate the universality of PointDP, we select six representative model architectures including PointNet qi2017pointnet , PointNet++ qi2017pointnet++ , DGCNN wang2019dynamic , PCT guo2020pct , CurveNet xiang2021walk , and PointMLP ma2022rethinking . These backbones either have representative designs (e.g., Transformer and MLP) or achieve SOTA performance on the ModelNet40 benchmark.

Adversarial Attacks. As briefly described in § 3.2, adversarial attacks could be roughly categorized into C&W- and PGD-styled attacks. C&W attacks involves the perturbation magnitude into the objective term of the optimization procedure, while PGD attacks set the perturbation magnitude as a firm constraint in the optimization procedure. Moreover, adversarial attacks by $\ell_{p}$ norm as the distance metric for the perturbation. Although a number of attacks measure Chamfer and Handoff “distances” in 3D point cloud xiang2019generating , they are not formal distance metrics as they do not satisfy the triangular inequality. Therefore, we still leverage $\ell_{2}$ and $\ell_{\infty}$, following most defense studies in both 2D and 3D vision tasks carlini2017towards ; sun2021adversarially . We also have designed adaptive attacks on our proposed method § 2.3. Besides naive C&W and PGD attacks, we leverage specific attacks designed to break the robustness of point cloud recognition such as $k$NN tsai2020robust and AdvPC hamdi2020advpc . We also apply strong adaptive AutoAttack croce2020reliable (i.e., APGD) in our evaluation. Moreover, we use SPSA uesato2018adversarial and Nattack li2019nattack as black-box adversaries, followed by the suggestion of Carlini et al. carlini2019evaluating . We also leverage EOT-AutoAttack. Point adding (PA) and dropping/detaching (PD) attacks are also evaluated in our study, followed by the setups in sun2021adversarially . We set the attack steps to 200 to maximize the adversarial capability and follow the settings in sun2021adversarially for other attack parameters by default.

Evaluation Metrics. We leverage two main metrics to evaluate the performance of our defense proposal, which are standard and robust accuracy. The standard accuracy measures the performance of the defense method on clean data, which is evaluated on the whole test set from ModelNet40. The robust accuracy measures the performance on adversarial examples generated by different attacks. Because of the high computational cost of applying adaptive and black-box attacks to our method, we evaluate robust accuracy for our defense on a fixed subset of 128 point clouds randomly sampled from the test set. Notably, robust accuracies of most baselines do not change much on the sampled subset, compared to the whole test set. We evaluate the robust accuracy on the whole test set for other adversarial attacks with acceptable overhead (e.g., C&W and PGD attacks).

Baseline. Without any defense applied to the original recognition models, the robust accuracy is mostly 0 for all models under $\ell_{2}$ and $\ell_{\infty}$ based attacks. DGCNN exceptionally achieves 64% on $\ell_{2}$-based PGD, AutoAttack, respectively, due to its dynamic clustering design, which adaptively discards outlier points. PA and PD are two weaker attacks and Table 1 presents the robust accuracy against these two attacks.

In this section, we first present the evaluation results of PointDPunder attacks on the plain models. We train the diffusion and 3D point cloud recognition models in a sequential order. Table 2 presents the detailed results of PointDP against attacks on six models. We find that PointDP overall achieves satisfactory results across all models and attacks. The average robust accuracy against adversarial attacks is above 75%. We observe a drop on the clean accuracy for the chosen models, which is expected. As mentioned before, diffusion models for 3D point cloud is a more difficult task than 2D image diffusion, which may lead to partial semantic loss. The average drop of standard accuracy is 4.9%. We find that DGCNN still achieves the best robustness combined with PointDP, which has a 79.9% of robust accuracy. We further compare the performance of PointDPwith adversarial training, IF-Defense, and LPC in the next section.

In this section, we demonstrate how lately proposed defense solutions fail when encountered with stronger (adaptive) adversarial attacks on 3D point cloud recognition models.

Adversarial training (AT) has been applied to PointNet, DGCNN, and PCT with the help of self-supervised learning sun2021adversarially that achieves satisfactory robustness. Such observations are consistent with the performance of AT for 2D perception models. However, we find that AT is, in fact, a weak defense solution in 3D perception models. First, as acknowledged by sun2021adversarially , point cloud models (e.g., PointNet++ and CurveNet) often leverage different sampling strategies to select anchor points, like furthest point sampling (FPS). Such sampling involves high randomness. AT either cannot converge with different random seeds in each iteration or overfits to a single random seed. Therefore, AT cannot fit these models. Moreover, we discover that the $k$NN layers will cause severe gradient obfuscation in point cloud models as well. Different from 2D models that are almost fully differentiable, except for the max pooling layer. As shown in Figure 3, $k$NN essentially applies top-$k$. Therefore, gradient backward propagation through $k$NN layers is indexing, which is not non-smooth. The heavy usage of $k$NN layers in DGCNN and PCT will drastically hinder the gradient flow. As mentioned in § 4.1, we exploit black-box SPSA and Nattack to validate our findings. Table 3 presents the results of AT. SPSA and Nattack can greatly lower the average robust accuracy (7.8%) than white-box attacks (55.6%) on DGCNN and PCT, which confirms the effect of gradient obfuscation. PointNet, however, achieves better robustness under black-box attacks because it only has one max pooling layer and does not employ $k$NN layers.

Existing purification-based defenses against 3D adversarial point clouds mainly leverage C&W-styled attacks in their evaluation. C&W attacks utilize the method of Lagrange multipliers to find tractable adversarial examples while minimizing the magnitudes of the perturbation. From the perspective of adversary, such attacks are desirable due to their stealthiness, while this does not hold from a defensive view. Defense methods should be evaluated against strong adaptive attacks carlini2019evaluating . IF-Defense and LPC are the SOTA adversarial purification methods for 3D point cloud models. We leverage PGD and AdvPC attacks, which assign constant adversarial budget in the adversarial optimization stage. We follow the original setups of IF-Defense and LPC in our study. Such evaluation is stronger than C&W attacks, while we note that they are not strict adaptive attacks since the adversarial target is still the classifier itself. Similar to PointDP , IF-Defense can be pre-pended to any point cloud classifier, but LPC uses a specific backbone. Table 4 presents the detailed evaluation results of IF-Defense under various settings and attacks. We find that PointDP achieves much better robustness than IF-Defense, which is on average an 12.6% improvements. However, IF-Defense achieves slightly higher clean accuracy (4.9%). This is because IF-Defense leverages SOR to smooth the point cloud zhou2019dup . However, such an operation has been demonstrated to be vulnerable sun2020adversarial . With specific adaptive attacks, there will be a even larger drop of robust accuracy for IF-Defense.

Figure 2 shows the comparison among PointDP and existing methods. PointDP overall achieves the best performance than prior arts, which are 12.6% and 40.3% improvements than IF-Defense and LPC, respectively. We find that even without adaptive attacks, adversaries with constant budgets can already hurt the robust accuracy by a significant margin. This suggests that IF-Defense and LPC fail to deliver strong robustness to 3D point cloud recognition models. Especially, LPC appears in the proceedings of CVPR 2022, but actually achieves trivial robustness, indicating that a rigorous benchmarking is highly required in this community.

We have so far illustrated that state-of-the-art defenses can be easily broken by (adaptive) adversarial attacks and PointDP consistently achieves the best robustness. In this section, we further extensively evaluate the robustness of PointDP on even stronger adaptive attacks to demonstrate the actual robustness realized by PointDP. As mentions in § 4.1, we leverage two types of adaptive attacks in our study, and Table 5 presents their results. We also leverage black-box SPSA and Nattack to validate our results. We find that BPDA-PGD the strongest adaptive attacks, which align well with previous study on 2D diffusion-driven purification nie2022diffusion . Even though with strong adaptive attacks, PointDP still achieves much better robustness. Besides, black-box attacks are much less effective. Although we admit that PointDP still relies on gradient obfuscation, the extremely high randomness will hinder the black-box adversaries finding correct gradients.