Perceptual Quality Assessment of Colored 3D Point Clouds

Although a large-scale point cloud database with reliable subjective quality scores is as essential to PCQA study as the LIVE database [72] to the IQA research[73], acquiring high quality point clouds is not as easy as capturing high quality images with a modern camera. A summary of publicly-available point cloud databases for different purposes can be found in the Point Cloud Library (PCL) website [63]. Here we are focused on point cloud databases for quality assessment, i.e. PCQA databases. Since 1994, Turk and Levoy [66] started to build the Stanford 3D scanning repository, in which some representative point clouds, such as “Stanford Bunny”, are still used in recent researches [14, 15, 9]. Later, MPEG point cloud database [64] and JPEG Pleno database [65] collect more point clouds from multiple resources, whose contents include cultural heritages, computer-generated objects, human figures, etc. The three point cloud databases provide a good number of visual stimuli for a series of subjective studies on point cloud quality [24, 13, 14, 15, 9, 23, 34, 36, 55, 49, 45, 48, 68, 52]. In fact, most point clouds used in existing publicly-available PCQA databases [24, 13, 14, 15, 9, 23, 34, 36, 55, 49, 45, 48, 68, 52] are selected from the three point cloud databases. This makes existing PCQA databases suffer from similar shortcomings. First, most point clouds in the Stanford 3D scanning repository [66] are colorless, and thus cannot simulate distortions associated with color/attribute degradation. Second, the scanning process does not capture the aesthetic quality of the objects, especially for cultural heritages. Typical examples include the “RomanOilLight” [65] as shown in Fig. 1 (a) and the “Head” [64] as shown in Fig. 1 (b). Third, the perceptual quality of some point clouds are inferior due to scanning noise (“Statue_Klimt” [64] as shown in Fig. 1 (c)), and irregular edges (“Phil” [74] as shown in Fig. 1 (d)), etc; Fourth, many point clouds in the three databases are scanned only from the front side, and thus cannot be watched from many viewpoints. Fig. 1 (e) shows an example of such point clouds observed from invalid viewpoints. In contrast, the human figure point cloud [75] as shown in Fig. 1 (f) is a good source content for its excellent quality and integrity. Nevertheless, high quality point clouds like this are rarely found in existing PCQA databases [24, 13, 14, 15, 9, 23, 52]. Fifth, existing PCQA databases exhibit relatively low content diversity. We wish to address these limitations in the proposed WPC database by generating new high quality point cloud contents and introducing various distortion types.

The 3D nature and irregular data representation of point clouds also add new complications to the design of subjective test. We summarize all existing subjective tests [7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 67, 38, 36, 34, 39, 55, 45, 48, 56, 68, 52] on PCQA in Table II, including both publicly-available and privately-held ones. From the table, we find that existing subjective tests vary on the scoring methodology, the viewing display, the interaction method, and the rendering mode. Below are several important observations. First, compared with Absolute Category Rating (ACR) and Pairwise Comparison (PC), Double Stimulus Impairment Scale (DSIS) [76] is more often used. One possible reason is that most point clouds used in the subjective tests are cultural heritages or computer-generated objects, which are not common in daily life. Therefore, subjects may not be able to detect all the distortions, especially the color impairments [20], in the ACR method without a reference. Second, most subjective tests prefer 2D monitors to the more advanced 3D monitors and HMDs. We speculate that the latter devices cause more 3D dizziness and sickness that may impair participants’ perception of quality [77, 78]. Third, there are mainly two methods for subjects to interact with and view point clouds from 360 degrees. The first method lets the subjects interact freely with the displayed point cloud using a mouse/head-mounted display (HMD), while the other one rotates the point cloud along a predetermined virtual path so that subjects can observe the point cloud from various viewpoints and distances. Since existing subjective tests show little bias on the interaction method, implying that both methods work similarly well, the passive method has an edge in repeatability and reproducibility over the interactive one. Fourth, the point-based rendering method is overwhelmingly popular in existing subjective tests due to its simplicity and low computational complexity. While there already exist various subjective test settings in the literature, which setting is the best remains an open-ended question. In this paper, we will adopt the most common settings, as shown in the last line of Table II, to conduct the subjective test.

A handful of algorithms have been proposed for PCQA, which can be roughly categorized as follows. From the perspective of distortion type, objective PCQA models can be classed as geometry distortion metrics [25, 28, 29, 26, 16, 79, 30, 33, 39, 40] and geometry-plus-color distortion metrics [28, 29, 35, 37, 38, 47, 46, 55, 49, 44, 12, 21, 23, 32, 45, 50, 51, 52, 53, 54, 56, 57, 58, 59, 60]. From the perspective of feature extraction, objective PCQA models can be classed as point-based models [25, 28, 29, 16, 30, 33, 39, 40, 35, 37, 38, 47, 46, 55, 49, 44, 50, 51, 54, 56, 57, 59, 60] and projection-based models [12, 21, 23, 32, 45, 52, 53, 58].

More specifically, point-to-point and point-to-plane metrics, both adopted by the MPEG PCC group, assess geometric distortions by Euclidean distances or projected errors along normal vector directions [25, 28, 29]. In [16], angular similarity metrics estimate the geometric distortions by calculating the similarity of local surface approximations. To obtain the geometric distortion, the PC-MSDM model [30] computes curvature similarities between original and distorted PCs. However, because PC normals are very sensitive to different normal acquisition methods, normals in the above metrics [25, 28, 29, 16, 30] may lead to uncertain performance. The generalized Hausdorff distance metric [33] exploits multiple rankings to identify the best performing quality model in terms of correlation with subjective quality, overcoming the oversensitivity of the classical Hausdorff distance to outliers. In [39], a PSNR-based metric [25, 28, 29] is proposed by including a normalization factor that accounts for changes in the intrinsic PC resolution after rendering. In [29], PSNR-based methods are modified by a density coefficient determined by the coordinate peak and the rendering resolution. In [40], a point-to-distribution quality assessment model is proposed by exploiting the correspondence between a point and a distribution of points from a small PC region. The main idea in this paper is to characterize the PC surface through the covariance of points within some local region, which is not extremely influenced by the number of reconstructed points after decoding, but rather by a statistical characterization of the point locations.

Compared with geometry-only PCs, colored PCs have a broad range of applications. Many point-based metrics for colored PCs have emerged recently [28, 29, 35, 37, 38, 47, 46, 55, 44, 49, 42, 6, 54, 51, 50, 57, 59, 60]. In [28, 29], point-to-point PSNR on the Y component (MPEG PSNR_Y) is used to estimate texture distortion of colored PCs, though such a direct extension of PSNR inevitably inherits the widely-known disadvantages of PSNR. Similarity measures [69] proven to be effective in general image quality assessment are extended to PCQA [35, 37, 55]. In these methods, geometry-based, color-based, normal-based and curvature-based features are extracted from both reference and distorted PCs, then both geometry and color feature similarities are evaluated and combined to overall objective scores. In [38], color histograms and correlograms are used to estimate the impairment of a distorted PC with respect to its reference. Geometry-only and color-only approaches are then combined to get a rendering-independent objective PCQA metric. More recently, statistics of a variant of the Local Binary Pattern (LBP) [47, 46], Perceptual Color Distance Patterns (PCDP) [50] and Local Luminance Patterns (LLP) [51] descriptors are introduced to PCQA. In [54], the BitDance metric uses color and geometry texture descriptors. The proposed method first extracts the statistics of color and geometry information of the reference and test PCs. Then, it compares the color and geometry statistics and combines them to estimate the perceived quality of the test PC. The GraphSIM approach[44, 60] uses graph signal gradient as a quality index to evaluate PC distortions. Considering the visual masking effect of PC’s geometric information and the color perception of human eyes, the CPC-GSCT metric [49] uses geometric segmentation and color transformation respectively to construct geometric and color features and then to estimate the PC quality. Similarly, the BQE-CVP metric [57] uses geometric feature, color feature and joint featrue to develop a blind quality evaluator. In [42], a reduced reference PCQA metric is developed that extracts geometry-based, normal-based and luminance-based features from the reference PC. Such features are then transmitted alongside the content, and are employed at the receiver side and compared with the distorted PC. The best combination of the features is obtained through a linear optimization algorithm. In [6], two color features are proposed to estimate three content dependent parameters, and then a reduced reference PCQA model is established. The reason is that the content has a masking effect on the coding distortion that is consistent with the characteristics of the human visual system. That is to say, the parameters are highly content dependent.

In addition to the aforementioned point-based models, there are also many projection-based models [32, 12, 21, 23, 45, 52, 53, 58], such as projection-based PSNR (PSNRp) [32, 12], projection-based structural similarity (SSIMp) [69, 32, 12], projection-based multi-scale structural similarity (MS-SSIMp) [80, 32, 12] and projection-based pixel-domain visual information fidelity (VIFPp) [81, 32, 12]. Yang et al. [45] choose to project the 3D point cloud onto six perpendicular image planes of a cube for the color texture image and corresponding depth image, and aggregate image-based global and local features among all projected planes for a final objective index. Wu et al. [52] propose two projection-based objective quality evaluation methods: a weighted view projection based model and a patch projection based model. He et al. [53] propose a method combining colored texture and curvature projection. Specifically, the colored texture information and curvature of PC are projected onto 2D planes to extract texture and geometric statistical features, respectively, so as to characterize the texture and geometric distortion. However, these methods treat background padding pixels on projected image planes the same way as the foreground ones, leading to inferior quality prediction accuracy [71]. Alexiou et al. [21, 23] develop an additional algorithm to remove the influence of background pixels, but the model complexity increases and the robustness declines.

Motivated by the lack of source 3D point cloud, we gather a collection of objects with diverse geometric and textural complexity, including snacks, fruits, vegetables, office supplies, and containers, etc. The selected contents are moderate in size and are omni-directionally acquirable. Fig. 2 shows snapshots of the objects in our point cloud dataset. The 3D point clouds are constructed using the following steps.

• •

  Image acquisition: Image acquisition is conducted in standardized laboratory environment which has a normal lighting condition without reflecting ceiling walls and floor. A single-lens-reflex camera and a turntable are employed to take photos of an object from a variety of perspectives. A graph illustration of acquisition process is shown in Fig. 3, where each photo is placed at its capture position relative to the object in the center.

• •

  3D reconstruction: We apply image alignment, sparse point cloud reconstruction, dense point cloud reconstruction and point cloud merging to each sequence of images with Agisoft Photoscan [82]. The resulting point clouds are further refined by Screened Poisson Surface Reconstruction [83] and resampling using CloudCompare [84].

• •

  Normalization: Each point cloud is normalized to a unit-cube with a step size of 0.001, where duplicated points are removed [84]. Finally, 20 voxelized point clouds are generated where the number of points range between 400K to 3M, with an average of 1.35M points and a standard deviation of 656K, respectively. Detailed specifications are given in Table III.

To test the capability of PCQA models in real-world applications, we chose to distort the source point clouds with the following processes.

• •

  Downsampling: We apply octree-based downsampling [84] to the normalized point clouds. Each dimension is uniformly divided into $2^{N}$ intervals, where $N$ represents the octree level. Then points located in the same cube are merged into one node. In this study, we set $N$ to be 7, 8, and 9, respectively, to cover diverse spatial resolutions.

• •

  Gaussian noise contamination: White Gaussian noise is added independently to both geometry and texture elements with standard deviation of $\left\{0,2,4\right\}$ and $\left\{8,16,32\right\}$, respectively. Then both geometry and texture elements are rounded to the nearest integer, followed by points removal by Meshlab [85].

• •

  MPEG-PCC: In 2017, MPEG issued a call for proposals on PCC methods for International Organization for Standardization [62]. Two technologies were chosen as test models: G-PCC for static content and dynamically capturing, and V-PCC for dynamic content, respectively. In this work, G-PCC (Trisoup) reference codec [86] is employed to encode the original point clouds with ‘max_NodeSizeLog₂’ of $\left\{10\right\}$, ‘NodeSizeLog₂’ of $\left\{2,4,6\right\}$ and ‘rahtQuantizationStep’ of $\left\{64,128,256,512\right\}$, respectively. G-PCC (Octree) [87] employs downsampling method to encode the geometry information, and is thus not performed redundantly. We set the ‘quantizationSteps’ of texture encoding as $\left\{16,32,48,64\right\}$. V-PCC reference codec [88] is employed to encode the original point clouds at three ‘geometryQP’ values and three ‘textureQP’ values, ranging from 35-50 and 35-50, respectively, followed by duplicated points removal [85].

Eventually, 740 distorted point clouds are generated in total by 5 distortion generators from 20 original point clouds. In total, there are 760 original and distorted point clouds in the WPC database.

Sampled distortion patterns in the WPC database are shown in Fig. 4. It is interesting to observe that the distorted point cloud not only exhibits loss of texture information similar to 2D images such as blockiness and blur, but also novel geometric distortion types. Specifically, hollow is caused by point cloud downsampling, where the point density is not sufficient to cover the object surface. Holes and collapses arise from unsuccessful triangulations and inappropriate downsampling in G-PCC (Trisoup), respectively. Even when the triangulation is successful, geometry distortion may still appear as a consequence of ill-conditioned triangles. A sample case is given in the bottom right part of Fig. 4 (d). Moreover, a large ‘geometryQP’ in V-PCC potentially results in gaps and burrs. All these distortions are point cloud-specific, which create new challenges to objective PCQA models.

We choose passive watching instead of interactive watching for subjective tests because the latter creates large variations and inconsistencies in terms of the viewpoints and viewing time between subjects and viewing sessions. We employ Technicolor renderer [89] to render each point cloud to a video sequence. The rendering window, point size and point type parameters are set to 960$\times$960, 1 and ‘point’, respectively. A horizontal and a vertical circle both with a radius of 5,000 are selected successively as the virtual camera path with the center of circles at the geometry center of an object. The remaining parameters are set as default. These settings preserves detail information as much as possible while maintaining the original point clouds to be watertight. One viewpoint is generated every two degrees on these circles, resulting in 360 image frames for each point cloud. Each distorted clip is then concatenated horizontally with its pristine counterpart into a 10-second video sequence for presentation. A screenshot is shown in Fig. 5.

Our subjective testing environment is the same as that for image acquisition. All video sequences are displayed on a 23.6” LCD monitor at a resolution of 1920$\times$1080 with Truecolor (32bit) at 60 Hz. The monitor is calibrated in accordance with ITU-R Recommendation BT.500-13 [76]. Double-stimulus impairment scale (DSIS) methodology is applied in our subjective test [76]. Videos are displayed in random order using a customized graphical user interface, where subjective scores of individual viewers are recorded.

A total of 60 naïve subjects, including 32 males and 28 females aged between 21 and 40, participated in the subjective test. All the subjects have normal or corrected-to-normal vision, and viewed videos from a distance of twice the screen height. Before the testing session, a training session is performed during which 18 videos that are different from the videos in the testing session are shown to the subjects. The same methods are applied to generate videos used in both the training and testing sessions. Therefore, subjects knew what distortion types and levels to expect before the testing session, and thus the learning effects are kept minimal. Considering the limited subjective experiment capacity, we employed the following strategy. Each subject is assigned 10 objects in a circular fashion. Specifically, if subject i is assigned objects 1 to 10, then subject i + 1 watch objects 2 to 11. Each video is scored for 30 times, and 22,800 subjective ratings, including 600 scores for reference point clouds, are collected in total. For each subject, the whole study takes about 2 hours, which is divided into 4 sections with three 5-minute breaks in-between to minimize the influence of fatigue effect. For finer distinctions between ratings, 100-point continuous scale is utilized instead of a discrete 5-point ITU-R Absolute Category Scale (ACR).

After converting subjective scores to Z-scores, we apply the outlier removal scheme suggested in [76]. No outlier detection is conducted participant-wise. Then Z-scores are linearly rescaled to lie in the range of [0, 100]. Mean opinion score (MOS) for each distorted point cloud is calculated by averaging the re-scaled Z-scores from all valid subjects. The histograms for the MOS and the associated standard deviation are shown in Fig. 6, which demonstrates that the distorted point clouds span most of the quality range. Considering the MOS as the “ground truth”, the performance of individual subjects can be evaluated by calculating the correlation coefficient between individual subject ratings and MOS values for each source point cloud, and then averaging the correlation coefficients of all source point clouds. Pearson linear correlation coefficient (PLCC) and Spearman rank-order correlation coefficient (SRCC) are employed as the evaluation criteria. Both criteria range from 0 to 1, where higher values denote better performance. Fig. 7 depicts the mean and standard deviation of the results. We can see that most individual subjects perform quite consistently with relatively low variations for different source point cloud. The average performance across all individual subjects is also given in the rightmost columns of Fig. 7.

Using the aforementioned database, we test the performance of 13 PCQA models, which are selected to cover a wide range of design methodologies. These models are chosen for two reasons. Firstly, geometry distortion metrics except the MPEG metrics are not included for assessing colored point clouds. Secondly, algorithms not publicly available are not included. The models include point-wise models: 1) point-to-point mean squared error-based PSNR ($\operatorname{PSNR}_{p2po,M}$) [26, 27], 2) point-to-point Hausdorff distance-based PSNR ($\operatorname{PSNR}_{p2po,H}$) [26, 27], 3) point-to-plane mean squared error-based PSNR ($\operatorname{PSNR}_{p2pl,M}$) [26, 27], 4) point-to-plane Hausdorff distance-based PSNR ($\operatorname{PSNR}_{p2pl,H}$) [26, 27], 5) point-to-point PSNR on color component ($\operatorname{PSNR}_{Y}$) [29, 28], 6) PCM_RR [42], 7) PointSSIM [35], 8) PCQM [37], 9) GraghSIM [44]; and projection-based models: 10) projection-based PSNR ($\operatorname{PSNR}_{p}$) [12], 11) projection-based structural similarity ($\operatorname{SSIM}_{p}$) [12, 69], 12) projection-based multi-scale structural similarity ($\operatorname{MS-SSIM}_{p}$) [12, 80], and 13) projection-based pixel-domain visual information fidelity ($\operatorname{VIFP}_{p}$) [12, 81]. The implementation of all models are obtained from the original authors or their public websites.

We use PLCC, SRCC and RMSE between MOSs and model predictions as quantitative measures, and the test results are shown in Table IV, V and VI. We summarize the key observations as follows. First, it comes as no surprise that all geometry distortion models performs unfavorably to the geometry-plus-color PCQA model. Second, projection-based model, such as $\operatorname{VIFP}_{p}$, provide the most promising results so far. However, they often fall short in making a distinction of the perceptual importance between the background region and the regions corresponding to points in a 2D projection of a 3D point cloud. This suggests that more accurate objective PCQA models may be developed by removing the influence of background pixels and considering the perceptual importance of regions corresponding to the points. Third, even the best PCQA model only moderately correlates with human perception, leaving large space for improvement.

A point cloud can be omni-directionally inspected from a view-sphere at a given distance, while it is both cumbersome and unnecessary to use a large number of viewpoints when acquiring its 2D snapshots. Icosphere, a unit geodesic sphere created by subdividing a regular icosahedron with normalized vertices, are employed to generate viewpoints due to the uniform distribution of vertices [21, 90]. The number of vertices that can be generated is

where $l$ represents the subdivision level. For any point in a 3D point cloud, let $\textbf{p}=(\textbf{g}\ \textbf{c})$ be a 6 dimensional row vector where g and c contain its 3D coordinates $\left(g_{x}\ g_{y}\ g_{z}\right)$ and attributed color information $\left(c_{r}\ c_{g}\ c_{b}\right)$, respectively. We use a series of transformations to obtain the projected images.

Firstly, we translate a point cloud to align its geometric center to origin $\left(0\ 0\ 0\right)$. Specifically, for each p

where $\textbf{g}_{t}$ represents the 3D coordinates after translation, and $\textbf{t}_{r}$ represents the translation vector equalling the geometric center coordinates of its corresponding reference point cloud. The reason $\textbf{t}_{r}$ is used instead of the geometric center of a distorted point cloud is that geometric distortion may cause changes of the upper and lower bounds of 3D coordinates, leading to misalignment of the projected images.

Secondly, we rotate the point cloud to obtain a number of viewpoints. More specifically, let $\textbf{n}_{v}$, a 3 dimensional row vector, be the unit normal and $\textbf{n}_{z}$ be $\left(0\ 0\ 1\right)$, then the rotation vector $\left(\textbf{r}\ \theta\right)$ can be calculated as

and

where $\|\cdot\|$ denotes the $l^{2}$ norm of a vector, r is the rotation axis, and $\theta$ is the axial angle. The rotation matrix R is obtained using $\left(\textbf{r}\ \theta\right)$. Then we use R to calculate $\textbf{g}_{r}$, the 3D coordinates after rotation, for each p,

Thirdly, scaling transformation is applied to make 2D snapshots of all reference point clouds approximately watertight meanwhile keeping details as much as possible. For each p, this operation can be expressed as

where $\textbf{g}_{s}$ represents 3D coordinates after scaling and $s$ is a scaling factor. Since the values of the coordinates are rounded to integer numbers, for which the maximum rounding error is bounded by half of the pixel spacing, the default value of $s$ is set to 1/2. Empirically, we also find this leads to the best performance.

Fourthly, we use orthogonal projection [12] and rasterization to obtain a projected image. For each p, the projected coordinates are given by

where

represents orthogonal projection matrix. If there are multiple points occupying the same location $\left(\widetilde{g}_{x}\ \widetilde{g}_{y}\right)$, the point with the largest value of $g_{z}$ will be maintained. To implement rasterization, we put c of all maintained points into their corresponding projected positions on a projection plane filled with c of $\left(127\ 127\ 127\right)$, and we obtain a projected image denoted as I.

Finally, we perform the above operations on each distorted point cloud $PC_{dis}$ and its reference $PC_{ref}$ to obtain $\textbf{I}_{dis}\left(n\right)$ and $\textbf{I}_{ref}\left(n\right)$ at the $n$th viewpoint of icosphere, respectively.

$\textbf{I}_{dis}\left(n\right)$ and $\textbf{I}_{ref}\left(n\right)$ have identical background pixels, and thus, the similarity between $\textbf{I}_{dis}\left(n\right)$ and $\textbf{I}_{ref}\left(n\right)$ is larger than that of $PC_{dis}$ and $PC_{ref}$. Consequently, for better distinction, it is useful to remove the influence of background pixels. It is also worth noting that in a 2D projection of a 3D point cloud, the perceptual importance of different regions changes significantly over space. In particular, the background region contains no information about the point clouds, and the importance of the regions corresponding to the points in the cloud also varies. The principle of information content weighted pooling [71] provides an excellent framework to account for such variations in importance, as exemplified by Fig. 8. Therefore, we propose to assess the perceptual quality of $PC_{dis}$, by

where the IW-SSIM evaluations [71] between all pairs of $\textbf{I}_{ref}\left(n\right)$ and $\textbf{I}_{dis}\left(n\right)$ are averaged to an overall quality measure of $PC_{dis}$, and we name the proposed method IW-SSIM${}_{\emph{p}}$, where all default parameters of IW-SSIM [71] in still image quality assessment are inherited.

We validate the proposed IW-SSIM${}_{\emph{p}}$ model using the WPC database presented in Section 4 and compare its performance against existing objective PCQA models. Note that IW-SSIM${}_{\emph{p}}$ does not involve a training process and is independent of any existing PCQA databases including the WPC database. Tables IV, V and VI summarize the PLCC, SRCC and RMSE evaluation results. We find that evaluation results of IW-SSIM${}_{\emph{p}}$ when $Nv=12,42,162$ are very close to each other, while the computational complexity is proportional to $Nv$. Therefore we use $Nv=12$ in all results reported here. It can be seen that the proposed model delivers the best performance in predicting subjective quality of 3D point cloud not only on the whole database but also on almost every subset. In addition, its PLCC and SRCC performance is at the same level as compared to an average human subject as in Fig.7.

To ascertain that the improvement of the proposed model is statistically significant, we carried out a statistical significance analysis by following the approach introduced in[73]. First, a nonlinear regression function is applied to map the objective quality scores to predict the subjective scores. We observe that the prediction residuals all have zero-mean, and thus the model with lower variance is generally considered better than the one with higher variance. We conduct a hypothesis testing using F-statistics. Since the number of samples exceeds 50, the Gaussian assumption of the residuals approximately hold based on the central limit theorem[91]. The test statistic is the ratio of variances. The null hypothesis is that the prediction residuals from one quality model come from the same distribution and are statistically indistinguishable (with 95% confidence) from the residuals from another model. We compare every possible pairs of objective models. The results are summarized in Table VII, where a symbol “1” means that the row model performs significantly better than the column model, a symbol “0” means the opposite, and a symbol “-” indicates that the row and column models are statistically indistinguishable.

There are several useful findings from the statistical significance analysis. First, existing geometry distortion metrics are statistically indistinguishable from each other. Second, most geometry-plus-color distortion metrics are statistically better than geometry distortion metrics. Third, the proposed IW-SSIM${}_{\emph{p}}$ model is statistically better than all existing models.

Finally, to investigate the generalization potential of the proposed model, we performed a cross-database validation on popular databases and compared our metric with the well-known state-of-the-art GraphSIM [44], PointSSIM [35], PCQM [37] and PCM_RR [42]. Table VIII, IX and X depict the results of cross-database validation. We can draw the following conclusions. First, IW-SSIM${}_{\emph{p}}$ performs well on all databases except the IRPC database. Notice that it seems most of the PCQA models performs not very well on this database. Second, PCQM is the best metric for SJTU-PCQA database, meanwhile, GraghSIM has the best performance on the IRPC and M-PCCD database. Third, GraphSIM is a competitive model for IW-SSIM${}_{\emph{p}}$, however, its time complexity is too high in the actual experiment.