Analytic-Splatting: Anti-Aliased 3D Gaussian Splatting via Analytic Integration

Before describing our proposed method, we first revisit the example of the integrated response of a one-dimensional Gaussian Signal within a window area for better understanding. Given a signal $g(x)$ and a window area $[x_{1},x_{2}]$, we aim to get the response by integrating the signal $\mathcal{I}_{g}=\int^{x_{2}}_{x_{1}}g(x)\text{d}x$ within this domain as shown in Fig. 2(a). For an unknown signal, Monte Carlo sampling within the window area is a feasible approach to approximate the integral as demonstrated in Fig. 2(b) and Fig. 2(c), and the approximation result $\mathcal{I}_{g}\approx\frac{x_{2}-x_{1}}{N}\sum^{N}_{i=1}g(x_{i}),\ x_{i}\in[x_{1},x_{2}]$ will be more accurate as the number of samples $N$ increases. Nonetheless, increasing the number of samples (i.e., super sampling) increases the computation burden significantly.

Fortunately, our goal in the 3DGS framework is to obtain the intensity response of the Gaussian signal within the window area $[x_{1},x_{2}]$. Given that the Gaussian signal is a continuous real-valued function, it is natural to derive an analytical approximation to the Gaussian definite integral (Fig. 2(a)) which is more accurate compared to the numerical integration (Fig. 2(b) and Fig. 2(c)). For instance, in Mip-Splatting, the window area is treated as a Gaussian kernel $g_{\text{w}}$, and the integral is approximated as the result of sampling after convolving the Gaussian signal with the Gaussian kernel $\mathcal{I}_{g}\approx g\circledast g_{\text{w}}$ ²²2the standard deviation $\sigma$ in $g_{\text{w}}$ is set to $0.1$. Note that the result of convolving a Gaussian signal with a Gaussian kernel is still a Gaussian signal as shown in Fig. 2(d). While this prefiltering uses the great convolution properties of Gaussian signals, this approximation will introduce a large gap when the Gaussian signal $g$ mainly consists of high-frequency components (i.e., with small standard variation $\sigma$).

To overcome these shortcomings, we are motivated to calculate the integral within the window area analytically. Specifically, the problem of evaluating the definite integral within $[x_{1},x_{2}]$ can be simplified to the subtraction of two improper integrals by applying the first part of the fundamental theorem of calculus. Let $G(x)$ be the cumulative distribution function (CDF) of the standard Gaussian distribution $g(x)$ defined by,

and the definite integral of $g(x)$ within $[x_{1},x_{2}]$ can be expressed as,

However, this CDF of the Gaussian function (defined as the error function erf) is not closed-form. We start by approximating the CDF $G(x)$ and find an important corollary, i.e.,

This analytic approximation also contains similar properties of the CDF $G(x)$:

1. 1.

   $S(x)$ is non-decreasing and right-continuous, satisfying

   $$\lim_{x\rightarrow-\infty}G(x)=\lim_{x\rightarrow-\infty}S(x)=0\quad\text{and}\quad\lim_{x\rightarrow+\infty}G(x)=\lim_{x\rightarrow+\infty}S(x)=1.$$

2. 2.

   The curve of $S(x)$ has 2-fold rotational symmetry around the point $(0,\nicefrac{{1}}{{2}})$,

   $$G(x)+G(-x)=S(x)+S(-x)=1,\ \forall x\in\mathbb{R}.$$

For any Gaussian signals with different standard deviations, we can approximate their CDFs by scaling $x$ in Eq. 9 by the reciprocals of their standard deviations. Once $x$ in $S(x)$ scales by the reciprocal of $\sigma$, we express it as $S_{\sigma}(x)$. For more details, please refer to the supplementary material.

In summary, given the sample $u$ and setting the window area as $1$, the integral $\mathcal{I}_{g}(u)$ of Gaussian signal $g(x)$ within the area $[u-\frac{1}{2},u+\frac{1}{2}]$ is defined as:

Besides, according to Eq. 9, we apply Eq. 9 and approximate the integral of Eq. 10 as:

In the following section, we will discuss how to employ the above approximation to 2D-pixel shading in Analytic-Splatting.

After revisiting the one-dimensional Gaussian signal integration, we tend to approximate the integral of projected 2D Gaussians within each pixel window area in Analytic-Splatting. Mathematically, we replace the sampled $g^{\text{2D}}(\bm{u})$ in Eq. 4 with the approximated integral $\mathcal{I}_{g}^{\text{2D}}(\bm{u})$. For the pixel $\bm{u}=[u_{x},u_{y}]^{\top}$ in 2D screen space, which corresponds to the window area $\Omega_{\bm{u}}$ as shown in Fig. 3(a), the integration of Gaussian signal in Eq. 3 is represented as:

Notably, handling the correlation term in this integral is intractable. To unravel the correlation term and thus feasibly solve the integral, we diagonalize the covariance matrix $\hat{\bm{\Sigma}}$ of the 2D Gaussian $g^{\text{2D}}$ and slightly rotate the integration domain as shown in Fig. 3(b), thus approximating the integral by multiplying two independent 1D Gaussian integrals.

In detail, we first perform eigenvalue decomposition on the covariance matrix $\hat{\bm{\Sigma}}$ (refer to Eq. 5) to obtain eigenvalues $\{\lambda_{1},\lambda_{2}\}$ and the corresponding eigenvectors $\{\bm{v}_{1},\bm{v}_{2}\}$. After diagonalization, for better description, we take $\hat{\bm{\mu}}=[\hat{\mu}_{x},\hat{\mu}_{y}]^{\top}$ (the mean vector of $g^{\text{2D}}$) as the origin and the eigenvectors $\{\bm{v}_{1},\bm{v}_{2}\}$ as the axis to construct a new coordinate system.

In this coordinate system, given a pixel $\bm{u}=[u_{x},u_{v}]^{\top}$, we rewrite the $g^{\text{2D}}$ in Eq. 6 as the multiplication of two independent 1D Gaussians:

where $\tilde{\bm{u}}=[\tilde{u}_{x},\tilde{u}_{y}]^{\top}$ denotes the diagonalized coordinate of the pixel center. After diagonalization, we further rotate the pixel integration domain $\Omega_{\bm{u}}$ along the pixel center to align it with the eigenvectors and get $\tilde{\Omega}_{\bm{u}}$ for approximating the integral. Thus the integral in Eq. 12 can be approximated as:

where $\sigma_{\ast}$ subscripts of $G$ and $S$ respectively correspond to Gaussian signals with the standard deviation $\sigma_{\ast}$. $\sigma_{1}=\sqrt{\lambda_{1}}$ and $\sigma_{2}=\sqrt{\lambda_{2}}$ denote the standard derivations of the independent Gaussian signals along two eigenvectors respectively. In summary, the volume shading in Analytic-Splatting is given by:

In this section, we first comprehensively explore the approximation errors in our scheme and then conduct an elaborate comparison against other schemes. It is worth noting that during training, the pruning and densification schemes proposed in 3DGS tend to maintain the standard deviations of Gaussian signals within an appropriate range (i.e. $\sigma\in[0.3,\ 6.6]$), and each Gaussian signal only responds to pixels within the $99\%$ confidence interval (i.e. $|x|<3\sigma$) for shading. Therefore, we only consider the Gaussian signals with standard deviations in $[0.3,\ 6.6]$, and merely discuss the approximation error of pixels within the $99\%$ confidence interval.

Referring to Eq. 7 and Eq. 9, we get the approximation error function about the CDF of the Gaussian function:

and referring to Eq. 10 and Eq. 11, the approximation error function regarding the 1-width window area integral response is:

the approximation error $\mathcal{E}_{\text{CDF}}$ and $\mathcal{E}_{\text{Int}}$ referring to different standard deviations are shown in Fig. 4(a) and Fig. 4(b) respectively. ³³3Since Eq. 16 and Eq. 17 are even functions, we show the results for the positive semi-axis over $0\leq x\leq 6$ in Fig. 4. Please note that the errors shown in Fig. 4 are scaled by a tiny factor of $1\mathrm{e}{-4}$.

For approximating the integral response over the 1-width window area, our scheme significantly outperforms other schemes. Fig. 5(a) and Fig. 5(b) respectively show that one-dimensional approximation error with different standard deviations and variable distributions. Our scheme is robust to these two conditions, especially when the standard deviations of Gaussian signals become smaller, our advantage becomes more obvious, which means that our scheme is better at capturing the high-frequency signals (i.e. details) of the scene, and our subsequent experimental results also verify this.

Last but not least, we employ this scheme to approximate the window integral responses of two-dimensional Gaussian signals, which requires rotating the integration domain from $\Omega_{\bm{u}}$ to $\tilde{\Omega}_{\bm{u}}$ as shown in Fig. 3 and inevitably introduces additional errors. To study this error, we record the approximation errors caused by rotating the integration domain from $0^{\circ}$ to $45^{\circ}$ ⁴⁴4Since we always hold $\sigma_{1}\geq\sigma_{2}$ in practice, thus we only consider the approximation error caused by the rotation angle from $0^{\circ}$ to $45^{\circ}$. under different standard deviations of Gaussian signals and distributions, as shown in Fig. 5(c). Although the approximation error of our scheme slightly increases as the rotation angle becomes larger, our scheme still surpasses other schemes.

To verify the anti-aliasing capability and versatility of Analytic-Splatting, we conduct experiments against state-of-the-art methods under the multi-scale training & multi-scale testing (MTMT) setting on Blender Synthetic [23, 2] and Mip-NeRF 360 [3] datasets. We further evaluate the performance of 3DGS and its variants under the super-resolution setting.