Diverse Sample Generation: Pushing the Limit of Generative Data-free Quantization

Since the performance of existing data-driven quantization significantly surpasses that of data-free quantization in usual, the original training data (real data) is generally considered as the upper limit of the performance of synthetic data. So as Eq. (1) shows, methods constrained by $\mathcal{L}_{\textrm{BN}}$ are dedicated to generating samples by confining the feature statistics of synthetic data around BN statistics of the pre-trained model. However, the data from the real world is always more diverse, while that generated by fitting BN statistics suffers constrained features as shown in Fig. 2 and Fig. 2. The activation distribution of data generated in ZeroQ and GDFQ well fit the normal distribution of BN statistics while that of real data deviates from BN statistics. When we calculate the Wasserstein distance between the Gaussian distribution with BN statistics and the feature distribution of synthetic data, the index values of the existing methods are significantly smaller than the real data (real data 0.120 vs ZeroQ 0.040 in PTQ, and real data 0.134 vs GDFQ 0.116 in QAT). We consider it as distribution-level homogenization causing by the strict constraint applied to the synthesizing process makes the synthetic data overfit BN statistics in each layer.

Besides homogenization at the distribution level, synthetic data in existing generative data-free quantization also suffer the homogenization at the sample level.

From the statistical perspective, in existing generative data-free methods, such as ZeroQ and GDFQ, all samples are constrained by the identical form of BN statistics loss. So there might be little difference among statistics of synthetic samples in one batch, while samples from the real world are diverse. As shown in Fig. 2 and Fig. 2, the points of the mean and standard deviation of ZeroQ and GDFQ data are significantly overlapping and all points gather together closely near BN statistics. However, statistics of real data and our DSG data have larger variance and are more dispersed. Taking PTQ as an example, the variance of the statistics of the real sample is 0.029, that much larger than the variance of data generated by ZeroQ, which is 0.009.

From the spatial perspective, the synthetic samples generated in different ways have different behaviors. For existing data-free methods, all samples are synthesized by the same generation strategy. The samples generated by GDFQ are concentrated in spatial perspective, as shown in Fig. 2, while samples of our DSG and real data are more scattered. The heatmap shows the degree of aggregation in the synthetic data sample space. The densest cluster of GDFQ samples appears red in the heatmap and the density index is up to 2052 near the cluster center, while the highest density index of the real sample is only 1681. For PTQ, the index is 1902, and its homogenization phenomenon is not significant as QAT but is more severe than the real data. We regard these as sample-level homogenization from the statistical and spatial perspectives, which leads to a significant performance drop for the networks quantized by synthetic data.

In a nutshell, for the generation process in data-free quantization, the homogenization exists in two levels, i.e., distribution level and sample level. Thus the models quantized by these homogenized data can hardly achieve high accuracy. In this work, we devote diversify the synthetic data to improve the generative data-free quantization. We propose a generic data-free quantization scheme to tackle the homogenization problems by diversifying synthetic data from different perspectives, which we call Diverse Sample Generation (DSG) scheme. The synthetic data generated by our scheme enables the quantized neural network to achieve high accuracy.

To deal with the distribution-level homogenization, we propose Slack Distribution Alignment (SDA). Specifically, we apply the relaxation constants during the generation process for matching original BN statistics, which can be intuitively regarded as margins for means and standard deviation. With the relaxation constants, the distribution statistics of synthetic samples do not need to fit the BN statistics strictly, and thus the data can be more diverse in distribution. The loss term of SDA is $\mathbf{l}_{\textrm{SDA}}=[{l}_{\textrm{SDA}_{1}},{l}_{\textrm{SDA}_{2}},\dots,{l}_{\textrm{SDA}_{N}}]^{T}$, where $N$ is the number of BN layers in the quantized neural network, and the ${l}_{\textrm{SDA}_{i}}$ for $i$-th BN layer is expressed as follow:

where $\delta_{i}$ and $\gamma_{i}$ denote the relaxation constants for the mean and standard deviation statistics of features at the $i$-th BN layers, respectively. After introducing relaxation to the constraint between statistics of synthetic data and BN layers, the mean and standard deviation of generated samples can fluctuate within a certain range. Benefiting from the relaxed constraints, the synthetic samples are more diverse and their feature distribution behaves closer to that of real data.

As the ideal data to quantize networks, our experimental evidence in Fig. 2 shows that the feature statistics of real data still exist gaps compared with BN statistics. Thus, we attempt to approximate the gap between the statistics of real data and BN statistics of the original model as a reference to determine the value of $\delta_{i}$ and $\gamma_{i}$. For real-world data, when the number of samples achieves a huge amount, the components and features of inputs approximate the Gaussian assumption. Based on the central limit theorem, we suppose the whole potential input set is generally consistent with a Gaussian distribution.

Thus, we set the initial value of $\delta_{i}$ and $\gamma_{i}$ by a batch of samples that were randomly initialized from the Gaussian distribution: First, we input 1024 samples initialized by Gaussian distribution with $\mu=0$ and $\sigma=1$ into the full-precision network and then save the feature statistics at each BN layer, including the mean and standard deviation of the feature distribution. Then, we measure the margin between saved statistics and the corresponding BN statistics, then take percentiles of the margin to initialize $\delta_{i}$ and $\gamma_{i}$:

where $\epsilon\in(0,1]$ is a hyper-parameter determining the degree of relaxation, and $\boldsymbol{\tilde{\mu}}^{0}_{i}/\boldsymbol{\tilde{\sigma}}^{0}_{i}$ are mean/standard deviation of the feature of Gaussian initialized data $\mathbf{x}^{0}$ at the $i$-th BN layer. $|\boldsymbol{\tilde{\mu}}^{0}_{i}-\boldsymbol{\mu}_{i}|_{\epsilon}$ and $|\boldsymbol{\tilde{\sigma}}^{0}_{i}-\boldsymbol{\sigma}_{i}|_{\epsilon}$ denote the $\epsilon$ percentile of $|\boldsymbol{\tilde{\mu}}^{0}_{i}-\boldsymbol{\mu}_{i}|$ and $|\boldsymbol{\tilde{\sigma}}^{0}_{i}-\boldsymbol{\sigma}_{i}|$, respectively. Intuitively, the larger the value of $\epsilon$, the more relaxing the constraints are in Eq. (5). We empirically set the default value of $\epsilon$ as $0.9$ to deal with outliers.

For the generation processes in existing generative data-free quantization approaches, all synthetic samples are always generated by the same initialization and optimization strategy. Specifically, the samples share the same objective function that directly sums the loss of each layer, i.e., all the BN layers receive the same degree of attention from each sample, and thus results in homogenization among the synthetic samples. We propose Layerwise Sample Enhancement (LSE) to tackle sample-level homogenization. LSE enables each sample to focus on the statistics of different layers by enhancing the specific loss term of the sample in the optimization process so that the synthetic samples of our DSG in one batch could be diverse.

Given a well-trained full-precision network containing the $N$ number of BN layers, earlier generative data-free quantization methods treat each layer the same way without particular predilection or bias. But in LSE, we engineer $N$ different loss terms and apply each of them to the specific data sample. Here, we suppose to generate $N$ samples in one batch, equaling the number of BN layers of the quantized network. The loss term of LSE for the $i$-th sample is as follows:

where $\mathbf{1}$ denotes a column vector of all ones, $\mathbf{h}_{i}$ is an ${N}$-dim one-hot column vector where the $1$ is in the $i$-th position, $\mathbf{l}=[l_{1},l_{2},...,l_{N}]^{T}$ is the vector of the original loss terms for all BN layers. The set of loss functions for all samples can be expressed as $\mathbf{X}_{\textrm{LSE}}\mathbf{l}=[{l}_{\textrm{LSE}_{1}},{l}_{\textrm{LSE}_{2}},...,{l}_{\textrm{LSE}_{N}}]^{T}$, which is an $N$-dim column vector and its $i$-th element represents the loss function for $i$-th sample in the batch. Thus, for the whole set of samples $\mathbf{x}^{s}$, the effect exerted by LSE can be formulated as

where $\mathbf{X}_{\textrm{LSE}}$ can be seen as the enhancement matrix for loss terms. In this way, we focus on every BN layer of the original model respectively and generate corresponding samples for each layer. By such layerwise enhancement, we optimize the whole batch of synthetic samples simultaneously.

The above two proposed methods are based on the relaxation of generation constraints, however, the state of samples at the spatial level still cannot be perceived and adjusted directly during the generation process. Therefore, in addition to improving the loss function constraining the statistics of synthetic data, we construct the unsupervised Sample Correlation Inhibition (SCI) loss for the samples to sample-level diversify the synthetic data from the spatial perspective. SCI applies the determinant point process loss to spatially disperse the intermediate features in the generator, thereby inhibiting the correlation among these synthetic samples.

The determinant point process $\mathcal{P}$ can express negative interactions among samples by the similarity kernel $\mathbf{K}_{\mathbf{S}}$ of the elements in a set ${\mathbf{S}}$, where a large value of $\mathbf{K}_{ij};i,j\in{\mathbf{S}}$ reduces the likelihood of both elements to appear together in a diverse subset [3, 29]. The related works are widely used to generate or select diverse samples in some existing generation tasks for diversifying samples closer to the real data [13, 6, 30, 33]. However, real data is inaccessible in generative data-free quantization, and thereby it cannot be used to construct the constraint of the diversity of synthetic data.

Therefore, we utilize random samples in our SCI instead of the real data, and the random data is applied as a lower bound of diversity to alleviate the homogenization of synthetic samples. Inspired by Theorem 1, each dimension of these vectors is initialized by the uniform distribution $U[0,1)$ independently and randomly. The vectors initialized in this way are for uniforming approximately on spatial, which are fixed during the quantization process and expected to compose a base set that covers the potential input domain uniformly and is de-homogenized. In this way, the homogenization of synthetic data is limited to a level lower than its initial state.

So far we are able to build the complete flow of our SCI. The intermediate features correspond to the data $\mathbf{x}^{s}$ in the generation process is expressed as $\mathbf{f}=\{f_{1},f_{2},...,f_{N}\}$, where $f_{i}$ is the feature of the corresponding synthetic sample. And for the set of features $\mathbf{f}$, we construct a set of noise vectors $\mathbf{r}=\{r_{1},r_{2},...,r_{N}\}$ of the same size. Then we construct the similarity kernels $\mathbf{K}_{\mathbf{f}}$ and $\mathbf{K}_{\mathbf{r}}$ of the features $\mathbf{f}$ and noise vector set $\mathbf{r}$ as the following decomposition proposed in [13]: $\mathbf{K}_{\mathbf{f}}={\phi(\mathbf{f}})^{\top}\phi(\mathbf{f})$ and $\mathbf{K}_{\mathbf{r}}={\phi(\mathbf{r})}^{\top}\phi(\mathbf{r})$, where all $\phi({f}_{i})\in\phi(\mathbf{f})$ and $\phi({r}_{i})\in\phi(\mathbf{r})$ are the $\ell_{2}$ normalized vectors that guarantees the similarity kernels $\mathbf{K}_{\mathbf{f}}$ and $\mathbf{K}_{\mathbf{r}}$ to be real positive semidefinite matrices. The main idea of our SCI is to learn a similarity kernel $\mathbf{K}_{\mathbf{f}}$ of feature $\mathbf{f}$ and make it less than the similarity kernel $\mathbf{K}_{\mathbf{r}}$ of random data $\mathbf{r}$. In this way, we keep the correlation among the features of synthetic samples lower than that among random vectors, thus alleviating the homogenization of synthetic data. Since matching two similarity kernels directly is an unconstrained optimization problem [31], we construct the loss using the major characteristics: eigenvalues and eigenvectors of kernels. Hence, our SCI loss term is defined as follows:

where $\lambda_{\mathbf{f}}^{i}$ and $\lambda_{\mathbf{r}}^{i}$ are the $i^{\textrm{th}}$ eigenvalues of $\mathbf{K}_{\mathbf{f}}$ and $\mathbf{K}_{\mathbf{r}}$, respectively. $v_{\mathbf{f}}^{i}$ and $v_{\mathbf{r}}^{i}$ are the eigenvectors, and $\hat{\lambda}_{\mathbf{r}}^{i}$ is the min-max normalized version of the eigenvalues applied to alleviate the effect of noisy structures.

We first analyze the effectiveness of SDA. As discussed earlier, hyper-parameter $\epsilon$ determines the degree of relaxation in SDA. Therefore, in Fig. 5, we verify the impact of different values of $\epsilon$ in the interval $(0,1]$ with a moderate step size 0.1, and further add $\epsilon=0$ to serve as the vanilla case that the fitting of BN statistics is constrained without relaxation. In PTQ (Fig. 5(a)), when the $\epsilon$ varies from 0 to 0.9, the final performance increases gradually from 26.04% to 33.39%. In QAT (Fig. 5(b)), the accuracy also presents an overall upward trend (from 60.52% to 61.08%) as the $\epsilon$ varies from 0 to 0.9. However, when $\epsilon=1$, the accuracy drops more or less in both scenarios, which is 3.61% in PTQ and 0.02% in QAT. These phenomena forcefully prove that by way of relaxing the constraints, the SDA method brings generated data a certain offset when fitting the BN statistic distribution, which contributes to feature diversity and consequently improves the performance. And when $\epsilon$ is set to 1, the degree of slack reaches the limit as all the outliers in $\boldsymbol{\tilde{\mu}}^{0}_{i}$ and $\boldsymbol{\tilde{\sigma}}^{0}_{i}$ are taken into account, which means the feature distribution of generated data might be out of the reasonable range. Therefore, it might result in better feature diversity in synthetic data but would harm the final accuracy of the quantized network. Therefore, we set the default value of $\epsilon$ as 0.9 empirically to balance divergence and the impact of outliers.

As the results shown in TABLE I, the vanilla data-free PTQ method without SDA suffers a severe accuracy degradation by 7.35%, and the SDA also provides 0.56% accuracy promotion in data-free QAT (vanilla 60.52% vs. SDA 61.08%). The results reflect that SDA is essential. Compared to the other two parts of our scheme, SDA may provides a major contribution to the final performance.

The TABLE I also shows that LSE can improve the performance in both data-free QAT and PTQ. As the ablation results show, in PTQ, LSE method gives a non-negligible increment compared with ZeroQ by 1.08%. As for in QAT, it also helps the DSG scheme to have a slight improvement compared with the vanilla method by 0.19%.

Then we evaluate SCI in data-free quantization. The motivation of our SCI is to alleviate the sample-level homogenization from the spatial perspective. TABLE I shows the effects of using SCI. In QAT, compared with the vanilla method, integrating with SCI helps to acquire better accuracy, which is 0.40% higher. Also, in PTQ, the SCI method gains 13.49% accuracy compared with using pure random inputs.

Intuitively, these three techniques are motivated by different observations, and also the processes are carefully engineered that they hardly interfere with each other. In short, SDA and LSE improve the loss related to BN statistics, aiming to prevent the generated samples from overfitting to BN statistics and makes the samples focus on the statistics of different layers. While the SCI for data-free quantization constructs an unsupervised loss to constrain the features in the generator to holding distances among them, so that mitigate the sample homogenization. The results show that, in data-free QAT, DSG scheme equipped with these techniques obtains 1.66% improvement in total with ResNet18 under W4A4 setting, which is up to 62.18%. As for data-free PTQ, DSG can also boost the performance to 13.86% with all the techniques.

To evaluate the advantage of the proposed scheme in PTQ, we first compare our DSG against other data-free PTQ methods (ZeroQ [5], DFQ [38], ACIQ [1], MSE [7], KL [49], SQuant [19], and OCS [58]) on CIFAR10 and ImageNet datasets. Among these methods, ZeroQ and SQuant are typical generative data-free PTQ methods, which reconstruct synthetic data and calibrate the quantized network in different ways. Thus, we evaluate the generation performance of our DSG in PTQ with calibration methods from ZeroQ and SQuant, denoted as DSG¹ and DSG². Other quantization methods use weight equalization or analytical clip range to improve the network performance. Besides, we additionally compare our method with OCS, which is also a PTQ method but requires real data for calibration. We evaluate these methods on various bit-width settings, and the results on CIFAR10 and ImageNet dataset are shown as TABLE II and TABLE IV(b), respectively.

Specifically, on CIFAR10 [28] dataset, we evaluate our DSG with ResNet20 [24] and VGG16bn [48]. The results are shown in TABLE II. Under all settings on the CIFAR10 dataset, our DSG far exceeds existing SOTA methods. And we highlight that the quantized network calibrated with DSG data completely surpasses the network calibrated with real data on various settings. For example, under the W4A4 settings of ResNet20 and VGG6bn, our method exceeds the calibration with real data by 0.41% and 0.39%, respectively.

As listed in TABLE IV(b), the results over various network architectures, including ResNet18/50 [23], SqueezeNext [16], InceptionV3 [50], and ShuffleNet [57], demonstrate that our proposed DSG significantly outperforms previous sample generation methods. As the results shown, the accuracy of quantized networks calibrated with our DSG data under the W8A8 setting is almost not decreased and even surpasses the full-precision (W32A32) baseline networks on the ResNet18 and InceptionV3 architectures. The higher accuracy might be attributed to making full use of the potential of the quantized network. Quantizing the network to W8A8 maintains greater representation ability of the network compared with the lower bit-width quantization (such as W4A4) and brings less quantization error. Thus, under this setting, our diversified data can alleviate the performance degradation of the network quantization while even leading to a better solution compared with the full-precision network. And the advantage of our DSG gets more evident when the bit-width becomes lower. For example, our DSG outperforms ZeroQ on SqueezeNext on the W6A6 setting by more than 26%, and even outperforms real data by 1.54%. And it is noteworthy that in W4A4 cases with ResNet18 architecture, our DSG surpasses ZeroQ by 13.86%, and even surpasses real data by a notable 1.45% with the compared SQuant calibration, which is up to 66.67%. Moreover, with InceptionV3 under W4A4 settings, our DSG helps to acquire 74.02% accuracy, which is higher than SQuant and real data by eminent 0.76% and 0.52%, respectively.

Furthermore, to demonstrate the applicability of our DSG scheme in data-free QAT, we compare it with existing QAT methods, such as DFC [21], RVQuant [41], Integer-Only [26], ZAQ [35], GZNQ [25], ZS-CGAN [9], and GDFQ [47]. These methods are engineered to finetune and update the network parameters. DFC is a finetuning method to recover accuracy for ultra-low bit-width cases, which uses Inceptionism [36] to facilitate the generation of data with random labels. The other mentioned methods finetune the quantized network to improve the accuracy of the network. Among them, ZAQ and GDFQ introduce a generator to synthesize data and use the generated data to finetune the network. We evaluate these methods on various bit-width settings in the more challenging large-scale image classification task. The results on ImageNet are shown as TABLE V(b).

We test these methods on ResNet18/50 [23], InceptionV3 [50], ShuffleNet [57], and MobileNetV2 [46]. The results in TABLE V(b) show that our DSG enjoys the best performance. Concretely, as can be seen, regardless of network architecture, our DSG method surpasses many previous methods, including DFC, RVQuant, Integer-Only, and GDFQ in various bit-width settings. And it is noteworthy that, after finetuning with our DSG data, quantized ResNet18 and InceptionV3 even surpass their full-precision counterparts under the W8A8 setting. Comparing with other data-free QAT training methods under W4A4 setting, our DSG outperforms GDFQ by 1.66% and 16.31% with ResNet18 and ResNet50 respectively, also higher than RVQuant with real data by 7.06% with ResNet50. Besides the mainstream neural architectures, we also evaluate our DSG over existing well-designed lightweight architectures. The proposed DSG shows an overwhelming advantage over GDFQ with a similar training pipeline, 9.16% higher with MobileNetV2 under W4A4, 7.93% higher with ShuffleNet under W4a4, and 1.62% higher with InceptionV3 under W4A4.

In short, our DSG scheme outperforms other competitors over a wide range of experiments in data-free PTQ and QAT, including various bit-width, different network architectures, and two datasets. All the results forcefully demonstrate that the diversity of generated data is significant to calibrate the model for higher accuracy, especially in ultra-low bit-width conditions. If the synthetic data are homogeneous or even identical at the distribution and sample level, it would be ineffectual when used to quantize the network. Our DSG effectively diversifies the synthetic data and thus improves the performance of the quantized neural network.

In this section, we further discuss the visualization results of our DSG scheme. First, for data-free PTQ, we show the distribution of statistics of the real data, vanilla data (ZeroQ), and DSG data (ours) in Fig. 6, which explains that SDA and LSE alleviate the homogenization from the aforementioned two levels. Then, for data-free QAT, we visualized samples of vanilla data (GDFQ), DSG data (ours), real data, and Gaussian data in Fig. 7 to show the additional effect of SCI that diversifies samples by inhibiting feature correlation. We also visualize the synthetic samples of our method and other generative data-free quantization methods (ZeroQ, GDFQ) in Fig. 8 to visually show the effect of data diversifying.

In Fig. 6, it can be obviously investigated that DSQ samples behave more like real data than vanilla data on the offset of mean and standard deviation statistics, which corresponds to the diversity at the distribution level of our generated data scheme. Especially, the SDA plays an important role to make the distribution diverse by slacking the constraint of statistics during the generation process. This phenomenon proves that our DSG scheme diversifies the synthetic data at the distribution level. Moreover, the DSG scheme also generates data with a larger variance compared to the vanilla scheme, which implies that our data samples are widely dispersed and more in line with the real situation. Especially, both SDA and LSE jointly promote diversity at the sample level, which might be useful in providing more content information.

The motivation of the SCI method is to inhibit the correlation among features in the generator, and thus the synthetic samples are separated, as shown in Fig. 7. We visualize the data generated by the vanilla method (GDFQ) and our SCI method, as well as the real data and Gaussian random data. At the right-bottom corner of each subplot, we measure the diversity of each set of features by summarizing the elements in the similarity kernel $\mathbf{K_{f}}$ as an index $s=\sum_{i,j\in\mathbf{f}_{k}}[\mathbf{K}_{i,j}]$, which indicates the feature distance between samples in spatial perspective. The larger the $s$, the more similar the samples. As can be seen from Fig. 7, the Gaussian random data is dispersive and stochastic. And it is also quite obvious that the real data is naturally distinct. However, samples generated by the vanilla GDFQ method, as shown in the subplot (a), are generally more concentrated in comparison, some of which are even overlapped. So it has the biggest $s$. Instead, our SCI utilizes the random Gaussian data as initialization and inhibits the correlation between samples to make the data more dispersed than Gaussian sampling. The samples generated by our method, which has the smallest $s$, are more dispersed than those generated by the vanilla method and seem close to real data. It demonstrates that the SCI method helps to avoid homogenization from the spatial perspective, and thus diversifies the synthetic samples from each other.

We visualize some synthetic samples of our method and other generative data-free quantization methods (ZeroQ, GDFQ). Take a closer look at Fig. 8, comparing pictures of DSG-PTQ and ZeroQ above, both of which generate images directly and update them iteratively. It is obvious that pictures generated by our DSG-PTQ method seem to have diverse colors with fine-grained textures and coarse-grained figures, while pictures of ZeroQ seem identical and have little difference in between. Meanwhile, below two sets of pictures are generated by DSG-QAT and GDFQ respectively. Owing to the generator, there are much more possibilities in the generation process and finally exhibit in both sets of images. It can be seen with naked eyes that samples of our DSG-QAT have distinct colors with higher saturation degrees, and white images inside the pictures have uncertain patterns. However, although samples generated by GDFQ have different colors and details inside the pictures as well, these samples have a uniform style in texture, which is tanglesome and poor in diversity.

Additionally, to verify the versatility and robustness of the performance gains of our DSG method, we evaluate our DSG scheme with different calibration methods and compare it with other data generation methods under the same setting. The calibration methods include Percentile, EMA, and MSE. As shown in TABLE V, no matter which calibration methods is integrated, our DSG scheme substantially outstrips ZeroQ by 6.52%, 9.36%, and 0.61%, respectively. The results strongly suggest that DSG can achieve the leading performance in various experimental settings, and thus the improvement is versatile and robust for various calibration methods.

We also evaluate the synthetic data with DFQ [38], which is a calibration method for data-free quantization. Specifically, DFQ has proposed cross-layer range equalization to equalize the different channel ranges of weight in per-layer quantization and bias correction which is to eliminate the biased quantization error. Both of the two techniques rely on the statistics of BN layers following the convolution layer. Therefore, BN layers are needed to calibrate the corresponding activations, so they have to proceed behind each convolution layer, which results in DFQ only working on specific network architectures and cannot be commonly practiced. Fortunately, generative methods, such as ZeroQ and our DSG, can work on arbitrary architectures, and the statistics of activations can take the place of BN statistics and thus be used in the DFQ method. TABLE VI shows the closeups of two generative data-free quantization methods, i.e., ZeroQ and DSG, in conjunction with the DFQ method. Results show that our DSG outperforms ZeroQ by 0.57% and 3.13% in W6A6 and W8A8 cases.

Experiments above are conducted on calibration methods that optimize the clipping value for activations. We further evaluate our DSG data with a novel data-driven PTQ method named Adaround [37], which utilizes a rounding approach to quantize weights. Meanwhile, we introduce two other data generation tricks into our DSG scheme, e.g., generating data with labels [22] provides class information from parameters, image prior [55] avoids generating unpractical scenes or unrecognizable patterns. We have conducted different bit-width for both weights and activations on different network architectures including ResNet18 and MobileNetV2 (TABLE VII). And we generate 1024 samples in every single experiment. TABLE VII shows that our DSG scheme outperforms ZeroQ by a wide margin when solely quantizing weights and preserving full-precision for activations. It is notable that in ultra-low bit-width settings (i.e., 3-bit), our DSG surpasses ZeroQ by 11.46% on ResNet18 and a surprising 34.33% on MobileNetV2. We also tried quantizing activations to 8-bit, and the results prove that our DSG is more robust to quantization of parameters, which outstrips ZeroQ by 6.06% with ResNet18 on ImageNet. Besides, we have the observation that the model gains further improvements when these tricks are applied together, even close the performance applying real data. Because our DSG is orthogonal with labels and image prior methods, so these methods can jointly boost the accuracy performance without any inconsistency.