A QP-adaptive Mechanism for CNN-based Filter in Video Coding

Given a simple filtering model(we focus on the weight, so the bias is dropped here.):

$$\bm{w}\ast\bm{y}=\hat{\bm{x}}$$

(4)

where $\bm{w}$ is the trained convolution kernel, $\bm{y}$ is the distorted image, and $\hat{\bm{x}}$ is the filtered image. It is known that the spatial domain convolution is essentially equivalent to frequency domain multiplication.

$$\mathcal{F}(\bm{w})\mathcal{F}(\bm{y})=\mathcal{F}(\hat{\bm{x}})$$

(5)

where the notation $\mathcal{F}$ represents the Fourier transform. We assume that this simple model can effectively remove the specific noise in $\bm{y}$ but can’t handle variable quantization noise. So it is approximated as the original image $\bm{x}$.

$$\mathcal{F}(\hat{\bm{x}})\approx\mathcal{F}(\bm{x})$$

(6)

As is known, the increase of the coding parameter QP represents the added noise that related to Qstep in the frequency domain. Here $\bm{\epsilon}$ represents the noise caused by the change of QP.

$$\bm{w}^{\prime}\ast(\bm{y}+\bm{\epsilon})=\hat{\bm{x}}^{\prime}$$

(7)

Therefore, a feasible solution to achieve the adaptiveness for various quantization noise is to find a changeable convolution kernel $\bm{w}^{\prime}$ to minimize the loss function between the filtered image $\hat{\bm{x}}^{\prime}$ and the original image $\bm{x}$. Similarly, (7) in frequency domain can be expressed as:

$$\mathcal{F}(\bm{w^{\prime}})(\mathcal{F}(\bm{y})+\mathcal{F}(\bm{\epsilon}))=\mathcal{F}(\hat{\bm{x}}^{\prime})$$

(8)

Here we choose classical mean square error (MSE) as the loss function.

$$\mathcal{L}=\mathbb{E}|\bm{x}-\hat{\bm{x}}^{\prime}|^{2}$$

(9)

where $\mathbb{E}$ is the notation of expectation. Considering (6) and Parseval’s theorem for the Fourier transform, the loss can be transformed into:

	$\displaystyle\mathcal{L}=$	$\displaystyle\mathbb{E}|\mathcal{F}(\bm{x})-\mathcal{F}(\hat{\bm{x}^{\prime}})|^{2}$
	$\displaystyle=$	$\displaystyle\mathbb{E}|\mathcal{F}(\bm{x})-\mathcal{F}(\bm{w}^{\prime})(\mathcal{F}(\bm{y})+\mathcal{F}(\bm{\epsilon}))|^{2}$
	$\displaystyle\approx$	$\displaystyle\mathbb{E}|\mathcal{F}(\bm{x})-\mathcal{F}(\bm{w}^{\prime})\left[\mathcal{F}(\bm{x})/\mathcal{F}(\bm{w})+\mathcal{F}(\bm{\epsilon})\right]|^{2}$		(10)

By taking the derivative w.r.t. $\mathcal{F}(\bm{w}^{\prime})$, we can obtain the solution:

$$\mathcal{F}(\bm{w}^{\prime})=\underbrace{\mathcal{F}(\bm{w})}_{\text{org. filter}}\underbrace{\left[\frac{1}{1+|\mathcal{F}(\bm{w})|^{2}\mathcal{F}(\bm{n})/\mathcal{F}(\bm{s})}\right]}_{\text{influence factor}}$$

(11)

where $\mathcal{F}(\bm{n})=\mathbb{E}|\mathcal{F}(\bm{\epsilon})|^{2}$ and $\mathcal{F}(\bm{s})=\mathbb{E}|\mathcal{F}(\bm{x})|^{2}$. Here, the first term $\mathcal{F}(\bm{w})$ is the original filter in (5), and the second term is equivalent to the influence factor that compensats for the increased quantization noise. It can be found that this solution is similar in form to Wiener deconvolution [23]. The difference lies in the motivations and forms. Wiener deconvolution hopes to recover the original signal from the distorted signal by using the priors of the input signal, noise, and degradation function. While this solution doesn’t have the concept of degradation function and it aims at making a specific filter become adaptive to the changing quantization noise.

From the perspective of the frequency domain, the extracted features from CNNs are equivalent to specific selections of the input image at different frequencies, which establishes the relationship between CNN and the frequency domain in the solution. For example, the Gaussian kernel is a low-pass filter and the Laplacian kernel is a high-pass one. The filter $\bm{w}^{\prime}$ of the entire frequency band can be decomposed into different sub-filters, and each sub-filter $w^{\prime}_{i}$ works in a selected frequency sub-band. With (11), the $\mathcal{F}(\bm{w}^{\prime})$ could be written as:

	$\displaystyle\mathcal{F}(\bm{w}^{\prime})$	$\displaystyle=\sum_{i}\mathcal{F}(w^{\prime}_{i})$
		$\displaystyle=\sum_{i}\mathcal{F}(w_{i})\left[\frac{1}{1+|\mathcal{F}(w_{i})|^{2}\mathcal{F}(n_{i})/\mathcal{F}(s_{i})}\right]$		(12)

The first term $\mathcal{F}(w_{i})$ in (3.2) is actually equivalent to the convolution kernel in CNN. Due to the decomposition, the second term $1/[1+|\mathcal{F}(w_{i})|^{2}$ $\mathcal{F}(n_{i})/\mathcal{F}(s_{i})]$ that represents the influence factor of each kernel can be regarded as working only in a sub frequency band. In this sub-band, we approximate $|\mathcal{F}(w_{i})|^{2}$ in the influence factor to a constant. The strength of the original signal $\mathcal{F}(s_{i})$ is also invariable in the task of adapting to different quantization noises, so $|\mathcal{F}(w_{i})|^{2}/\mathcal{F}(s_{i})$ can be approximated as a constant $k_{i}$. For the intensity of the quantization noise $\mathcal{F}(\bm{n})$, it is proportional to the square of Qstep at all frequencies with the default coding setting. Similarly, the decomposed noise $\mathcal{F}(n_{i})$ has the same pattern at the selected frequency.

$$\frac{|\mathcal{F}(w_{i})|^{2}}{\mathcal{F}(s_{i})}\mathcal{F}(n_{i})\approx k_{i}\mathcal{F}(n_{i})\propto Q_{step}^{2}$$

(13)

We use trainable parameters $\bm{\theta}$ to represent the proportional relationship here. Therefore, (3.2) can be rewritten as follows.

$$\mathcal{F}(\bm{w}^{\prime})\approx\sum_{i}\mathcal{F}(w_{i})\left[\frac{1}{1+\theta_{i}Q_{step}^{2}}\right]$$

(14)

where the $\theta_{i}$ indicates a specific parameter in set $\bm{\theta}$. We assume that $N$ represents the number of feature maps of the CNN, then the number of parameters introduced by this method should be $O(N^{2})$, which is the same order as the number of the kernels. Inspired by DSC [16] that uses depthwise convolution instead of standard convolution, we apply the influence factor to the feature maps instead of the convolution kernels. Fig. 1 shows the schematic diagram of our proposed method. Thus the parameter quantity becomes $O(N)$, which is the same order as the number of the biases (the QP map method [20]).

Same as HEVC[1], the relationship between QP and Qstep in VVC can be written as [24]:

$$Q_{step}=2^{(QP-4)/6}$$

(15)

so the square of Qstep is:

$$Q_{step}^{2}=2^{(QP-4)/3}$$

(16)

Due to the trainable multiplier $\theta_{i}$ in (14), multiplying different $Q_{step}^{2}$ by the same constant does not affect the performance of the model. A normalization of using $2^{(QP-32)/3}$ to replace $Q_{step}^{2}$ is adopted, which can avoid the gradient vanishing problem caused by large $Q_{step}^{2}$. Besides, the parameter $\theta_{i}$ should be greater than 0 because both $|\mathcal{F}(w_{i})|^{2}$ and $\mathcal{F}(\bm{s})$ are greater than 0. When $\bm{\theta}$ are $\bm{0}$, the proposed model will turn into the original CNN filter. There are two common methods to solve this: 1. Using the reparametrization of $\bm{\theta}=exp(\bm{\eta})$, where $\bm{\eta}$ are the unconstraint trainable parameters. 2. Directly truncating $\bm{\theta}$. We adopt the second one in this paper.

The order of the filtering process of H.266/VVC with CNN filter is the luma mapping with chroma scaling(LMCS), DB, CNN filter, SAO, and ALF. Both SAO and ALF need to add some bits to indicate the offsets and coefficients. By putting the CNN-based filter before SAO and ALF, a filtered image with higher quality can be sent to SAO and ALF, thereby reducing the number of coded bits. For the DB, which depends on the preset thresholds to perform filtering, putting the CNN filter before it may need to modify its thresholds accordingly. Therefore, instead of using this strategy, we choose to put the CNN filter after it.

By integrating our method into different models, the BD-rate [25] reduction of the proposed method could be tested in various situations, such as complexity, activation, serial or parallel structure, etc. Here we chose four different models, including Liu et al. [5], VRCNN [6], DCAD [7], and Tucodec [8], as the backbones for our experiment. The CNN-filter was integrated into the Versatile Video Coding Test Model (VTM)-6.3 [26] and put between DB and SAO. The DIV2K dataset [27] was used to train and validate all of the mentioned CNN filters. We divided 900 pictures in DIV2K into 800 as the training set and 100 as the validation set. Four QPs including 22, 27, 32, and 37 in common test condition (CTC[28]) were used to encode these pictures. By cutting these pictures into $64\times 64$ blocks, we obtained 522,877 samples for training and 66,712 samples for validation under each QP. With framework Keras [29] and optimizer Adam [30], about 40,000 iterations were trained for each QP with batch size 128. In the test phase, the 1-st frames from HEVC test sequences were used for evaluating the performance of the aforementioned filters. It is worth mentioning that these test sequences were not overlapped with the datasets used in the training or validation phase.

From the results shown in Table 1, the ”Global” column of using a single model has the lowest BD-rate reduction. By using multiple models shown in the ”Separate” column, the overall BD-rate reduction of the CNN-filter has been improved, but the number of required parameters has increased fourfold. From the ”Proposed” column, it can be seen that our proposed method enables a single model to have excellent performance for all four backbones while only increasing a small number of parameters. This shows the versatility and flexibility of our proposed method. Its BD-rate reduction is almost the same as that of the separate method on the luma component. And it achieves extra 0.2% BD-rate reduction on the chroma components. This fully demonstrates that our method effectively improves the generalization ability of the model, because we only use the luma component for training. The performance of the chroma components is completely dependent on the generalization ability of the model. As shown in Fig. 2, the PSNR gains of the methods relative to the VTM baseline are also plotted. The dotted line represents the separated method, and the QP in its legend represents the dataset used for training. We can find that the lines peak in the QP of the corresponding training dataset but perform poorly on the other QPs. Especially for lower QP, it may even lead to a negative impact on performance. The blue solid line represents the global method, which can obtain an ordinary BD-rate gain at higher QP, but similarly, it hurts the reconstructed image at lower QP. On the contrary, the proposed method has a significant filtering performance in a wide range of QPs and almost reaches the optimal performance of using multiple separate models. This further demonstrates the effectiveness and versatility of our model.

In Table 1, the parameter comparisons of different methods are shown in the ”Param.” column. In addition, Table 2 shows comparisons of relative decoding complexity than VTM anchor, which shares the same test setting with Section 4.2. Although the separate method uses more models than the global method, for the filtering of determined QP, they both use a single model with the same structure, so their complexity should be the same. From Table 2, the decoding complexity of our proposed method only increases by about 2% compared with the global method. Therefore, the impact of our proposed method on complexity is minimal. This lays a good foundation for the practical application of our method.

The performance comparison of the proposed method and the QP map method (Song et al.[20]) are shown in Fig. 3. These two methods achieve similar relative PSNR gain with the backbone of Liu et al. [5], DCAD [7]. For VRCNN[6], Song et al.[20] has a negative impact on lower QPs, whereas our proposed method still achieves a minor gain when $QP=22$. Our method performs more robust, probably because it provides the quantization information for each convolution layer but Song et al.[20] only does it for the input one. This comparison demonstrates the robustness and versatility of our method.