A Reduced Codebook and Re-Interpolation Approach for Enhancing Quality in Chroma Subsampling

At the server side, suppose the input image is a RGB full-color image. We first introduce the five traditional chroma subsampling methods, namely 4:2:0(A), 4:2:0(L), 4:2:0(R), 4:2:0(DIRECT), and 4:2:0(MPEG-B) [28]. Then, we introduce five state-of-the-art chroma subsampling combinations [43], [33], [4], [22].

Among the five traditional chroma subsampling methods, 4:2:0(DIRECT) subsamples the top-left chroma pair of $B^{CbCr}$ as the subsampled $(Cb,Cr)$-pair of $B^{CbCr}$. For simplicity, 4:2:0(DIRECT) is abbreviated as 4:2:0(D). 4:2:0(MPEG-B) determines the subsampled $(Cb,Cr)$-pair by performing the 13-tap filter with mask [2, 0, -4, -3, 5, 19, 26, 19, 5, -3, -4, 0, 2]/64 on the top-left location of $B^{CbCr}$. 4:2:0(L) and 4:2:0(R) subsample the $(Cb,Cr)$-pairs by averaging the chroma components in the left and right columns of $B^{CbCr}$, respectively. 4:2:0(A) subsamples the $(Cb,Cr)$-pair of $B^{CbCr}$ by averaging the four $(Cb,Cr)$-pairs of $B^{CbCr}$. The subsampled chroma positions of 4:2:0(D) and 4:2:0(MPEG-B) are located at (0, 1); the subsampled chroma positions of 4:2:0(L), 4:2:0(A), and 4:2:0(R) are located at (0, $\frac{1}{2}$), ($\frac{1}{2}$, $\frac{1}{2}$), and (1, $\frac{1}{2}$), respectively, and the four subsampled chroma positions are marked by the four red bullets in Figs. 2(a)-(d).

According to the new edge-directed interpolation (NEDI) [19] based chroma upsampling process which improved the previous method [1], Zhang et al. [43] proposed an interpolation-dependent image downsampling (IDID) based chroma subsampling method. Their combination is expressed as IDID-NEDI. To improve IDID-NEDI, Wang et al. [33] deployed the palette mode [27] in their JCDU (joint chroma downsampling and upsampling) based chroma subsampling method, and their best combination is expressed as JCDU-BICU, where BICU denotes the bicubic interpolation based chroma upsampling process. The experimental data demonstrated that JCDU-BICU outperforms IDID-NEDI and JCDU-BILI, in which BILI denotes the bilinear interpolation based chroma upsampling process, particularly for screen content images (SCIs) [23].

Following the COPY-based chroma upsampling process and the differentiation technique used in [20], but considering the demosaiced RGB full-color block-distortion as the criterion, Lin et al. [22] proposed a modified 4:2:0(A) chroma subsampling method which selects the best case among the four subsampled $(Cb,Cr)$-pairs of $B^{CbCr}$ by considering the ceiling operation-based 4:2:0(A) and the floor operation-based 4:2:0(A). Naturally, Lin et al.’s chroma subsampling method is suitable for the input RGB full-color image. At the client side, they improved the chroma upsampling process [22] by considering the distance between each missing chroma value and its three neighboring known (TN) pixels to achieve good quality performance. Their combination is expressed as “modified 4:2:0(A)-TN”.

Differing from the the chroma subsampling-first luma modification method [5], in [4], based on the subsampled chroma parameter-pair, at the server side, a BILI-based chroma estimation of $B^{CbCr}$ is deployed in the block-distortion function with two chroma parameters and four luma parameters. Next, using a multiple linear regression technique, a joint chroma subsampling and luma modification (CSLM) method [4] was proposed to determine the subsampled $(Cb,Cr)$-pair of $B^{CbCr}$ and the four modified luma values of $B^{Y}$ simultaneously. Their combination is expressed as CSLM-BILI. Experimental data indicated the quality superiority of CSLM-BILI relative to IDID-NEDI [43], JCDU-BICU [33], and modified 4:2:0(A)-TN [22].

For $I^{Bayer}$, we mainly introduce the four state-of-the-art chroma subsampling methods [20], [7], [22], [8].

Chen et al. [3] observed that in Eq. (2), the R value is dominated by the Y and V values, and the B value is dominated by the Y and U values, and then the subsampled $(Cb,Cr)$-pair of $B^{CbCr}$ equals $(U_{3},V_{2})$ by considering the Bayer CFA pattern. Although their method benefits the R and B components of the reconstructed Bayer CFA image, it does not benefit the G components at all. To overcome this disadvantage, based on the COPY-based upsampling process to estimate the four chroma-pairs of $B^{CbCr}$ at the server side, Lin et al. [20] proposed a 2$\times$2 Bayer CFA block-distortion function to measure the distortion between the original 2$\times$2 Bayer CFA block and the estimated one. Using a distortion minimization (DM) technique, Lin et al. derived a closed form to determine the subsampled $(Cb,Cr)$-pair of $B^{CbCr}$. Their combination is expressed as DM-COPY.

In the gradient descent-based (GD) method [7], the $2\times 2$ Bayer CFA block-distortion function used in the DM method [20] is proved to be a convex function. Then, according to the shape similarity of the convex function in the real domain to that in the integer domain, an iterative procedure, in which the closed form derived in DM [20] was used as the initially subsampled $(Cb,Cr)$-pair of $B^{CbCr}$, was proposed to better improve the DM method. In each iteration, the GD method applied the BILI method to estimate the four $(Cb,Cr)$-pairs of each neighboring chroma block in the eight neighboring chroma blocks, and then the GD method selected the best one with the minimal 2$\times$2 Bayer CFA block-distortion. Their combination is expressed as GD-BILI [7], and it has better quality than DM-COPY and GD-COPY.

As introduced in the last paragraph of Subsection I.A, Lin et al.’s modified 4:2:0(A) chroma subsampling method [22] only considers the demosaiced RGB full-color block-distortion as the distortion minimization criterion. After performing the modified 4:2:0(A)-TN combination on the demosaiced RGB full-color image, by Eq. (1), the reconstructed Bayer CFA image can be extracted from the reconstructed RGB full-color image. Therefore, ‘modified 4:2:0(A)-TN’ is also applicable for the input Bayer CFA image.

In [8], based on the BILI-based chroma upsampling process at the server side, combining chroma subsampling, luma modification, and the Bayer CFA pattern together, a $CSLM^{Bayer}$-BILI combination was proposed. For each 2$\times$2 YCbCr block $B^{YCbCr}$, $CSLM^{Bayer}$-BILI determined the best solution of the subsampled $(Cb,Cr)$-pair and the modified luma values for $B^{YCbCr}$ simultaneously. In particular, after analyzing all the sixteen (= $2^{4}$) luma-selection cases, only two luma parameters, namely $Y_{1}$ and $Y_{2}$, are modified such that the 2$\times$2 Bayer CFA block-distortion could be minimized. Experimental data demonstrated that the $CSLM^{Bayer}$-BILI combination [8] outperforms DM-COPY [20], GD-BILI [7], and modified 4:2:0(A)-TN [22].

From the introduction of the related chroma subsampling works for $I^{RGB}$ and $I^{Bayer}$, we find that under the current coding system in Fig. 1, the traditional and state-of-the-art combinations tend to suffer from the coordinate-inconsistency (CI) problem and/or the chroma upsampling process-inconsistency (UPI) problem because at the client side, the decoder is blind to the chroma subsampling process used at the server side and the future chroma upsampling process prefered by the chroma subsampling method. The two problems will be defined in Section III in detail.

The CI and UPI problems lead to the quality degradation of the reconstructed images, which will be explained in Section III. The two problems prompted us to develop a systematic approach to analyze them, and then to propose an effective approach to solve them for enhancing the quality of the reconstructed images.

In this paper, we first analyze the subsampled chroma positions of all considered chroma subsampling methods, and then we partition all these chroma subsampling methods into four classes. Based on the four partitioned chroma subsampling classes and the allowable chroma upsampling processes, we propose a flow diagram approach to analyze the coordinate-inconsistency (CI) problem and the upsampling process-inconsistency (UPI) problem occurring in the traditional and state-of-the-art combinations. We also explain why the two problems lead to the quality degradation of the reconstructed images.

To solve the CI and UPI problems, we propose an effective reduced codebook and re-interpolation (RCRI) approach for enhancing the quality of the reconstructed images. Based on the testing RGB full-color images and Bayer CFA images collected from the IMAX [14] dataset, the Kodak [16] dataset, and the Video [32] dataset, the thorough experimental results justified the significant quality enhancement effects of our RCRI approach against the CI and UPI problems for the traditional and state-of-the-art chroma subsampling methods. For $I^{RGB}$ and $I^{Bayer}$, the average CPSNR (color peak signal-to-noise ratio) gain and the average PSNR gain of our RCRI approach are at least 1.4 dB and 2.4 dB, respectively.

The rest of this paper is organized as follows. In Section II, based on the subsampled chroma positions, all considered chroma subsampling methods are partitioned into four classes. In Section III, the proposed flow diagram approach is presented to analyze the CI problem and the UPI problem. In Section IV, the proposed RCRI approach is presented to solve the two problems. In Section V, the thorough experimental results are illustrated to justify the significant quality enhancement using our RCRI approach. In Section VI, some concluding remarks are addressed.

Without the loss of generality, we take the CSLM-BILI combination [4] as the example to define the UPI problem.

In CSLM-BILI, as introduced in Subsection I.A, at the server side, a BILI-based chroma estimation of each 2$\times$2 CbCr block $B^{CbCr}$ is deployed in the 2$\times$2 RGB full-color block-distortion function with the parameter-pair, $(Cb,Cr)$.

At the client side, instead of ‘BILI’, if the decoder adopts the other upsampling process, e.g. ‘COPY’, to upsample the received subsampled chroma image, it causes an upsampling process-inconsistency (UPI) problem because the future chroma upsampling process ‘BILI’ preferred by CSLM-BILI is misused as ‘COPY’ by the decoder.

Let the set symbol ‘CU’ denote the four chroma upsampling processes, namely COPY, BILI, NEDI, and BICU, used at the client side. Let CSxCU, where the symbol ‘x’ indicates the cross product operator, denote all combinations over the product of CS and CU. For each combination in CSxCU, we propose a flow diagram approach to analyze whether a true CI problem occurs in that combination and to report the coordinate displacement of the true CI problem. Furthermore, we explain why the true CI problem leads to the quality degradation of the reconstructed images.

1) The proposed flow diagram to analyze the true CI problem and to calculate the coordinate displacement: After performing one chroma subsampling $cs$ $\in$ CS on the chroma image $I^{CbCr}$, it yields a subsampled chroma image $I^{sub,CbCr}_{cs}$, which can be expressed as $I^{sub,CbCr}_{cs\in C_{top-left}}$, $I^{sub,CbCr}_{cs\in C_{left}}$, $I^{sub,CbCr}_{cs\in C_{mid}}$, or $I^{sub,CbCr}_{cs\in C_{right}}$. The four possible subsampled chroma images are depicted in Fig. 5(a). For compression, as depicted in Fig. 5(b), each subsampled chroma image $I^{sub,CbCr}_{cs}$ is further rearranged to a quarter-sized subsampled chroma image $I^{q,CbCr}_{cs}$ which is stored in an array data structure under an integer coordinate system.

However, at the server side, for compression, moving the subsampled chroma pair of each 2$\times$2 chroma block from the subsampled chroma position ($\in$ SCP) to the new position, namely (0, 1), often causes a coordinate displacement problem. Generally, as depicted by the four arrows between Fig. 5(a) and Fig. 5(b), the corresponding four coordinate displacements equal (0, 0) (= (0, 1) - (0, 1)), (0, $\frac{1}{2}$) (= (0, 1) - (0, $\frac{1}{2}$)), (-$\frac{1}{2}$, $\frac{1}{2}$) (= (0, 1) - ($\frac{1}{2}$, $\frac{1}{2}$)), and (-1, $\frac{1}{2}$) (= (0, 1) - (1, $\frac{1}{2}$)) corresponding to $I^{q,CbCr}_{cs\in C_{top-left}},I^{q,CbCr}_{cs\in C_{left}},I^{q,CbCr}_{cs\in C_{mid}},andI^{q,CbCr}_{cs\in C_{right}}$, respectively. We conclude that for one chroma subsampling method $cs\in C_{left}\cup C_{mid}\cup C_{right}$, at the server side, preparing the quarter-sized subsampled chroma image $I^{q,CbCr}_{cs}$ for compression causes a CI problem due to its nonzero coordinate displacement (NCD) in the set NCD = (0, $\frac{1}{2}$), (-$\frac{1}{2}$, $\frac{1}{2}$), (-1, $\frac{1}{2}$).

After receiving the compressed quarter-sized subsampled chroma image $I^{q,CbCr}_{cs}$ by the decoder at the client side, each subsampled chroma pixel $I^{q,CbCr}_{cs}(i,j)$ is moved to $I^{rec,CbCr}_{cs}(2i,2j)$, where $I^{rec,CbCr}_{cs}$ indicates the upsampled chroma image, for constructing the initially upsampled chroma image $I^{ini,CbCr}_{cs}$, as depicted in Fig. 5(c). Because from Fig. 5(a) to Fig. 5(b), for $cs\in C_{left}\cup C_{mid}\cup C_{right}$, it causes a CI problem in the subsampled chroma image $I^{q,CbCr}_{cs}$, the initially upsampled chroma image $I^{ini,CbCr}_{cs}$ in Fig. 5(c) thus inherits the CI problem in Fig. 5(b) and the associated nonzero coordinate displacement in NCD.

Further, all missing chroma pixels in $I^{ini,CbCr}_{cs}$ of Fig. 5(c) are reconstructed using the adopted chroma upsampling process $`cu^{\prime}$ in CU. We first consider the chroma upsampling process ‘COPY’. After performing the COPY-based upsampling process on each initially subsampled chroma image in Fig. 5(c), the reconstructed chroma pixels are denoted by black bullets of the upsampled chroma image in Fig. 5(d). For each 2$\times$2 chroma block in $I^{ini,CbCr}_{cs}$, as depicted in Fig. 5(d), the three missing chroma pixels are reconstructed by copying the top-left subsampled chroma-pair of that block. According to the analysis from Fig. 5(a) to Fig. 5(d), we conclude that for any combination ‘$cs$-COPY’, where ‘cs’ is in CS, no true CI problem occurs.

Next, we consider the chroma upsampling process ‘$cu$’ $\in$ {BILI, NEDI, BICU}. After performing the upsampling process ‘$cu$’ on each initially subsampled chroma image in Fig. 5(c), the reconstructed chroma pixels are denoted by black cross-marked symbols of the reconstructed chroma image in Fig. 5(d). In Fig. 5(d), each missing chroma pixel in $I^{ini,CbCr}_{cs}$ is reconstructed by the upsampling process ‘$cu$’ referring to the neighboring chroma pairs of that missing chroma pixel. Consequently, we conclude that for any combination in {$C_{left}$, $C_{mid}$, $C_{right}$}x{BILI, NEDI, BICU}, a true CI problem occurs eventually, as depicted in Fig. 5(e). On the other hand, among the sixteen combinations in CSxCU, the true CI problems occur in only nine combinations in {$C_{left}$, $C_{mid}$, $C_{right}$}x{BILI, NEDI, BICU}.

From the above analysis of the true CI problems, we find that there are only three distinct coordinate displacements, namely (0, $\frac{1}{2}$), (-$\frac{1}{2}$, $\frac{1}{2}$), and (-1, $\frac{1}{2}$), corresponding to the three combinations in $C_{left}$x{BILI, NEDI, BICU}, the three combinations in $C_{mid}$x{BILI, NEDI, BICU}, and the three combinations in $C_{right}$x{BILI, NEDI, BICU}, respectively. Due to the nonzero coordinate displacement problem, for each 2$\times$2 upsampled chroma block in Fig. 5(d), the four temporarily upsampled chroma pairs should be replaced by the four correct chroma pairs, as depicted by the four red triangles in Fig. 5(d). The detailed re-interpolation based recovery strategy in our RCRI approach will be presented in Section IV.B.

2) Why the true CI problem degrades the quality of the reconstructed image: Because for any combination, at the server side, we do not subsample the luma image $I^{Y}$ at all, the luma image has no CI problem from the chroma subsampling step to the chroma upsampling step at the client side. For any combination in {$C_{left}$, $C_{mid}$, $C_{right}$}x{BILI, NEDI, BICU}, we know a true CI problem occurs in the reconstructed chroma image $I^{rec,CbCr}$. Consequently, at the client side, the luma pixel $I^{Y}(i,j)$ and the upsampled chroma pixel $I^{rec,CbCr}(i,j)$ lead to a coordinate displacement problem each other. Finally, after converting the upsampled YCbCr image to the reconstructed RGB full-color image (or the reconstructed Bayer CFA image) by Eq. (2), it degrades the quality of the reconstructed image.

From Fig. 4, we know there are four chroma subsampling classes, namely $C_{top-left}$, $C_{left}$, $C_{mid}$, and $C_{right}$, in CS and there are four chroma upsampling processes, namely COPY, BILI, NEDI, and BICU, in CU. To record the necessary information of each combination in CSxCU, a reduced 4-bit codebook is depicted in Table I. In Table I, for each codeword, the first two bits are used to represent the chroma subsampling class, which corresponds to the chroma subsampling method used at the server side, and the last two bits are used to record the future chroma upsampling process preferred at the client side. For example, the combination ‘JCDU-BICU’ corresponding to ‘$C_{top-left}$-BICU’ is expressed as the 4-bit codeword ‘0011’.

To increase the robustness to tolerate one bit error against the communication interference, we deploy three redundant bits in the 4-bit codeword in Table I to form a (7, 4)-Hamming code [11]. Let each 4-bit codeword in Table I be denoted by ‘$m_{1}m_{2}m_{3}m_{4}$’. Using the error correcting code technique, the corresponding (7, 4)-Hamming code is represented as a 7-bit codeword “$r_{1}r_{2}m_{1}r_{3}m_{2}m_{3}m_{4}$” in which the redundant 3-bit ‘$r_{1}r_{2}r_{3}$’ is used for correcting the one bit error. For easy exposition, let $p_{1}p_{2}p_{3}p_{4}p_{5}p_{6}p_{7}$ = $r_{1}r_{2}m_{1}r_{3}m_{2}m_{3}m_{4}$. Based on the three equations with even parity: $p_{1}+p_{2}+p_{4}+p_{5}=0\pmod{2}$, $p_{2}+p_{3}+p_{6}+p_{7}=0\pmod{2}$, and $p_{1}+p_{3}+p_{5}+p_{6}=0\pmod{2}$, the corrupted one bit can be detected and corrected.

For example, suppose the decoder received the 7-bit error correction code, namely $p_{1}p_{2}p_{3}p_{4}p_{5}p_{6}p_{7}$ = 1100000. From the equality: $p_{1}p_{2}p_{3}p_{4}p_{5}p_{6}p_{7}$ = $r_{1}r_{2}m_{1}r_{3}m_{2}m_{3}m_{4}$, it yields ‘$m_{1}m_{2}m_{3}m_{4}$ = 1000’. Based on three even parity equations, it yields $p_{1}+p_{2}+p_{4}+p_{5}=0\pmod{2}$, $p_{2}+p_{3}+p_{6}+p_{7}=1\pmod{2}$, and $p_{1}+p_{3}+p_{5}+p_{6}=1\pmod{2}$. From the resultant 3-bit ‘011’ calculated from the above three even parity equations, we know that the value of $p_{3}$ is corrupted. Therefore, the value of $m_{1}$ is corrected from 0 to 1. Equivalently, the corrected 7-bit code should be 1110000, and the corrected 4-bit codeword equals 0001. Taking the first two bits of the corrected 4-bit codeword, namely 00, as a key to query Table I at the client side, the chroma subsampling class used at the server side is reported as $C_{top-left}$. Taking the last two bits of the corrected 4-bit codeword, namely 01, by Table I, the future chroma upsampling process preferred at the client side is reported as BILI. Table II illustrates the error correcting (7, 4)-Hamming codebook used in our RCRI approach. Consequently, for any combination in CSxCU, Table II can be used to solve the UPI problem in a robust way. Table II is kept by the server side and the client side simultaneously.

We first consider the three combinations in $C_{left}$x{BILI, NEDI, BICU}. As depicted in Fig. 5, we know their common nonzero coordinate displacement vector is (0, $\frac{1}{2}$). To solve the true CI problem which occurred in the three combinations, based on the coordinate displacement vector (0, $\frac{1}{2}$), for each 2$\times$2 upsampled chroma block in the temporarily reconstructed chroma image, the four temporarily upsampled chroma pairs, which are denoted by one black bullet and three cross-marked black bullets in Fig. 5(d), are replaced by re-interpolating the four chroma pairs marked by the four red triangles in $I^{rec,CbCr}_{cs\in C_{left}}$ of Fig. 5(d).

Next, we consider the three combinations in $C_{mid}$x{BILI, NEDI, BICU}. As depicted in Fig. 5, we know their common nonzero coordinate displacement vector is (-$\frac{1}{2}$, $\frac{1}{2}$). To solve the true CI problem which occurred in the three combinations, based on the coordinate displacement vector (-$\frac{1}{2}$, $\frac{1}{2}$), for each 2$\times$2 upsampled chroma block in the temporarily reconstructed chroma image $I^{rec,CbCr}_{cs\in C_{mid}}$ of Fig. 5(d), the four temporarily upsampled chroma pairs are replaced by re-interpolating the four chroma pairs marked by the four red triangles in $I^{rec,CbCr}_{cs\in C_{mid}}$.

By the same argument, for each 2$\times$2 upsampled chroma block in $I^{rec,CbCr}_{cs\in C_{right}}$ of Fig. 5(d), the four temporarily upsampled chroma pairs can be recovered by re-interpolating the four chroma pairs marked by the four red triangles in $I^{rec,CbCr}_{cs\in C_{right}}$. Consequently, the deviationly upsampled chroma image occurred in the nine combinations (= {$C_{left}$, $C_{mid}$, $C_{right}$}x{BILI, NEDI, BICU}) can be recovered using our coordinate displacement-based re-interpolation strategy.

For $I^{RGB}$, this subsection presents the quality enhancement effects of our RCRI approach against the UPI and true CI problems for the traditional and state-of-the-art combinations [43], [33], and [4]. In the five traditional chroma subsampling methods, 4:2:0(L) and 4:2:0(R) are selected to balance our discussion of the experiments. In addition, the quality enhancement effect of our RCRI approach to the state-of-the-art combination [22] is also investigated.

Table III illustrates the CPSNR gains of the reconstructed RGB full-color images using our RCRI approach against the UPI and true CI problems existing in the concerned combinations. For clarifying the quality enhancement effect of our RCRI approach, the CPSNR value of the reconstructed RGB full-color images using our RCRI approach for each combination is tabulated in the parenthesis ‘()’.

After deploying our RCRI approach in IDIDxCU, except for IDID-NEDI [43] without UPI and true CI problems, the CPSNR gains are 5.1511 (= 45.2343 - 40.0832 ) dB, 0.01 (= 45.2343 - 45.2248) dB, and 0.7249 dB w.r.t. IDID-COPY, IDID-BILI, and IDID-BICU, respectively. Suppose the probability of selecting each chroma upsampling process in CU at the client side is the same and equals $\frac{1}{4}$. For IDID-NEDI, the average CPSNR gain using our RCRI approach equals 1.4715 (= $\frac{1}{4}$(5.1511 + 0.01 + 0.7249)) dB, achieving a clear quality enhancement effect.

Similarly, after deploying our RCRI approach in JCDUxCU, except for JCDU-BICU [33] without UPI and true CI problems, the CPSNR gains using our RCRI approach are 4.8367 dB, 0.2341 dB, and 0.6037 dB w.r.t. JCDU-COPY, JCDU-BILI, and JCDU-NEDI, respectively; the average CPSNR gain equals 1.4186 (= $\frac{1}{4}$(4.8367 + 0.2341 + 0.6037)) dB, also achieving a clear quality enhancement effect.

After deploying our RCRI approach in 4:2:0(L)xCU, 4:2:0(R)xCU, and CSLMxCU, the average CPSNR gains equal 2.0526 (= $\frac{1}{4}$(3.4798 + 1.5036 + 1.6860 + 1.5410)) dB, 3.5774 (= $\frac{1}{4}$(2.7823 + 3.7786 + 3.8396 + 3.9092)) dB, and 4.4471 (= $\frac{1}{4}$(4.9532 + 3.9743 + 4.0299 + 4.8309)) dB, respectively, also achieving significant quality enhancement effects.

In addition, our experimental results indicate that for $I^{RGB}$, after deploying our RCRI approach in ‘modified 4:2:0(A)’x(CU $\cup$ {TN}), except for ‘modified 4:2:0(A)’-TN [22], the CPSNR gains for the five combinations are 2.1622 dB, 2.3352 dB, 2.5043 dB, and 2.2331 dB, respectively. Accordingly, the average CPSNR gain of our RCRI approach equals 1.84696 (= $\frac{1}{5}$(2.1622 + 2.3352 + 2.5043 + 2.2331)) dB, also achieving a clear quality enhancement effect.

Besides the CPSNR improvement, Table III also demonstrates the $SSIM_{c}$ and $FSIM_{c}$ improvements of our RCRI approach for $I^{RGB}$. In fact, more chroma upsampling processes [42], [37], [36], [10], [30], [34] can be included in our study to justify the quality enhancement effect of the proposed RCRI approach.

For $I^{Bayer}$, this subsection presents the quality enhancement effects of our RCRI approach against the UPI problems and true CI problems for 4:2:0(L), 4:2:0(R), and the state-of-the-art combinations [20], [7], [8], and [22] under the current coding environment.

From Table IV, we observe that after deploying our RCRI approach in DMxCU, GDxCU, 4:2:0(L)xCU, 4:2:0(R)xCU, and $CSLM^{Bayer}$xCU, the average PSNR gains equal 4.8134 (= $\frac{1}{4}$(5.2940 + 5.6031 + 8.3564)) dB, 5.7962 $\frac{1}{4}$(= (0.1047 + 7.5368 + 7.2515 + 8.2917)) dB, 2.4466 (= $\frac{1}{4}$(2.7829 + 2.3125 + 2.7198 + 1.9712)) dB, 4.1248 $\frac{1}{4}$(= (3.2360 + 4.3958 + 4.4658 + 4.4017)) dB, and 10.1278 (= $\frac{1}{4}$(9.8250 + 10.0740 + 9.7277 + 10.8843)) dB, respectively, achieving significant quality enhancement effects.

In addition, our experimental results indicate that for $I^{Bayer}$, after deploying our RCRI approach in ‘modified 4:2:0(A)’x(CU $\cup$ {TN}), except for ‘modified 4:2:0(A)’-TN [22], the CPSNR gains are 2.0860 dB, 3.1487 dB, 3.5034 dB, and 2.8881 dB, respectively. Accordingly, the average PSNR gain of our RCRI approach equals 2.3252 (= $\frac{1}{5}$(2.0860 + 3.1487 + 3.5034 + 2.8881)) dB, also achieving a clear quality enhancement effect.

Besides the PSNR improvement, Table IV also demonstrates the SSIM and FSIM improvements of our RCRI approach for $I^{Bayer}$.