RingCNN: Exploiting Algebraically-Sparse Ring Tensors for Energy-Efficient CNN-Based Computational Imaging ^††thanks: This work was supported by the Ministry of Science and Technology, Taiwan, R.O.C., under Grant no. MOST 109-2218-E-007-034.

A ring $R$ is a fundamental algebraic structure which is a set equipped with two binary operations $+$ and $\cdot\,$. Here we consider the set of real-valued $n$-tuples, i.e. $R=\left\{x=(x_{0},...,x_{n-1})^{t}\mid x_{i}\in\mathbb{R}\right\}$. For clarity, $x$ is a ring element and $x_{i}$ is its real-valued component. And we simply use the component-wise vector addition for the ring addition $+$.

As for ring multiplication $\cdot\,$, it plays an important role for the properties of different rings. Given

where $z,g,x\in R$, we consider it has a bilinear form to have a general formulation for fast algorithms, which will be discussed in Section III-B. In particular, the components of the three ring elements are related by

where $M$ is a 3-D indexing tensor with only $1$, $0$, and $-1$ as its entries. In other words, the products of input ring components in form of $g_{k}x_{j}$ are distributed to output components $z_{i}$ through $M_{ikj}$. With the bilinear form (3), the ring multiplication (2) will be isomorphic to a matrix-vector multiplication

where the matrix $G$ has entries $G_{ij}=\sum_{k=0}^{n-1}M_{ikj}g_{k}$. Without loss of generality, we will use $g$ for filter weights and $x$ for feature maps in the following.

After having definitions of $+$ and $\cdot\,$, we still need to define a unary non-linear operation $f$ for a ring to construct neural networks. A conventional choice, which is usually adopted by previous methods for full-rank or algebraic sparsity, is a component-wise ReLU

where $\max(0,\cdot)$ is the commonly-used real-valued ReLU.

Now we will integrate transform-based full-rank sparsity into this framework. For the bilinear-form ring multiplication (3), from [46] we know that its optimal general fast algorithm over real field can be expressed by the following three steps

where $T_{g}$ and $T_{x}$ are $m\times n$ transform matrices for $g$ and $x$ respectively, and $T_{z}$ is $n\times m$ for $z$. And $\circ$ represents a component-wise product, i.e. $\tilde{z}_{i}=\tilde{g}_{i}\tilde{x}_{i}$ for $i=0,1,...,m-1$ for the three $m$-tuples $\tilde{z}$, $\tilde{g}$, and $\tilde{x}$. If the transform matrices involve only simple coefficients, e.g. $\pm 1$ or $0$, then they can be implemented by adders, and the component-wise product will dominate computation complexity. In particular, the number of real-valued multiplications can be reduced from the general $n^{2}$ for matrix $G$ to $m$ in (7). Therefore, the complexity of fast ring multiplication depends on how we decompose the indexing tensor $M$ or its isomorphic matrix $G$ into (6)-(8).

When $G$ is diagonalizable over real field, i.e. $G=T^{-1}DT$, this complexity can be minimized as $m=\mbox{rank}(G)$ for $\tilde{g}=\mbox{diag}(D)$. The proof is given in Appendix A. In this perspective, a ring $R_{H}$ alike to HadaNet has a full-rank $G$, i.e. $\mbox{rank}(G)=n$, which is diagonalized by Hadamard transform. Another example is a ring $R_{I}$ equivalent to group convolution which applies component-wise products for a diagonal full-rank $G$, and its invertible $T$ is simply the identity matrix $I$.

In contrast, if $G$ is not diagonlizable over real field, we can instead apply the tensor rank decomposition for the indexing tensor $M$ as mentioned in [20]. However, the complexity is usually larger than $\mbox{rank}(G)$ in this case, and the generic rank (grank) represents the lower bound for real-valued multiplications: $m\geq\mbox{grank}(M)$. For example, the rotation matrix for complex field $\mathbb{C}$ leads to three real-valued multiplications as its $\mbox{grank}(M)=3$ while $\mbox{rank}(G)=2$, and the circulant matrix in CirCNN also belongs to this category. The related properties of $\mathbb{C}$ as well as $R_{H}$ and $R_{I}$ with ring dimension $n=2$ are as shown at the top of Table I.

In addition to the rings at hand, we would like to search more proper variants for in-depth analysis. We make three practical assumptions to confine the scope of discussion. The first one is exclusive sub-product distribution: each input sub-product $g_{k}x_{j}$ in (3) is distributed to one output component $z_{i}$ exclusively. It provides complete and non-redundant information mixing between ring components for maintaining compact model capacity. Then $G$ has full rank and can be formulated by a sign matrix $S$ and a permutation indexing matrix $P$:

where $S_{ij}\in\{1,-1\}$, and each row or column of $P$ is a permutation of $\{0,1,...,n-1\}$. For example, the rotation matrix $\big{(}\begin{smallmatrix}g_{0}&-g_{1}\\ g_{1}&g_{0}\end{smallmatrix}\big{)}$ for $\mathbb{C}$ has $S=\big{(}\begin{smallmatrix}1&-1\\ 1&1\end{smallmatrix}\big{)}$ and $P=\big{(}\begin{smallmatrix}0&1\\ 1&0\end{smallmatrix}\big{)}$.

Furthermore, given the existence of a ring unity $\mathbf{1}$, we consider the following explicit condition:

Without loss of generality, the condition on the first column of $G$ and $\mathbf{1}$ is drawn from the permutation definition of $P$ and $g\cdot\mathbf{1}=g$. The diagonal of $G$ is then derived by $\mathbf{1}\cdot g=g$ which states that the isomorphic matrix of $\mathbf{1}$ is the identity matrix, and therefore $G=g_{0}I$ if $g=g_{0}\mathbf{1}$.

The second assumption is commutativity. It is not necessary for constructing neural networks, e.g. quaternions $\mathbb{H}$ are not commutative. But it is sufficient to enable the demanded associativity for a ring, together with the exclusive sub-product distribution and an additional condition on commutative permutation. The details are discussed in Appendix B. Then, by examining the matrix form $Gx=Xg$ for $g\cdot x=x\cdot g$, we have a cyclic-mapping condition for reducing candidates:

Finally, the last assumption is that a smaller $\mbox{grank}(M)$ is preferred for saving computations and leads to this rule:

In practice, for each $P$ satisfying (C1) and (C2) we ran the CP-ARLS algorithm [6] in MATLAB to evaluate $\mbox{grank}(M(S^{\prime};P))$ for all possible $S^{\prime}$ and determined ring variants based on the results.

In the following, we consider moderate sparsity for computational imaging with $n=2\mbox{ and }4$. We searched new ring variants as mentioned above and determined their transform matrices as discussed in Section III-B. Our findings are listed in Table I where we distinguish ring symbols by indicating $n$ in the subscripts for clarity. For $n=2$, only $R_{H2}$ and $\mathbb{C}$ can satisfy . For $n=4$, we found, by exhaustion, that there are two such non-isomorphic permutations. After applying (C3), the minimum $\mbox{grank}(M)$ of them is found to be 4 and 5. The grank-4 permutation leads to two ring variants: $R_{H4}$ and $R_{O4}$ which are diagonalized respectively by Hadamard transform $H$ and a reflected Householder matrix $O=2L_{1}(I-2vv^{t})$ where $L_{1}=\mbox{diag}(\left(\begin{smallmatrix}1&-1&-1&-1\end{smallmatrix}\right)^{t})$ and $v=\frac{1}{2}\left(\begin{smallmatrix}1&1&1&1\end{smallmatrix}\right)^{t}$. On the other hand, there are four grank-5 ring variants. Two of them, $R_{H4\mbox{\scriptsize{-I}}}$ and $R_{H4\mbox{\scriptsize{-II}}}$, have transform matrices related to $H$, and the other two, $R_{O4\mbox{\scriptsize{-I}}}$ and $R_{O4\mbox{\scriptsize{-II}}}$, are similarly connected to $O$. In particular, $R_{H4\mbox{\scriptsize{-I}}}$ applies circular convolution as CirCNN and needs five real multiplications for complex Fourier transform. The details of isomorphic $G$ and fast algorithms are summarized in Table II.

Now we can systematically examine the benefits of these rings in terms of hardware resources. For concise hardware analysis, we assume that different algebraic structures have the same bitwidths for layer inputs and parameters. Then the weight storage is directly proportional to the degrees of freedom (DoF), and the multiplier complexity can be evaluated on the same basis. Regarding the amount of filter weights, a real-valued network would require $n^{2}$ weights for an $n$-tuple pair of input and output features. But using $n$-tuple rings instead will only need $n$ real-valued weights to represent the matrix $G$, i.e. DoF of $G$ is reduced from $n^{2}$ to $n$. Therefore, the efficiency of weight storage with respect to the real-valued networks is $n\times$, e.g. $2\times$ and $4\times$ for 2- and 4-tuple rings respectively. Similarly, the corresponding efficiency in terms of real-valued multiplications can be derived as $n^{2}/m$. In Table I, only $R_{I}$, $R_{H}$, and $R_{O4}$ can reach the maximum efficiency, $n\times$, for full-rank $G$.

More importantly, for practical implementation we need to consider fixed-point computation and include the bitwidths of the multiplications for precise evaluations. Fig. 3 shows such an example for the fast algorithm (6)-(8). The main overheads brought by the transforms are the increased bitwidths for $\tilde{x}$ and $\tilde{g}$, e.g. $T_{x}$ and $T_{g}$ will transform $w$-bit $x$ and $g$ into wider $w_{x}$-bit $\tilde{x}$ and $w_{g}$-bit $\tilde{g}$. The circuit complexity of a multiplier can be approximated by the product of its input bitwidths. We further consider this factor, $w_{x}\times w_{g}$, for evaluating the multiplier complexity for 8-bit features and weights as shown in the rightmost column of Table I. In this case, only $R_{I}$ can reach the maximum efficiency for using identity transforms, and the other rings all suffer the corresponding overheads induced by their transforms. For example, $R_{H4}$ and $R_{O4}$ merely achieve $2.6\times$ efficiency which is $1.6\times$ worse than $R_{I4}$.

The above discussions only involve linear operations of neural networks. For non-linearity, the component-wise ReLU $f_{cw}$ is conventionally adopted even when we actually operate on $n$-tuples. As a result, $R_{I}$ will have the worst model capacity, although it has the best hardware efficiency. It is because the information between different components of an $n$-tuple is not communicated or mixed, which is the same as the discussion on group convolution in [52]. This is also the reason why we assumed the complete information mixing property for searching ring multiplications in Section III-C. In the following, we apply algebraic-architectural co-design to have the hardware advantages of $R_{I}$ while recovering the model capacity.

By examining the fast algorithm (6)-(8), we found that the information is in fact mixed by the transforms for data, $T_{x}$, and reconstruction, $T_{z}$. In addition, for neural networks this should be required only near non-linearity because cascaded linear operations will simply degrade to another single linear operator. Based on these two observations, we propose to mix information only before and after non-linearity and thus can adopt $R_{I}$ for linear operations to have its architectural benefits. This proposal leads to a novel algebraic function for ring non-linearity: directional ReLU $f_{dir}(y)\triangleq Uf_{cw}(Vy)$, where $U$ and $V$ are two $n\times n$ matrices for an input $n$-tuple $y$. It is equivalent to performing non-linearity in the directions of the row vectors of $V$, instead of the conventional standard axes, and then turning the axes to the column vectors of $U$. Thus the components of an $n$-tuple are considered as a whole, not separately, for non-linearity.

The computation of $U$ and $V$ induces complexity overheads. But they are only linearly proportional to the number of output channels, unlike the bitwidth-increased products in (7) which grow quadratically. To further reduce the overhead, we consider the simple Hadamard transform in Table II and propose a novel ring $(R_{I},f_{H})$ with the directional ReLU as shown in Fig. 4:

For $n=4$, another similar variant $(R_{I4},f_{O4})$ with $f_{O4}(y)\triangleq Of_{cw}(Oy)$ is also possible. They have the same hardware advantages as $R_{I}$ and possess better model capacity for additional information mixing. Note that for constructing neural networks they are different from $R_{H}$ and $R_{O4}$, especially when skip connections exist or some convolutions are not followed by non-linearity.

We propose a unified RingCNN framework to include all the considered rings for in-depth comparisons on quality-complexity tradeoffs. By extending ring algebra to ring tensors $\mathbb{z}$, $\mathbb{g}$, and $\mathbb{x}$, we formulate a $K\times K$ ring convolution (RCONV):

where $c_{o}$ and $c_{i}$ are indexes for output and input $n$-tuple channels, $p$ and $q$ for feature positions, and $s$ and $t$ for weight positions. Then a real-valued convolution layer, either with non-linearity or not, can be converted into an RCONV layer as shown from Fig. 5(a) to (b). In this way, we can convert any existing real-valued model structure into an RingCNN alternative.

An RingCNN model can be treated as a conventional real-valued CNN if we implement it in form of the matrix-vector multiplication (4). Then the Backprop algorithm can flow gradients as usual without any special treatment. For the completeness of ring algebras, we can also represent the gradients in terms of ring operations and then express Backprop using only the ring terminology. For example, we have $\nabla_{x}L=G^{t}\nabla_{z}L$ from (4) for a training loss $L$. Then $\nabla_{x}L=g\cdot\nabla_{z}L$ for $R_{I}$, $R_{H}$, and $R_{O4}$ since $G$ is symmetric for them. Similarly, the gradient $\nabla_{x}L$ equals to $g^{c}\cdot\nabla_{z}L$ for $R_{H4\mbox{\scriptsize{-I}}}$ and $g^{*}\cdot\nabla_{z}L$ for $\mathbb{H}$, where $g^{c}$ and $g^{*}$ represent circular folding and quaternion conjugate of $g$ respectively. The same approach can be applied to express $\nabla_{g}L$ in ring operations.

Dynamic fixed-point quantization. We prefer fixed-point computation for hardware implementation. It has been shown effective to apply dynamic quantization with separate per-layer Q-formats [1] for real-valued feature maps and parameters [21]. We found that this approach also works well for the RingCNN models that adopt the component-wise ReLU. But when the directional ReLU is applied, image quality is deteriorated in many cases. It is because after this non-linearity different ring components have different dynamic ranges, and using one single Q-format for them causes large saturation errors. Therefore, for the directional ReLU we propose to use component-wise Q-formats for feature maps to address this issue. In other words, there are $n$ different feature Q-formats in one layer, and each component of $n$-tuple features follows its corresponding Q-format.

Fast algorithm. We use the fast algorithm to formulate a fast ring convolution (FRCONV):

where $\tilde{\mathbb{g}}$ and $\tilde{\mathbb{x}}$ are the ring tensors after the transforms $T_{g}$ and $T_{x}$ respectively. For minimizing overheads, we avoid redundant transform operations by applying $T_{g}$, $T_{x}$, and $T_{z}$ only once for each of weight, input, and output ring elements respectively. Then each RCONV layer can then be efficiently implemented in hardware by applying FRCONV to its fixed-point model as shown in Fig. 5(c). Note that for $R_{I}$ FRCONV is the same as RCONV for its identity transform matrices.

Training setting and test datasets. For quality evaluations, we use the advanced ERNets for eCNN [21] as the real-valued backbone models. Then RingCNN models are converted from them as shown from Fig. 5(a) to (b). To fairly compare RingCNNs and real-valued CNNs, we evaluate their best performance by increasing their initial learning rates as high as possible before training procedures become unstable. Note that the real-valued ERNets in this paper will therefore perform better than those in [21] because of using higher learning rates. The models are trained using the lightweight settings as summarized in Table III if not mentioned. Finally, we test denoising networks on datasets Set5 [7], Set14 [48], and CBSD68 [36], and super-resolution ones on Set5, Set14, BSD100 [36], and Urban100 [22].

Quality comparison for different rings. Fig. 9 compares image quality in PSNR for the rings in Table I. When the component-wise ReLU is used, $R_{I}$ performs the worst due to the lack of information mixing. The two traditional algebra alternatives $\mathbb{C}$ and $\mathbb{H}$ also do not perform well, considering more real-valued multiplications are required. Between the two grank-4 variants for $n=4$, the newly-discovered $R_{O4}$ performs better than the HadaNet-alike $R_{H4}$. Similar results can be found for their corresponding grank-5 variants, e.g. the newly-discovered $R_{O4\mbox{\scriptsize{-I}}}$ better than the CirCNN-alike $R_{H4\mbox{\scriptsize{-I}}}$. However, by using the directional ReLU, the proposed $(R_{I},f_{H})$ can give better quality and constantly outperform the others. Since $(R_{I4},f_{O4})$ shows inferior quality, we therefore focus on $(R_{I},f_{H})$ and adopt it for our implementation.

Ablation study between $(R_{I},f_{H})$ and $R_{H}$. They share similar structures but have two major differences. First, $(R_{I},f_{H})$ multiplies input features by weights $\mathbb{g}$ directly while $R_{H}$ does that after applying the filter transform. Second, $(R_{I},f_{H})$ applies Hadamard transform only when non-linearity is required, but $R_{H}$ always does that and results in a redundant structure. Therefore, $R_{H}$ can imitate $(R_{I},f_{H})$ by making up the differences: first training on transformed weights $\tilde{\mathbb{g}}$ and then modifying model structures accordingly. Fig. 10(a) shows an example for modifying a residual block, and Fig. 10(b) illustrates typical PSNR results using two SR$\times$4 networks as examples. Training on $\tilde{\mathbb{g}}$ is occasionally helpful, but structure modification improves image quality most of the time. Therefore, the compact model structure is the main reason why $(R_{I},f_{H})$ outperforms $R_{H}$ for computational imaging.

Comparison with weight pruning. We also compare image quality between the proposed algebraic sparsity and the unstructured magnitude-based weight pruning. While RingCNNs are trained directly, real-valued CNNs are first pre-trained, then pruned, and finally fine-tuned. Fig. 11 shows the comparison results, and RingCNNs over $(R_{I},f_{H})$ can deliver better image quality than the weight pruing for compression ratios 2$\times$, 4$\times$, and 8$\times$. In particular, the 2-tuple networks can even outperform the original (1$\times$) real-valued networks in many cases. This shows that the algebraic sparsity can serve strong prior for CNN models. As a result, $(R_{I},f_{H})$ not only provides more regular structures but also achieves better quality for computational imaging. A case for recognition tasks, though not the focus of this paper, is also studied in Appendix C, where convolutions and corresponding non-linear functions are implemented with $(R_{I},f_{H})$, and batch normalization is remained as real-valued operations.

Fixed-point implementation. For comparisons in hardware efficiency, we implemented highly-parallel FRCONV engines, as depicted in Fig. 5(c) with non-linearity, for different rings. Their RTL codes are synthesized with 40 nm CMOS technology. For quality comparison, the models are quantized in 8-bit and then fine-tuned using the setting at the bottom of Table III. The quality loss due to quantization is similar for each ring, and Fig. 12 shows the comparison results. The area efficiencies are very close to the estimated 8-bit complexity in Table I because convolutions dominate the areas. The proposed $(R_{I},f_{H})$ can provide the smallest area and the best quality at the same time. Compared to the CirCNN-alike $R_{H4\mbox{\scriptsize{-I}}}$ and HadaNet-alike $R_{H4}$, it has nearly 0.1 dB PSNR gain for the SR$\times$4 task and provides 1.8$\times$ and 1.5$\times$ area efficiencies respectively. In summary, $R_{I}$ can save area efficiently, and $f_{H}$ can recover image quality significantly.

Training setting and model selection. To show competitive image quality, we further train models using the polishment setting in Table III with two large datasets, DIV2K [2] and Waterloo Exploration [33]. We consider two throughput targets for hardware acceleration: HD30 for Full HD 30 fps and UHD30 for 4K Ultra-HD 30 fps. For each throughput target and application scenario, we adopt the compact ERNet configuration for the real-valued eCNN in [21]. It has been optimized over model depth and width in terms of PSNR, and we build its corresponding RingCNN models with $(R_{I},f_{H})$.

Floating-point models. The PSNR results are shown in Table IV. The RingCNN models show significant gains over the traditional CBM3D [11] for denoising and VDSR for SR$\times$4. Compared to the advanced FFDNet and SRResNet, the models for eRingCNN-n2 can outperform them with PSNR gains up to 0.15 dB at HD30 and have similar quality at UHD30. With 75% sparsity, the models for eRingCNN-n4 still give superior quality and only show noticeable PSNR inferiority for denoising at UHD30 due to the shallow layers.

Dynamic fixed-point quantization. On the top of Fig. 13, we show the effect of the 8-bit dynamic quantization which is used to save area. The quality degradation for ring tensors is around 0.11-0.12 dB of average PSNR drops, which is similar to the case of using real numbers. We also show the effect of applying sparse ring algebras (eCNN$\Rightarrow$eRingCNN) at the bottom of Fig. 13. The degradation is not obvious for $n=2$, and the models for eRingCNN-n2 even outperform those for eCNN by 0.01 dB on average. For $n=4$, those for eRingCNN-n4 only suffer small PSNR degradation in 0.11 dB.

Implementation and CAD tools. We developed RTL codes in Verilog and verified the functional validity based on bit- and cycle-accurate simulations. Then the verified RTL codes were synthesized using Synopsys Design Compiler with TSMC 40 nm CMOS library. And SRAM macros were generated by ARM memory compilers. We used Synopsys IC Compiler for placement and routing and generating layouts for five well-pipelined macro circuits which constitute eRingCNN collectively. Finally, we performed time-based power consumption using Synopsys Prime-Time PX based on post-layout parasitics and dynamic signal activity waveforms from RTL simulation.

Hardware performance. We show the design configurations and layout performance in Table V. The areas of eRingCNN-n2 and eRingCNN-n4 are 33.73 and 23.36 $\mbox{mm}^{2}$ respectively, and the corresponding power consumptions are 3.76 and 2.22 W. They mainly differ in the ring dimension, and eRingCNN-n4 uses only a half number of MACs and a half size of weight memory compared to eRingCNN-n2.

Area and power breakdown. The details are shown in Table VI. For eRingCNN-n2, the convolution engines contribute 57.42% of area and 86.51% of power consumption for the highly-parallel computation. And for eRingCNN-n4 their contributions go down to 45.63% and 76.56%, respectively, because of the saving of MACs. In addition, for a larger $n$ the directional ReLU uses more adders and causes wider bitwidths. Therefore, for the RCONV-3$\times$3 engines it occupies only 3.4% of area for eRingCNN-n2 but up to 8.9% for eRingCNN-n4. Accordingly, the inference datapath in eRingCNN-n4 is also 0.53 $\mbox{mm}^{2}$ larger than that in eRingCNN-n2.

Comparison with eCNN. As shown in Fig. 14, the RCONV engines reach near-maximum hardware efficiencies $(\cong n)$. Those in eRingCNN-n2 achieve 2.08$\times$ area efficiency and 2.00$\times$ energy efficiency. And those in eRingCNN-n4 further increase the efficiency gains of area and energy to 3.77$\times$ and 3.84$\times$. The numbers for the whole accelerator are as high as 1.64$\times$ and 1.85$\times$ for eRingCNN-n2, and 2.36$\times$ and 3.12$\times$ for eRingCNN-n4. In addition, we compare their quality-energy tradeoff curves in Fig. 15. The eRingCNN accelerators show clear advantages over eCNN; in particular, the low-complexity eRingCNN-n4 is preferred when less energy is allowed to be consumed for generating one pixel. Finally, as eCNN, the eRingCNN accelerators demand only 1.93 GB/s of DRAM bandwidth for high-quality 4K UHD applications.

Comparison with Diffy. Table VII compares the hardware performance of Diffy¹¹1We project the silicon area and power consumption of Diffy under 40 nm technology based on the scaling comparison of 65 nm in [45]: 2.35$\times$ gate density and 0.5$\times$ power consumption under the same operation speed. [34], another state-of-the-art accelerator for computational imaging, along with eCNN. Diffy applies optimization on bit-level computation which is hard to compare with eRingCNN directly; therefore, we perform comparison based on the same application target: FFDNet-level inference at Full-HD 20fps. In this case, the energy efficiencies of eRingCNN-n2 and eRingCNN-n4 over Diffy are 2.71$\times$ and 4.59$\times$ respectively by running at 167 MHz.

Comparison with SparTen, TIE, and CirCNN. Table VIII compares with the state-of-the-arts for different sparsity approaches: SparTen (natural) [16], TIE (low-rank) [12], and CirCNN (full-rank). Here we compare synthesis results because only such numbers are reported for SparTen and CirCNN. For comparing over different compression ratios, we consider an equivalent throughput which corresponds to the computing demand of the target uncompressed or real-valued model. With only 2-4$\times$ compression, our eRingCNN accelerators already provide competitive energy efficiencies as equivalent 19.1-28.4 TOPS/W. In contrast, SparTen merely achieves 2.7 TOPS/W due to significant overheads for handling irregularity. Although TIE is very efficient for highly-compressed fully-connected (FC) layers, it shows inefficiency for the CONV layers with lower compression ratios. Finally, CirCNN only provides 10.0 TOPS/W using as high as 66$\times$ compression. The potential of algebraic sparsity is therefore demonstrated, in particular on moderately-compressed CNNs for computational imaging.

Among the proper rings found in Section III-C, $R_{H}$ and $R_{O4}$ directly have the associative property of multiplication by Corollary B.3.1 since they are diagonalizable over $\mathbb{R}$. The rest of them ($\mathbb{C}$, $R_{H4\mbox{\scriptsize{-I}}}$, $R_{H4\mbox{\scriptsize{-II}}}$, $R_{O4\mbox{\scriptsize{-I}}}$, and $R_{O4\mbox{\scriptsize{-II}}}$) all possess the commutative property of permutation matrices and thus have the associative property as well by Theorem B.3.

Recent pruning techniques for image recognition have mainly focused on structured pruning for their practical efficiency. In Fig. C-1, we compare RingCNN models to LeGR [10] which is the state-of-the-art for structured filter pruning. RingCNN models show their potential by outperforming LeGR for high computation efficiency (left) and for a wide range of compression efficiency (right). The study on hardware accelerators in this aspect will be our future work.