Quantum Sparse Coding and Decoding Based on Quantum Network

Sparse coding and decoding refers to compressing high-dimensional data sets into their lower-dimensional representations sparsely and reconstructing higher-dimensional data sets from low-dimensional ones WOS:000961147200001 ; WOS:000406868800017 ; WOS:000258641500006 , which plays a crucial role in the fields of image compression, storage and processing WOS:000085444700052 ; WOS:000486237600006 ; WOS:000277884900014 . Sparse coding originates from the fact that the receptive field of the visual stripe cortex produces a sparse response to visual information. To be specific, most neurons of the visual stripe cortex are in a static resting state, while a small number of neurons are in the stimulated state WOS:A1996UQ65700051 . Mathematically, based on data structure analysis, digital image processing, and advanced algebra, CSC algorithms are developed gradually, where an overcomplete dictionary set is trained by various training algorithms, and sparse representation is a linear combination of the overcomplete dictionary set and active neurons in a high-dimensional space originally with redundant information WOS:000277884900014 . More explicitly, the sparse solutions of underdetermined equations are able to compress the data to a certain extent allowing little information loss of restoration. Currently, CSC algorithms have been implemented by hardware friendly, owing to their simple linear matrix operation in classical computer.

In contrast, quantum computing is an emerging frontier with higher information security and faster parallel acceleration. Specifically, quantum algorithms are expected for its potential advantages in quantum transmission, storage and encryption WOS:000343620500001 ; WOS:000431301800015 ; WOS:000454369600010 ; WOS:000514356400004 ; WOS:000209372300002 ; WOS:000340140200011 ; WOS:000646064000002 ; WOS:000888558600009 ; WOS:000685625800001 ; WOS:000935994100004 . Therefore, with the breakthrough of quantum machine learning algorithms and quantum circuits WOS:000410555900033 ; WOS:000390793900027 ; WOS:000476555600001 , more cutting-edge quantum algorithms are gradually moving toward hardware WOS:000940796200001 ; WOS:000711864600001 ; WOS:000367200400001 ; WOS:000340611600001 ; WOS:A1994NV63100015 ; WOS:000830702800001 ; WOS:000604141000011 ; WOS:000471037100002 ; WOS:000491132700075 ; WOS:000467374300005 ; WOS:000466034400001 ; WOS:000457547900068 , which is currently a possible development that challenges Moore’s Law WOS:000888210300020 . Meanwhile, there is no efficient CSC algorithm to handle quantum states. Inspired by CSC algorithms WOS:000486237600006 and quantum circuit models WOS:000390793900027 ; WOS:A1995QT40100017 , we propose a quantum algorithm named Quantum Sparse Coding and Decoding (QSCD), which can sparsely code quantum states from high to low dimensional Hilbert space and then decode them inversely. Specifically, the quantum networks are trained by gradient-based descent method WOS:000444202100003 ; WOS:000246314600001 ; WOS:001001538500030 , and can be realized by universal multi-port design interferometers in optical quantum circuits WOS:000390793900027 ; WOS:000316614600026 ; WOS:000289982600014 . The simulations fully demonstrate that our QSCD algorithm is able to deal with the real and complex quantum states efficiently, which can be applied to further industrial problems as a cutting-edge technology.

are the normalized pixels which satisfy $\sum_{n=0}^{N-1}r_{mn}^{2}=1$ for all $m=1,\cdots,M$. It is worth noting that $\left\{r_{mn}\right\}$ alone does not contain the complete information of the original pixels, but when combined with the normalization factor $\sigma_{m}$, all the pixels of the $m$th grayscale image can be fully reconstructed.

In quantum sparse coding of grayscale images, we only need to focus on the normalized pixel vector $\left\{r_{mn}\right\}$ as it contributes to almost all the cost associated with image transmission and storage. It is convenient to write a general $N$-dimensional quantum state in the computational basis as follows

Here, each basis state $|n\rangle\in\{|00\cdots 00\rangle,\ |00\cdots 01\rangle,\ \ldots,\\ \ |11\cdots 11\rangle\}$ corresponds to the binary representation of an integer $n\in[0,N-1]$, and the expansion coefficients are normalized, $\sum_{n}r_{n}^{2}=1$. In this way, classical information can be properly encoded into the expansion coefficients $\{r_{n}e^{i\phi_{n}}\}$ of $\ket{\psi}$ which, further serves as an input of our quantum sparse coding network. In particular, the quantum representation of normalized pixel vectors $\{\psi_{m}\}$ now become $\ket{\psi_{m}}=\sum_{n}r_{mn}e^{i\phi_{mn}}|n\rangle$, with phase parameters being 0, $\phi_{mn}=0$.

We proceed to detail our network architecture. Our sparse coding network aims to transform a collection of $N$-dimensional quantum states into lower dimensional ones while preserving, as much as possible, the information encoded in the input states. Considering that distinct classical information must be encoded in orthogonal states to be perfectly distinguishable under quantum measurements, our basic assumption in this work is that a coding network preserves orthogonality. In other words, we focus on quantum networks that implement unitary transformations. Furthermore, our main interest is in two particular types of quantum network structures, called Order type and Cross type. Order type links quantum gates sequentially, while Cross type interconnects them in a cross-pattern. Numerical analyses show Order type excels at lower layers, whereas Cross type is better for deeper networks (see supplementary information I).

General unitary transformations on an $N$-dimensional Hilbert space $\mathcal{H}_{N}$ can be factored into a series of unitaries on 2-dimensional subspaces of $\mathcal{H}_{N}$ (single-qubit gates), which leave quantum states in the respective $(N-2)$-dimensional complementary subspaces invariant. Each specific factoring then determines a specific network structure, and the network parameters are exactly the variables describing individual single-qubit unitary gates. Notably, the network structures based on different ways of factoring unitary transformations may have different advantages and limitations. In the following, our discussions will revolve around the optimal factoring proposed in Cross-type network (see supplementary information I), which is comparatively more efficient for deep networks. Its physical realization based on optical circuits consists of $l_{E}$ layers of unitary gates on two neighboring optical modes, involving approximately $(N-1)l_{E}$ unitary gates in total WOS:000390793900027 . More specifically, a unitary gate $U_{n}$ on two optical modes $\{\ket{n},\ \ket{n+1}\}$ can be implemented by a phase shifter at one mode (with phase parameter $\alpha_{n}\in[0,2\pi]$) followed by a two-mode Mach-Zehnder interferometer (with parameter $\theta_{n}\in[0,\pi/2]$)

Each layer of our coding network consists of $N/2$ parallel single-qubit unitary gates on the optical modes $\{\ket{2(k-1)},\ket{2k-1}\}\ (k=1,2,\cdots,N/2)$, followed by another $N/2-1$ parallel single-qubit unitary gates on the modes $\{\ket{2k-1)},\ket{2k}\}\ (k=1,2,\cdots,N/2-1)$ (see Fig. 1 for an example and supplementary information I for more details). Then, the $l$-th layer of our coding network implements the transformation

The overall coding network then essentially implements $l_{E}$ unitary transformations consecutively

which is described by $2l_{E}\times(N-1)$ real parameters WOS:000390793900027 . In supplementary I, we also discuss another network structure and compare it with the above one numerically.

As for the decoding network, a direct choice is simply the mirror inversion of the coding network, which performs the inverse transformation to recover the initial states from the output of the coding network. However, if some input state (2) is not transformed into the output space completely, say a $d$-dimensional Hilbert space $\mathcal{H}_{d}$ spanned by the last $d$ optical modes, then the actual outputs are described by the (unnormalized) states

with $P_{0}=\sum_{n=0}^{d-1}|n\rangle\langle n|$ the projector onto $\mathcal{H}_{d}$. This projector is critical for realizing sparse coding. It retains prominent features of the initial set of states while discarding unimportant information. In accordance, input photons of the decoding network are prepared into the normalized states $\{\braket{\chi_{m}}{\chi_{m}}^{-\frac{1}{2}}\ket{\chi_{m}}\}$ which, after some decoding transformation $T_{D}$, become

In the above, $\{R_{mn}\}$ are real parameters, and $R_{mn}^{2}$ is the probability of detecting a photon at the $n$th output mode when prepared in the $m$th initial state.

We introduce the overall loss function $L$ for reconstructing the initial states $\{\ket{\psi_{m}}\}$ from outputs of the decoding network as follows

where $J=R_{mn}e^{i\varphi_{mn}}-r_{mn}e^{i\phi_{mn}}$. With this loss function, we can update the parameters in our network during the training process to minimize potential errors. In the ideal case $L=0$, the information encoded in the initial states can be fully extracted without error. If the classical information is encoded only in the real coefficients $\{r_{mn}\}$ of the initial states $\{\ket{\psi_{m}}\}$ (2) and the decoding network implements the inverse transformation, a simpler and more convenient loss function is

with $P_{1}=\sum_{n=d}^{N-1}|n\rangle\langle n|$ denoting the projection onto the subspace of $\mathcal{H}_{N}$ that is complementary to the output space $\mathcal{H}_{d}$. Note that $P_{0}$ and $P_{1}$ sum up to the identity operator on $\mathcal{H}_{N}$ and quantum states are normalized $\braket{\psi_{m}}{T_{E}^{-1}P_{0}T_{E}}{\psi_{m}}+\braket{\psi_{m}}{T_{E}^{-1}P_{1}T_{E}}{\psi_{m}}=1$, thus $L_{inv}$ is exactly the probability that output photons are not detected at the last $d$ modes. $L_{inv}=0$ necessarily means that the encoding network successfully transforms all the initial states into the output space $\mathcal{H}_{d}$, $\ket{\chi_{m}}=T_{E}\ket{\psi_{m}}$, and the decoding network would perfectly recover those initial states.

We remark here that if the initial states span a space with a dimension larger than the dimension $d$ of the output space, the loss function $L_{inv}$ would never approach 0 in the training process. Consequently, there is also no reason to believe that a decoding network implementing the inverse transformation $T_{D}=T_{E}^{-1}$ is optimal for minimizing $L_{inv}$ or, equivalently, maximizing the fidelity between the initial states and final states. This is in fact a much more general situation. To mitigate error, we may properly train the decoding network to exploit, besides the features of individual input states, also the structure of the input set. Moreover, properly extending the depths of our coding and decoding networks is expected to further enhance the overall fidelity.

In conclusion, we fashion the decoding network of the QSCD algorithm either by inverting the coding network or through parameter training of the decoding network. In the case of the decoding network with parameter training, the QSCD algorithm can be smoothly implemented in experiments, ensuring that parameter values remain within their designated range. Additionally, our approach to updating network parameters allows for concurrent updating of both the sparse coding network and the decoding network, solely through measuring the output states of the decoding network, eliminating the need for intermediate state measurements.

To see the performance of our network numerically, next we apply it to the $5\times 5$ binary images of the 26 English capital letters as shown in Fig. 2a. Each binary image requires 5 qubits to be faithfully represented $\left(D_{1}D_{2}=25,N=2^{5}\right)$. We choose a coding network with depth $l_{E}=20$ and a decoding network with depth $l_{D}=25$, the number of sparse coding channels is $d=4$. Thus, the coding network and decoding network involve $480$ and $600$ rotation parameters $\theta_{k}^{l}$, respectively (see supplementary information II). Further, we set the learning rate to be $\eta=0.01$ and update the network parameters through 150 iterations. In each iteration, the output states of the decoding network are described by Eq. (7), with which we are able to compute the respective loss function according to Eq. (8). After each iteration, we then utilize the Quantum Natural Gradient Descent (QNGD) algorithm (see supplementary information III) to update the network parameters to reduce loss. The evolution of the real quantum states can be implemented by optical circuits to train and adjust the quantum gate parameters (see Methods). On the basis of the decrease of the loss function (Fig. 2c.), the reconstruction loss gradually tends to 0, and the gradient renewal of $\theta$ gradually approaches 0 (Fig. 2d), with the parameters $\theta$ gradually leveling off in the ranges of $\left[0,\frac{\pi}{2}\right]$. Eventually, the quantum network training is completed. The measurement results of output quantum states are converted to classical data $\hat{x}_{mn}$ (Fig. 2b):

where $\sigma_{m}=(\sum_{n}x_{mn}^{2})^{1/2}$ is normalization factor of the respective pixel vector $X_{m}$ as defined in Eq. (1). The implementations mean that the classical information can be sparsely encoded and decoded through the QSCD algorithm, where the data flow is $X_{m}\rightarrow r_{mn}\rightarrow\left|\psi_{m}\right\rangle\rightarrow T_{D}P_{1}T_{E}\left|\psi_{m}\right\rangle\rightarrow R_{mn}\rightarrow\widehat{X}_{m}$. Undoubtedly, it also makes sense to achieve sparse coding and decoding of high-frequency or low-frequency signals, which provides effective ideas for pattern recognition.

The superiority of the QSCD algorithm over traditional QNN based on Hamiltonian evolution is embodied in its lower computation complexity, better hardware-friendly performance, and ability to deal with higher dimensional information. The current optical quantum computing devices are more suitable for sparse quantum unitary transformation in our QSCD network, which increases the experimental efficiency and is expected to solve more quantum problems such as calculating eigenstates. Moreover, the differences between QSCD and CSC root in their computation frameworks, where the QSCD is constructed by multiple sparse matrices with the same dimensions, and the CSC is composed of one matrix full of weight parameters. In more detail, the two real parameters are updated together in each sparse matrix in the QSCD algorithm, nevertheless, the full weights are adjusted in a dictionary set in the CSC algorithm.