Lean classical-quantum hybrid neural network model for image classification

Let’s first organize our theoretical framework. The quantum machine learning algorithm proposed in this study is a classical quantum classical process. We position the classical convolutional neural network at the front end of VQCs for feature extraction while reducing the spatial dimension to meet the quantum bit requirements of LCQHNN[40]. Although we have reduced the spatial dimension of the image, CNN maps the information that needs to be focused on to a higher feature dimension by increasing the number of channels, which is used to extract hidden features and improve accuracy[41]. Therefore, the features extracted by the CNN algorithm can be used as inputs for VQCs, where classical state data becomes quantum states. VQCs is the core of our algorithm, similar to the general approximation theorem in Artificial Neural Network. VQCs represents a circuit that can fit the objective function. During the training process, VQCs will repeat the feedforward process and compare the results with the true labels of the training data. By iteratively adjusting the parameters of the variational quantum circuit, the cost function is minimized in terms of the difference between the predicted label and the true label. Finally, the decoded measurement results provide the classification results of the input data, at which point the data state returns from quantum state to classical state.

As shown in Figure 1, we construct the quantum algorithm model from five aspects to achieve image classification. Firstly, the feature extraction model of LCQHNN consists of two convolutional layers and two pooling layers. The convolutional layers are used to extract information to ensure reliable information transmission throughout the entire network architecture. Reducing the spatial dimension of the pooling layer can decrease the number of parameters and computational complexity in subsequent layers, making the network more efficient. In the second aspect, to ensure non-linear features, we added an activation layer composed of ReLU function between each convolutional layer and pooling layer. This work achieves nonlinear mapping of feature vectors. We assume that $f$ is the input image, $g$ is the convolution kernel, $(i,j)$ are the coordinates of the input image, and $(m,n)$ are the coordinates of the summation. So the convolution result should be $(f*g)(i,j)=\sum_{m=-\frac{k-1}{2}}^{\frac{k-1}{2}}\sum_{n=-\frac{k-1}{2}}^{\frac{k-1}{2}}f(m,n)g(i-m,j-n)$. Where k is the size of the convolution kernel, we set it to 3. The formula for ReLU is $max(0,x)$. We assume that $Window(i,j)$ represents a pooling window centered around $(i,j)$, and $P(i,j)$ is the value of the pooled output feature map at position $(i,j)$. So the extracted feature results are as follows[42].

Thirdly, after feature extraction, we set up a transition model composed of fully connected networks, which reduces the dimensionality of features from 256 to 4 to accommodate quantum computing constraints. Due to the property of parameter sharing in convolutional neural networks, neurons in the network may develop complex co adaptation relationships. Therefore, we incorporate Dropout as a regularization technique into the module, which reduces co adaptation relationships among neurons by randomly discarding them. Intuitively described, it can be seen that each Dropout generates a different "sparse" network, which is equivalent to training many different networks and averaging their prediction results, which usually helps prevent overfitting of the training data by the network.

Fourthly, we use quantum encoding technology to transform the 4-dimensional feature vector into a 4-dimensional 4-bit quantum state, followed by a series of carefully designed single bit operations. Finally, by measuring the transition of quantum states to classical states, the classification results are obtained. We will provide a detailed description of the quantum algorithm in Section 2.2.

We employ quantum encoding techniques to process classical data[43]. Within the quantum deep learning model, the single-qubit unitary layer serves as the core structure for quantum state transformation, comprising a series of meticulously designed single-qubit operations[44]. These operations include, but are not limited to rotation gates, phase gates, and Hadamard gates, which collectively act on qubits to facilitate efficient processing and learning of quantum data. In the encoding layer of LCQHNN, we introduce Hadamard (H) gates and U1 gates for each qubit[45]. By appropriately selecting the rotation angle of the U1 gate in conjunction with the superposition characteristics of the H gate, we can effectively adjust and manipulate individual qubit states. Additionally, our model incorporates the controlled-not (CNOT) gate-a key multi-qubit operator that enables non-local state transfer through interactions between control and target qubits[46]. A notable feature of the CNOT gate is its parameter-free operation it relies solely on the current states of both qubits when they are in a superposition state. The application of this CNOT gate results in entanglement between them, introducing powerful non-linear characteristics into quantum computation. Through this innovative encoding structure, as depicted in Figure 2, the LCQHNN not only achieves efficient quantum encoding of classical data but also fully leverages principles from quantum mechanics such as superposition and entanglement, significantly enhancing both its learning capacity and expressive power.

After the data have undergone the encoding phase, they proceed to a crucial transition step, the parametrized layer, which is composed of tunable quantum gates that can perform delicate adjustments and transformations on the quantum state. The core function of the parametrized layer is to capture and enhance the intrinsic features of the input data through the evolution of quantum states. Within this layer, each qubit undergoes a rotation around the Y-axis, achieved by applying RY gates. The parameters of the RY gates are trainable, determining the proximity of the qubits to the $|0\rangle$ and $|1\rangle$ basis states in the quantum state space[47]. The modulation of this proximity directly affects the model’s representational power, significantly influencing the depth and breadth of feature extraction. These parameters can be precisely adjusted through classical optimization algorithms to suit specific classification tasks. During the training process, the algorithm iteratively optimizes the parameters to minimize the discrepancy between the model’s predictions and the actual labels. This process not only enhances the model’s sensitivity to data features but also strengthens its capability to address complex classification problems. The VQCs algorithmic procedure is formulated in Equations (2-7).

We first apply four Hadamard gates (H) to transform the four qubits from their initial state $|0\rangle$ to a uniformly stacked state $|\psi_{1}\rangle$. Among them, Hadamard transforms $|0\rangle$ into $\frac{1}{\sqrt{2}}(|0\rangle+|1\rangle)$[48].

Then four single qubit rotation gates $U_{1,i}(\delta i)$ are applied to each qubit of $|\psi_{1}\rangle$. The rotation angle $\delta_{i}$ is twice the input data $x[i]$. This operation rotates the phase of each quantum bit by $\delta_{i}$ radians, thereby generating a new quantum state $|\psi_{2}\rangle$.

Here three controlled NOT gates (CNOT gates) are designed in VQCs for the application of $|\psi_{2}\rangle$. The CNOT gate $CX_{i,j}$ is controlled by the i-th quantum bit and targets the j-th quantum bit. This operation entangles the information between quantum bits together, resulting in a new quantum state $|\psi_{3}\rangle$.

Afterwards, we apply four single qubit Y-axis rotation gates $RY(\eta_{i})$ to each qubit of $|\psi_{3}\rangle$. The rotation angle $\eta_{i}$ is a model parameter. This operation causes the state of each quantum bit to rotate by an angle of $\theta_{i}$ on the Y-axis of the Bloch sphere, thereby changing the phase and amplitude of the quantum state.

Then, we apply a series of CNOT gates to the $|\psi_{4}\rangle$ state, in order from right to left. This means that the rightmost quantum bit (3rd quantum bit) serves as the control bit first, followed by the 2nd quantum bit, and finally the 1st quantum bit. This sequential CNOT gate operation can be seen as an effect of "canceling" previous CNOT gate operations, as they are applied in the opposite order.

Finally, we apply Hadamard gates $H_{0}$ to the first qubit of the $|\psi_{5}\rangle$ state, while keeping the other qubits unchanged (using unit gates $I$). This operation is typically used in the final stage of quantum circuits to prepare for measurements.

To observe the impact of quantum gates and operations on the quantum state more intuitively, we have mapped the output states of the two circuits onto the Bloch sphere. The specific distribution of quantum states is shown in Figure 7. After the input bits pass through the H gates and U1 gates of the encoding circuit, they are uniformly distributed on the equatorial plane. At this point, the quantum bits are in an equal superposition state of the 0 state and 1 state. The uniform distribution of quantum states implies that the quantum information is uniformly encoded in the quantum state space, which is beneficial for quantum entanglement and quantum error correction. Moreover, the uniform distribution of quantum states also indicates that our encoding strategy does not introduce any specific preferences when processing sample data. This is crucial for enhancing the model’s generalization capability. Such uniformity helps ensure that the model does not overfit to specific features in the training data, thereby maintaining high predictive accuracy and robustness when facing new, unseen data. Figure 7(b) illustrates the distribution of quantum states after rotation around the Y-axis. The parameters of the RY gate determine the final position of the quantum states on the Bloch sphere. Since the Bloch sphere can only represent a single quantum bit, a uniform distribution does not mean that there is no specific entanglement formed between the quantum bits.

To substantiate and evaluate the effectiveness of the methodologies proposed within this manuscript, we have selected accuracy (Acc) as the evaluative criterion to measure the performance of disparate models. The computation of accuracy is defined by Equation (2), where N denotes the total number of classes, T signifies the total number of samples, the true class of each sample is represented by $y_{i}$, and the class predicted by the model is indicated by $\hat{y_{i}}$[53].

At the same time, we also compared the convergence speed of different models. Assuming $E_{A}$ represents the number of epochs for model A to converge and $E_{B}$ represents the number of epochs for model B to converge, the percentage improvement $Q_{AB}$ in the convergence speed of model A compared to model B can be calculated using the following formula.

We compared the LCQHNN against CNNs featuring fully connected modules with 4, 8, and 16 parameters, tracking the evolution of Accuracy throughout the training process. As can be seen from Figure 4(a), LCQHNN converges faster than a traditional cnn with 4 variable parameters, and the accuracy exceeds 99% by the sixth epoch, which is 75% quicker than its CNNs counterpart with the same parameters. Traditional CNNs necessitate a larger number of parameters to match this performance. Figure 4(b) displays the accuracy of the four models after 10 epochs of training. It is apparent that the model introduced in this paper performs on par with, and occasionally exceeds, the capabilities of sophisticated traditional neural network models on the digit classification task. This hints at the VQCs’s superior linear transformation capacity in comparison to the classical fully connected layers, allowing the model to concentrate on finer details, thereby attaining higher predictive precision. The data suggest that the LCQHNN possesses greater potential compared to conventional models.

To further elucidate the differences in model performance, we trained four models on the more challenging FashionMNIST dataset and evaluated their classification accuracy on a separate test set, analogous to the structure of the previously discussed MNIST dataset. The experimental results are presented in Figure 5. Figure 5(a) illustrates the fluctuation in accuracy throughout the training process for all four models, with LCQHNN’s rapid convergence being particularly evident. The LCQHNN exceeded an accuracy threshold of 90% on the test set after only five epochs of training, representing a 70.59% improvement in convergence speed compared to traditional CNN models with an equivalent number of parameters. Even models with a larger array of parameters, such as CNN-8 and CNN-16, did not demonstrate such notable efficacy. This phenomenon can be attributed to VQCs’s adeptness at exploring high-dimensional feature spaces, facilitating quicker comprehension and adaptation to the underlying characteristics of the data. Figure 5(b) delineates each models precision on the FashionMNIST test set after ten epochs these outcomes substantiate that within an identical constrained training period, LCQHNN achieves superior classification precision, corroborating that incorporating quantum attributes significantly enhances classification proficiency within a reduced training duration.

The complete training data is presented in Table 1. The MNIST dataset is relatively straightforward and is primarily used to explore the quantum characteristics within the LCQHNN. In terms of experimental evaluation metrics, there is no significant difference compared with traditional CNNs. However, on the more complex FashionMNIST dataset, it is evident that the LCQHNN outperforms the other three traditional CNNs of varying complexities in both convergence speed and final accuracy. The results indicate that the final test accuracy of the LCQHNN on FashionMNIST is 0.78% higher than that of CNN-16, 2.34% higher than CNN-8, and shows a significant improvement of 5.07% over CNN-4.

Neural network models are frequently considered as black-box systems due to their inherently complex nonlinear transformations and opaque internal structural parameters. This lack of transparency prompts researchers to investigate the focal points of these models. To enhance our understanding of the model’s focus in this paper, we introduce Gradient-weighted Class Activation Mapping (Grad-CAM), an interpretability technique that leverages gradient information to identify regions in an image that contribute most significantly to the model’s predictions for a specific class. By combining gradients with activation maps from convolutional layers to generate Class Activation Maps (CAMs), areas with substantial activation are superimposed onto the original image, thereby highlighting regions that play a critical role in class prediction. We visualize the training process through heatmaps, which provide clearer insights into shifts in the model’s attention direction. The detailed computational formula is presented subsequently[54].

We first calculated the partial derivative of the model output $y$ with respect to the k-th convolutional layer output $A^{k}$. $y$ is usually the score of the model for a specific category. This partial derivative represents the contribution of each convolutional layer output to the final classification result.

Then, on the feature map $A^{k}$ of the k-th convolutional layer, calculate the average of all partial derivatives $\alpha_{k}$ within the window centered on $(i,j)$. This average value $\omega_{k}(i,j)$ represents the average contribution of the k-th convolutional layer to the final classification result within the window.

Finally, multiply the contribution degree $\omega_{k}$ of all convolutional layers by their corresponding feature maps $A^{k}$, and sum up all layers. Then, apply the ReLU function to the results to ensure that only positive contributions are retained. The final $G$ is a heatmap that displays the region in the input image that contributes the most to the classification results of a specific category.

The visualization interpretation results of the LCQHNN on the MNIST dataset are depicted in Figure 6. We selected the last convolutional layer as the target layer to observe the significant areas the model uses for predictive concepts. Figure 6(a) presents the input original image, while Figures 6(b), 6(c), and 6(d) illustrate the visualization interpretation results of the model at different training stages. We use the heatmaps to represent the importance of various regions in the image for the target category. The closer the heatmap pixel color is to dark red, the more critical the model considers that area for predicting the target category. Initially, for the untrained model (Figure 6(b)), the visualization shows that the model’s focus is scattered and chaotic, with no discernible pattern. This indicates that the model’s initial cognition of the target category is inaccurate, and it has not yet learned effective features related to prediction. Subsequently, the model in the middle of training (Figure 6(c)) begins to learn some feature information from the training set data, the model starts to focus on the edge contours of the handwritten digits, signifying that it is gradually understanding some important features of the digit shapes through training. Finally, the model after completion of training (Figure 6(d)) shows that the areas of focus largely coincide with the shape of the handwritten digits. This demonstrates that the model has learned the feature information of the data and associated these features with the target categories after training. The model’s ability to accurately identify and locate the important areas of the handwritten digits confirms the effectiveness of the LCQHNN on this task. Moreover, this indicates the practicality of the VQCs in neural networks, enhancing the model’s expressive power and performance by combining classical and quantum computing advantages. By leveraging the unique features of quantum computing, VQCs can offer advantages in handling complex data patterns and tasks, providing superior predictive and interpretive capabilities.

Furthermore, our investigation extends to visualizing the decision-making process of the LCQHNN on the FashionMNIST dataset. As depicted in Figure 3, the visualization outcomes allow us to distinctly perceive the progression of the model’s focal points of attention. Initially, during the nascent training phase, the model’s attention was divided, and as the training progressed, the model gradually redirected its attention to the key components of the image, eventually accurately identifying the object in question. Nonetheless, contrary to the MNIST dataset, the heatmaps on FashionMNIST do not manifest a saturated phenomenon even after successive training iterations, a discrepancy potentially owing to the visual intricacy inherent in FashionMNIST images. By juxtaposing the heatmaps that evolve with each training epoch against the accuracy curves, a pronounced correlation emerges. As the model’s concentration on the predictive targets intensifies, there is a concomitant rise in test accuracy. This correlation substantiates that the model indeed seizes upon critical image data features throughout the training regimen, with these features being instrumental in the amelioration of classification proficiency. The visual analytics within this research not only offer a clear vantage point for deciphering the LCQHNN’s decision-making apparatus but also underscore the model’s capacity to refine its performance through attentive readjustments during the learning trajectory when confronted with sophisticated image data. These revelations furnish persuasive evidence for delving deeper into the nascent potential of quantum computation within the realm of deep learning, thereby establishing a groundwork for the innovation of hybrid quantum-classical models that are both more efficacious and precise.