Circuit Design based on Feature Similarity for Quantum Generative Modeling

QCBMs are a subclass of parameterized quantum circuits (PQCs) used for generative modeling and they contain a set of parameters, which are adjusted during a training process according to a loss function $\mathcal{L}(\theta)$. The QCBM aims to encode the target joint distribution $p(y)$ of a discrete dataset $y\in\{1,0\}^{n}$ into an $n$-qubit parameterized quantum state $\ket{\psi_{\theta}}$, which is, without loss of generality, evolved from the all zero input state $\ket{0}^{\otimes n}$, i.e., $\ket{\psi_{\theta}}=U(\theta)\ket{0}^{\otimes n}$ using a PQC $U(\theta)$. Measuring the output state yields $x\in\{1,0\}^{n}$ with $x$ sampled from the variational model distribution $q_{\theta}=|\Braket{x}{\psi_{\theta}}|^{2}$ which is called the Born rule. Unlike in classical models, one does not have access to the explicit model distribution. Therefore the loss function has to be estimated using quantum circuit measurements. Since the widely used Kullback-Leibler (KL) divergence, as an explicit loss function, requires a high number of samples, and is shown to be not trainable for large-scale QCBMs in Ref. [68], we use the implicit loss function, maximum mean discrepancy (MMD) [69]

where $\kappa(x,y)$ is a freely chosen kernel function [70]. As a standard choice in recent literature [28, 9, 68], we employ the Gaussian mixture kernel defined as

The parameters $\sigma_{i}$ are the bandwidths to adjust the width of the Gaussian kernel and $\|\cdot\|^{2}_{2}$ refers to the $l_{2}$-norm. For optimizing schemes, both gradient-free and gradient-based procedures have been proposed, although gradient-free schemes failed when scaling to larger number of parameters [27]. Therefore we use the stochastic gradient descent algorithm [71] and ADAM as meta-optimizer [72]. For the MMD loss, it is possible to obtain the gradient with respect to the $k$-th model parameters $\theta_{k}$ as

by shifting $\theta_{k}$ and performing a projective measurement to obtain samples. The distributions $q_{\theta_{k}}^{\pm}$ are generated by sampling from the parameter-shifted circuit with parameters $\theta^{\pm}_{k}=\theta_{k}\pm\frac{\pi}{2}$ [73].

In the field of quantum mechanics, the development of efficient classical representations of quantum wave functions has been a subject of extensive research, whereby tensor networks (TN) have demonstrated notable effectiveness. Matrix product states (MPS) are a particular type of TNs, whose tensors are arranged in a one-dimensional geometry, facilitating the representation of one-dimensionally entangled quantum states. The MPS provides means to efficient computation of quantum circuits with limited entanglement [74] via the density matrix renormalization group (DMRG) algorithm [75]. Although the original application of DMRG is ground state computation, it has also been successfully applied to unsupervised learning tasks [61], which enables the use of MPS as a pretraining method for quantum generative modeling. In this work, we implemented the synergistic pretraining framework [56], as schematically shown in Fig. 1, to mitigate the problem of barren plateaus in random initialization.

First of all, the MPS is classically trained to find a high-quality solution by a DMRG-like algorithm that minimizes negative log-likelihood [61, 76]. Theoretically, any probability distribution of an $n$-bit system can be represented through an MPS using exponentially large bond dimensions. By removing the components with small singular values in the MPS, we can obtain an approximate MPS, which is known to be locally optimal at each neighboring matrices in the MPS [77]. This truncation corresponds to dropping off minor components of the entanglement spectrum. In the present study, we did not explicitly set the upper bound of the bond dimensions, but instead we used a truncation cutoff of $5\times 10^{-5}$ to truncate the bond dimensions. We set the training iterations to 10 and the learning rate to $0.05$. In both of our experiments in Sec. III and Sec. IV the training concluded after 10 iterations with a realized maximum bond dimension of 5 for the bars and stripes dataset and 25 for the financial dataset. After classical pretraining, the obtained MPS solution is then transferred to a quantum circuit ansatz by analytical decomposition [62, 78]. In order to introduce inductive bias to the model, the next step consists of extending the decomposed quantum circuit by additional gates that change its connectivity, therefore changing its entanglement capabilities. Consequently, the extended circuit should be subsequently retrained on quantum hardware. In this work, we explore the effect of different circuit extensions on this subsequent training process. However, we use numerical simulations instead of real quantum hardware and leave it as a task for future research.

Since the MPS solution exhibits a high-quality solution only for a linear entanglement structure, we do not expect it to significantly improve the solution with limited entanglement even when retraining on a quantum device. However, the MPS solution can be used as a baseline. To learn long-range correlations, it is necessary to introduce more entanglement capacity, which raises the question of how to extend the baseline circuit in a way that uses a minimal amount of gates. For small datasets, connecting the qubits in an all-to-all fashion provided good results [56]. However, the number of parameters gets high with increasing qubits, which leads to deep circuits that are infeasible to train. Instead of all-to-all connections, we introduce an idea called the metric-based extension heuristic to reduce the gate count and introduce inductive bias by utilizing characteristics of the dataset.

The BAS dataset exhibits strong correlations between bits in the same row or column. In order to capture these correlations it is necessary to extend the circuit connectivity from the one-dimensional MPS model to a more connected ansatz. Since the BAS dataset only consists of a low number of observations for each binary feature, we use the hamming distance in Eq. (4) as a metric to determine the connections between the qubits. Computing this distance between each possible qubit combination gives us a distance matrix, which is visualized in Fig. 3 (a). By setting a threshold of 0.5, we decide the maximum distance between two qubits that are allowed to be connected, which results in the connections shown in Fig. 3 (b).

In our experiment, we compare the circuit extensions, shown in Fig. 4. The initial linear connections that result from decomposing the MPS can be chosen by a good guess. For the BAS dataset, we therefore switched the features {3,5} to prevent diagonal connections on the two-dimensional grid resembling the BAS-image. The nearest-neighbor extension can then be constructed by connecting each qubit to its neighboring qubits on that grid. This results in two connections for corner qubits, three connections for edge qubits, and four connections for inner qubits. Therefore, we need a total of 4 additional gates to extend the initial circuit. The nearest-neighbor extension can be seen as a problem-informed ansatz, as it aims to resemble the two-dimensional BAS grid. The metric-based extension, however, requires 10 additional connections. We further compare it to random connections with the same amount of extension gates. The problem-agnostic all-to-all extension connects each qubit to each other, requiring 28 extension gates.

The experiment was conducted using the statevector simulator from the IBM Qiskit Aer simulator and consists of 5 independent training runs with the same initial conditions for each circuit connectivity. The configuration included 9 qubits, with 1000 shots per iteration, and a total of 1000 iterations. The data was split into a training set and a test set with a ratio of 80/20, thus the train set consists of 11 images while the test set includes 3 images. To compute the loss function and its gradient the full training set was used in each iteration, and the kernel bandwidth was set to $\sigma=1.0$. In Fig. 5, we compare the performance of different circuit designs with respect to the MMD loss on the train and test set. The MMD values for both train and test set were computed after each training iteration to validate the model performance during the training process.

The circuit connectivity significantly influences the training process. Through evaluation of the MMD loss on the train data in each iteration, we observe a clear separation between each extension method according to the number of their extension gates. With zero extension gates, the linear baseline circuit performs the worst, closely followed by the nearest-neighbor extension with 4 additional gates. Compared to nearest-neighbor the metric-based extension with 10 extension gates shows a large improvement on the train set. However, random connections perform worse than metric-based, despite having the same number of parameters. The train set performance of the all-to-all is the best with 28 extension gates. Despite its good performance on the train set, the all-to-all extension shows an increasing MMD curve on the test set, which is a clear sign of overfitting. We attribute this to the fact that the all-to-all extension includes more parameters without any relation to the underlying dataset characteristics, which makes it more likely to overfit. The metric-based extension on the other hand shows the best performance on the test set, closely followed by the nearest-neighbor extension and random connections. While these observations hint at better generalization capabilities of the metric-based extension, an increasing slope can be observed. Therefore, the question whether these results hold with increasing training iterations or increasing circuit depth, remains open. However, despite having the same amount of parameters as the random connections, the metric-based circuit seems to resemble a lower bound on the MMD loss. These results further motivate our use of the metric-based extension heuristic on a real-world financial dataset.

We consider three types of JGBs, which differ by their maturity periods of 5, 10, and 20 years. The dataset can be freely downloaded from the official website of the Ministry of Finance Japan and consists of observations in daily frequency for the interest rates of each JGB, whereas we use a period from 2000-01-04 to 2025-02-28, as visualized in Fig. 6.

The first step in preprocessing the dataset is differencing the interest rates time series $S_{t}$ by $D_{t}=S_{t}-S_{t-1}$, to represent the day-by-day changes and ensure stationarity. Since the quantum circuit-born machine can only represent data in binary format, as each qubit represents a value of either 0 or 1 upon measurement, it is necessary to quantize the data distributions to a desired bit resolution. As commonly used in recent literature, we adapt the uniform quantization [86]. We convert a decimal value $(x)_{10}$ into its $n$-bit binary equivalent $(x)_{2}$ by

where $x_{\text{min}}$ and $x_{\text{max}}$ are the minimum and maximum values in the decimal dataset, respectively. The floor operation $\lfloor\cdot\rfloor$ ensures that the result remains an integer. The conversion from the integer representation of the binary number to the actual bitstring is straightforward. Conversely, to transform the binary value back to its decimal representation, we apply

In our experiments, we restrict the algorithm to utilize 12 qubits, therefore each of the 3 features has to be quantized to a 4-bit resolution.

In order to apply the metric-based extension heuristic, we rely on the variation of information metric $\widetilde{VI}$ in Eq. (II.3.2), since financial datasets are known to exhibit a low signal-to-noise ratio, as well as non-linear correlations that are not captured by linear metrics [88]. The distance matrix between the features is shown in Fig. 7 (a), and the connections between the qubits for a threshold of 0.95 are shown in Fig. 7 (b).

The particular threshold was set by the elbow rule to allow for sufficient connectivity, while maintaining a low number of parameters, as seen in Fig. 8.

As a good guess for the initial connections, we suggest to establish a linear connection between the bits that belong to one time-series feature, e.g. {0,1}, {1,2}, and {2,3}. By applying the metric-based extension heuristic we extend the circuit by 13 additional gates, as shown in Fig. 9. From visual inspection we see that bits within the same feature tend to be connected: {0,1,2,3} for the 5-year rate, {4,6} for the 10-year rate, {8,9,10,11} for the 20-year rate. These connections are expected since the bits represent the same feature in decimal format. The connections between different features are less frequent, although we observe that the bits {4,5} for the 10-year rate and {8,9,10} for the 20-year rate, are all-connected to each other. This could be attributed to the fact that the short end of the interest rate curve is more volatile and shares therefore less correlation with the long end, which is known to be more stable and less affected by short-term market movements [87].

The experiment was conducted using the statevector simulator of the Aer Simulator from IBM Qiskit and consists of 5 independent training runs with the same initial conditions for each extension method. The configuration included 12 qubits and 3 time series features, using 1000 shots per iteration over a total of 1000 iterations. The data was split into a training set and a test set with a ratio of 80/20, thus the train set consists of 4932 samples while the test set includes 1233 samples. The loss function was computed on the full training set but for evaluating its gradient we used a mini-batch size of 1000 samples which were randomly drawn from the train set in each iteration. The MMD kernel bandwidth was set to $\sigma=\frac{n}{4}=3$ as recommended in [68]. We compare the MMD loss during training, evaluated after each iteration on the train and the test set, for the linear, random, and metric-based extension, as shown in Fig. 10.

While being close to each other, the metric-based extension shows a better performance than random, which confirms the similar results of the BAS dataset. However, the separation between both loss curves is not as clear on the test set, which is why we did not include the graph. Nevertheless, we suggest these results indicate the metric-based extension is a useful approach to improve the training process also on real-world data that exhibit more noisy observations. Especially, with regards to restricting the number of parameters for trainability, the metric-based extension heuristic provided valuable guidelines for circuit design.