Quantum Kernel t-Distributed Stochastic Neighbor Embedding

t-SNE [28] is a non-parametric unsupervised learning method to visualize data. More concretely, t-SNE maps high-dimensional data $\bm{x}_{i}$ to lower dimensional data $\bm{y}_{i}$ while preserving the similarities between high- and low-dimensional data. The high-dimensional data similarities $p_{ij}$ between data $\bm{x}_{i}$ and $\bm{x}_{j}$ are defined using a Gaussian distribution:

where $N$ denotes the number of data, and $\sigma_{i}$ is a parameter determined by making a quantity called perplexity $\mbox{Perp}_{i}=2^{-\sum_{j}p_{j|i}\log_{2}{p_{j|i}}}$ equal to a user-specified value. The low-dimensional data similarities are defined using t-distribution with one degree of freedom:

In t-SNE, the initial positions of low-dimensional data are determined at random. The low-dimensional data are then iteratively moved to minimize the cost function $C$ defined by Kullback-Leibler(KL) divergence between the similarities of data distributions in the high- and low- dimensional spaces:

After the optimization, we can visualize data by plotting the resulting low-dimensional data. A caveat of t-SNE is that we cannot add data after the visualization; to add data to the visualization, we must redo the whole process from the beginning. This short-coming is resolved by modified versions of t-SNE which are discussed in the following subsections.

Parametric t-SNE [29] is the variant of t-SNE that can be applied to unseen data. It explicitly constructs a map from a high-dimensional space to a low-dimensional space by a neural network. Let us denote weights of a neural network as $\bm{w}$. A neural network $\bm{y}=f(\bm{x}|\bm{w})$ is trained to minimize the cost function $C$ in Eq. (3). The trained network allows us to visualize data, including unseen ones. In numerical experiments, we use parametric t-SNE to compare the accuracy of our proposed method.

We have proposed to use a quantum neural network instead of a neural network in parametric t-SNE to visualize both classical and quantum data [21]. Specifically, we apply a parametrized quantum circuit to a set of quantum states $\{\ket{\psi_{i}}\}$ to visualize them. The states $\{\ket{\psi_{i}}\}$ can either be intrinsically quantum, e.g. quantum states provided from physical experiments or from other quantum circuits, or those that encode classical data $\{\bm{x}_{i}\}$ through some quantum feature map. In this method, the high-dimensional data similarities are defined as

where

and $\|\cdot\|$ is Euclidean norm. On the other hand, the low-dimensional data $y_{\mu}$ is given by

for classical data, and

for quantum data. Note that $U(\bm{x}_{i},\bm{\theta})$ or $U(\bm{\theta})$ is a parameterized quantum circuit, $\{O_{\mu}\}_{\mu=1}^{d}$ are observables, and $d$ is the dimension of low-dimensional space. Also, note that $\beta_{\mu}$ is used as a trainable parameter instead of a hyper-parameter in Ref. [21] to increase expressibility in our numerical experiments.

Kernel t-SNE is the variant of t-SNE that can be applied to unseen data [24, 25]. Given a training dataset $\{\bm{x}_{i}\}$, the task here is to visualize a high-dimensional vector $\bm{x}$ on a $d$-dimensional space along with $\{\bm{x}_{i}\}$. The kernel t-SNE maps $\bm{x}$ to $d$-dimensional data (low-dimensional data) $\bm{y}$ as

where $N$ is the number of training data, $k$ is a kernel function, and $\bm{\alpha}_{i}\in\mathbb{R}^{d}$ are trainable parameters. In classical machine learning, a Gaussian kernel $k(\bm{x_{i}},\bm{x})=\exp{(-\|\bm{x_{i}}-\bm{x}\|^{2}/(2{\sigma^{\prime}}^{2}))}$ is often employed as the kernel function. The parameters $\bm{\alpha}_{i}$ are determined by minimizing the t-SNE cost function [24, 25]. This method is expected to accelerate the mapping of all data points through learning the map of t-SNE consisting of the subset. In this paper, we use this method to accelerate the visualization in terms of the number of access to a quantum computer. Specifically, we propose to use quantum kernels defined by inner products of quantum states. This allows us to visualize them without optimizing a parametrized quantum circuit in contrast to our previous work [21].

Before we describe our proposed method, we first refer to our notation. We are interested in visualizing a quantum state $\ket{\psi}$ along with a set of quantum states $\{\ket{\psi_{i}}\}_{i=1}^{N}$ provided as a training dataset. As in Sec. II.3, the states $\{\ket{\psi_{i}}\}$ and $\ket{\psi}$ can either be intrinsically quantum, e.g. quantum states provided from physical experiments or from other quantum circuits, or those that encode classical data $\{\bm{x}_{i}\}$ and $\bm{x}$ through some quantum feature map.

The outline of our proposed method is shown in Fig. 1. The procedure of our proposed method is described as follows:

1. 1.

   Calculate the high-dimensional data similarities $p_{ij}$ for all data pairs by Eqs. (II.3) and (5).

2. 2.

   Calculate the low-dimensional data $\bm{y}$ through Eq. (6) with $k(\ket{\psi_{i}},\ket{\psi})=|\langle\psi_{i}|\psi\rangle|^{2}$.

3. 3.

   Calculate the low-dimensional data similarities $q_{ij}$ and the cost function $C$ defined as Eq. (3).

4. 4.

   Update $\bm{\alpha}_{i}$ to minimize the cost function $C$.

After the optimization, we can visualize data by plotting the low-dimensional data $\bm{y}$ using the optimized $\{\bm{\alpha}_{i}\}$.

In this section, we explain the advantages of our proposed method compared to our previous approach [21]. Specifically, we compare the number of accesses to a quantum computer. Let us define $p$ as the number of parameters in a quantum circuit and $m$ as the number of iterations required to optimize the parameters within a circuit in our previous approach.

First, we compare the number of accesses to a quantum computer when visualizing classical data. In the present method, we need to evaluate $O(N^{2})$ fidelities to calculate the kernel function during training. For our previous approach [21], we need to use $O(Np)$ accesses for each iteration to evaluate the cost function and its gradient. So, $O(Npm)$ accesses are required for the whole optimization. The present method is therefore advantageous roughly when $mp>N$ in terms of the number of accesses to a quantum computer compared to the previous work. This condition is satisfied in a common situation. For example, in Ref. [30], they use a quantum neural network with $15$ or $60$ parameters and optimize the parameters with $400$ iterations to classify $360$ data into two labels. If we visualize the data, the present method performs visualization more efficiently than the previous one under the condition that the latter requires roughly the same number of iterations for the optimization of the quantum neural network as in Ref. [30].

Next, we compare the number of accesses when visualizing quantum data. We need $O(N^{2})$ accesses in the present method. On the other hand, the previous one [21] needs $O(N^{2})$ accesses to evaluate fidelities to calculate high-dimensional data similarities. Additionally, it needs $O(Np)$ accesses for each iteration to evaluate the cost function and its gradient. In total, $O(N^{2})+O(Nmp)$ accesses are required for the whole optimization. Therefore, our proposed method is more efficient for the visualization of quantum data than our previous work [21] without any assumptions in contrast to the case of classical data. We summarize the results in Table 1.

Here, we visualize hand-written digits data set provided in scikit-learn [27] as an example of visualizing classical data. Specifically, we split $80\%$ of the given data as training data and the remaining $20\%$ as test data. We employ the quantum feature map defined by a quantum circuit described in Fig. 2 with $12$ qubits. As a preprocess, we reduce the dimension of the data to $12$ by PCA. Additionally, we normalize each principal component from $0$ to $\pi/2$ so that the data can be input as the angles of quantum gates. As shown in Fig. 2, we input the $i$th principal component $x_{i}$ as angles of single-qubit rotations acting on qubit $i$ for $i=0,\ldots,n-1$, which are followed by $n$ CZ gates with control qubit $i$ and target qubit $(i+1)\pmod{n}$ to entangle the quantum states. Furthermore, input the principal components through single-qubit rotation gates’ angles again, namely data re-uploading, to improve the visualization performance. We optimize parameters $\bm{\alpha}_{l}$ in Eq. (6) with Adam [32]. The perplexity is set to $30$ and the initial values of $\bm{\alpha}$ are taken from uniform random values between $0$ and $1$. We show the resulting low-dimensional data in Fig. 3. The left figure visualizes the training data, and the right one visualizes the test data. We see that the present method can nicely form the clusters by the data belonging to each class representing digits.

Next, we quantitatively evaluate the visualization performance of our proposed method. Specifically, we investigate it by the classification accuracy of low-dimensional data generated by quantum and classical methods. If the visualization works well, the low-dimensional data form clusters according to the belonging classes, and, in addition, the clusters are separated from the others. In such a case, a low-dimensional point is surrounded by points belonging to the same class; therefore, $k$-nearest neighbor algorithm is effective for evaluating the visualization performance. In Ref. [29, 33], the visualization performance is evaluated by the accuracy of $k$-nearest neighbor algorithm of low-dimensional data points. In our numerical experiments, we classify the low-dimensional data using $k$-nearest neighbor algorithm and compare $5$-fold cross-validation accuracy with four conventional models. Specifically, the models to be compared are a kernel t-SNE with Gaussian kernels (t-sne(gauss)), a parametric t-SNE with a quantum neural network (t-sne(qnn)), a parametric t-SNE with a neural network with a single-hidden layer (t-sne(nn1)), and that with three hidden layers (t-sne(nn3)). We describe their details in Appendix A.

The accuracies are shown in Table 2. The accuracy of t-sne(qnn) is the worst for all values of $k$. It is because we could not use many parameters due to the long optimization time it causes. From Table 2, our model achieves almost the same accuracy with t-sne(gauss) but results in a lower accuracy than t-sne(nn1) and t-sne(nn3). While the present method could not beat the methods based on neural networks, we believe that achieving the comparable performance with respect to the gaussian kernel which can be seen as a basic traditional technique is an encouraging result; in fact, the state-of-the-art result in recognizing hand-written digits using quantum machine learning techniques is at this level [34]. This showcases the effectiveness of our approach.

In this section, we present an example of visualizing quantum data. More concretely, we visualize quantum states generated within optimization processes of VQE applied to transverse field Ising model:

where we set $n=8,J=-1.0,h=-0.75$. The task of the VQE here is to find the ground state of $H$ which we denote by $\ket{\psi_{\text{g}}}$. We use the so-called hardware-efficient type ansatz [9, 15], which is shown in Fig. 4. We set the depth $d=6$ and use three different initial parameters drawn from a uniform distribution between $0$ and $2\pi$. The parameters are optimized by BFGS algorithm implemented in SciPy [35] with $100$ iterations. We visualize the quantum state at each iteration of the optimization by our kernel t-SNE method. The optimization of parameters in Eq. (6) is performed by Adam [32] setting the perplexity to $10$ and the initial values of $\bm{\alpha}$ to uniform random values between $0$ and $0.1$.

We show the visualization result in Fig. 5. The initial states exist around the origin of the two-dimensional plane because Eq. (6) returns values that are close to zeros when a quantum state has low similarity to all other quantum states in the data unless $\alpha_{l}$ is large. They move away from the origin as the optimizations proceed. The trajectories of hardware efficient ansatz converge to the same final state $\ket{\psi_{\text{g}}}$. The trajectories of trajectory0 and trajectory2 first diverge but then they meet with each other after $30$ iterations. Their fidelities are actually larger than $0.90$ in this range. The trajectory of trajectory1, in contrast to the above two, passes nearby the $1$st excited state. Making this kind of observation has been impossible with the existing works. For instance, our previous approach [21] demands too much computational resource. Also, the approach proposed in [20] only looks at the parameters of the circuit and does not visualize quantum states themselves. With further experiments of this kind, we might be able to extract useful information in characterizing the optimization process of variational quantum algorithms, which may then lead to improved optimizers and ansatz designs.