Quaternion-based machine learning on topological quantum systems

The phase classification using machine-learning (ML) based techniques has been attracting intense attentions since the pioneering work in 2017 [1]. In addition to the classical phase detections [2, 3] where each phase is well defined by the corresponding order parameters, detecting topological phase transitions [2] is interesting and challenging [4] due to the lack of local order parameters. Recently, the phase detections and classifications have been performed via different ML techniques for classifying various topological invariants [5, 6, 7, 8, 9, 9, 10, 11, 12, 13, 14, 15, 16, 17, 6, 18, 19, 10, 20, 21, 22, 23, 24, 16, 25, 21, 18, 26, 18, 17, 27, 28, 29, 30, 6, 10, 4, 31, 32, 33, 34, 35, 26, 36], including the Chern number [14, 15, 16, 17, 6, 18, 19, 10, 20, 21, 22, 23], winding number [24, 16, 25, 21, 18, 26, 18], $\mathbb{Z}_{2}$ index [17, 27, 28, 29, 30, 6, 10, 4, 31, 32, 33, 34, 35, 26, 36], to name a few. In addition to the applied ML architectures, the forms of the inputs for training the machine also play a crucial role in determining the resulting performance of the topological phase detections [4].

For the topological systems with the Chern numbers or the winding numbers as the topological invariants, various types of inputs are used to perform the phase classifications. For instance, the quantum loop topography (QLT) is introduced to construct multi-dimensional images from raw Hamiltonians or wave functions as inputs [14, 17]. The Bloch Hamiltonians are arranged into an arrays to feed the neural networks [24, 16]. In addition, the real-space particle densities and local density of states [15] and the local projections of the density matrix [6] are also used as inputs. From cold-atom experiments, momentum-space density images were generated as inputs for classifications [20]. The time-of-flight images [10, 19], spatial correlation function [10], density–density correlation function [10] and the density profiles formed in quantum walks were also proposed as appropriate inputs [23]. Furthermore, the spin configurations [18] and the Bloch Hamiltonians over the Brillouin zone (BZ) have also been treated as inputs for the neural networks [21, 18]. For these forms of inputs mentioned above, various ML techniques with distinct real-valued neural networks have been applied to discriminate different topological phases.

As the development of artificial neural networks becomes mature, a raise of representation capability of machines is anticipated by generalizing real-valued neural networks to complex-valued ones [37, 38]. Specifically, a quaternion number, containing one real part and three imaginary parts, and the corresponding quaternion-based neural networks [39, 40, 41, 42] are expected to enhance the performance on processing of data with more degrees of freedom than the conventional real-number and complex-number systems. There have been various proposals about quaternion-based neural networks in ML techniques and applications in computer science, such as the quaternion convolutional neural network (qCNN) [43, 44, 38], quaternion recurrent neural network [45], quaternion generative adversarial networks [46], quaternion-valued variational autoencoder [47], quaternion graph neural networks [48], quaternion capsule networks [49] and quaternion neural networks for the speech recognitions [50]. However, the ML-related applications of the quaternion-based neural networks on solving problems in physics are still limited, especially in the topological phase detections, even though the quaternion-related concepts have been applied in some fields in physics [51, 52, 53].

In this work, we perform the Chern-insulator classifications from both supervised- and unsupervised-learning aspects based on the inputs transformed via the quaternion algebra. For the unsupervised learning, we encode the quaternion-transformed eigenstates of Chern insulators via a convolution function as inputs and study them using the principal component analysis (PCA). We found that using only the first two principal elements is not enough to fully classify the Chern insulators, consistent with Ming’s work [23]. Further studies show that the performance can be improved by including more principal components. For the supervised learning, we construct a quaternion-based neural network in which the first layer is a quaternion convolutional layer. We then show that this quaternion-based machine has better performance than a conventional CNN machine. Our machine is good not only for testing datasets but also for identifying data points that have different distributions from those seen by our machine in the training processes. The good performance can be attributed to the similarities between the formula of the Berry curvatures and our quaternion-based setup. Therefore, our work demonstrates the power of the quaternion algebra on extracting relevant information from data, paving the way to applications of quaternion-based ML techniques in topological phase classifications.

The outline of the remaining part of this work is as follows. In Sec. II, we introduce the model Hamiltonian, generating the data for our classification tasks, and the quaternion convolution layer used in this work. PCA analysis of the quaternion-transformed eigenstates is discussed in Sec. III. The data preparations, the network structures and the performance of the quaternion-based supervised learning task are given in Sec. IV. Discussions and Conclusions are presented in Sec. V and Sec. VI, respectively. We have three appendixes. Appendix A shows the details of data preparation. Appendix B provides a brief introduction to the quaternion algebra. Some properties of functions in Sec. III are included in Appendix C.

Principal component analysis (PCA) is a linear manifold learning that is to find the relevant basis set among data [56, 57].

We prepare eigenstates $\ket{u_{\pm}}$ of Eq. (1), where $+\ (-)$ stands for the upper (lower) band. For a topologically nontrivial state, the phase cannot be continuous over the whole BZ. Therefore, we can divide the whole BZ into two parts, in each part of them the topological wave function has continuously well-defined phase. We then pick up a gauge by choosing two regions according to the sign of $h_{3}$ in Eq. (3):

and

where $h_{\pm}=\pm\sqrt{h_{1}^{2}+h_{2}^{2}+h_{3}^{2}}$. In this choice of gauge, the first (second) component of $\ket{u_{+}}$ $(\ket{u_{-}})$ is real-valued when $h_{3}\geq 0$, and the second (first) component of $\ket{u_{+}}$ $(\ket{u_{-}})$ is real-valued when $h_{3}<0$.

By translating $\ket{u_{\pm}}\doteq(\alpha_{\pm},\beta_{\pm})^{T}$ with $\alpha_{\pm},\beta_{\pm}\in\mathbb{C}$, into a quaternion number of four components, we have

To see the correlation of states over $\vec{k}$, we define the quantity $F$ to be the quaternion-based convolutions:

where $q^{*}$ is the conjugate of $q$. It can be proved that $F$ is real-valued. Therefore, $F(\vec{p})$ of all $\vec{p}$ in the BZ based on a given Hamiltonian can be analysed by using PCA.

We collected various $F$ of all $\vec{k}\in$BZ within seven topological phases as the dataset for PCA. For each topological phase, 30 $F$’s were prepared, so the total amount of data was 210. The data for six non-trivial phases were generated based on Eq. (3) with $m=\pm 1$ (the sign of $m$ determines the sign of $C$). For the trivial phase, we prepared five data points from each of six combinations of $\{c,m\}$, where $c\in\{1,2,3\}$ and $m\in\{3,-3\}$, and then there are totally 30 data. To augment the number of data, we add Gaussian noises $\delta\mathbf{h}$ at every $\vec{k}$ of the model [Eq. (3)] such that $\mathbf{h}\to\mathbf{h}+\delta\mathbf{h}$ without closing the band gap.

In Fig. 4, we present various noiseless $F$ generated from Eq. (3) with different $c$ and $m$.

It is notable that $F$ for $C=0$ are featureless, $F$ for $C=\pm 1$ have a dipole moment, and $F$ for $C=\pm 2$ have a quadruple moment, and $F$ for $C=\pm 3$ seemingly have a primary dipole and a secondary quadruple moment. The remarkable features imply that the convolution function $F$ is a good choice for topological classifications.

We examine data with the standard deviation (SD) equal to 0, 0.1, 0.2 and 0.3 respectively, and show the first two PCs of 210 pieces of data for each SD in Fig. 5. In Fig. 5, data are clustered into four groups and their variances increase with SD. It is successful to separate different topological phases into different clusters via PCA. However, some clusters contain two topological phases of Chern numbers: $\{+1,-3\}$, $\{-1,+3\}$, and $\{+2,-2\}$. This $C$ modulo 4 resemblance has also be observed in a previous study [23].

We find that including more PCs helps separate different classes in each cluster. Figure 6 shows the first six PCs of data in topologically non-trivial phases, where PC$x$ denotes the $x$-th PC component. One can find that PC1 and PC2 in each pair of $\{+1,-3\}$, $\{-1,+3\}$, and $\{+2,-2\}$ are nearly identical, as also shown in Fig. 5. By incorporating more PCs up to PC6, all topological classes are completely classified. Via the proposed convolution, topological states can be successfully classified by using PCA, a linear classification machine.

Compared to the eigenstates, the spin configurations $\mathbf{n(}\vec{k})$ are gauge-invariant. Therefore, it is desired to classify the topology of the spin configurations via PCA. Unfortunately, the performance was not good, which will be discussed later. In order to directly classify the spin configurations, in the following, we train a qCNN machine via the supervised learning algorithm to discriminate spin configurations with different topological phases.

In this work, we apply the quaternion multiplication laws to both PCA (unsupervised learning) and qCNN (supervised learning). The two methods take different inputs; the former one takes scalar function $F(p)$, which is something to do with a convolution of the wave function, and the second one takes the pure quaternion function $(0,\mathbf{n}(\vec{k}))$, where real part is zero and the imaginary part is the spin vector. We will explain physical intuitions and comment the mechanisms in this section.

On PCA, we did not take $\mathbf{n}$ simply as the input because the representation of the vector $\mathbf{n}$ depends on coordinates but the topology is not. We believed that the topology as a global geometry should be embedded in the correlations. The correlation of dot products of $\mathbf{n}$ turned out to fail since relative angles of two spins were not informative to understand the swirling of $\mathbf{n}$ on $S^{2}$. If one tries the quaternion in the convolution Eq. (15) by $q=(0,n_{x},n_{y},n_{z})$, the result is still inappropriate for the convolution is independent of the sign of $m$ to discriminate topological states (see Appendix C). Eventually, we found that the $F(p)$ defined in Eq. (15) was a proper quantity to characterize topology after PCA. The $F$ has the property that it is featured (featureless) when the wave function is unable (able) to be globally continuous that happens in the nontrivial (trivial) phases. Unfortunately, the $F(p)$ is not gauge invariant. The results were based on the choice of gauge in Eqs. (12) and (13) that made the wave function continuous locally and discontinuous at $k$ where $n_{z}(\vec{k})=0$. We had examined other choices of gauge and found that the present gauge exhibited the PCA features most clearly (results not shown). We remark that our PCA results looked good because the inputs were ingeniously designed and the PCA method might not be more practical than the qCNN method.

For qCNN, it is interesting to understand the mechanism behind. There are several possible factors promoting the performance of our supervised learning machine. The first one is that the size of kernel in the first convolutional layer is 2 × 2 with stride = 1, which means the machine can collect spin information among four nearest neighbors [see Fig. 11(b)]. We know that the Chern number is the integral of the Berry curvature in the BZ, and the Berry curvature is twice of the solid angle. A solid angle $\Omega$ subtended by three unit vectors $\vec{a}$, $\vec{b}$, and $\vec{c}$ is obtained by

Our choice of the size of the kernel in the first hidden layer is the minimal of $2\times 2$ that mixes only the nearest-neighboring spins. In this way, it is very possible to enforce the machine to notice the solid angle extended in this plaquette. The second factor is the quaternion product. Recall that the conventional CNN might correlate spins $\mathbf{n}^{\prime}s$ in neighboring $\vec{k}^{\prime}s$ due to the feature map through the kernel. However, the map does not mix the components of spins. In comparison, the qCNN is more efficient for it directly entangle spins via the quaternion product. It is this entanglement of spin components by the quaternion product that makes the scalar and vector products in calculating the solid angle (see Eq.  (17)) become possible to be realized by the machine. As a solid angle involves at least three spins and the feature map by the kernel is just linear, a nonlinear transformation is crucial to create high-order (three spins) terms in the expansion. This is possible and proved in Ref. [60] that multiplication of variables can be accurately realized by simple neural nets with some smooth nonlinear activation function. Therefore, the third factor is the non-linear activation function, arctan in this work. We expect that using arctan as the activation function can further help the machine to learn correct representations because the calculation of a solid angle involves the arctan operation in Eq. (17). This belief is indeed supported by the results shown in Fig. 12, where the arctan activation function outperforms the ReLU and tanh activation functions over nine different datasets. In summary, several factors are combined to enhance the performance of our machine as follows. The quaternion-based operations in the q-Conv layer mix not only spins with their neighbors but also components of spins. When these linear combinations are fed into the non-linear activation functions in our qCNN, the output can be viewed as an expansion of a non-linear function, which may contain a term having both the scalar- and vector-product of neighboring spins, similar to that in Eq.  (17). Therefore, after the optimization process, the machine may keep increasing the weight of a solid-angle-related term and eventually learn to classify the topological phases.

Also, adding some noises to the training dataset helped our supervised-learning machine to learn the generic feature of our data. We found that when the training data was generated directly from Eq. (3) without adding any noise, the machine worked well for training and testing datasets but had poor performance on all the prediction dataset. This could be understood by noting that the topological invariant is determined by the sign of $m$, which appears in the $z$ component in Eq. (3). By using the dataset without noise, the machine might naively regard the $z$ component as the judgment of topology when the training data does not contain wide distribution. We note that the topology is invariant when the spin texture is uniformly translated or rotated. So we trained our machine with randomly translated and rotated data to avoid incorrect learning. (See data preparation in Appendix A.) From our observations, the performance on the prediction dataset was remarkably enhanced when the noise was included, which supports our idea.

In summary, we classify topological phases with distinct Chern numbers via two types of machine-learning techniques. For the unsupervised part, we propose a quaternion-based convolution to transform the topological states into the input data. With this convolution, distinct topological states are successfully classified by PCA, a linear machine for classification.

We then go to the supervised learning part where, in contrast to the conventional CNN, we successfully use the qCNN to classify different topological phases. This work demonstrates the power of quaternion-based algorithm, especially for the topological systems with the Chern number as the topological invariants.