Feature-energy duality of topological boundary states in multilayer quantum spin Hall insulator

The two-dimensional (2D) quantum spin Hall effect has attracted significant attention over the past two decades due to the unconventional spin propagation on the edge of the materials and its potential application in spintronics devices as well as quantum computations [1, 2, 3, 4, 5]. Quantum spin Hall insulator (QSHI), characterized by spin Chern number (SChN, $C_{s}$), are considered prime candidates for realizing the quantum spin Hall effect in condensed matter materials. The term QSHI has been utilized to describe the $\mathbb{Z}_{2}$ class of the topological insulator in 2D, attributed to the existence of gapless topological nontrivial helical edge states resulting from the bulk-boundary correspondence [6, 7]. However, this understanding is somewhat misleading. The $\mathbb{Z}_{2}$ index is ill-defined in bulk band structures if the original gapless edge states open a gap in the absence of time-reversal symmetry [7, 8, 9]. In contrast, the SChN can be defined in the thermodynamic limit and generalized the Chern number without time-reversal symmetry [10]. Consequently, the time-reversal breaking quantum spin Hall effect emerges as a form of QSHI classified by SChN [4, 8, 9, 10]. Following the well-established concept of bulk-boundary correspondence in non-trivial topological phases, we anticipate the appearance of a gapless topological edge state on the boundary of QSHI without time-reversal symmetry. However, research on bulk-boundary correspondence under symmetry-breaking conditions of QSHI has remained elusive for many years. In addition to time-reversal breaking in monolayer QSHI, bilayer QSHI with each layer carrying SChN $C_{s}=1$, exhibit $\mathbb{Z}_{2}=0$ and $C_{s}=2$ in the entire system. In this scenario, interlayer coupling hybridizes the gapless edge states of each layer. Since the electron spins are not generally conserved, such hybridization results in gapped edge state in bilayer QSHI with $C_{s}=2$ , challenging the concept of bulk boundary correspondence [11]. These two cases underscore the invalidity of the bulk-boundary correspondence in QSHI when the original gapless edge states are gapped by a perturbation.

In this work, we investigate the bulk-boundary correspondence via feature spectrum topology [10, 12, 13, 14] with $\hat{S}_{z}$ feature operator. We first introduce a perturbation involving a Zeeman exchange filed in monolayer Kane-Mele model [6, 7, 15]. Our calculations reveal the expected gapped edge state of QSHI in the E-k map under time-reversal symmetry-breaking perturbation. Concurrently, a non-trivial edge state emerges, connecting the spin-up and down sectors separated by a gap in the $\hat{S}_{z}$ feature spectrum. Moving on to the bilayer Kane-Mele model [16, 17, 18] with QSHI phase possessing $C_{s}=2$, we observe an energy gap in the edge states within the E-k map due to interlayer coupling. In contrast, a pair of non-trivial edge states manifest within the gap of the bulk feature spectrum, which equilavent to the SChN in the bilayer Kane-Mele model. To investigate the feature spectrum topology in a realistic material, we study the feature eigenvalues, topological invariants, and corresponding edge states in a van der Waals material $\rm{Bi_{4}Br_{4}}$, employing first-principles calculations. Our results demonstrate a feature-energy duality, revealing that the total number of gapless edge states in the E-k map and non-trivial edge states in the feature spectrum equals the SChN in multilayer $\rm{Bi_{4}Br_{4}}$. Furthermore, we note an increase in the SChN with stacking layers, demonstrating the presence of a QSHI insulating phase with a high-spin Chern number in $\rm{Bi_{4}Br_{4}}$.

Very recently, an alternative perspective to understanding the essence of topological phases of matter, namely the feature spectrum topology, has been proposed [12, 13]. This approach involves selecting a particular quantum operator $\hat{O}$ through which the projection operator $(P)$ projects the occupied Bloch state wave functions onto a quantum operator $\hat{O}$, forming the feature operator $\hat{\mathbb{F}}_{O}=P\hat{O}P$. This operation partitions the wavefunction of occupied states into distinct sectors. The expectation value of $\hat{\mathbb{F}}_{O}$ is termed the feature spectrum $O_{mk}$, serving as an alternative representation of band structure, which replaces the Hamiltonian acting on Bloch states $\hat{H}\left|\psi_{nk}\right\rangle=E_{nk}\left|\psi_{nk}\right\rangle$ by the feature operator $\hat{\mathbb{F}}_{O}\left|\widetilde{\psi}_{mk}\right\rangle=O_{mk}\left|\widetilde{\psi}_{mk}\right\rangle$, where $\left|\widetilde{\psi}_{mk}\right\rangle$ denotes the eigenstates of $\hat{\mathbb{F}}_{O}$ at each momentum vector $k$ and index of sector $m$. Similar to the Chern number in Chern insulator [19, 20], the corresponding topological invariants $C_{\hat{O}}$ in the feature spectrum represent the Chern number defined by particular feature sectors associated with the feature operator $\hat{\mathbb{F}}_{O}$ in $\mathbb{Z}$ class [10, 12, 13, 14]. Moreover, the universality of the feature operator leads to a well-defined feature spectrum even when the system lacks the symmetry corresponding to the selected quantum operator.

In examining the feature spectrum topology of QSHI in the Kane-Mele model, we select $\hat{S}_{z}$ as the quantum operator to formulate the feature operator $\hat{\mathbb{F}}_{S_{z}}=P\hat{S}_{z}P$. The four-band Kane-Mele tight-binding Hamiltonian [6, 7] is expressed as

For simplicity, we initially focus on the first two terms. The resulting band structure is illustrated in Fig. 1(a) with black lines. The gap of the band structure is induced by the strength of intrinsic SOC $\lambda_{I}$, leading to a nontrivial topological phase of QSHI. Each Bloch state wavefunction $\left|\psi_{nk}^{s}\right\rangle$ with opposite spins is doubly degenerate due to the preservation of both time-reversal symmetry and spatial-inversion symmetry in the Hamiltonian. Here $n$ is the band index, $k$ is the momentum vector in the Brillouin zone (BZ), and $s$ is the spin index for the $\hat{S}_{z}$ spin-up and spin-down states. As the Rashba SOC is absent, the Hamiltonian can be decomposed into two spin-diagonal blocks. Based on the $\hat{S}_{z}$ characteristic equation $\hat{S}_{z}\left|\psi_{nk}^{s}\right\rangle=\pm\frac{\hbar}{2}\left|\psi_{nk}^{s}\right\rangle$, each Bloch state of the $\hat{S}_{z}$-conserved Hamiltonian can be categorized into $\pm\frac{\hbar}{2}$ eigenvalues in $\hat{S}_{z}$ quantum number. Notably, the eigenvalues of feature operator $\hat{\mathbb{F}}_{S_{z}}$ reveal two distinct flat bands divided into $\pm\frac{\hbar}{2}$ two sectors (black lines in Fig. 1(b)). This non-dispersive feature band structure indicates the conservation of $\hat{S}_{z}$ in the Hamiltonian.

The calculation of the SChN for the QSHI within the framework of feature spectrum topology can be determined based on the eigenstates of each sector in the feature spectrum [6, 9, 21, 22, 23, 24]. We compute the Chern number for each spin $C_{s}^{\pm}$ by using the well-established method, spin-resolve Wilson’s loop [10, 12, 13, 14, 25]. The calculation result is shown in Fig. 1(c). The trajectories of the Wannier center charge for spin-up and spin-down sectors display opposite open curves traversing the entire BZ once, indicating the $C_{s}^{+}=1$ and $C_{s}^{-}=-1$. The total SChN is defined by $C_{s}=(C_{s}^{+}-C_{s}^{-})/2$. Fig.1 1(c) shows the $C_{s}=1$, indicating the presence of a QSHI. It is well-known that the QSHI is associated with a topological insulator (TI) in 2D classified by the $\mathbb{Z}_{2}$ index [7, 21], where $\mathbb{Z}_{2}=(C_{s},\rm{mod\ 2})$. The calculated $\mathbb{Z}_{2}=1$ in this case confirms a $\mathbb{Z}_{2}$ TI.

The brown lines in Fig. 1(a) show the band structure when the Rashba SOC is introduced. The feature spectrum of $\hat{\mathbb{F}}_{S_{z}}$ is no longer exhibits +1 and -1 while displaying dispersive eigenvalues due to the coupling between spin-up and spin-down induced by Rashba SOC (brown lines in Fig. 1(b)). The eigenstates of the feature spectrum in this situation consist of the linear combination of the occupied states of Bloch wavefunctions with opposite spins. Although $\hat{S}_{z}$ is no longer conserved in the Hamiltonian, it is noteworthy that the sectors of the feature eigenvalues remain separated by a gap. Consequently, the SChN is well-defined in this non-$\hat{S}_{z}$-conserved system [8, 10, 14]. Our calculation shows $C_{s}$ and $\mathbb{Z}_{2}$ both equal 1.

The feature spectrum topology provides an advanced perspective for comprehending the bulk-boundary correspondence. Since it depends only on valence bands within the band structure, it is well-suited for the evaluation in first-principles calculations, allowing for quantitative predictions in condensed matter materials. Here, we consider $\rm{Bi_{4}Br_{4}}$, which is a van der Waals material predicted to be a 2D topological insulator with $\mathbb{Z}_{2}=1$ in monolayer [11, 22]. In the following, we demonstrate that multilayer $\rm{Bi_{4}Br_{4}}$ exemplifies the bulk-boundary correspondence of the feature spectrum topology in the Kane-Mele models we discussed earlier.

The $\beta$-$\rm{Bi_{4}Br_{4}}$ crystallizes in the monoclinic space group $C_{2h}^{3}(C2/m)$ with AA stacking between alternating layers (Fig. 3(a)). Fig. 3(b) presents the band structure of monolayer $\rm{Bi_{4}Br_{4}}$, decomposed into atomic orbitals of Bi atoms, via first-principles calculations (See method). The band inversion between external Bi ($\rm{Bi_{ext}}$) and internal Bi ($\rm{Bi_{in}}$) atoms around the Fermi level at $R$ point in the BZ originates from the strong spin-orbital coupling. The gapless topological edge state in Fig. 3(d) reflects the non-trivial bulk-boundary correspondence in monolayer $\rm{Bi_{4}Br_{4}}$, which is consistent with previous calculations [11, 22]. Considering a time-reversal symmetry breaking perturbation, an in-plane Zeeman exchange field $M_{x}$, this gapless edge state opens a gap and transforms into an insulating phase in the E-k map (Fig. 3(e)). The $\hat{S}_{z}$ feature spectrum of monolayer $\rm{Bi_{4}Br_{4}}$ is shown in Fig. 3(c). The dispersive feature eigenvalues result from the complicated non-$\hat{S}_{z}$-conserved SOC in real material (e.g. Rashba SOC). The SChN of monolayer $\rm{Bi_{4}Br_{4}}$ is calculated using the method of spin-resolve Wilson’s loop in each spin sector of gapped feature spectrum. The result $C_{s}=1$ indicates a QSHI phase (inset of Fig. 3(c)), corresponding to the well-konwn $\mathbb{Z}_{2}=1$ in monolayer $\rm{Bi_{4}Br_{4}}$ [11, 22, 23, 24]. The SChN $C_{s}=1$ remains in our calculation even with the introduction of $M_{x}$ as a perturbation, indicating monolayer $\rm{Bi_{4}Br_{4}}$ is a time-reversal symmetry breaking QSHI. The edge feature spectrum with the 1D ribbon structure of monolayer $\rm{Bi_{4}Br_{4}}$ shows two non-trivial edge states in the bulk gap of feature spectrum (Fig.3 3(f)). This feature edge state flows in the feature spectrum (green line in Fig. 3(f)), corresponding the characteristics of the $\hat{S}_{z}$ spin filp on the gapped edge band structure in the E-k map (green arrows in Fig. 3(e)), as introduced in the monolayer Kane-Mele model with time-reversal symmetry breaking perturbation.

In bilayer $\rm{Bi_{4}Br_{4}}$ (AA stacking of monolayer), the originally paired gapless edge bands of each layer give rise to an insulator state due to non-negligible interlayer coupling (Fig. 3(g)), showing a $\mathbb{Z}_{2}=0$ trivial phase. By contrast, the feature spectrum topology analysis provides a non-trivial SChN $C_{s}=2$. This is because each $\rm{Bi_{4}Br_{4}}$ layer undergoes a band inversion, such that the SChN of each layer is $C_{s}=1$, resulting in total $C_{s}=2$. We observe that two pairs of non-trivial edge states corresponding to the $C_{s}=2$ in the edge feature spectrum flow from the $\hat{S}_{z}$ spin-up sector to spin-down sector along the $k$-direction (green lines in Fig. 3(h)). Our result reveals that bulk-boundary correspondence is still valid in bilayer $\rm{Bi_{4}Br_{4}}$.

Proceeding to trilayer $\rm{Bi_{4}Br_{4}}$. The band structure of the edge states in trilayer $\rm{Bi_{4}Br_{4}}$ closely resembles a combination of the edge states from the monolayer and bilayer counterparts (Fig. 4(a)). One topological Dirac edge state with time-reversal symmetry preserved at $\bar{\Gamma}$ point and two attached additional gapped edge bands around the Fermi level support a 2D topological insulator phase with $\mathbb{Z}_{2}=1$ in trilayer $\rm{Bi_{4}Br_{4}}$. The SChN $C_{s}$ of trilayer $\rm{Bi_{4}Br_{4}}$ calculated from the eigenstates of the spin sector in the feature spectrum is determined to be $C_{s}=3$ (Fig. 4(c)). The edge feature spectrum calculation presents pairs of non-trivial edge states manifesting in the gap of bulk feature eigenvalues (Fig. 4(b)), which can be explained as the $\hat{S}_{z}$ spin flip on the gapped edge band structure of trilayer $\rm{Bi_{4}Br_{4}}$ in the E-k map (green arrows in Fig. 4(a)). Our calculations display the total number of non-trivial edge states in trilayer $\rm{Bi_{4}Br_{4}}$ as three, one non-trivial state from the E-k map and two non-trivial states from the feature spectrum, which is equivalent to the SChN $C_{s}=3$. Most importantly, this is an unanimous signature of feature-energy duality. As the Zeeman exchange field $M_{x}$ is involved, the topological Dirac edge state in the E-k map opens a gap due to time-reversal symmetry breaking (Fig. 4(d)). Concurrently, the third non-trivial edge state around $\bar{\Gamma}$ point emerges in the edge feature spectrum (Fig. 4(e)). These instances not only illustrate the feature-energy duality in realistic material but also demonstrate that the bulk-boundary correspondence remains universally valid based on the combination of the non-trivial states in the E-k map and the feature spectrum (Fig. 5).

Fig. 4(f) presents the topological invariant numbers, $C_{s}$ and $\mathbb{Z}_{2}$, as functions of the stacking layers of $\rm{Bi_{4}Br_{4}}$. Notably, the SChN $C_{s}$ exhibits linear increases with the number of layers, while $\mathbb{Z}_{2}$ index oscillates between 1 and 0 for odd and even layers, respectively. This phenomenon arises from the requirement of symmetry protection for the existence of gapless edge states in $\mathbb{Z}_{2}$ topological insulators, such as time-reversal or crystalline symmetry. In an even number of layers, the interlayer coupling induces the wavefunction hybridization of the topological edge states originating from each layer. In the lacking of additional crystal symmetry protection, this hybridization effect opens a gap in the E-k map, resulting in $\mathbb{Z}_{2}=0$. In contrast, the SChN $C_{s}$ exhibits feature-energy duality, independent of symmetry protection, whose value is equal to the total number of gapless edge states in the E-k map and non-trivial edge states in the $\hat{S}_{z}$ feature spectrum. In simpler terms, the emergence of non-trivial edge states of QSHI in the E-k map is not a prerequisite unless the system also qualifies as a $\mathbb{Z}_{2}$ topological insulator.

We discuss the universality of the feature-energy duality and the bulk-boundary correspondence in multilayer QSHI, as depicted in Fig. 5. In monolayer QSHI, the gapless edge state emerges in the E-k map, while the edge feature spectrum exhibits a gap. Corresponding, topological invariant numbers are $C_{s}=1$ and $\mathbb{Z}_{2}=1$. In the second scenario, the edge state opens a gap due to time-reversal symmetry breaking. A non-trivial edge state appears in the edge feature spectrum, connecting the spin-up and spin-down sectors of bulk feature eigenvalues. Moving on to bilayer (the third case in Fig. 5), the edge states open a gap in the E-k map due to interlayer coupling. Conversely, a pair of non-trivial edge states emerge in the edge feature spectrum. Consequently, the topological invariant numbers of bilayer QSHI are $C_{s}=2$ and $\mathbb{Z}_{2}=0$. The last two cases exhibit the same topological invariant numbers ($C_{s}=3$ and $\mathbb{Z}_{2}=1$), yet their edge state behaviors differ significantly. In the fourth case, one gapless edge state and two attached gapped edge bands appear in the E-k map, while two non-trivial edge states manifest in the edge feature spectrum. The edge states open a gap in the E-k map result from the time-reversal symmetry breaking perturbation in the fifth case. Simultaneously, the third non-trivial edge state emerges in the edge feature spectrum. These examples collectively demonstrate that the SChN $C_{s}$ equals the total number of gapless edge states in the E-k map and non-trivial edge states in the feature spectrum.

Our investigation reveals that the SChN $C_{s}$ can always be well-defined through the feature spectrum topology [10, 12], even when the $\hat{S}_{z}$ is non-conserved under gap-opening perturbations such as Rashba SOC and Zeeman exchange field in the Hamiltonian. As a result, we speculate that the topological phases classified by the feature operator may extend beyond conventional symmetry constraints. This concept aligns with the quantum anomalous Hall effect, where the Chern number does not necessitate additional symmetry protection. The feature spectrum topology offers a robust perspective that broadens the classification of the topological phase to a more fundamental level, transcending the limitations of symmetry-based considerations.

In conclusion, our exploration of bulk-boundary correspondence through feature spectrum topology, employing the $\hat{S}_{z}$ feature operator in both monolayer and bilayer Kane-Mele models, has revealed a feature-energy duality. Our calculation unveils that the combination of gapless edge states in the E-k map and non-trivial edge states in the feature spectrum equals the SChN in multilayer QSHI. Additionally, we propose the van der Waals material $\rm{Bi_{4}Br_{4}}$ as a candidate. Our results validate the feature-energy duality and further identify Bi4Br4 as a layer-dependent high-spin Chern insulator. The discovery of the feature spectrum topology not only deepens our comprehension of the classification of topological phases but also opens avenues for the exploration of QSHI with high-spin Chern number.

The first-principle calculations on the $\rm{Bi_{4}Br_{4}}$ thin film were performed using the Vienna ab Initio simulation package (VASP) [26, 27, 28] with the projector augmented wave approach and the generalized-gradient approximation (GGA) [29]. The spin-orbit coupling (SOC) was included self-consistently in the calculations of electronic structures with a Gamma-centered mesh 11×11×1. The experimental lattice parameters of $\rm{Bi_{4}Br_{4}}$ were used, with a=6.998$\rm{\AA}$, b=4.338$\rm{\AA}$, and 71.945^∘ angle between a-b axis. The interlayer distance between each layer is 2.456$\rm{\AA}$ in multilayer systems. To calculate the feature spectrum, edge state, spin Chern number, and spin Hall conductivity, we constructed the tight-binding Hamiltonian, where the matrix elements were calculated by projecting onto Wannier orbitals, which used the VASP2WANNIER90 interface [30, 31, 32]. The Bi $p$ and Br $p$ orbitals were used without performing the procedure for maximizing localization.

T.-R.C. was supported by the 2030 Cross-Generation Young Scholars Program from the National Science and Technology Council (NSTC) in Taiwan (Program No. MOST111-2628-M-006-003-MY3), National Cheng Kung University (NCKU), Taiwan, and National Center for Theoretical Sciences, Taiwan. This research was supported, in part, by Higher Education Sprout Project, Ministry of Education to the Headquarters of University Advancement at NCKU. The work at Northeastern University was supported by the National Science Foundation through NSF-ExpandQISE award #2329067 and it benefited from Northeastern University’s Advanced Scientific Computation Center and the Discovery Cluster.