Moiré flat Chern bands and correlated quantum anomalous Hall states
generated by spin-orbit couplings in twisted homobilayer MoS₂

Introduction.— The discovery of possible correlated insulating states Cao1 and superconductivity Cao2 in magic-angle twisted bilayer graphene has paved a new avenue toward engineering electronic structures where interactions play a decisive role Carr ; MacDonald1 ; MacDonald2 ; Adrian1 ; Noah ; Koshino ; Isobe ; Wu1 ; Vafek1 ; Yang ; Biao ; Gonzalez ; Xie ; Andrei ; Efetov ; Perge ; Yazdani1 . Lately, nontrivial topological properties brought about by the moiré patterns were also unveiled Jianpeng1 ; Song . Under appropriate symmetry breaking conditions, flat bands in twisted graphene acquire nonzero Chern numbers Zaletel ; Senthil1 ; Senthil2 ; Wenyu , which manifest experimentally as quantum anomalous Hall (QAH) states when spin or valley degeneracies are lifted spontaneously through electronic correlations Sharpe ; Young ; Yazdani2 . The interplay between correlation and topology uncovered in these systems indicates possibilities for novel topological phases beyond conventional non-interacting theories.

Motivated by advances in twisted graphene systems, explorations into moiré superlattices formed by other two-dimensional materials, such as transition-metal dichalcogenides (TMDs) Wu2 ; Wu3 ; LeRoy ; Fai1 ; Wang ; Dean ; Bi and copper oxides Marcel ; Pixley , have also seen rapid progress recently. In particular, moiré flat bands in a twisted homobilayer TMD formed by the valence band (VB) states in each $K$-valley were shown to carry nonzero Chern numbers Wu2 . The nontrivial band topology arises from combined effects of moiré potential and interlayer coupling, which act together as a layer-pseudospin magnetic field and create skyrmionic pseudospin textures in superlattice cells. The strong spin-valley locking due to giant Ising-type spin-orbit coupling (SOC) of order $\sim$ 100 meVs Xiao ; GuiBin further entails a possible quantum spin Hall state.

While flat bands from VB states have enjoyed wide interest, moiré physics arising from conduction band (CB) states in twisted TMDs remains largely unexplored. One possible complication lies in the relatively weak SOC in the conduction bands on the scale of a few to tens of meVs GuiBin ; Rossier , which may cause crossings between spin-up and spin-down moiré bands. Recent works on untwisted TMDs, on the other hand, revealed that the small spin-orbit splitting in CB states, when combined with Rashba SOC introduced by mirror symmetry breaking, can lead to nontrivial Berry phase effects Benjamin ; Taguchi ; Lee . Notably, upon assembling two identical TMD layers into a twisted bilayer, the mirror symmetry is broken locally in each layer (Fig.1(a)-(b)) and Rashba SOC is expected to arise. How this SOC would affect the band topology of twisted TMDs is an outstanding question.

In this Letter we establish a novel topological phase generated by SOC in CB moiré bands in twisted TMDs, which is essentially different from those in twisted graphene Jianpeng1 ; Song ; Zaletel ; Senthil1 ; Senthil2 ; Wenyu where SOC is negligible, and those in the VB moiré bands of twisted TMDs where SOC plays no essential role in creating nonzero Chern numbers Wu2 . Focusing on the specific case of a twisted homobilayer MoS₂ we predict that an interplay among angular twist, local symmetry breaking and spin-orbit coupling in the CB states creates moiré flat bands with nonzero Chern numbers $\mathcal{C}=\pm 2$, and density-density interactions generically drive the system at $1/2$-filling into a valley-polarized correlated QAH state. We further show that the Chern bands generated by SOC stay robust against displacement fields which would otherwise destroy the nontrivial topology from layer-pseudospins Wu2 .

Continuum model for twisted homobilayer MoS₂.— Stacking two monolayers of MoS₂ with a small relative twist angle $\theta_{0}$ results in a moiré superlattice with lattice constant $a_{M}=a/\theta_{0}$, where $a$ is the lattice constant of each monolayer. The system has a symmetry group of $D_{3}\otimes\{U_{v}(1),\mathcal{T}\}$, where $U_{v}(1)$ stands for valley conservation, $\mathcal{T}$ for time-reversal symmetry, and the point group $D_{3}$ is generated by the three-fold rotation about the $z$-axis ($C_{3z}$) and a two-fold rotation about the $y$-axis ($C_{2y}$) (Fig.1(a)). For an aligned bilayer at $\theta_{0}=0$, the mirror symmetry $\mathcal{M}_{z}$ about the horizontal 2D mirror plane, which is respected in a monolayer, is broken locally on each layer. This generates uniform electric fields of opposite signs in different layers, which remain effective upon a small angular twist with $\theta_{0}\sim 1^{\circ}$ (Fig.1(b)). Based on models of monolayer MoS₂ with broken $\mathcal{M}_{z}$ Benjamin ; Taguchi ; Lee , the effective Hamiltonian for CB minima at $\pm K$ valleys of two twisted isolated layers can be written as:

where $\xi=\pm$ denotes the valley index, $l=1(2)$ labels the top (bottom) layer, $\alpha,\beta$ label the spin indices with the $\sigma$-matrices representing the usual Pauli matrices for spins. $\bm{K}_{1}\equiv\bm{K}_{m,+}$ and $\bm{K}_{2}\equiv\bm{K}_{m,-}$ are the shifted $+\bm{K}$-points on layer 1 and layer 2 (Fig.1(c)) due to the angular twist of $\theta_{0}$ equivalent to rotating the top (bottom) layer by an angle $\theta_{0}/2$ ($-\theta_{0}/2$) about the $z$-axis. In Eq.1, $m^{\ast}\approx 0.5m_{e}$ is the CB effective mass at $K$, $\mu$ is the Fermi energy measured from band minima. The $\alpha_{\rm so}$-term and $\beta_{\rm so}$-term account for the Rashba SOC and the Ising SOC, respectively. The Ising SOC has a valley-dependent sign due to time-reversal symmetry and causes a splitting of $\Delta_{\rm so}=2|\beta_{\rm so}|$ in the spin-up and spin-down CB states. The Rashba SOC has a layer-dependent sign imposed by $C_{2y}$ which swaps the two layers. Spatial variations of $\alpha_{\rm so}$ are discussed in Supplemental Material (SM).

Spatial modulations due to the formation of moiré pattern can be captured by introducing coupling terms between states at $\bm{k}$ and $\bm{k}+\bm{G}_{M,j}$ Wu2 ; Senthil1 , where $\bm{G}_{M,j}=-\frac{4\pi}{\sqrt{3}a_{M}}(\cos\frac{(j-1)\pi}{3},\sin\frac{(j-1)\pi}{3})$ are the reciprocal vectors of the moiré superlattice. The momentum-space moiré potential is given by

where $V_{l,\bm{G}_{M,j}}=V_{M}(\cos\psi-i(-1)^{j}l\sin\psi)$ are complex coupling parameters with amplitude $V_{M}$ and phase $\psi$.

The effective inter-layer tunneling for states at $K$-valleys is modeled following the general recipe of two-center approximation MacDonald1 ; Wu2 , which can be written as

where $w_{0}$ denotes the effective inter-layer tunneling amplitude near $K$, which is extrapolated to be $w_{0}\approx 10$ meV using a combined approach of empirical models and Slater-Koster methods (see the Supplemental Material SM for details). The total non-interacting continuum model thus reads: $\mathcal{H}_{0}=\sum_{\xi=\pm}\sum_{l=1,2}\mathcal{H}^{\xi}_{0,l}+\mathcal{H}^{\xi}_{T}+\mathcal{H}_{M}$ with parameters tabulated in Table 1.

Moiré flat Chern bands and skyrmionic spin textures.— From general considerations, level spacings among the lowest moiré bands correspond roughly to the quantization energy $\Delta E\sim\hbar^{2}\pi^{2}/(2m^{*}a_{M}^{2})$ of an electron confined in a superlattice cell with size $a_{M}^{2}=a^{2}/\theta^{2}_{0}$. For $\theta_{0}\sim 1^{\circ}-2^{\circ}$, $\Delta E\sim 1-10$ meV in twisted MoS₂, which is comparable to the spin-orbit splitting $\Delta_{\rm so}\equiv 2|\beta_{\rm so}|=3$ meV caused by the Ising SOC GuiBin . If spin is conserved (e.g., in the absence of Rashba SOC), spin-up and spin-down moiré bands are expected to cross in general.

By setting $\alpha_{\rm so}=0$ in $\mathcal{H}_{0}$, we find that for $\xi=+$, the 1${}^{\text{st}}$ spin-down and the 2${}^{\text{nd}}$ spin-up moiré bands cross each other for $\theta_{0}\in(1.25^{\circ},1.5^{\circ})$. As a specific example, spin-resolved moiré bands at $\theta_{0}=1.4^{\circ}$ with $\alpha_{\rm so}=0$ are shown in Fig.2(a). With spin conservation in this case, the spin Chern number is well-defined, and the spin-up and spin-down bands involved in the level crossing have Chern numbers ($\mathcal{C}_{1,\downarrow}=+1$, $\mathcal{C}_{2,\uparrow}=-1$), which originate from the layer-pseudospin magnetic fields as in moiré bands from VB states Wu2 . Upon turning on $\alpha_{\rm so}$, crossing points are gapped out by the layer-dependent Rashba SOC which mix the spin-up and spin-down species (Fig.2(b)). As a result of this mixing, the original $\mathcal{C}_{1,\downarrow}$ and $\mathcal{C}_{2,\uparrow}$ from layer pseudospins annihilate each other; meanwhile, a new pair of flat bands (2${}^{\text{nd}}$ and 3${}^{\text{rd}}$ bands in Fig.2(b)) with Chern numbers $\mathcal{C}=\pm 2$ emerge (see SM SM for details on Chern number calculations). The nontrivial gap induced by Rashba SOCs is further signified by giant Berry curvatures of order $10^{4}$Ų in the MBZ (Fig.2(c)).

The unusual Chern numbers $\mathcal{C}=\pm 2$ arise from a unique combination of angular twist and local mirror-symmetry breaking: as moiré bands at opposite $\bm{K}_{m,+}$ and $\bm{K}_{m,-}$ points originate from states in different layers (Fig.1(c)), the layer-dependent Rashba SOCs (Eq.1) create an in-plane spinor field with opposite vorticities at $\bm{K}_{m,+}$ and $\bm{K}_{m,-}$ in the MBZ. This unsual pattern causes the in-plane spin to wind twice as one goes around a loop enclosing the $\Gamma_{m}$-point in the MBZ (Fig.2(d)). As we confirm numerically, the spin textures of the 2${}^{\textrm{nd}}$ and 3${}^{\textrm{rd}}$ moiré bands are characterized by nonzero skyrmion numbers $\mathcal{S}=\int d^{2}\bm{k}\hat{n}_{\bm{k}}\cdot(\partial_{k_{x}}\hat{n}_{\bm{k}}\times\partial_{k_{y}}\hat{n}_{\bm{k}})/4\pi=\pm 2$ ($\hat{n}_{\bm{k}}$: spin orientation of state at $\bm{k}$), and these two bands describe a two-level system with one-to-one correspondence between Chern numbers $\mathcal{C}$ and skyrmion number $\mathcal{S}$ Bernevig .

Correlated QAH state at $1/2$-filling.— Given the narrow bandwidth $W\approx 1$ meV of moiré Chern bands (Fig.2(b)), the characteristic Coulomb energy scale $g_{C}\simeq e^{2}/\epsilon a_{M}\approx 10$ meV overwhelms the kinetic energy and correlation physics is expected to arise (assuming $\theta_{0}=1.4^{\circ}$ and dielectric constant $\epsilon=10$ Bi ). Motivated by the observation that twisted graphene at $3/4-$filling can behave as an SU(4)-quantum Hall ferromagnet Senthil1 ; Sharpe ; Young ; Yazdani2 , and the fact that all CB moiré bands in Fig.2(b) involves only a two-fold valley degeneracy (note that spin SU(2)-symmetry is already broken by SOC), we examine correlation effects when the 2${}^{\text{nd}}$ moiré band is half-filled. For simplicity, we drop the band index $n$ in the following.

Including density-density interactions, the low-energy effective Hamiltonian: $\mathcal{H}_{\rm eff}=\mathcal{H}_{0,\rm eff}+\mathcal{H}_{I,\rm eff}$, where $\mathcal{H}_{0,\rm eff}=\sum_{\bm{k},\xi=\pm}E_{\xi}(\bm{k})c^{\dagger}_{\xi}(\bm{k})c_{\xi}(\bm{k})$ is the non-interacting part with band energies $E_{\xi}(\bm{k})$, and the interacting Hamiltonian compatible with $U_{v}(1)$ and $\mathcal{T}$ reads

where $V(\bm{q})$ is the Fourier transform of the two-body interaction, $A$ is the total area of the system, and $\Lambda^{\xi}(\bm{k}+\bm{q},\bm{k})\equiv\braket{u_{\xi,\bm{k}+\bm{q}}}{u_{\xi,\bm{k}}}$ ($\ket{u_{\xi,\bm{k}}}$: periodic part of the Bloch states) are the form factors arising from projecting the density-density interaction to active bands. The general trial Hartree-Fock ground state at $1/2$-filling has the form: $\ket{\Phi}=\prod_{\bm{k}\in\text{MBZ}}[w_{+}(\bm{k})c^{\dagger}_{+}(\bm{k})+w_{-}(\bm{k})c^{\dagger}_{-}(\bm{k})]\ket{\Phi_{0}}$, where $\ket{\Phi_{0}}$ is the vaccuum state with all lower-lying bands filled, and $w_{\xi}(\bm{k})$ are the variational parameters.

Minimizing $E^{\Phi}\equiv\braket{\Phi}{\mathcal{H}_{\rm eff}}{\Phi}$ with mean-field approximations for Fock exchange terms (see SM SM ) leads to a set of self-consistent equations:

Here, $J_{0}$ and $J_{1}$ denote the mean-field intra-valley and inter-valley exchange coupling constants, $\Delta_{\xi}=J_{0}\bar{n}_{\xi}$ ($\bar{n}_{\xi}$: mean occupancy for valley $\xi$) and $\Delta_{+-}$ are the intra-valley and inter-valley order parameters, which characterize the average energy gains from intra-valley and inter-valley exchange interactions, respectively. $\delta(\bm{k})\equiv E_{+}(\bm{k})-E_{-}(\bm{k})$ denotes energy difference between bands from two valleys $\xi=\pm$, $D(\bm{k})=\sqrt{\Delta^{2}_{+-}+[\delta(\bm{k})-\Delta_{M}]^{2}/4}$, where we introduce the valley magnetic order $\Delta_{M}\equiv\Delta_{+}-\Delta_{-}=J_{0}(\bar{n}_{+}-\bar{n}_{-})$.

Solutions of Eq.S27 are divided into two classes: (i) the valley-polarized (VP) state Adrian1 ; Senthil1 , in which one out of the two nearly degenerate moiré bands from two valleys is fully filled, with spontaneous $\mathcal{T}$-symmetry breaking: $\Delta_{M}=\xi J_{0},\Delta_{+-}=0,w_{\xi}(\bm{k})=1$; (ii) the inter-valley coherent (IVC) state, in which the Slater determinant is formed by coherent superpositions of states from the two valleys, with spontaneous U${}_{v}(1)$-symmetry breaking Adrian1 ; Senthil1 ; Senthil2 : $\Delta_{M}=0,\Delta_{+-}\approx J_{1}/2$, $w_{+,-}(\bm{k})\neq 0$. By comparing the energies of the VP and IVC states, we obtain the mean-field $J_{0}-J_{1}$ phase diagram for the range $0.3g_{C}\leq J_{0},J_{1}\leq g_{C}$ (Fig.3(a)). The IVC state is favored throughout the entire $J_{1}\geq J_{0}$ regime, while the VP phase takes up most of the $J_{1}<J_{0}$ regime. By tuning $J_{0}/J_{1}$ across the phase boundary (green dashed line in Fig.3(a)), a first-order transition occurs at $J_{0}\simeq J_{1}$ (Fig.3(b)).

It is worth noting that given a general repulsive interaction $V(\bm{q})>0$, the inequality $2|\Lambda^{+}(\bm{k}^{\prime},\bm{k})||\Lambda^{-}(\bm{k},\bm{k}^{\prime})|\leq|\Lambda^{+}(\bm{k}^{\prime},\bm{k})|^{2}+|\Lambda^{-}(\bm{k},\bm{k}^{\prime})|^{2}$ together with $\mathcal{T}$-symmetry leads to $J_{0}\geq J_{1}$, regardless of details of the form factors and the microscopic interaction SM . Thus, electron correlations most likely drive the system at $1/2$-filling into a $\mathcal{T}$-broken VP state, which realizes a correlated QAH state with $\mathcal{C}=\pm 2$. The case of general filling away from 1/2 is discussed in SM SM .

Effects of displacement fields and comparison with Chern bands from layer pseudospins.— As shown in Fig.2(a), with $\alpha_{\rm so}=0$, nonzero Chern numbers also arise in CB moiré bands due to layer pseudospin magnetic fields, with the same origin as those in VB moiré bands Wu2 . Thus, two different mechanisms for moiré Chern bands, one from layer pseudospin and the other from SOCs, coexist in the CB moiré bands. In contrast, the Rashba SOC is unimportant in the VB moiré bands where spin-up and spin-down bands are separated by $100-400$ meV GuiBin , and the VB moiré band topology is governed by layer pseudospin alone.

The nontrivial topology from layer pseudopsin is shown to be prone to displacement fields which can destroy the skyrmionic pseudospin textures by polarizing the layer-pseudospins Wu2 . On the other hand, the SOC mechanism has an independent origin in the avoided level crossings between spin-up and spin-down bands and does not require nontrivial layer pseudospin textures. Thus one would expect the nontrivial band topology in the CB moiré bands to be more robust against displacement fields than its VB counterpart.

To demonstrate the effects of displacement fields, we follow Refs.Wu2 and introduce a layer-dependent potential $V_{D,l}=(-1)^{l}V_{z}/2,(l=1,2)$ in $\mathcal{H}_{0}$. The moiré bands at $\theta_{0}=1.4^{\circ}$ under finite $V_{z}$ are shown in Fig.4(a), where we set $V_{z}=4$ meV strong enough to destroy the nontrivial topology generated by layer pseudospins (Fig.4(b)). Clearly, the avoided level crossings still occur and a pair of Chern bands with Chern numbers $(+1,-1)$ remain. The Chern number is reduced from $\mathcal{C}=\pm 2$ to $\mathcal{C}=\pm 1$ under $V_{z}$ because the displacement field drives a band inversion near $\bm{K}_{m,+}$ and mediates a Chern number exchange $\Delta\mathcal{C}=\pm 1$ between the 1${}^{\text{st}}$ and 2${}^{\text{nd}}$ moiré bands.

The topological phase diagram for CB moiré bands as a function of $\theta_{0}$ and $V_{z}$ is shown in Fig.4(b). Due to the extra mechanism from SOCs, the critical displacement field for the nontrivial topological regime is enhanced to be $V^{c2}_{z}$, which is 2-4 times of the critical $V^{c1}_{z}$ (yellow dashed line) needed to destroy the layer pseudospin mechanism for $\theta_{0}\sim 1^{\circ}$ ($V^{c1}_{z}$ obtained by setting $\alpha_{\rm so}=0$ in $\mathcal{H}_{0}$). Importantly, there is a wide parameter regime in the phase diagram (depicted in orange) where the layer pseudospin mechanism proposed in Ref.Wu2 fails while the CB moiré bands remain topological, which confirms that the CB states are more robust than VB states against displacement fields.

Conclusion and Discussions. — While on-going activities in twisted TMDs have focused mainly on VB moiré bands Wu2 ; Wu3 ; LeRoy ; Fai1 ; Wang ; Dean ; Bi , our proposal of moiré Chern bands generated by SOC opens a new pathway into the largely unknown territory of CB moiré physics. When these Chern bands are half-filled, according to our predictions, electron interactions can lead to a spontaneously $\mathcal{T}$-broken VP phase and realize a correlated QAH state, which will manifest itself through quantized Hall conductance in transport measurements.

To realize the topological moiré bands generated by SOC, any finite Rashba SOC which is allowed by the $D_{3}$-symmetry of the system would be sufficient, in principle, to induce a nontrivial gap. Notably, as MoS₂ is intrinsically semiconducting (with Fermi level $\sim 0.8$ eV below the conduction band edge) Xiao ; GuiBin ; Fai2 , it is necessary to gate the chemical potential such that the CB moiré bands are filled in the first place. With the dual-gate setup used widely in experiments Cao1 ; Dean , local mirror symmetry breaking can be further enhanced by interfacial contact with gating electrodes, which enhances the SOC gap in the non-interacting bands. Due to the correlated nature of the QAH state at 1/2-filling, the actual topological gap to be manifested experimentally is not solely determined by the SOC gap in the non-interacting bands; instead it should be largely governed by the intra-valley exchange coupling $J_{0}\sim 10$ meV for $\theta_{0}\sim 1^{\circ}$ SM . This sizable correlation-induced topological gap will reduce complications of disorder and thermal effects in experimental detection of the predicted correlated QAH state. However, as $\theta_{0}$ increases, correlation effects become weaker and the moiré bands more dispersive, thus the predicted correlated QAH phase would be less robust in the large twist angle regime.

Acknowledgement. — B.T.Z. thanks Zefei Wu and Ziliang Ye for inspiring discussions on experimental aspects of twisted MoS₂. This work was supported by NSERC and the Canada First Research Excellence Fund, Quantum Materials and Future Technologies Program. B.T.Z. further acknowledges the support of the Croucher Foundation.