Low-lying Electronic Structure of Rare-Earth Based Topological Nodal Line Semimetal Candidate DySbTe

The rapidly expanding field of topological quantum materials (TQMs) has undergone significant advancements since the initial discovery of three-dimensional (3D) topological insulators [1, 2, 3], semimetals [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19], and superconductors [20, 21]. The identification of various topological semimetals, including Dirac semimetals [4, 5, 6], Weyl semimetals [7, 8, 9, 10, 11], and nodal-line semimetals (NLSMs) [12, 13, 14, 15, 16, 17, 18, 19], has been driven by a surge of both theoretical and experimental research. The identification of nodal-line topological states in ZrSiS has significantly expanded interest in ZrSiS-type materials [14, 22, 23]. These materials, characterized by nonsymmorphic topological fermions from the square-net plane of group-IV elements [14, 22], exhibit remarkable electronic properties such as, unconventional magnetotransport behavior [24], flat optical conductivity [25], and enhanced effective mass [26] which has further motivated the exploration of additional members in this material family. These studies of nodal line structures in this family also underscore the need for further exploration into the effects of electron hybridization, including interactions influenced by magnetism, 4$f$-electron correlations, and strong spin-orbit coupling (SOC). Factors such as these uniquely affect the nodal line features and demand deeper investigation to understand their impact on the topological and magnetic characteristics of these materials.

The $Ln$SbTe systems, part of the ZrSiS-type family of materials, have attracted significant interest due to the less-studied Sb-square net, potential magnetic ordering from $Ln$ magnetic moments, and correlation effects from the 4$f$ electrons of $Ln$ elements. Nonsymmorphic symmetry, involving fractional lattice translations with operations like screw axes or glide planes, has been associated with symmetry-protected topology [27, 28, 29]. However, materials with non-accidental nodal lines without SOC are limited [12, 14, 30, 27, 22]. Recent studies proposed continuous Dirac nodal points, validated by observing nodal line phases in ZrSiS and other MZX (M$=$ Transition metals, Z$=$ Si, Ge, Sb, Sn, X$=$S, Se, Te) materials with PbFCl-type structures [14, 22]. Replacing transition metals with rare earth elements alters topological properties due to 4$f$ electron correlations and magnetism. Generally, lighter lanthanide compounds exhibit gapless nodal lines, as seen in LaSbTe, where the nodal lines along R−X and A−M directions remain robust against SOC due to nonsymmorphic symmetry [31]. In contrast, CeSbTe exhibits meta-magnetic transitions, charge density waves (CDW), and weak Kondo effects [32, 33, 34]. SmSbTe displayed multiple Dirac nodes alongside Kondo effects and electronic correlations [35, 36], while PrSbTe and NdSbTe revealed gapless nodal lines and diamond-shaped structures [37], with the latter showing magnetic studies indicating metamagnetic transitions, Kondo localization [38], and enhanced correlations [39]. GdSbTe exhibits ZrSiS-like nodal line characteristics at different energy positions [40], with early work revealing nodal-line and AFM Dirac states protected by broken time-reversal and rotoinversion symmetries [40]. While extensive studies have focused on lighter lanthanide-based compounds, the stronger SOC and more localized 4$f$-electron interactions in heavier elements, like Dy, have yet to reveal significant changes in their electronic structures, leaving much of their potential properties underexplored. The SOC gap in HoSbTe at the $\overline{\text{X}}$ point suggests it as a weak topological insulator, with magnetism-driven band structure changes near $4$K [41]. HoSbTe also shows SOC-induced gaps along specific high-symmetry directions, driving a Lifshitz transition that leads to a Fermi surface (FS) with four electron pockets centered at the $\overline{\text{X}}$ point [41, 42, 43]. In contrast, lighter $Ln$SbTe retains a gapless nodal line even with SOC [44, 45], highlighting Dy as a transition between gapless and gapped topological states across the series.
DySbTe is another representative of the $Ln$SbTe family that was characterised by means of thermodynamic and transport measurements as an AFM semimetal [46]. Recent neutron diffraction experiment revealed on DySbTe competing magnetic phases within the AFM state [47]. In this paper, we report on our magnetic susceptibility, heat capacity, and electrical resistivity measurements performed on high-quality single crystals of DySbTe. The ARPES measurements, conducted in the paramagnetic (PM) phase at $15$ K, reveal persistent nodal line features along several high-symmetry directions, and measurements are further complemented by first-principles band structure calculations. Notably, DySbTe marks a significant case within the $Ln$SbTe family as it transitions from gapless to gapped states, showing gapped band crossings along the $\overline{\text{X}}-\overline{\Gamma}-\overline{\text{X}}$ and $\overline{\text{M}}-\overline{\Gamma}-\overline{\text{M}}$ directions. Additionally, the nodal line along the bulk X−R direction is also gapped. These findings are consistent with first-principles calculations and provide key insights into the electronic structure and topological states of DySbTe, establishing a strong foundation for understanding the interplay between topology and $Ln$ elements in the $Ln$SbTe family of quantum materials.

Single crystals of DySbTe were grown from Sb flux as described in the Supplementary Material (SM) to this manuscript [48]. Their quality and chemical composition were proved by standard techniques (see SM [48]). Measurements of the magnetic susceptibility, heat capacity and electrical resistivity were carried out through the usage of Quantum Design PPMS-9 and MPMS-XL. STM and ARPES experiments were conducted and analysed as described in SM [48]. First-principles calculations were performed within the framework of density functional theory (DFT), using the Vienna Ab initio Simulation Package (VASP), based on projector augmented wave (PAW) potentials (See SM [48] for additional information) [49, 50, 51, 52].

The tetragonal crystal structure of DySbTe is shown in Fig.1(a). The compound is isostructural with other $Ln$SbTe materials which crystallize within a $P4/nmm$ (No. 129) nonsymmorphic space group. The lattice parameters reported in the literature are $a=b=4.240$ Å and $c=9.168$ Å [46], and our theoretical crystal structure optimization yielded similar values: $a=b=4.286$ Å and $c=9.235$ Å. In the crystallographic unit cell, the Sb atom occupies the Wyckoff position $2b$ (1/4,3/4,1/2), while the Dy and Te atoms are located at the $2c$ (3/4,3/4,$z$) sites, where $z=0.2743$ and $0.6238$ for Dy and Te, respectively. The Dy-Te double layers are situated between the Sb atoms arranged in a square lattice configuration. These layers weakly bonded to neighboring sheets via van der Waals interactions, resulting in a natural (001) cleaving plane.
Fig.1(b) illustrates the three-dimensional bulk Brillouin zone (BZ) and its projection along the (001) surface, where ARPES experiments were conducted, showing the corresponding high symmetry points. The calculated bulk band structures, without and with the inclusion of SOC, are shown in Figs.1(c) and1(d). In Fig.1(c), locations of high interest are indicated by a black box and a possible gap formation due to the inclusion of SOC is indicated by a blue arrow, and other gapped features are indicated by a red and green arrow.
The bulk thermodynamic and electrical transport properties of the DySbTe single crystals are presented in Fig.1(e-g). Magnetic measurements performed in a magnetic field of 0.1 T applied in the tetragonal plane of the compound corroborated the AFM order at low temperatures, first reported in Refs. [46, 47]. However, the Neel temperature $T_{\rm N}$ = 7.45 K, defined at the maximum in the temperature dependence of the molar magnetic susceptibility (see the inset to Fig.1(e)) is slightly higher than the values given in the literature (7 K [46], 6.7 K [47]). Noteworthy is the rapidity of the decrease in the magnetic susceptibility just below $T_{\rm N}$. In the ordered state, the magnetic susceptibility was found nearly independent of the magnetic history of the specimen, in line with the AFM character of the electronic ground state of DySbTe. However, a small change in the slope of $\chi$($T$) measured in the field-cooled regime can be detected near 7.1 K. This latter feature was not observed in the previous studies.
In the paramagnetic state, the magnetic susceptibility follows the Curie-Weiss law (note the red line in Fig.1(e)) with the effective magnetic moment $\mu_{\rm eff}$ = 10.2 $\mu_{\rm B}$ and the paramagnetic temperature $\theta_{\rm p}=-2.4$ K. The experimental value of $\mu_{\rm eff}$ is fairly close to the theoretical prediction for a free trivalent Dy ion (10.65 $\mu_{\rm B}$), calculated assuming the Russell-Saunders coupling scenario. Some tiny deviation of $\chi^{-1}$($T$) from a straight line, occurring below about 40 K, probably results from crystalline electric field interactions. The negative sign of $\theta_{\rm p}$ complies with the AFM coupling of localized paramagnetic moments.
The magnetic phase transition at $T_{\rm N}$ manifests itself as a distinct anomaly in the temperature-dependent specific heat of the compound. As can be inferred from Fig.1(f), the peak in $C_{\rm p}$($T$) is extremely sharp, resembling first-order transitions. Remarkably, another small peak appears in the specific heat data at $T_{\rm N1}$ = 7.15 K (see the inset to Fig.1(f)), which coincides with the results of the magnetic measurements. Above 200 K, $C_{\rm p}$($T$) saturates at a value of 75 J/(mole K), which is equal to the Dulong-Petit limit 3$nR$, where $n$ is the number of atoms in the formula unit and $R$ stands for the gas constant. The observed unevenness of $C_{\rm p}$($T$) above 210 K probably results from the contribution of Apiezon N grease [53], which was quite significant in measuring the heat capacity of a very small DySbTe crystal.
As shown in Fig.1(g), the studied crystals of DySbTe were found to exhibit semimetallic character of the electrical transport in terms of the magnitude of the electrical resistivity, its little variation with temperature, and the overall shape of $\rho$($T$). As the temperature is lowered below 300 K, the resistivity first increases, passes through a broad maximum centered near 170 K, and then decreases with further temperature reduction. When approaching $T_{\rm N}$, $\rho$($T$) shows a plateau and then jumps suddenly at the onset of the AFM state. The sharpness of this anomaly corresponds very well with the first-order-like character of the peak in $C_{\rm p}$($T$). Interestingly, $\rho$($T$) flattens out below $T_{\rm N1}$ and the resistivity remains constant down to 2 K. This behavior is rather unusual for AFM materials and may result from the complexity of the magnetic structure of DySbTe that features two propagation vectors and was interpreted in terms of two-phase model [47].
It is worth noting that the overall behavior of $\rho$($T$) in the PM region is similar to that found earlier [46], but the resistivity is about four times smaller than in the previous report. The key difference is in the ordered state, namely, in contrast to our finding, a slight decrease in the resistivity was observed below $T_{\rm N}$ in Ref. [46]. This dissimilarity hints at a possible difference in the stoichiometry of the crystals investigated in the two studies, which is a common feature of the $Ln$SbTe compounds (see for example Ref. [54] and references therein).
Fig.2(a) shows an STM topograph of DySbTe cleaved in ultra-high vacuum along the (001) plane. A rectangular atomic lattice is visible, consistent with our theoretically predicted crystal structure. Additionally, there are two types of atomic defects present, attributed to Te vacancies (dark, red arrow) and adatoms or substitutions (bright, orange arrow) [55]. Atomic Bragg peaks in the FFT (inset) indicate lattice constants of $a=4.1\pm 0.2~{}\text{\AA}$ and $b=4.0\pm 0.2~{}\text{\AA}$. Fig.2(b) shows STS spectra taken on and away from the defects, displaying a lack of a hard gap within a 800 meV window around the Fermi level ($V_{bias}=0$), consistent with our band structure calculations.
To further investigate the compound, we turn to a detailed examination of the electronic band structure of DySbTe through ARPES measurements. In this comprehensive study of the band structure of DySbTe, we present the experimentally obtained constant energy contours (CECs) from ARPES at the Fermi level and various binding energies using an incident photon energy of 75 eV, as shown in 3(a). The leftmost panel of 3(a) reveals the FS characterized by a diamond-shaped Fermi pocket centered around the $\Gamma$ point, a distinctive feature of ZrSiS-type materials. The FS shows small pockets around the $\overline{\text{X}}$ points, with a narrow pocket extending towards neighboring $\overline{\text{X}}$ points to form the diamond shape. Notably, the FS is gapped specifically along the $\overline{\text{M}}-\overline{\Gamma}-\overline{\text{M}}$ direction with a pronounced decrease in spectral intensity, while the corners of the diamond pocket remain ungapped, highlighting areas of persistent electronic states. As the binding energy increases, we observe that the band along the $\overline{\text{M}}-\overline{\Gamma}-\overline{\text{M}}$ direction begins to fill around $-$150 meV. The band features within the CECs become more pronounced with increasing binding energies. At approximately $-$250 meV below the Fermi level, a hole pocket containing a double sheet and features near the $\Gamma$ point begin to form. This evolution continues, with the diamond-shaped pocket developing into a triple sheet band around the $\overline{\text{X}}-\overline{\Gamma}-\overline{\text{X}}$ direction at around $-$350 meV and then reverting to a double sheet nature at around $-$450 meV. These experimental CECs can be compared to the calculated CECs presented in 3(b). At the Fermi energy, our experimental data reveal a gap along the $\overline{\text{M}}-\overline{\Gamma}-\overline{\text{M}}$ direction without the double sheet feature observed in theoretical results. Both the experimental and theoretical results show open, diamond-shaped Fermi pockets; however, the experimental pockets close rapidly at higher binding energies within the Brillouin zone, as opposed to the theoretical calculations.
To further investigate the electronic structure of DySbTe, we focus on the dispersion maps along various high symmetry directions. Specifically, Fig.4(a) presents the experimental dispersion cut along the $\overline{\text{M}}-\overline{\text{X}}-\overline{\text{M}}$ direction. The bands observed along this direction reveal a significant gapped feature at the $\overline{\text{X}}$ point with a band gap around 300 meV, which differs to other lighter members in the $Ln$SbTe family that have little to no gapped feature along this direction [37, 39, 36, 40]. This gap can be further visualized in the second derivative plot shown in Fig.4(b). These experimental observations are compared with the calculated band structure along the $\overline{\text{M}}-\overline{\text{X}}-\overline{\text{M}}$ direction, as depicted in Fig.4(c), which shows good agreement with the experimental data. Next, we explore the band features along the $\overline{\text{M}}-\overline{\Gamma}-\overline{\text{M}}$ direction using an incident photon energy of 75 eV, as shown in Fig.4(d). In this dispersion cut, a very sharp, prominently linearly dispersive band with a high band velocity of around 4.74 eV Å is observed. This band extends close to the Fermi level, terminating around 0.1 eV below it. This feature again remains unseen in other lighter $Ln$SbTe members which have their linear dispersive bands extend up to the Fermi level [37, 39, 36, 40]. The second derivative plot in Fig.4(e) highlights this linear band further, showing its movement away from the $\overline{\Gamma}$ point. Additionally, starting around 0.8 eV below the Fermi level, multiple bands with varying band velocities are seen crossing near the $\overline{\Gamma}$ point. These findings are corroborated by the calculated band structure shown in Fig.4(f), which predicts that the bands should extend up to the Fermi level; however, experimental results reveal a gap instead. Despite this discrepancy, the calculated and experimental data are otherwise in good agreement.
We then investigated the dispersion map along the $\overline{\text{X}}-\overline{\Gamma}-\overline{\text{X}}$ direction, as presented in Fig.5(a), using an incident photon energy of 75 eV. Similar to other ZrSiS-type materials, we anticipated multiple Dirac crossings along the $\overline{\text{X}}-\overline{\Gamma}-\overline{\text{X}}$ direction. To verify if these Dirac crossings persist in DySbTe, we examined the second derivative plot shown in Fig.5(b). From both Figs.5(a) and (b), we observe linear dispersive bands around the $\overline{\text{X}}$ point, along with a gap located approximately 0.1 eV below the Fermi level (surrounded by a dashed red rectangle in Fig.5(b)). This is consistent with our STS measurements in Fig.2(b), which show a minimum in the d$I$/d$V$ spectroscopy at $V_{bias}\approx-100$ meV. This observation is distinct from other $Ln$SbTe members, where a nodal line crossing typically exists near the Fermi level [37, 39, 36, 40]. Further analysis along the $\overline{\text{X}}-\overline{\Gamma}-\overline{\text{X}}$ direction with an incident photon energy of 90 eV, as shown in Fig.5(c), reveals that the induced gap remains near the $\overline{\text{X}}$ point. These experimental findings are compared to the calculated band structure in Fig.5(d), showing reasonable agreement.
In conclusion, we have investigated the electronic structure of DySbTe through high-resolution ARPES and STM/S measurements with complementary DFT calculations. Our experimental data reveal a diamond-shaped Fermi pocket centered around the $\overline{\Gamma}$ point, characteristic of the ZrSiS-type materials with the nonsymmorphic $P4/nmm$ space group, where distinct band features evolve at various binding energies. Notably, STM/S results indicate a minimization of the density of states at around 100 meV below the Fermi level, and we observed, through ARPES, a gap feature along the $\overline{\text{M}}-\overline{\text{X}}-\overline{\text{M}}$ and $\overline{\text{M}}-\overline{\Gamma}-\overline{\text{M}}$ direction with a additional gap located at the $\overline{\text{X}}$ point along the $\overline{\text{X}}-\overline{\Gamma}-\overline{\text{X}}$, distinguishing DySbTe from other lighter $Ln$SbTe members. The low-temperature thermodynamic and electrical transport measurements, combined with STS, confirmed an AFM character of the electronic ground state in DySbTe and a semimetallic nature of the compound. All our findings provide a deeper understanding of the influence of SOC on the electronic structure and topological properties within the $Ln$SbTe family.
M.N. acknowledges the support from the National Science Foundation (NSF) CAREER Award No. DMR-1847962, and the NSF Partnerships for Research and Education in Materials (PREM) Grant No. DMR-2424976. M.Y. acknowledges support from the National Science Foundation (NSF) MRSEC 2308979. T.R., S.E., G.C., and D.K. are supported by the National Science Centre (Poland) under research grant 2021/41/B/ST3/01141. A.P. acknowledges the support by National Science Centre (NCN, Poland) under Project No. 2021/43/B/ST3/02166. This research utilized resources of the Stanford Synchrotron Radiation Lightsource (SSRL), SLAC National Accelerator Laboratory, which is a DOE Office of Science User Facility under Contract No. DE-AC02-76SF00515. We express our gratitude to Dr. Makoto Hashimoto and Dr. Donghui Lu for providing valuable beamline support at SSRL. We acknowledge Milo Sprague and Anup Pradhan Sakhya for their help during the experiment and fruitful discussion during the manuscript preparation.