Chemical tuning of a honeycomb magnet through a critical point

Initial attempts at partial arsenic substitution with vanadium in trigonal BCAO [space group R$\overline{3}$H (148)] resulted in a series of compositions with clear, progressive shifts in both structural and magnetic properties. The powder diffraction patterns measured for each composition, shown in Figure S1, show shifts to higher 2$\theta$ for the majority of (h0l), (0kl), and (00l) reflections, and a slight shift to lower 2$\theta$ for the singular (hk0) reflection around 35.8 degrees. Assuming random incorporation of vanadium into the structure, Rietveld refinement of the experimental pXRD and SCXRD data in the R$\overline{3}$H space group resulted in similarly good fits for all compositions upon initial and secondary heating (Tables S1-S4, S5-S10). Several compositions initially possessed a small ($\sim$3-5 wt%) \chBa3(AsO4)2 impurity phase, which was typically removed via a secondary heating step above the materials’ melting point. Reflections corresponding to this impurity phase were also observed to shift with increasing nominal V-substitution of the target phase, indicating that partial substitution of arsenic with vanadium occurs for both phases present in the initial product mixture. Up to x = 0.70, the roughly 0.02 Å increase in M-O bond lengths for tetrahedrally-coordinated vanadium is observed to produce a gradual contraction of the unit cell along the c-axis, as well as a gradual expansion of the a-axis, by 0.835 Å and 0.054 Å respectively (Figure 1d). Compared to BCAO, these relative shifts $\Delta c/c=-3.6\%$ and $\Delta a/a=1.1\%$ correspond to a volumetric contraction of $\Delta v/v=-1.4\%$. When substituted for the arsenate groups present in pure BCAO, the longer V-O bond lengths in the BCAO-V compositions produce both a compression of the interplanar region along the stacking axis and an additional tilting of the Co octahedra within the honeycomb plane. Beyond x = 0.70, the tetragonal structure of the pure \chBaCo2(VO4)2 (BCVO) phase [space group I4₁/acd (142)] begins to appear, with slight shifts in the observed reflections due to partial arsenic substitution for vanadium, and the trigonal BCAO structure gradually disappears. Similar to previous reports of vanadium substitution for phosphorus in \chBaCo2(PO4)2 (BCPO), the effective upper limit of vanadium uptake in the trigonal BCAO phase appears to be around 70%.¹¹

More subtle structural changes on increasing vanadium substitution can be observed in the refined SCXRD data. Relative to pure BCAO, the average Co-O-Co bridging angle between neighboring \chCo^2+ ions increases by 0.7%, while the average Co-O-As bridging angles decrease by -0.6% and -0.2%, respectively, up to x = 0.50 (Figure S2). Laue diffraction patterns of representative BCAO-V single crystals, shown in Figure S3, were used to confirm the crystalline quality of each composition and show no signs of twinning or diffuse scattering, up to x = 0.20.

The true degree of vanadium substitution in each composition was both estimated from refinement of the experimental diffraction data and confirmed via ICP-OES analysis. Calculated V-substitution levels obtained via Rietveld refinement of the As-site in BCAO after the first and second heating steps are shown in Figure S8a. Vanadium occupancies on the As-site typically refine close to nominal values for powder samples produced via initial heating of the reaction mixture, while the secondary heating step is observed to either meet or slightly exceed the expected occupancy. This can likely be attributed to the incorporation of the V-substituted \chBa3(AsO4)2 impurity phase from the initial reaction mixture into the BCAO structure upon further annealing.

ICP-OES analysis of synthesized BCAO-V samples agrees well with the V-substitution levels estimated via Rietveld powder and single crystal structural refinement, shown in Figure S8b and summarized in Table S12. No pronounced recovery of vanadium is observed from the initial powder synthesis to the secondary crystallization step, indicating that initial V-substituted \chBa3(AsO4)2 impurity concentrations are actually generally minor. The measured V-content in the maximally-substituted composition suggests an upper limit of around 56.0% before the tetragonal structure becomes more prevalent. A second sample of the x = 0.10 composition was also analyzed and was found to possess closer to 17% V. This sample was grown by the vertical Bridgman method, where the initial powder product was placed in a pointed alumina crucible, sealed under vacuum in a quartz tube, melted and slowly recrystallized. Due to the vertical temperature gradient, the slow re-solidification may have resulted in inhomogeneous distributions of vanadium throughout the grown crystal mass, and as a result these crystals were not used for further characterization.

As the shifts in both lattice parameters appear to be progressive over the studied composition range and the shift observed along the a-axis is small relative to that observed along the c-axis, it is likely that vanadium does not substitute for arsenic randomly in the structure, but rather as discrete domains. Attempts to generate accurate unit cells for each composition, particularly at low substitution levels, would thus be dependent on the manner in which these vanadium clusters are distributed throughout the structure. This would also imply a wider distribution of distinct cobalt coordination environments, producing a more complex magnetic ground state. Due to the difficulty in making this determination concretely, refinements on both pXRD and SCXRD data presented in this work were performed under the assumption of random substitution. Regardless of the true vanadium distribution, the progressive trends in the pXRD and SCXRD data and color changes in the grown crystals from pink (pure BCAO) to red (0 $<$ x $\leq$ 0.40) and gradually to black (x $\geq$ 0.40), together indicate average changes in the crystal field splitting energy of the octahedrally-coordinated Co and potentially also in the trigonal distortion thereof.

Representative Raman scattering spectra collected from BCAO-V single crystals provide additional evidence for the gradual tuning of the Co coordination environment upon substitution of the interlayer polyanion group (Figure 2). In pure BCAO, the symmetry of the trigonal R$\overline{3}$H structure is well-described by the $C_{3i}$ point group. Of the symmetrically-allowed optical modes, 18 are expected to be Raman-active (12 in the (ab) plane (6$A_{g}$ + 6$E_{g}$)) and 10 of which have been previously observed below 900 cm^-1.¹² The polarization dependence of band intensities was used to distinguish between $A_{g}$ and $E_{g}$ modes, which were observed to be strongest in the (x,x) and (x,y) configurations, respectively (Figure S4).¹²

Upon partial substitution of As with V, this polarization dependence remains relatively consistent, with singlet $A_{g}$ modes initially predominating in the parallel polarization (x,x) configuration, doublet $E_{g}$ modes in the (x,y) configuration, and the overall measured intensity remaining higher for the former. The measured frequency shifts and linewidths for each vibrational mode as a function of substituted vanadium content are shown in Figure S5. Up to x = 0.075, all $E_{g}$ and lower-energy $A_{g}$ modes begin to harden, while the $A_{g}$(5) mode begins to soften, all without notable broadening. The higher-energy $E_{g}$ modes also begin to decrease in intensity.

At x = 0.10, the intensity of the measured spectra in both configurations drops considerably, with that of the (x,y) polarization decreasing to less than half of that observed in either the x = 0.075 or x = 0.15 compositions. Interestingly, the intensity of the two highest-frequency $A_{g}$(5) and $E_{g}$(5) modes decreases by over an order of magnitude relative to the same samples, indicative of more considerable changes to the local bonding environment in BCAO. While reproducible across several single crystal samples, powder samples of this composition do not possess such a drop in magnitude, as shown in Figure S6. This effect likely then arises as a consequence of a distinctive clustering order for the x = 0.10 phase that manifests only across larger domains. Beyond x = 0.10, the majority of singlet modes begin to soften and the majority of the bands begin to markedly broaden. The higher-energy $E_{g}$ modes are not observed to shift in frequency, but rather decrease in intensity and are entirely absent by the x = 0.20 composition. At and above this level of substitution, the intensity ratio of the remaining $A_{g}$ and $E_{g}$ modes is no longer dependent on the experimental configuration and the spectra differ only in magnitude. An additional broad feature also appears at $\sim$875 cm^-1 at higher substitution levels and is most prominent for the x = 0.20 composition. This likely indicates a more heavily disordered bonding environment that introduces greater spin freezing in this region.

Upon chemical substitution, the majority of systems display noticeable broadening of Raman active modes, indicative of random displacement of the substituted atom and subsequently higher disorder throughout the material.^{13, 14} While slight broadening of several modes is observed upon partial V-substitution of BCAO, the bands remain comparatively sharp, indicating that vanadium does not purely substitute randomly and further supporting that it may do so in a periodic manner.

Although not explicitly assigned in previous reports, the highest frequency of these likely correspond to breathing modes of the two distinct oxygen sites present in the structure.¹⁵ The frequencies at which these modes appear agree well with those previously attributed to honeycomb lattice materials containing edge-sharing Co-octahedra.^{16, 14} As increasing vanadium incorporation is accompanied by a larger contraction of the unit cell along the stacking axis than expansion within the honeycomb plane, the oxygen vibrations occurring between cobalt layers would be expected to be more substantially impacted by greater substitution. The most impacted, highest-energy oxygen modes [$A_{g}$(5) and $E_{g}$(5)] are thus attributed to the apical oxygen of each arsenate/vanadate group not bound within the honeycomb plane, labeled O2 in Figure 1a. The lower frequency oxygen modes [$A_{g}$(4) and $E_{g}$(4)], then correspond to the oxygens directly bound to Co atoms, labeled O1 in Figure 1a. In the absence of broader structural change, these modes would be less impacted by substitution outside of the honeycomb planes.

The anomalous frequency and linewidth trends observed for vibrational modes in the x = 0.10 composition, combined with the sharp decrease in measured intensity, provide evidence for a critical point at which signs of a more symmetric bonding environment emerge, despite the retention of the trigonal R$\overline{3}$H structure suggested by the diffraction data. Whereas the gradual shifts observed in the calculated lattice parameters for each composition provide evidence for the amount of vanadium substituting into the BCAO structure, this critical point illustrates that the way in which that vanadium is incorporated is not consistent throughout the series. This composition clearly delineates the BCAO-V series between regions of order and disorder, driven by increased steric pressure about the honeycomb plane.

Temperature-dependent DC magnetic susceptibility measurements of V-substituted BCAO powder compositions, the ordering transition temperatures of which are shown in red in Figure 3, suggest a gradual tuning of the magnetic ground state of the system with increasing vanadium content. In pure BCAO, long-range incommensurate antiferromagnetic (AFM) order is suppressed above T = 5.4 K. From 0.025 $\leq$ x $\leq$ 0.09, the observed transition to the incommensurate ordered state decreases in temperature to T $\approx$ 3.0 K, about half that of the parent material. From 0.10 $\leq$ x $\leq$ 0.20, no transition is typically observed, but rather the susceptibility rounds off and stagnates, down to at least T = 0.4 K. Above x = 0.20, the transition reappears and gradually increases in temperature to exceed that of either pure BCAO or BCVO, before again decreasing to match that observed in the fully-substituted end-member with the dominance of the tetragonal-type phase.

To better understand how the magnetic ground state of the system evolves with increasing vanadium content, particularly for the low (x = 0.025-0.20) V-substituted samples, temperature-dependent DC magnetic susceptibility measurements were also performed on single crystals of these compositions. With the field applied within the honeycomb plane ($\mu_{0}\text{H}\perp$ c, Figure S9), the incommensurate ordering transition declines to T = 4.06 K in the x = 0.075 sample, slightly higher than that observed in the powder measurements. At and above x = 0.10, the susceptibility instead plateaus below T = 3.0 K, indicating that the incommensurate ordered state is suppressed in these compositions. Some variability is observed for samples of the x = 0.10 composition, with a weak antiferromagnetic transition sometimes appearing between T = 2.0 - 3.0 K before the susceptibility plateaus. As this occurred even for samples from the same batch and other measurement techniques did not show large variability, this can likely be attributed to the proximity of this composition to a critical point. The inherent instability of such a state leaves the magnetic behavior of these samples highly sensitive to a number of synthetic and handling conditions, such as furnace temperature gradient, cooling rate of the melt and sample handling, all of which can produce additional strains and alter magnetic ground state degeneracies. The higher-substituted samples also exhibit a bifurcation between the zero-field-cooled (ZFC) and field-cooled (FC) susceptibility measurements below T = 2.0 K, indicating some degree of spin freezing. The ZFC susceptibility for the x = 0.15 composition trends negative below $T_{\text{f}}$, a feature typically attributed to materials containing multiple, highly anisotropic magnetic sublattices.¹⁷ As \chCo^2+ is the only magnetic cation present in the BCAO-V system, this composition must possess a higher proportion of discrete cobalt clusters which produce a net negative magnetization at low temperatures.

Curie-Weiss analysis at high temperatures (T = 150 K – 300 K) for $\mu_{0}\text{H}\perp$ c yields a Curie-Weiss temperature $\theta_{\text{CW}}$ $\approx$ 37(1) K for all compositions and an effective magnetic moment $p_{\text{eff}}$ $\approx$ 5.5(2) $\mu_{B}$ (Table 1, Figure S11), which lies between the effective moments 3.87 $\mu_{B}$ and 6.54 $\mu_{B}$ calculated for the ${}^{4}F_{9/2}$ term of the Co²⁺ $3d^{7}$ ion in the spin only versus the full spin-orbital limits, respectively. The relatively stable positive Curie-Weiss temperature indicates the preservation of dominant FM coupling between adjacent \chCo^2+ ions. Yet $\theta_{\text{CW}}$ is $\approx$ 20% larger for the x = 0.10 and x = 0.15 samples, suggesting a net enhancement of ferromagnetism with V-substitution. The effective moment $p_{\text{eff}}$ for these compositions is also slightly lower than for either the other compositions or the parent BCAO, indicating additional minor quenching of the \chCo^2+ orbital angular momentum due to an increase in orbital symmetry. Although the substitution of the apical polyanion groups would not be expected to greatly impact the in-plane magnetic exchange pathways, these results, as well as the observed structural changes, indicate modification of both the FM $J_{1}$ and the AFM $J_{3}$ exchange interactions and the degree of distortion of the cobalt octahedra.

When the field is instead applied along the stacking axis ($\mu_{0}\text{H}\parallel$ c), more substantial changes are observed in the measured DC magnetic susceptibility and calculated Curie-Weiss parameters (Figure S10). As in the in-plane measurements, increasing vanadium substitution results in a suppression of the incommensurate ordering transition to T = 4.06 K by x = 0.075, beyond which the susceptibility plateaus below T = 3.0 K. Curie-Weiss analysis of the high-temperature susceptibility results in negative $\theta_{\text{CW}}$ of variable magnitude, as well as reduced effective magnetic moments, relative to pure BCAO (Table S13, Figure S11). All derived $\theta_{\text{CW}}$ for compositions in this range are smaller than that of the parent phase. The $\theta_{\text{CW}}$ for both the x = 0.025 and x = 0.20 crystals, T = -67.8 K and T = -62.8 K respectively, represent local minima in the series. Changes in superexchange pathways as well as in the spin-orbital crystal field levels could both be important factors. With increasing substitution, the effective moment $p_{\text{eff}}$ is more variable, however generally trends downward. This may arise due to the variable number of distinct cobalt coordination environments created as progressively more vanadium is incorporated into the structure. A wider distribution of coordination environments would possess more variability in the average degree of orbital quenching amongst \chCo^2+ ions in each honeycomb layer and potentially a lower net effective moment as a result.

A clearer comparison between the observed trends in relative magnetic interaction strength for spin components within and perpendicular to the honeycomb plane can be drawn from the Curie-Weiss-normalized temperature-dependent magnetic susceptibility of each composition, where C/[($\chi-\chi_{0}$) $|\theta|$] = T/$|\theta|$ + 1 for dominant AFM interactions, and C/[($\chi-\chi_{0}$) $|\theta|$] = T/$|\theta|$ - 1 for dominant FM interactions (Figure 4). For all compositions, spin components parallel to c exhibit AFM deviations from Curie-Weiss behavior above the respective $T_{\text{N}}$, below which FM deviations are observed. These low-temperature deviations are attributed to the development of net AFM planar spin correlations, and decrease in magnitude with increasing vanadium substitution. When the field is applied within the honeycomb plane ($\mu_{0}H\perp c$), FM deviations from Curie-Weiss behavior dominate for temperatures above T $\approx$ 150 K. Below this temperature, AFM correlations become more prevalent. The full suppression of the incommensurate AFM ordering transition for the x = 0.10 composition is apparent in the absence of a thermal anomaly in both $\chi_{\parallel c}$ and $\chi_{\perp c}$.

A convenient measure of the magnetic anisotropy and its dependence on substitution and temperature lies in the susceptibility ratio, $\chi_{\perp}c/\chi_{\parallel}c$, which varies as a function of V-substitution (Figure S11c). In pure BCAO, $\chi_{\perp}c/\chi_{\parallel}c\approx$ 62, a value which drops considerably to 9 by x = 0.025, before slowly increasing as a function of further substitution. At x = 0.10, there is an additional slight decrease in the observed anisotropy. The trends in the magnetization measured along each of the principal axes are however not directly correlated. The magnetization gradually decreases when measured along the stacking axis with increasing V-content, while in-plane measurements are relatively consistent, with two sharper declines at x = 0.025 and x = 0.10. As no substitution occurs within the honeycomb lattice, but rather adjacent to it, little variation would be expected in the strength of the $J_{1}$ superexchange interactions between neighboring cobalt atoms in the ab plane, though its spin-space anisotropy could change. With substitution of a minimal amount of vanadium into the structure and the corresponding contraction of the unit cell along the c axis, Co²⁺ ions within the honeycomb layer are pushed closer to and then further from perfect octahedral coordination. Further incorporation into and distortion of the parent structure progressively tunes the competing $J_{1}$/$J_{3}$ exchange interactions, with the x = 0.10 composition representing a critical point at which the incommensurate state is suppressed without the introduction of considerable glassiness to the system.

One feature commonly observed among magnetic cobaltates is the presence of metamagnetic transitions in field-dependent magnetization measurements, which are associated with the strong anisotropy of the Co²⁺ Kramers doublet ground state.^{18, 19, 20} In pure BCAO, this manifests as a pair of low-field metamagnetic phase transitions ($\mu_{0}\text{H}_{c1}$ = 0.26 T and $\mu_{0}\text{H}_{c2}$ = 0.53 T) which together produce the dumbbell-shaped hysteresis observed at T = 1.8 K.⁶ Field-dependent magnetization measurements performed on representative BCAO-V powder compositions, shown in Figure 5 as the derivative of magnetization with respect to the applied field strength, display a clear reduction in both the magnitude and applied field strength at which the \chCo^2+ metamagnetic transitions appear. Upon any amount of V-substitution, the dumbbell-shaped hysteresis observed in pure BCAO at T = 2.0 K is no longer present, with the two metamagnetic transitions gradually merging into one and broadening. At and above x = 0.25, minor remanence reemerges and the system begins to behave more like the fully-substituted BCVO phase. The suppression of metamagnetism in this system is indicative of a gradual shift toward more perfect octahedral coordination and an increase in orbital symmetry of the Co²⁺ cations. This would also be consistent with the suppression of the incommensurate ground state observed in pure BCAO. The disorder associated with V-substitution may also play a role, as it can impede phase transitions between the long wave length modulated spin structures associated with the metamagnetic transitions in pure BCAO.

Measurements performed on lower V-content BCAO-V single crystal compositions provide additional insight into the evolution of the observed transitions with increasing substitution. In-plane ($\mu_{0}\text{H}\perp$ c) magnetization measurements (Figure 6) show no sign of hysteresis in any composition at either T = 2.0 K or T = 5.0 K, with all saturating by $\mu_{0}\text{H}$ = $\pm$ 1 T. At T = 0.4 K, only a weak remanence is present from x = 0.10-0.20, which can likely be attributed to the weak FM coupling between neighboring Co²⁺ ions within the honeycomb plane. The two sharp transitions observed in pure BCAO are broadened in all substituted compositions and gradually merge by the x = 0.10 sample, leaving only a single smoothed curve. This indicates some alteration of the average cobalt coordination environment in the substituted structure upon increasing V-content and subsequently also to the ground state degeneracy of the system. Alternatively, the $J_{1}/J_{3}$ ratio may be passing through a critical value driven by V-substitution. Out-of-plane ($\mu_{0}\text{H}\parallel$ c) magnetization measurements show only a linear field-dependence and no notable hysteresis at both T = 2.0 K and T = 5.0 K, as in the parent material (Figure S12).

The evolution of the cobalt metamagnetic transitions in V-substituted BCAO is also observed in the in-plane, field-dependent AC magnetic susceptibility data, shown in Figure S13. At x = 0.025, two transitions are visible at $\mu_{0}$H = 0.25 T and $\mu_{0}$H = 0.43 T, with the former being weaker. Upon further substitution, the higher-field transition gradually shifts to meet and merge with that occurring at lower field, in agreement with a gradual suppression of the incommensurate state. The magnitude of the combined transition then continuously decreases with increasing V-content. In contrast to the DC magnetization data, the magnitude of the AC susceptibility of several x = 0.10 samples also notably decreases, relative to the other compositions, another indication of a more complex interplay between competing exchange interactions at low temperatures.

This discrepancy between the DC and AC response of several compositions is also apparent in temperature-dependent AC magnetic susceptibility measurements of BCAO-V powder samples (Figure S14). As in the DC susceptibility, substitution of vanadium for arsenic in BCAO results in a gradual shift in the observed transition temperature, down to T = 3.6 K in the x = 0.075 composition, albeit with greater broadening. However, from x = 0.10 - 0.20, a narrower transition appears from T = 2.5 - 2.7 K, lower in temperature than other members of the series. Above this level of V-substitution, the AC transition begins to weaken, broaden, and shift higher in temperature. When performed on single crystals of the less-substituted compositions, a similar trend appears, with variability again observed in the susceptibility of x = 0.10 samples (Figure 7). Often samples of this composition possessed sharper transitions, relative to those of the x = 0.075 and x = 0.15 samples, providing additional evidence for a critical point in this compositional regime.

Notable differences between the magnetic response of a material via AC and DC characterization techniques typically indicate spin glass-like behavior, and subsequently varied spin relaxation rates in a material.²¹ In order to better understand the evolution of the magnetic ground state of the BCAO-V system, the temperature dependence of the AC magnetic susceptibility was measured as a function of applied AC frequency for the lower-substituted single crystal compositions (Figure 7, zoomed view in Figure S15). From x = 0.025-0.075, there is no frequency dependence to T${}_{\text{N}}$ from $f$ = 50-10,000 Hz, whereas each of the x = 0.10, x = 0.15, and x = 0.20 compositions display minimal shifts over the same range. The abrupt shift from the absence to appearance of a frequency-dependent T${}_{\text{N}}$ is a common signature of the onset of spin freezing, and when considered with the change observed in the DC susceptibility over the same range further suggests the presence of a critical point between the x = 0.075 and x = 0.15 compositions. Similar to the variability observed in DC susceptibility measurements, samples of the nominal x = 0.10 composition display a variable frequency dependence of the AC susceptibility, both within and between batches. The typical frequency dependence of this composition is similar to that observed in the higher-substituted samples, however several crystals closer to the critical point display a reduced dependence. The comparative consistency observed across the other compositions further indicates a proximity to a critical point around x = 0.10.

Plotting of this frequency dependence as 1/T vs. ln($\omega$) yields a linear trend for each of the three higher V-content samples and indicates the presence of some degree of spin freezing (Figure S16, Table S14). While the uncertainty associated with determination of the precise transition temperature produces considerable error in the calculated activation energies and characteristic frequencies for the x = 0.10, x = 0.15, and x = 0.20 compositions, both trend upward with increasing x. From x = 0.10-0.20, linear fitting yields an Arrhenius-type activation energy between E${}_{\text{a}}$/k_B = 91.8 - 107.5 K and a characteristic frequency on the order of $\omega_{0}$ = 10¹⁶ - 10¹⁸ Hz, consistent with a cluster glass ground state. ^{22, 23, 24} The magnetic ground state of the BCAO-V system is then likely best understood as a gradual suppression of the incommensurate order down to around x = 0.10, followed by the introduction of increased glassiness after its disappearance, as has been commonly reported for phases possessing high degrees of magnetic frustration and substitution-induced disorder.²⁵ With respect to the interaction strength between magnetic Co²⁺ ions in the BCAO-V structure, the x = 0.10 composition then represents a critical point about which strong spin correlations persist down to low temperatures without the transition to the incommensurate state. The bifurcation of FC/ZFC magnetization data for T $\approx$ 1.48 K (Figure S9d) indicates some degree of spin freezing at the critical point, as is generally expected in the presence of disorder. In the analogous \chBaCo2(PO4)_2-x(VO4)_x system, similar behavior was attributed to the presence of a spin-liquid-like state stabilized by strong quantum fluctuations.¹¹

Further evidence for a spin-glass-like ground state in this composition space is observed in the heat capacity of representative BCAO-V polycrystalline samples, shown in Figure 8. With increasing V-content, the transition attributed to the onset of long-range incommensurate order in pure BCAO again substantially broadens and declines in both magnitude and temperature, down to T $\approx$ 4.0 K from x = 0.075-0.20. In the absence of an applied field, all substituted compositions possess singular broad humps, higher in temperature than the transitions observed in both DC and AC susceptibility. Beyond this level of substitution, the transition shifts higher in temperature, in agreement with that observed in the AC susceptibility measurements and indicative of the reemergence of an ordered ground state in the system.

Additional hints to the glassy nature of the magnetic ground state in the intermediate substitution regime can be observed in the field-dependent heat capacity of single-crystalline BCAO-V compositions. When the field is applied within the honeycomb plane ($\mu_{0}\text{H}\perp$ c), all compositions display a linear trend of further broadening and suppression of the transition to higher temperature, with the application of as small as a $\mu_{0}\text{H}$ = 0.5 T field being sufficient to partially suppress the magnetic entropy (Figure S17). This provides an additional indication of spin freezing in the magnetic ground state and is consistent with the observed stability of the in-plane FM correlations upon V-substitution. Conversely, application of the field along the stacking axis yields a more varied response, with progressively larger fields being required for any suppression of the feature to occur (Figure 9). In the x = 0.10 sample, a $\mu_{0}\text{H}$ = 2.5 T field is sufficient to suppress the transition by about $\Delta T$ = 3.9 K, whereas for the x = 0.15 sample this produces a shift of only $\Delta T$ = 1.7 K and for the x = 0.20 sample this decreases to $\Delta T$ = 1.1 K. However when the applied field is increased to $\mu_{0}\text{H}$ = 5.0 T, the anomaly present for all three compositions is suppressed to a comparable degree. This suggests a finite barrier to spin reorientation which increases from the x = 0.10 to the x = 0.20 composition, supporting the presence of two distinct ground state phases - the latter behaving as a cluster glass and the former as a more complex intermediate between the regions of order and pure glassiness bounding the critical point.

Estimates of the magnetic contribution to the measured heat capacity were attempted via subtraction of the non-magnetic analogue of each V-substituted composition. Although \chBaMg2(AsO4)2 and its V-substituted derivatives could be easily synthesized, both direct subtraction and rational scaling of the Mg-analogues from the measured heat capacity of the cobalt phases ultimately did not suggest full recovery of the expected magnetic entropy by T = 50 K. Subsequent measurement of the Raman scattering spectrum of \chBaMg2(AsO4)2, compared to that of BCAO in Figure S7, reveals that the substitution of the much smaller Mg atom for Co produces variable, non-uniform shifts in the frequencies of the respective vibrational modes. While the two phases share the same structure, the frequencies of phonon modes in pure and substituted BCAO then differ too considerably for reliable approximation via subtraction of the Mg-analogue. However, an estimate of the recovered magnetic entropy for each composition was obtained by scaling of the BMAO-V phase, such that the measured $C_{\mathrm{p}}/T$ converged with that of the Co-analogue by T = 50 K (Figure S18). Integration of this estimated contribution over the same temperature range yields an entropy rise greater than the predicted $\Delta S_{\mathrm{mag}}$ = 1/2Rln2 for an ideal Kitaev system and less than the $\Delta S_{\mathrm{mag}}$ = Rln2 expected for a typical S = 1/2 magnet.²⁶ Modeling of the evolution of the BCAO phonon spectrum as a function of V-substitution will ultimately be necessary to determine the true magnetic entropy rise in this temperature regime.

Reports of other substituted honeycomb phases exhibiting such intermediate spin relaxation rates have typically attributed this behavior to either the presence of stacking faults or magnetic site defects.²⁷ Due to the higher stability of the high-spin configuration in most Co²⁺ compounds, as well as low likelihood of further oxidation to the Co³⁺ oxidation state upon V-substitution, the presence of abundant magnetic site defects is unlikely. In either case, the calculated effective magnetic moment would also be expected to decrease more than is observed. For each composition, the substitution of clusters of arsenic atoms by vanadium would generate distinct stacking sequences within the structure akin to more random stacking faults, leading to more variability in the balance of $J_{1}$ superexchange and $J_{3}$ third nearest-neighbor interactions within the honeycomb plane. At low substitution levels, this increases the overall degree of geometric spin frustration, while at higher substitution levels this frustration is relieved by typical spin-freezing. Between these two regions, the stacking sequence in the x = 0.10 composition produces a near-optimal tuning of the $J_{1}$/$J_{3}$ exchange interactions, fully suppressing the incommensurate-ordered state before the introduction of more considerable spin freezing.

In solid solutions such as the BCAO-V system, qualitative changes in material properties are typically well described by percolation theory, wherein channels between impurity atoms form above some threshold impurity concentration.²⁸ These channels produce deviations from the expected trends where impurity atoms enter the structure but largely do not interact with one another. On the two-dimensional honeycomb lattice, this threshold has been reported to be 0.7, which in the BCAO-V series is found to be the upper substitutional limit before transformation to the tetragonal \chBaCo2(VO4)2 structure.²⁹ However, magnetization and Raman scattering measurements show that the system passes through a critical point around x = 0.10, suggesting that standard percolation theory can’t be used to effectively describe the evolution of magnetic properties in this system.

Several alternative models of exchange disorder via dilution or magnetic substitution have also been proposed for systems believed to possess dominant Kitaev exchange.^{30, 31} For the honeycomb iridate family of materials, these models typically agree well with experimental reports, where substitution tunes the magnetic ground state directly from long-range order to that of a typical spin glass. In $\alpha$-\chRuCl3, substitution on the ruthenium site initially suppresses the long-range zigzag order before passing through a narrow region of short-range, quasistatic order and finally to the onset of increased spin freezing.^{32, 33, 34} While substitution in these systems occurs on the magnetic honeycomb lattice, vanadium incorporation into the BCAO structure occurs only on the arsenic site bounding the honeycomb lattice and not for the cobalt within it. As our results are not well-described by any existing model for this type of substitution and direct honeycomb substitution in these related systems has also been claimed to tune the magnitude of off-diagonal exchange, the local minima observed in the x = 0.10 composition likely arise from tuning of the $J_{1}$/$J_{3}$ exchange interactions via exchange disorder.

The impact of increasing disorder on the $J_{1}$/$J_{3}$ exchange can be understood as both a consequence of the gradual compression of the structure along the stacking axis and a reduction in the covalency of the polyanion orbitals mediating $J_{3}$ exchange. In compressing the structure along the c-axis, the Co-O-Co superexchange pathway responsible for the dominant FM first neighbor interaction is widened beyond the 87.7 degrees observed in pure BCAO (Figure S2a). As predicted by the Goodenough-Kanamori rules, superexchange through an oxygen bridge between half-filled orbitals is expected to be dominant FM when the bridging angle is 90° and gradually attain more AFM character as this angle nears 180°.^{35, 36} As more vanadium is substituted into the structure, the average Co-O-Co bond angle widens, reducing the magnitude of the FM $J_{1}$ superexchange interaction.

However, the weaker $J_{3}$ exchange proceeding through the more complex polyanion group can’t be as easily rationalized by these rules (Figure 1g).^{8, 37} In pure BCAO, the bounding arsenate group physically bridges third-nearest neighbor Co²⁺ ions via a Co-O-As-O-Co pathway, however due to the large, delocalized orbitals of the arsenic center, bonding in the complex is highly covalent.³⁸ The polyanion group thus behaves akin to a superatom, with dominant superexchange-like interactions through the two oxygen atoms linking the cobalt and arsenic atoms. In BCAO, this interaction is AFM, and the competition between it and the FM $J_{1}$ exchange ultimately drives the transition to a long-range incommensurate state at low temperatures. In contrast to the strong covalency of the arsenate group, the smaller vanadate group is bonded more ionically, resulting in reduced orbital overlap and a more complex exchange pathway between third-nearest neighbor Co²⁺ ions.³⁸ Increasing the vanadate content of the system would then also decrease the magnitude of the AFM $J_{3}$ exchange. This is supported by the decrease in the Co-O-As bond angles observed on increasing vanadium substitution (Figure S2b). While both $J_{1}$ and $J_{3}$ are expected to decrease with increasing vanadium substitution, the combination of the decrease in Co-O-As bond angle, the more ionic character of the introduced vanadate group, and the subsequently reduced orbital overlap suggests a faster decline in $J_{3}$. The progression of magnetic properties observed across the BCAO-V series thus indicates that the competition between the weakening $J_{1}$ and $J_{3}$ exchanges gradually tunes away the incommensurate ground state described by the XXZ-$J_{1}$-$J_{3}$ model, toward that of the standard easy-plane XXZ model with the more dominant FM $J_{1}$.³⁷ The anomalous behavior observed about the critical point is then a consequence of a particular concentration and arrangement of vanadate groups within the structure where $J_{1}/J_{3}$ approaches a critical value and the incommensurate state is suppressed.