Molecular optomechanics in the anharmonic regime: from nonclassical mechanical states to mechanical lasing

Recently, in response to surprising experimental results [1, 2], it has been suggested that Raman scattering of light from molecules in plasmonic cavities can be cast as an optomechanical process [3, 4, 5, 6, 7, 8], with the molecular vibrations modes playing the role of ultra-high frequency mechanical resonators. This realization brought the vast set of tools developed for canonical cavity optomechanics to the field of Surface- or Tip-Enhanced Raman Scattering (SERS and TERS) research [9, 10, 11, 12]. The resulting formalism of molecular optomechanics led to new insights into the correlations of the inelastically scattered Raman light [4, 13], control over the quantum-mechanical description of the single- and multi-mode plasmonic cavities [5, 14, 15, 6], or the dynamics of systems with multiple molecules [16]. It also enabled the theoretical proposals [17], and experimental demonstrations of new THz detection techniques [18, 19].

Simultaneously, molecular optomechanics stretched the landscape of the conventional cavity optomechanics [20] towards the largely unexplored regimes of ultra-high mechanical frequencies characteristic of molecular vibrations, complex (multimode) optical spectrum, and to systems with hundreds, or thousands of homogeneous, and both directly and indirectly coupled mechanical modes. It also brought optomechanics closer to the elusive limit of the strong single-photon coupling [21, 22], by confining molecules in ultra-small volume optical cavities [23].

Unlike in the canonical optomechanical systems, the dynamics of the THz molecular vibrations involves only a few, lowest-energy mechanical states, thanks to the combination of low thermal population ($n_{b}^{\mathrm{th}}<0.1$) [10, 9, 24], large mechanical losses, and small populations of the optical cavities, which render the mechanical amplification mechanism ineffective [3, 4, 23]. Therefore, in the original formulation of molecular optomechanics [3, 4], and follow-up contributions, the molecular vibrations was routinely approximated by a harmonic model of the potential (see the schematic representation in Fig. 1). However, recent experiments (see e.g. Ref. [7]) are beginning to explore the more efficient amplification mechanisms, setting up questions about the role of the intrinsic anharmonicity of the mechanical potential [25, 26, 27]. To date, these effects were simply neglected, and their potential experimental observations attributed with other effects, such as the bond breaking [7]. At the same time, new types of hybrid plasmonic-dielectric cavities, characterized with small optical mode volumes and sharp spectral features [6, 14, 28, 29] offer realistic designs of systems which could resolve these anharmonicities.

Therefore, in this work we ask if this anharmonicity can be harnessed for new physics, and how do the predictions of cavity optomechanics hold up in a system with a strong mechanical nonlinearity: Can the anharmonic mechanical potential open up a pathway towards vibrational quantum nonlinearity [30], and expand the toolbox of nonlinear mechanical phenomena explored to date in optomechanics [31, 32, 33, 34]? Can we use it to engineer nonclassical mechanical states of vibrations [35] without employing external nonlinear elements [36, 37], or can the current experiments induce coherent mechanical lasing [38, 39, 40] in the presence of the mechanical linearity?

The manuscript is structured as follows: In Section II we introduce the formalism, and corrections to the conventional framework of molecular optomechanics, including the redefined coupling mechanism. We then show how this anharmonicity can be harnessed to prepare the molecular vibrations in a nonclassical state (Section III), and how the anhamornicities modify the mechanical amplification, and the onset of mechanical lasing in molecular systems (Section IV).

In the elementary formulation of molecular optomechanics, we consider a single quantized optical mode, coupled through nonlinear interaction to a single mechanical mode [3, 4]. The dynamics of this setup is described by the sum of the optical, mechanical, and interaction Hamiltonians:

The optical mode has resonant frequency $\omega_{a}$, and is coherently driven with amplitude $\Omega$ and frequency $\omega_{l}$, so that in the frame rotating with $\omega_{l}$ we have $\hat{H}_{\text{opt}}=\hbar(\omega_{a}-\omega_{l})\hat{a}^{\dagger}\hat{a}+\Omega(\hat{a}+\hat{a}^{\dagger})$. To explicitly consider the harmonic or anharmonic characteristics of molecular vibrations, here we explicitly write the mechanical Hamiltonian as

with $V_{M}$ representing the Morse potential

The eigenfrequency of the $k$th, out of $N$ bound states of $\hat{H}_{M}$, is

with $\delta\omega_{b}=\hbar\omega_{b}^{2}/(4D_{e})$, and $N=(\omega_{b}/\delta\omega_{b}-1)/1$.

The optomechanical interaction between molecular system (characterized by the position operator $\hat{x}$) and the optical cavity mode (with electric field operator $\hat{\mathbf{E}}$) is mediated by the Raman dipole, induced in the molecule with Raman polarizability tensor $\mathbf{R}$ by the optical cavity field as

The explicit connection to the molecular vibrations is given by representing $\hat{x}$ in the basis of the eigenstates $\left|\phi_{k}\right\rangle$ of the mechanical Hamiltonian:

where $\hat{\sigma}_{kj}=\left|\phi_{k}\right\rangle\left\langle\phi_{j}\right|$ denotes the transition operator. The analytical expressions for the matrix elements $x_{kj}=\left\langle\phi_{k}\right|\hat{x}\left|\phi_{j}\right\rangle$ are given in Appendix A. Note that in contrast to the harmonic model of optomechanics, the anharmonic potential introduces diagonal components to the $\hat{x}$ operator, represented by $\hat{x}^{(0)}$. The interaction Hamiltonian of the system takes the form [41, 15, 5]:

where $\hat{\mathbf{E}}(\mathbf{r})=\mathbf{E}_{0}(\mathbf{r})\hat{a}^{\dagger}+\text{h.c.}$, and we carried out the rotating wave approximation to remove the optical mode-squeezing terms ($\propto\hat{a}^{2},(\hat{a}^{\dagger})^{2}$).

We note that this framework explicitly assumes an off-resonant nature of the Raman scattering from a single molecule — see Refs. [8, 42, 38] for the models of the Surface-Enhanced Resonant Raman Scattering (SERRS), and Ref. [15] for the extension of the off-resonant molecular optomechanics towards multiple molecules.

Since the cavity is coherently driven, and only weakly coupled to the mechanical system, the optical mode fluctuates around a coherent state with amplitude $\left\langle\hat{a}\right\rangle\approx\alpha=\Omega/[-i(\omega_{a}-\omega_{l})-\kappa/2]$, where $\kappa$ is the optical decay rate. We can then linearize the interaction Hamiltonian by separating the coherent part of the optical mode from the fluctuations $\hat{a}=\delta\hat{a}+\left\langle\hat{a}\right\rangle\approx\delta\hat{a}+\alpha$ as:

where $G_{0}=\mathbf{E}_{0}(\mathbf{r}_{d})\mathbf{R}\mathbf{E}_{0}^{*}(\mathbf{r}_{d})$, with $\mathbf{r}_{d}$ denoting the position of the molecule.

From here, the optomechanical linearization neglects the terms $\propto\delta\hat{a}^{\dagger}\delta\hat{a}$ to write $\hat{H}_{om}\approx\hat{H}_{om}^{(0)}+\hat{H}_{om}^{(\pm)}$ with

where we assumed, without the loss of generality, that $\alpha$ can be made real. The diagonal contributions to the displacement operator $\hat{x}^{(0)}$ define

The second term in $H_{om}^{(0)}$ introduces a shift of the frequency of each vibrational level $\left|\phi_{i}\right\rangle$, mentioned also in Ref. [43], proportional $\alpha^{2}G_{0}x_{k,k}$. We thus dress the Morse potential eigenfrequencies given in Eq. (4) as

For the Morse potential we can find a good approximation for the diagonal matrix elements $x_{k,k}$ (see Appendix A). For reference, we note that the difference between the neighboring eigenfrequencies is

and can be approximated using the analytical expressions for the diagonal matrix elements $x_{k,k}$. In particular, as we show in Appendix A, the shift due to the diagonal term is nearly constant for all $k$.

To prepare nonclassical states of molecular vibrations in an incoherently driven anharmonic system, we need to suppress its the excitation beyond the two lowest-order states $\left\{\left|\phi_{0}\right\rangle,\left|\phi_{1}\right\rangle\right\}$ [30] — that is, form a phonon blockade. Similar schemes have been explored in other branches of physics, most famously in circuit QED, where the Kerr nonlinearity enables the generation of nonclassical states of superconducting circuits [46]. However, that functionality is enabled by the presence of the coherent microwave drive, which induces transitions between specific levels of the anharmonic ladder. In the molecular optomechanics, as well as the canonical cavity optomechanics, all transitions are due to incoherent processes, and so we turn to engineering the rates of these incoherent processes $\Gamma_{\pm}^{(k)}$ defined above, by structuring the optical spectrum of the system.

An example of a system with the desirable spectrum is depicted in Fig. 2(c), where we show a (not to scale) schematic of a hybrid metallic-dielectric cavity, explored in several recent studies [6, 14, 28, 29]. Here, a dimer of gold nanoparticles supports a lossy ($Q\sim 10$) plasmonic mode with dramatically reduced effective mode volumes $\sim 10^{-6}\lambda^{3}$ [23], and coupled to a high-Q dielectric microresonator in the form of toroidal [6], or nanobeam cavity [14]. The optical response $S_{\text{opt}}$ of this system, defined in the previous section, is shown in Fig. 2(a), and exhibits a strong Fano feature due to the off-resonant interaction between the high- and low-Q optical modes. Expressions used to model $S_{\text{opt}}$ in the presence, and absence of coupling between the modes (depicted with dashed line in Fig. 2(a)), and parameters used in this work, are listed in Appendix E.

In an idealized scheme, we realize the incoherent blockade by tuning the first Stokes frequency $\omega_{l}-(\tilde{\omega}_{1}-\tilde{\omega}_{0})$ to matche the peak of the Fano feature, so that state $\left|\phi_{1}\right\rangle$ can be efficiently populated from $\left|\phi_{0}\right\rangle$; additionally, if the second Stokes frequency $\omega_{l}-(\tilde{\omega}_{2}-\tilde{\omega}_{1})$ matches the dip in the optical spectrum, transitions to the $\left|\phi_{2}\right\rangle$ state become suppressed. This incoherent blockade should be therefore governed by the contrast between the optical spectra calculated at $S_{\text{opt}}(\omega_{l}-\tilde{\omega}_{1}+\tilde{\omega}_{0})$ and $S_{\text{opt}}(\omega_{l}-\tilde{\omega}_{2}+\tilde{\omega}_{1})$. A similar scheme, involving multiple high-$Q$ optical modes for controlling the state of a MHz mechanical oscillator with Kerr nonlinearity, was proposed by Rips et al. [35].

To characterize the population of the mechanical states, and its non-classical statistics, in a manner most resembling the usual second quantization language of populations and statistics of the harmonic system, in Appendix C we employ the position operator $\hat{x}$ as the key observable, used to define the steady-state mechanical populations and intensity correlations

We note that as the system becomes anharmonic, and we turn away from the second quantization framework, $n_{x}$ losses its exact definition as the phonon population. Nevertheless, we embrace this language for simplicity, and a direct mapping to the harmonic setup. Furthermore, if we measured the direct IR emission from the transitions between the mechanical states, these magnitudes would characterize the intensity, and statistics of the emitted IR light. Further details about the calculations of $n_{x}$ and $g^{(2)}_{x}(\tau)$ are listed in Appendix D.

In Fig. 2(d,e) we plot with solid lines the populations $n_{x}$ and intensity correlations $g^{(2)}_{x}(0)$, as a function of the laser frequency $\omega_{l}$. The former are visibly suppressed when laser is chosen to have its first Stokes frequency $\omega_{l}-(\tilde{\omega}_{1}-\tilde{\omega}_{0}$) match the dip in the optical cavity spectrum, around 498 GHz. Conversely, when the second Stokes transition at $\omega_{l}-(\tilde{\omega}_{2}+\tilde{\omega}_{1}$) matches the dip in $S_{\text{opt}}$ for the laser around 495 THz, we approach the blockade condition described above, the system demonstrates sub-Poissonian statistics $g^{(2)}_{x}(0)<1$. Away from these features, the system acquires the thermal statistics $g^{(2)}_{x}(0)\sim 2$. In Fig. 2(f,g) we show the same magnitudes, calculated as a function of nonlinearity $\delta\omega_{b}$, and coherent cavity population $|\alpha|^{2}$. Here again the statistics diverges from the thermal one towards sub-Poissonian (denoted as a blue region), when the second Stokes transition frequency is tuned to the dip of the Fano feature in the optical spectrum $S_{\text{opt}}$. Since that anti-Stokes frequency explicitly depends on $|\alpha|^{2}$ due to the dressing by the coherent field (see Eq. (12)), the region of sub-Poissonian statistics is largely diagonal. In Appendix F we discuss how this region of sub-Poissonian statistics changes with the laser frequency.

To gain analytical insight into these effects, we consider the analytical solution to the coupled equations for the populations $\rho_{k,k}$ of the three lowest-energy states (see derivation in Appendix D.1), approximate the numerator and denominator in the definition of $g^{(2)}_{x}(0)$ (Eq. (18)) by $2\rho_{2,2}$ and $\rho_{1,1}$, respectively, to find

We plot the function given in the first line in Fig. 2(e) with the dashed line, finding a qualitative agreement of the spectral range corresponding to the sub-Poissonian statistics with the full calculations (solid line). We have verified that the discrepancy is due to the truncation to the three states.

The second line represents a far more crude approximation, where we assume that the anti-Stokes transitions rates $\bar{\Gamma}_{-}^{(k)}$ are largely independent of $k$, and far larger than the second anti-Stokes rate $\bar{\Gamma}_{+}^{(1)}$. The first fraction in this expression directly characterizes the contrast in the anti-Stokes due to the Fano feature of the optical cavity.

As we discuss in more detail in Appendix F, we can further suppress the intensity correlations by increasing the relative role of the optomechanical feedback over the thermal pumping. This can be achieved by employing a larger intensity of the optical driving, although one needs to account for the $|\alpha|^{2}$ dependence of the dressed mechanical frequencies (Eq. (12)), to ensure that the transitions $\left|\phi_{0}\right\rangle\rightarrow\left|\phi_{1}\right\rangle$ and $\left|\phi_{1}\right\rangle\rightarrow\left|\phi_{2}\right\rangle$ match the peak, and the trough of the optical spectrum. One could also explore hybrid resonators featuring larger contrasts of the peak and troughs of the Fano resonance.

In numerical modelling, we assumed a single molecule exhibiting optomechanical coupling $g_{0}/2\pi=2$ THz, consistent with the values reported for picocavities in Ref. [23], and below the estimates for coupling with a small ensemble of about 100 molecules in nanocavities in Ref. [7]. The non-classical state of vibrations in Fig. 2(g) was reported for aharmonicities as small as $\delta\omega_{b}/2\pi=0.1$ THz, which yields the anharmonicity parameter $\xi=\delta\omega_{b}/\omega_{b}\approx 5\times 10^{-3}$ — arguably large, but reportedly accessible with molecular systems investigated in the context of SERS [43]. We also choose to plot the results against the population of the driven optical (plasmonic) mode, rather than the input laser intensity. The low populations explored here are typical for a strongly driven, lossy plasmonic cavities [23, 7], and should offer good approximation to the characteristics of hybrid systems under the assumption that the laser predominantly drives the plasmonic mode.

Finally, we note that since our scheme is based around harnessing the changes to the optical spectrum between the two Stokes transitions, it does require the mechanical system to exhibit a strong nonlinearity $\delta\omega_{b}$ to resolve the features of the optical spectrum. However, it does not require the mechanical system to operate in the conventional phonon blockade regime $\delta\omega_{b}>\gamma$.

Anharmonicity of molecular vibrations should also have a strong effect on the opposite regime of molecular optomechanics, where the system is strongly optically driven, in the effort to amplify the mechanical mode, and boost the intensity of Stokes emission [10, 12]. In this scenario, the amplification is likely to be suppressed until it saturates the ladder of bound states of the vibrations, or the non-resonant model of Raman scattering breaks down. We explore these effects in Section IV.1.

Moreover, beyond the amplification regime, canonical optomechanical systems exhibit a transition to mechanical lasing or dynamical instability, where the mechanical component exhibits coherent oscillations [39, 38]. In Section IV.2 we explore similar effects in the context of molecular optomechanics, analyzing the impact of anharmonicity on the threshold, amplitude, and the trajectory of these oscillations.

In this work, we show that by the framework of molecular optomechanics offers pathways towards genuine quantum engineering of molecular vibrations, far beyond the typical applications to mechanical amplification or Surface-Enhanced Raman Scattering.

In particular, by embracing the anharmonicity of the molecular vibrations, and engineering the optical spectrum of hybrid cavities, we show how mechanical systems can be driven into the weakly populated, nonclassical states. We present a complete theoretical framework for designing and describing such systems, and formal connection that can serve to characterize the nonclassical statistics of the emitted THz photons.

In the opposite regime of strong driving, we show that the current experimental setups should be capable of inducing coherent oscillations, or mechanical lasing, of the molecular vibrations. This effect should lead to an observable narrowing of the Raman scattering lines, and further the mapping between molecular, and canonical optomechanics. Furthermore, we show that even weak anharmonicities can dramatically change the optical response of the system, reshaping the dependence of the Stokes intensity on driving power.

These findings should significantly expand the toolbox and the impact of the formalism of molecular optomechanics, towards applications in quantum sensing and microscopy, and call for revisiting of the reported experimental results, to analyze the possible impact of the anharmonicities.