Periodically-driven quantum thermal machines from warming up to limit cycle

Introduction.– A tantalizing goal of quantum thermodynamics is to investigate, analyze and ultimately design thermal machines on the quantum scale Scovil and Schulz-DuBois (1959); Geva and Kosloff (1992); Gemmer et al. (2009); Seifert (2012); Pekola (2015); Kosloff (2013); Goold et al. (2016); Vinjanampathy and Anders (2016); Benenti et al. (2017); Binder et al. (2018); Seifert (2019). Among the various routes towards this vision, devising periodically-driven quantum thermal machines (PD-QTMs) in which work can be extracted through externally driving a quantum working substance represents a paradigmatic one. This field has attracted much attention in recent years with intriguing theoretical proposals Brandner et al. (2015); Brandner and Seifert (2016); Proesmans et al. (2016); Newman et al. (2017); Alecce et al. (2015); Perarnau-Llobet et al. (2018); Mohammady and Romito (2019); Kloc et al. (2019); Pezzutto et al. (2019); Mohammady and Romito (2019); Park et al. (2019); Solfanelli et al. (2020); akmak and Altintas (2020); Wiedmann et al. (2020); Shirai et al. (2021); Strasberg et al. (2021) together with delicate experimental realizations in various platforms Abah et al. (2012); Roßnagel et al. (2014); Dechant et al. (2015); Krishnamurthy et al. (2016); Ro nagel et al. (2016); Peterson et al. (2019); Klatzow et al. (2019); de Assis et al. (2019); Pekola and Khaymovich (2019); von Lindenfels et al. (2019).

Existing theories on PD-QTMs are largely carried out in the limit-cycle stage of operation, where the working substance should recoil to its original state after a driving cycle (see, e.g., Refs.Feldmann and Kosloff (2004); Brandner et al. (2015); Brandner and Seifert (2016); Insinga et al. (2016); Proesmans et al. (2016); Insinga et al. (2018); Mohammady and Romito (2019); Shirai et al. (2021); Menczel and Brandner (2019)). Although the limit-cycle regime enables theoretical simplicity, it faces potential challenges. On the one hand, recent studies have pointed out that the state of the working substance needs not inherit the periodicity of external drivings at finite times Menczel and Brandner (2019); Pezzutto et al. (2019); Wiedmann et al. (2020), namely, there may exist a transient warming-up phase for the operation of PD-QTMs before a limit cycle sets in. On the other hand, the limit-cycle stage is not immediately accessible experimentally, which may lead to discrepancy between theoretical predictions and experimental measurements. Addressing these challenges requires a unified thermodynamic description of both the warming-up and limit-cycle phases of PD-QTMs, which is, however, still missing.

In this work, we present a thermodynamic framework that deals with cycle-number dependent thermodynamic quantities. As such, it allows us to characterize in a unified manner the performance of PD-QTMs in both the warming-up and limit-cycle phases. Our theory relies on the observation that periodicity can break down at the ensemble average level, namely, $\langle\mathcal{O}(t)\rangle\neq\langle\mathcal{O}(t+\mathcal{T})\rangle$ with $\mathcal{O}$ being a periodic observable $\mathcal{O}(t)=\mathcal{O}(t+\mathcal{T})$ and $\mathcal{T}$ the driving period. This breakdown is trivial in the transient warming-up phase where even the state of the working substance does not behave in a periodic manner. However, nontrivially, this breakdown can show up in the limit-cycle phase if $\mathcal{O}$ involves bath degrees of freedom, a scenario which is frequently encountered in studies of quantum thermodynamics (see recent discussions Talkner and Hänggi (2020); Seshadri and Galperin (2021) and reference therein).

Building upon this observation, we find that the thermodynamic characterization of PD-QTMs generally acquires a new contribution in both the warming-up and limit-cycle phases, besides the conventional notions of work and heat. Particularly, through completing the first law of thermodynamics [cf. Eq. (4)] and ensuring its validity across the whole parameter space at all times, we identify this extra term (denoted as $\mathcal{A}$ hereafter) as the change of an ensemble average $\langle H_{S}(t)+H_{I}(t)\rangle$ within a cycle [cf. Eq. (5)]; $H_{S(I)}$ denotes the Hamiltonian for the working substance (system-bath coupling). Intriguingly, this $\mathcal{A}$ term vanishes only in the limit-cycle phase provided that the system-bath coupling is rather weak. We further uncover that contributions from the $\mathcal{A}$ term play a nontrivial role in shaping behavior of the thermodynamic efficiency [cf. Eq. (6)] in both the warming-up and limit-cycle phases, especially at strong coupling, compared with existing characterizations, which largely neglect it.

We illustrate our general framework using the quantum Otto heat engine (see Fig. 1 for a sketch), with a driven resonant electronic level as the working substance.

Based on simulations, we confirm our theoretical predictions of the behaviors of thermodynamic quantities incorporating the $\mathcal{A}$ term, including the first law of thermodynamics and the efficiency (see Fig. 2). In comparison, conventional definitions without $\mathcal{A}$ fail to capture the correct thermodynamics, especially at strong coupling where the $\mathcal{A}$ term is substantial in both warm-up and limit-cycle regimes. The framework presented here allows the description of PD-QTMs even before the limit cycle stage, and at strong coupling, both aspects crucial for describing experimental systems and advancing the field of quantum thermodynamics.

Quantum thermodynamics of generic PD-QTMs.– Generally, PD-QTMs can be modeled as ($\hbar=1$, $k_{B}=1$ hereafter)

Here, $H_{S}(t)$ denotes a semiclassically-driven working substance with the understanding that the working substance exchanges work with an external agent which we do not explicitly include within the quantum description. $H_{B}=\sum_{v=h,c}H_{B}^{v}$ consists of a hot (h) and a cold (c) thermal bath and $H_{I}(t)=\sum_{v}H_{I}^{v}(t)\equiv\sum_{v}\lambda_{v}(t)H_{I}^{v}$ denotes a time-dependent system-bath coupling. Here, $\lambda_{v}(t)$ is a time-dependent coefficient describing the contact switching of the $v$ reservoir, used to implement thermodynamic strokes (see, e.g., Refs. Wiedmann et al. (2020); Pancotti et al. (2020); Shirai et al. (2021)). We assume periodic protocols $H_{S,I}(t)=H_{S,I}(t+\mathcal{T})$ such that $H(t)=H(t+\mathcal{T})$ with $\mathcal{T}$ the period of the driving.

To analyze the quantum thermodynamics of the PD-QTM, we introduce the definitions of heat and work based on the complete Hamiltonian dynamics, $H(t)$ Bruch et al. (2016); Oz et al. (2020); Wiedmann et al. (2020); Cangemi et al. (2021). This approach naturally avoids the problem of the partitioning of the system-bath interaction. Denoting by $\rho_{\mathrm{tot}}$ the total density matrix of the composite system, the integrated injected work $\mathcal{W}(m)$ and absorbed heat $\mathcal{Q}_{v}(m)$ from the $v$ bath during the $m$-th cycle $[m\mathcal{T},(m+1)\mathcal{T}]$ with a non-negative integer $m\geq 0$ are uniquely defined as Bruch et al. (2016); Oz et al. (2020); Wiedmann et al. (2020); Cangemi et al. (2021)

Here, $\dot{\mathcal{X}}(t)\equiv d\mathcal{X}(t)/dt$ for an arbitrary operator $\mathcal{X}(t)$, the trace is performed over system and bath degrees of freedom. The introduction of the cycle-number ($m$) dependent thermodynamic quantities allows us to characterize the performance of PD-QTMs in both the warming-up and limit-cycle stages. Note that the expression for absorbed heat $\mathcal{Q}_{v}$ is consistent with the definition employed in quantum transport scenarios Esposito et al. (2015a, b); Bergmann and Galperin (2021). For the work definition, it naturally incorporates work costs for implementing a time-dependent system-bath coupling Newman et al. (2017). In our convention, a heat engine corresponds to $\mathcal{W}<0$, $\mathcal{Q}_{h}>0$ and $\mathcal{Q}_{c}<0$.

Since the integrand in Eq. (2) is just $d\langle H(t)\rangle/dt$ with $\langle H(t)\rangle\equiv\mathrm{Tr}[H(t)\rho_{\mathrm{tot}}(t)]$ the internal energy of the composite system, we can express Eq. (2) as $\mathcal{W}(m)=\mathrm{Tr}\big{\{}H(t)[\rho_{\mathrm{tot}}((m+1)\mathcal{T})-\rho_{\mathrm{tot}}(m\mathcal{T})]\big{\}}$. This immediately leads to an intriguing observation: $\rho_{\mathrm{tot}}(m\mathcal{T})\neq\rho_{\mathrm{tot}}((m+1)\mathcal{T})$, generally valid since $\mathcal{W}(m)\neq 0$ for a PD-QTM. Note that this difference does not prevent the existence of a limit-cycle phase for the reduced system density matrix $\rho_{S}$, $\rho_{S}(m\mathcal{T})=\rho_{S}((m+1)\mathcal{T})$ Feldmann and Kosloff (2004); Brandner et al. (2015); Brandner and Seifert (2016); Insinga et al. (2016); Proesmans et al. (2016); Insinga et al. (2018); Mohammady and Romito (2019); Shirai et al. (2021); Menczel and Brandner (2019), since the bath state needs not be a periodic function at all. This simple observation has a profound consequence: An operator $\mathcal{O}$ involving bath degrees of freedom generally shows $\langle\mathcal{O}(m\mathcal{T})\rangle\neq\langle\mathcal{O}((m+1)\mathcal{T})\rangle$, even when $\mathcal{O}(m\mathcal{T})=\mathcal{O}((m+1)\mathcal{T})$. Namely, the periodicity can break down at the ensemble average level.

We now turn to the first law of thermodynamics for PD-QTMs. In terms of our definitions for heat and work, it takes the following exact form ¹¹1One can obtain the present first law by combining the expression $\mathcal{W}(m)=\int_{m\mathcal{T}}^{(m+1)\mathcal{T}}dt\frac{d\langle H(t)\rangle}{dt}$ with the Hamiltonian decomposition of Eq. (1).,

Crucially, besides the conventional notions of work and heat, an extra quantity appears, $\mathcal{A}(m)$,

Here, we denote by $H_{SI}(m\mathcal{T})\equiv H_{S}(m\mathcal{T})+H_{I}(m\mathcal{T})$. In arriving at the first law Eq. (4), we emphasize that no approximations were invoked. The $\mathcal{A}$ term reflects the change to ${\rm Tr}\left\{H_{SI}\,\rho_{{\rm tot}}\right\}$ after a cycle, highlighting the nontrivial role of the system-bath coupling in quantum engines Shirai et al. (2021); Talkner and Hänggi (2020). In the warming-up phase where even the system density matrix $\rho_{S}$ does not inherit the periodicity of the driving protocol, one naturally expects a nonzero $\mathcal{A}(m)$. In the limit-cycle phase, $\mathcal{A}(m)$ vanishes only when the system-bath coupling is negligible compared to the system energy scale since then $\mathcal{A}(m)\simeq\mathrm{Tr}_{S}\Big{\{}H_{S}(m\mathcal{T})[\rho_{S}((m+1)\mathcal{T})-\rho_{S}(m\mathcal{T})]\Big{\}}=0$. Otherwise, at strong coupling, we still have $\mathcal{A}(m)\neq 0$ as $\langle H_{I}(m\mathcal{T})\rangle\neq\langle H_{I}((m+1)\mathcal{T})\rangle$ [see discussion before Eq. (4)]. This result contradicts previous studies (see, e.g., Ref. Wiedmann et al. (2020)), which neglected the contribution from $\mathcal{A}(m)$ regardless of system-bath coupling strength. This first law and the nontrivial behavior of $\mathcal{A}(m)$ represent our first main result. As for the second law of thermodynamics, we still observe the conventional Clausius inequality Esposito et al. (2010); Brandner and Seifert (2016); Seshadri and Galperin (2021), $\mathcal{S}(m)\equiv[\mathcal{S}_{S}((m+1)\mathcal{T})-\mathcal{S}_{S}(m\mathcal{T})]-\sum_{v}\beta_{v}\mathcal{Q}_{v}(m)\geq 0$ with $\mathcal{S}_{S}(m)=-\mathrm{Tr}[\rho_{S}(m\mathcal{T})\ln\rho_{S}(m\mathcal{T})]$ the von Neumann entropy of the working substance.

Using Eq. (4), the averaged thermodynamic efficiency over a cycle, $\eta(m)\equiv-\frac{\mathcal{W}(m)}{\mathcal{Q}_{h}(m)}$, reads

Here, $\eta_{0}=1+\mathcal{Q}_{c}/\mathcal{Q}_{h}$ is the common definition of efficiency Alecce et al. (2015); Mohammady and Romito (2019); Park et al. (2019); Solfanelli et al. (2020); akmak and Altintas (2020), which has been applied to strong coupling scenarios Perarnau-Llobet et al. (2018); Wiedmann et al. (2020). Intriguingly, $\eta(m)$ gains an additional contribution $\mathcal{A}(m)/\mathcal{Q}_{h}(m)$, compared to $\eta_{0}(m)$. we point out that Eq. (6) is completely general as no assumptions were invoked in its derivation: It can be applied to arbitrary PD-QTMs in both the warming-up and limit-cycle phases, regardless of the system-bath coupling strength. On the contrary, $\eta_{0}(m)$ can only be applied to the weak-coupling scenario in the limit-cycle phase where $\mathcal{A}(m)$ vanishes [see discussion below Eq. (5)]. This working definition for the efficiency is our second main result.

Example: Driven resonant level engine.– To substantiate and exemplify our main results [cf. Eqs. (4), (5) and (6)] we present a numerical example of a quantum Otto heat engine (see Fig. 1) that extracts work from heat baths with a resonant level model Ludovico et al. (2014); Esposito et al. (2015a, b); Ochoa et al. (2016); Bruch et al. (2016); Thingna et al. (2017); Dou et al. (2018); Oz et al. (2019); Dou et al. (2020); Oz et al. (2020),

Here, the working substance consists of a single spinless electronic level with the annihilation operator $c_{d}$ and time-dependent energy $\epsilon(t)$. The dot is alternating its coupling to two fermionic baths (leads) $v=h,c$ of different temperatures. $c_{kv}$ annihilates an electron with energy $\epsilon_{kv}$ in the $v$-lead that couples to the central level with $t_{kv}$ as the tunneling rate. The $v$-lead is occupied according to the Fermi-Dirac distribution with an inverse temperature $\beta_{v}$ and a chemical potential $\mu_{v}$; for our purpose, we set $\beta_{h}\neq\beta_{c}$ and $\mu_{h,c}=0$. The coupling of the central level to the leads is given by the spectral densities $\Gamma_{v}(\epsilon)=2\pi\sum_{k}t_{kv}^{2}\delta(\epsilon-\epsilon_{kv})$. In what follows, we consider the wideband limit such that $\Gamma_{v}(\epsilon)=\Gamma_{v}$ and adopt symmetric couplings, $\Gamma_{h,c}=\Gamma$. The time-dependent coefficients $\lambda_{v}(t)$ equal $1$ when the coupling to the $v$-lead is turned on and $0$ otherwise, thus enabling thermalization strokes in the Otto cycle.

The complete Otto cycle with period $\mathcal{T}$ consists of four strokes; see Fig. 1 for an illustration: (1) Stroke 1 ($0\leq t<t_{1}$): In this isochoric thermalization step, the central level with a fixed energy $\epsilon_{1}$ is coupled to the hot lead. (2) Stroke 2 ($t_{1}\leq t<t_{1}+t_{2}$): The energy of the isolated central level is shifted from $\epsilon_{1}$ to $\epsilon_{2}$. (3) Stroke 3 ($t_{1}+t_{2}\leq t<t_{1}+t_{2}+t_{3}$): In this isochoric thermalization step, the central level with a fixed energy $\epsilon_{2}$ is coupled to the cold lead. (4) Stroke 4 ($t_{1}+t_{2}+t_{3}\leq t<\mathcal{T}$): The energy of the isolated central level is tuned back from $\epsilon_{2}$ to $\epsilon_{1}$. We remark that the adopted cycle protocol does not enforce a periodicity on the system density matrix a priori as we just tune the energy of the central level. The engine needs a transient warming-up process before settling into a limit-cycle operation mode. Moreover, we do not enforce an adiabatic work-extracting process, namely, strokes 2 and 4 can span finite times. In fact, the amount of work generated during strokes 2 and 4 is independent of the detailed functional form of $\epsilon(t)$ connecting fixed end points $\epsilon_{1,2}$ as the charge occupation of the central level during these two strokes remains fixed; in Ref. SM , we exploit this fact together with an equilibrium charge occupation expression Liu and Segal (2020) (note that we attach the central level to two leads alternatively in the Otto cycle) to estimate the heat-engine operation regime and determine relevant values of $\epsilon_{1,2}$ to be used in simulations.

The cycle number-dependent thermodynamics is simulated by employing the driven Liouville von-Neumann equation method Ajisaka et al. (2012); Zelovich et al. (2014); Chen et al. (2014); Zelovich et al. (2017); Chiang and Hsu (2020) whose performance for addressing quantum thermodynamics up to strong coupling has been carefully assessed Oz et al. (2019, 2020); we relegate simulation details, including numerical implementation, initial condition at $t=0$ and adopted parameter values, to SM . A typical set of thermodynamic results is depicted in Fig. 2. Quantities at $m=0$ are evaluated for the first cycle $[0,\mathcal{T}]$.

The behavior of $\mathcal{A}(m)$ along with the work $\mathcal{W}(m)$ is presented in Fig. 2 (a); results for the heat exchange can be found in SM . We find that as the coupling strength increases, the magnitude of $\mathcal{A}(m)$ becomes more pronounced. From this it is evident that based on our heat and work definitions, the complete first law of thermodynamics at all times should read $\mathcal{W}+\sum_{v}\mathcal{Q}_{v}-\mathcal{A}=0$ as depicted in Fig. 2 (b). In contrast, the sum $\mathcal{W}+\sum_{v}\mathcal{Q}_{v}$ considered in other studies (see, e.g., Refs. Mohammady and Romito (2019); Wiedmann et al. (2020)) approaches zero only in the weak-coupling scenario, and in the limit-cycle stage (see red line for $\Gamma=0.02$ in Fig. 2 (b) at $m=5$ where the engine is reaching the limit-cycle phase; this is confirmed in SM with a long-time simulation for $\Gamma=0.02$). Importantly, beyond the weak coupling regime, the faulty first law that neglects the $\mathcal{A}$ term deviates from zero at all times. Perhaps most surprisingly, Fig. 2 (c) reveals that if one adopts the incomplete definition of efficiency, i.e. Eq. (6) without $\mathcal{A}$, at strong coupling one would predict no heat engine behavior in this regime (see inset of Fig. 2 (c)). However, by properly including the $\mathcal{A}$ contribution, thus accounting for the effect of system-bath coupling, we find that the system does operate as a heat engine, consistent with the negative injected work value reported in the inset of Fig. 2 (a).

Besides the aforementioned basic features that reinforce the validity of our general theory, there are a few points worth discussing further: (i) Weak coupling scenarios need more transient cycles to warm up than the strong coupling counterparts before a limit-cycle phase sets in. Comparing the behaviors of $\mathcal{A}(m)$ and $\eta(m)$ as a function of $m$, we argue that the former can assess the number of transient cycles based on its magnitude variation with increasing $m$. For instance, from Fig. 2 (a), we observe that an engine of $\Gamma=0.05$ requires at least 4 transient cycles; note $m+1$ is the number of completed cycles. In contrast, an engine with $\Gamma=0.5$ needs just one cycle to warm up. We have verified that the number of transient cycles is independent of the initial charge occupation of the central level. (ii) The thermodynamic efficiency $\eta(m)$ does not reach a stationary value at strong couplings with increasing $m$ (see curve for $\Gamma=0.5$ in Fig. 2 (c)), noting that the heat $\mathcal{Q}_{h,c}(m)$ depends on the state of the bath, which needs not be a periodic function; see SM for the behavior of $\mathcal{Q}_{h,c}(m)$ while varying $m$ and the coupling strength. This finding implies that a characterization based on a single cycle in the limit-cycle phase may not be applicable at strong couplings. (iii) In our simulations, $\eta(m)$ increases as the coupling strength increases. This seems at odds with some studies Perarnau-Llobet et al. (2018); Newman et al. (2020), which have observed the opposite trend. However, we point out that these studies used the incomplete definition $\eta_{0}$. Indeed, the inset of Fig. 2 (c) shows that $\eta_{0}$ decreases as $\Gamma$ increases, until it becomes invalid for very large coupling strength. This suggests that interactions with the baths are in fact helpful in the heat-work conversion process Shirai et al. (2021). The dramatic qualitative difference between the conventional definition $\eta_{0}$ and the correct expression $\eta$ demonstrates why it is crucial to adopt the complete expressions, including the $\mathcal{A}$ term.

Discussion.– One may argue that the $\mathcal{A}$ term [cf. Eq. (5)] can be absorbed by redefining heat and work (see discussions in Refs. Bruch et al. (2016); Ochoa et al. (2016); Dou et al. (2018)). We would like to point out that (i) the work definition we adopted [cf. Eq. (2)] already comprises the internal energy change of the total composite system, and (ii) introducing an effective bath Hamiltonian with the replacement $H_{B}^{v}\to H_{B}^{v}+H_{I}^{v}/2$ Bruch et al. (2016); Ochoa et al. (2016); Dou et al. (2018) and accordingly modifying the heat definition [cf. Eq. (3)], one is still not able to eliminate the $\mathcal{A}$ term, by comparing the definitions Eq. (3) and (5). After all, as we argued, the $\mathcal{A}$ term originates from the fact that periodicity breaks down at the ensemble average level even in the limit-cycle phase.

Recognizing the generic presence of $\mathcal{A}$ , it is tempting to explore whether the inclusion of work costs for the attachment and detachment of the working substance to and from the baths can affect the behavior of $\mathcal{A}$. While here we switched between strokes by tuning $\lambda_{v}(t)$ [see Eq. (Periodically-driven quantum thermal machines from warming up to limit cycle)] using a sharp step function alternating between $0$ and $1$, one could adopt a smooth function. We leave this point to future investigations.

Although our thermodynamic framework is applicable to general PD-QTMs, a direct measurement of $\mathcal{A}$ is experimentally challenging since it requires measuring energy change associated with the system-bath coupling. However, the utility of our framework is that it provides predictions for efficiency as a function of the cycle number, enabling a direct comparison with experiments. Exploring feasible measurement schemes for quantifying the role of $\mathcal{A}$, in light of recent experimental advances in single-electron and superconducting circuits Pekola (2015); Pekola and Khaymovich (2019), is an interesting future direction.

In summary, we have presented a unified, consistent thermodynamic theory, with cycle-number dependent thermodynamic quantities, which describes the operation of PD-QTMs both before and during the limit-cycle phase, covering strongly-coupled thermal machines. The unearthed $\mathcal{A}$ term (i) provides a complete description of the first law of thermodynamics and (ii) substantially modifies the efficiency of the engine, particularly at strong coupling, compared with existing characterizations. With its ability to describe PT-QTMs from the warming up phase to their limit cycle and from weak to strong coupling, we expect this framework to promote the experimental development of quantum thermal machines.

Acknowledgement.– J.L. and D.S. acknowledge support from the Natural Sciences and Engineering Research Council (NSERC) of Canada Discovery Grant and the Canada Research Chairs Program.

Supplemental material: Periodically-driven quantum thermal machines from warming up to limit cycle

In this supplemental material, we first identify the parameter regime where the driven resonant level model operates as a quantum Otto heat engine. Next, we provide simulation details regarding the numerical implementation of the driven Liouville von-Neumann (DLvN) approach. We also present additional simulation results that complement those included in the main text.