Fluctuation Theorems and Thermodynamic Inequalities for Nonequilibrium Processes Stopped at Stochastic Times

Stochastic thermodynamics extends classical thermodynamics to individual trajectories of non-equilibrium processes, encompassing stationary or transient systems with or without external driving forces [1, 2, 3, 4]. A first-law-like energy balance equality and various second-law-like thermodynamic inequalities can be derived from fluctuating trajectories. Fluctuation theorems emerging from stochastic thermodynamics, as equality versions of the second law, impose constraints on probability distributions of thermodynamic functionals along single stochastic trajectories [5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19].

Recently, a gambling demon, which stops the processes at random times, has been proposed for non-stationary stochastic processes without external driving force and feedback of control under an arbitrary deterministic protocol [20, 21]. The demon employs martingales, a concept that has been proposed in probability theory for more than $70$ years. The authors constructed a martingale for dissipative work, and obtained a stopping-time fluctuation theorem by applying the well-known optional stopping theorem (or Doob’s optional sampling theorem), which states that the average of a martingale at a stopping time is equal to the average of its initial value [22].

On the other hand, we already know that there are three faces in stochastic thermodynamics [23, 10, 24, 25], namely, (total) entropy production, housekeeping heat (non-adiabatic entropy production) and free energy dissipation (adiabatic entropy production). In a system with no external driving force, the housekeeping heat vanishes and the entropy production is equal to the free energy dissipation. However, in general non-stationary stochastic processes with an external driving force as well as an time-dependent protocol, we are curious about whether different martingales can be constructed for entropy production and free energy dissipation separately, while the martingale for housekeeping heat is straightforward to construct without any compensated term [26]. Both entropy production and free energy dissipation belong to a class of functionals along a single stochastic trajectory, i.e. general backward thermodynamic functionals, which has been rigorously defined in [27]. Housekeeping heat belongs to another class, called forward thermodynamic functionals [27].

Therefore, in this paper, we propose a systematic strategy for constructing martingales applicable to general backward thermodynamic functionals, with a focus on entropy production, free energy dissipation, and dissipative work as illustrative examples. Notably, the construction of martingales for forward thermodynamic functionals has been previously established in [27]. By leveraging our constructed martingales, we derive stopping-time fluctuation theorems that hold for general backward thermodynamic functionals, followed by second-law-like thermodynamic inequalities for arbitrary stopping times. When the stochastic stopping time reduces to a deterministic one, we exploit the additional degree of freedom present in our constructed martingales, enabling us to obtain a sharper nonnegative bound for the mean thermodynamic functional. In particular, we obtain a stronger inequality for the dissipative work than that obtained through classic Jarzynski equality.

Stopping-time fluctuation theorems and thermodynamic inequalities First, we will give an even more general definition of the backward thermodynamic functional than [27]. We consider a stochastic thermodynamic system with temperature $\beta=\frac{1}{k_{B}\mathbf{T}}$. We denote the state (discrete or continuous) of the system at time $s\geqslant 0$ by $X(s)$, whose stochastic dynamics is governed by a prescribed deterministic protocol $\Lambda=\{\lambda(s)\colon s\geqslant 0\}$. For a given duration $[0,t]$, the trajectories are traced by the coordinates in phase space, denoted by $x_{[0,t]}\equiv\{x(s)\}_{0\leqslant s\leqslant t}$. We further denote the probability of observing a given trajectory $x_{[0,t]}$ by $\mathcal{P}^{X}(x_{[0,t]})$, and the probability density of $X(s)$ by $\varrho^{X}(x,s)$ at any given time $s$. The general backward thermodynamic functional in the duration $[0,t]$ is defined by $\{X(s)\}_{0\leqslant s\leqslant t}$ and another stochastic process $\{Y(s)\}$ with protocol $\tilde{\Lambda}=\{\tilde{\lambda}(s)\colon s\geqslant 0\}$ (can be either the same as or different from $\Lambda$). The only condition is that the processes $\{X(s)\}_{0\leqslant s\leqslant t}$ and $\{Y(s)\}_{0\leqslant s\leqslant t}$ are absolutely continuous with each other, i.e. the probability $\mathcal{P}^{X}(x_{[0,t]})>0$ if and only if $\mathcal{P}^{Y}(x_{[0,t]})>0$ for any given trajectory $x_{[0,t]}$. We define a third process $\{Z^{t}(s)\}_{0\leqslant s\leqslant t}$ driven by the time-reversed protocol $\tilde{\Lambda}^{r,t}=\{\tilde{\lambda}(t-s)\colon 0\leqslant s\leqslant t\}$ of $\{Y(s)\}$ up to time $t$. The probability density of $Z^{t}(s)$ is denoted by $\varrho^{Z^{t}}(x,s)$ for any given time $s\leqslant t$. Note that there is also an additional degree of freedom, i.e. the arbitrary choice of the initial distribution $\varrho^{Z^{t}}(x,0)$ of $\{Z^{t}(s)\}_{0\leqslant s\leqslant t}$ for any $t$, because for different $t$, only the protocols inherited from $\{Y(s)\}$ are closely related to each other, not the initial distributions.

The probability of observing a given trajectory $x_{[0,t]}$ in $\{Z^{t}(s)\}_{0\leqslant s\leqslant t}$ is denoted by $\mathcal{P}^{Z^{t}}(x_{[0,t]})$. We define a general backward thermodynamic functional by

where $\tilde{x}_{[0,t]}\equiv\{x(t-s)\}_{0\leqslant s\leqslant t}$ denotes the time reversal of $x_{[0,t]}$ in the duration $[0,t]$.

It is straightforward to derive the fluctuation theorem for $F_{t}$:

However, $F_{t}$ is generally not a martingale [27].

For any given time interval $[0,T]$, we would like to add a compensated term $\delta_{t}$ as a function of $X(t)$ and $t$, to $F_{t}$, so that $e^{-\beta(F_{t}+\delta_{t})}$ be a martingale, i.e.

for any $t\in[0,T]$.

Then we propose

in which $\tilde{\varrho}^{Z^{T}}(X(t),T-t)$ is the distribution of $Z^{T}(t)$ with any arbitrary initial distribution $\tilde{\varrho}^{Z^{T}}(\cdot,0)$, which is not necessarily the same as $\varrho^{Z^{T}}(\cdot,0)$ and contributes another extra degree of freedom.

We apply the optional stopping theorem in martingale theory to derive the general stopping-time fluctuation theorem

where $\tau$ is a stopping time, defined by any stopping mechanism to decide whether to stop a process based on the current position and past events.

By Jensen’s inequality,

The left-hand-side is independent of $\tilde{\varrho}^{Z^{T}}$. Hence we can improve the above inequality into

A special situation is when $\tau=t$ with probability $1$, followed by

and

in which $\left\langle\ln\frac{\varrho^{X}(X(t),t)}{\varrho^{Z^{t}}(X(t),0)}\right\rangle$ is the relative entropy of $\varrho^{X}(\cdot,t)$ with respect to $\varrho^{Z^{t}}(\cdot,0)$. The inequality (5) is stronger than the traditional inequality $\left\langle F_{t}\right\rangle\geqslant 0$ derived from the well-known fluctuation theorem $\left\langle e^{-\beta F_{t}}\right\rangle=1$, as long as $\varrho^{X}(\cdot,t)$ is different from $\varrho^{Z^{t}}(\cdot,0)$.

As a corollary, we can derive certain bound for the infimum of $F_{t}+\delta_{t}$ following the strategy in [28], which holds for both equilibrium processes and general nonequilibrium processes. According to Doob’s maximal inequality, we find the following bound for the cumulative distribution of the supremum of $e^{-\beta(F_{t}+\delta_{t})}$,

for any $\lambda\geqslant 0$. It is equivalent to a lower bound on the cumulative distribution of the infimum of $\beta(F_{t}+\delta_{t})$ in the given duration $[0,T]$, i.e.

for $s\geqslant 0$. It implies the random variable $-\inf_{0\leqslant t\leqslant T}\{\beta(F_{t}+\delta_{t})\}$ dominates stochastically over an exponential random variable with the mean of $1$. Thus, we find the following universal bound for the mean infimum of $\beta(F_{t}+\delta_{t})$, i.e.

Applications The thermodynamic functional $F_{t}$ becomes the (total) entropy production $S_{\text{tot}}(t)$ up to time $t$ if the process $\{Y(t)\}$ is driven by exactly the same protocol as $\{X(t)\}$, and the initial distribution of $Z^{t}$ is taken to be the distribution of $X(t)$ [29, 8], i.e. $\varrho^{Z^{t}}(x,0)=\varrho^{X}(x,t)$. Then

and $e^{-\beta(S_{\text{tot}}(t)+\delta^{S_{\text{tot}}}_{t})}$ is a martingale. It is followed by

for any stopping time $\tau$, and $\left\langle S_{\text{tot}}(\tau)\right\rangle\geqslant-\left\langle\delta^{S_{\text{tot}}}_{\tau}\right\rangle$.

The thermodynamic functional $F_{t}$ becomes the free energy dissipation (adiabatic entropy production) $f_{d}(t)$ if the process $\{Y(t)\}$ is taken to be the adjoint process of $\{X(t)\}$, and also the initial distribution of $Z^{t}$ is set as the distribution of $X(t)$, i.e. $\varrho^{Z^{t}}(x,0)=\varrho^{X}(x,t)$ [10, 24, 25, 23]. Then

and $e^{-\beta(f_{d}(t)+\delta^{f_{d}}_{t})}$ is a martingale. It is followed by

for any stopping time $\tau$, and $\left\langle f_{d}(\tau)\right\rangle\geqslant-\left\langle\delta^{f_{d}}_{\tau}\right\rangle$.

Let $\pi^{X}(t)$ be the pseudo-stationary distribution of $X(t)$ corresponding to the protocol $\lambda(t)$, i.e. the stationary distribution of $\{X(t)\}$ if the protocol is fixed at $\lambda(t)$. The thermodynamic functional $F_{t}$ becomes the dissipative work $W_{d}(t)$ up to time $t$, if the initial distribution of $X(t)$ is $\pi^{X}(0)$, the process $\{Y(t)\}$ is taken to be the adjoint process of $\{X(t)\}$, and the initial distribution of $Z^{t}$ is taken as the pseudo-stationary distribution of $X(t)$, i.e. $\varrho^{Z^{t}}(x,0)=\pi^{X}(x,t)$ [30, 20]. Then

and $e^{-\beta(W_{d}(t)+\delta^{W_{d}}_{t})}$ is a martingale. It is followed by

for any stopping time $\tau$, and $\left\langle W_{d}(\tau)\right\rangle\geqslant-\left\langle\delta^{W_{d}}_{\tau}\right\rangle$.

For the mean $W_{d}$ up to any fixed time $t$, we can obtain a stronger inequality than $\left\langle W_{d}\right\rangle\geqslant 0$. Applying (5), we have

Actually, this inequality can be derived from the equality $\frac{dH(t)}{dt}=-f_{d}(t)+W_{d}(t)$ with the inequality $f_{d}(t)\geqslant 0$ from [23], in which $H(t)$ is exactly $\left\langle\ln\frac{\varrho^{X}(X(t),t)}{\pi^{X}(X(t),t)}\right\rangle$. In [20], the thermodynamic functional under investigation is $W_{d}$ but the $\delta_{t}$ they defined is the same as $\delta^{S_{\text{tot}}}_{t}$. The mathematical derivation here implies that we should use different $\delta_{t}$ for different thermodynamic functionals.

Numerical verifications Many mesoscopic biochemical processes such as the kinetics of enzyme or motor molecules, can be modeled in terms of transition rates between discrete states. We apply our theory to a simple stochastic process with only three states. The time-dependent transition rates between different discrete states are set as follows

in which the chemical driven energy

For the three thermodynamic functionals $S_{\text{tot}}$, $f_{d}$ and $W_{d}$, the stopping strategy for $\tau$ is set as follows: the process is stopped at $\tau<T$ only when the functional reaches a given threshold value before $T$; while the process is stopped at the final time $\tau=T$ if the threshold value is never reached during the duration $[0,T]$.

Fig. 1(a-c) shows the numerical results of $\left\langle S_{\text{tot}}(\tau)\right\rangle$ versus $-\left\langle\delta^{S_{\text{tot}}}_{\tau}\right\rangle$, $\left\langle f_{d}(\tau)\right\rangle$ versus $-\left\langle\delta^{f_{d}}_{\tau}\right\rangle$, and $\left\langle W_{d}(\tau)\right\rangle$ versus $-\left\langle\delta^{W_{d}}_{\tau}\right\rangle$, as functions of the threshold value. The initial distribution is set to be uniform among the three states in Fig. 1(a-b), and concentrated on the second state in Fig. 1(c). Fig. 1(d-f) test the stopping-time fluctuation relations (7), (9) and (11), with and without the compensated term $\delta_{t}$.

In the special situation that $\tau=t$ with probability $1$, Fig. 1(g) shows that the inequality (12) for $\left\langle W_{d}(t)\right\rangle$ is not only stronger than the inequality $\left\langle W_{d}(t)\right\rangle\geqslant-\langle\delta^{W_{d}}_{t}\rangle$, but also the traditional Jarzynski inequality $\left\langle W_{d}(t)\right\rangle\geqslant 0$.

Another example is the stochastic dynamics of a colloidal particle with diffusion coefficient $D$ in a time-dependent potential $V(t)$. The dynamics obeys the Langevin equation

where $\xi$ is a Gaussian white noise with zero mean and autocorrelation $\left\langle\xi(t)\xi(t^{\prime})\right\rangle=2D\delta(t-t^{\prime})$.

In such a stochastic system, the housekeeping heat equals to zero and thus the entropy production $S_{\text{tot}}(t)$ coincides with the free energy dissipation $f_{d}(t)$. We follow the same stopping strategy as in the discrete model of Fig. 1, and show the numerical results of $\left\langle S_{\text{tot}}(\tau)\right\rangle$ versus $-\left\langle\delta^{S_{\text{tot}}}_{\tau}\right\rangle$ in Fig. 2(a) and $\left\langle W_{d}(\tau)\right\rangle$ versus $-\left\langle\delta^{W_{d}}_{\tau}\right\rangle$ in Fig. 2(b) with $T=3$. Fig. 2(c-d) test the stopping-time fluctuation relations (7) and (11), with and without the compensated term $\delta_{t}$.

In the special situation that $\tau=t$ with probability $1$, Fig. 2(e) shows that the conclusion (12) for $\left\langle W_{d}(t)\right\rangle$ is stronger than the inequality $\left\langle W_{d}(t)\right\rangle\geqslant-\langle\delta^{W_{d}}_{t}\rangle$ and the Jarzynski inequality $\left\langle W_{d}(t)\right\rangle\geqslant 0$.

In Fig. 1 and 2, the averaged thermodynamic functionals may be negative under certain stopping strategy, but the general stopping-time fluctuation relations and related thermodynamic inequalities always hold.

Derivation First, we notice that

is a martingale, where $\widetilde{X}_{[0,t]}\equiv\{X(t-s)\}_{0\leqslant s\leqslant t}$ denotes the time reversal of $X_{[0,t]}$ in the duration $[0,t]$, and $\mathcal{P}^{\tilde{Z}^{T,t}}(x_{[0,t]})$ denotes the probability of observing a given trajectory $x_{[0,t]}$ in $\{\tilde{Z}^{T,t}(s)=Z^{T}(s+T-t)\}_{0\leqslant s\leqslant t}$. The distribution of $Z^{T}(0)$ is $\tilde{\varrho}^{Z^{T}}(\cdot,0)$.

Since

we have

For $0\leqslant u\leqslant s\leqslant T$, let $\widetilde{X}_{[0,T]}(u,s)$ be the part of the trajectory $\widetilde{X}_{[0,T]}$ in the duration $[u,s]$, then $X_{[t,T]}$ and $\widetilde{X}_{[0,T]}(0,T-t)$ are exactly the time reversal of each other. Thus

in which the last equality comes from the definition of $\mathcal{P}^{\tilde{Z}^{T,t}}(\widetilde{X}_{[0,t]})$. So

which is exactly the definition of martingale for (13).

Second, we show that $\{e^{-\beta(F_{t}+\delta_{t})}\}_{0\leqslant t\leqslant T}$ is exactly the martingale (13). By the definition of $F_{t}$, we have

Since

and notice that $\{\tilde{Z}^{T,t}(s)\}_{0\leqslant s\leqslant t}$ and $\{Z^{t}(s)\}_{0\leqslant s\leqslant t}$ are driven by the same protocol $\{\tilde{\lambda}(t-s)\colon 0\leqslant s\leqslant t\}$, we have

which implies

Combining (14) and (15) shows that $\{e^{-\beta(F_{t}+\delta_{t})}\}_{0\leqslant t\leqslant T}$ is exactly the martingale (13), then the general stopping-time fluctuation theorem (2) follows from the optional stopping theorem.

When $\tau=t$ with probability $1$, we decompose

By Jensen’s inequality, we know

Furthermore, for any given $t$, we can choose $\tilde{\varrho}^{Z^{T}}(\cdot,0)$ such that $\tilde{\varrho}^{Z^{T}}(x,T-t)=\varrho^{X}(x,t)$, which leads to

Conclusion In summary, our study contributes a general framework for understanding martingales constructed upon thermodynamic functionals. We have successfully derived and proven the stopping-time fluctuation theorems, accompanied by second-law-like inequalities for mean thermodynamic functionals stopped at stochastic times. Our results generalize the recent gambling strategy and stopping-time fluctuation theorems [20] to a very general setting. Our framework encompasses the general definition of thermodynamic functionals, accommodates various types of stochastic dynamics, and allows for arbitrary stopping strategies. The validity and applicability of our framework are supported by numerical verifications conducted in stochastic dynamics with both discrete and continuous states.

Furthermore, we highlight the significance of the additional degree of freedom introduced through the compensated term $\delta_{t}$, which leads to a strengthening of the inequality for dissipative work compared to the well-known Jarzynski inequality when the stopping time is reduced to a deterministic one. Overall, our results provide novel insights, new interpretations, and improved bounds for the fundamental principles underlying the Second Law of Thermodynamics in the context of stochastic processes.

H. Ge is supported by NSFC 11971037 and T2225001.