Parameter inference and nonequilibrium identification for Markovian systems
based on coarse-grained observations

Introduction — Mesoscopic molecular systems are widely modeled as a Markov process with a large number of microstates Cornish-Bowden (2013); Sakmann (2013); Golding et al. (2005). In experiments, it often occurs that only a set of coarse-grained clusters can be detected, while the internal microstates of the system are often inaccessible or indistinguishable. For instance, using live-cell imaging, one can obtain the time trace of the copy number of a protein in a single cell; however, it is difficult to determine whether the gene is in an active or an inactive state Golding et al. (2005). Due to the inability to accurately identify all microstates, the data obtained is usually the time trace of some coarse-grained states, which only retain a small degrees of freedom of the system Seifert (2019); Esposito (2012); Teza and Stella (2020). Other well-known examples of partial observations include ion channel opening Sakmann (2013), molecular docking and undocking to a sensor Skoge et al. (2013), and flagellar motor switches Berg (2003).

Given a sufficiently long trajectory of coarse-grained states, two natural and crucial questions arise: (i) is it possible to determine the transition topology and even all transition rates between all microstates? (ii) If complete recovery of transition rates is impossible, is it still possible to determine whether the system is in an equilibrium state or in a nonequilibrium steady state (NESS) and even estimate the values of some macroscopic thermodynamic quantities? The detection of nonequilibrium is important since in an NESS, there are nonzero net fluxes between microstates, indicating that the system is externally driven with concomitant entropy production Qian and Qian (2000); Li and Qian (2002); Witkoskie and Cao (2006); Tu (2008); Amann et al. (2010); Jia and Chen (2015); Skinner and Dunkel (2021); Harunari et al. (2022); Van der Meer et al. (2022); Van Vu and Saito (2023); Ghosal and Bisker (2023). Over the past two decades, numerous studies have partially answered these questions; however, there is still a lack of a unified theory for general systems.

Among these studies, some Bruno et al. (2005); Flomenbom et al. (2005); Flomenbom and Silbey (2006, 2008) focus on transition topology inference; some Deng et al. (2003); Xiang et al. (2006, 2024) focus on transition rate inference with a given transition topology; some Qian and Qian (2000); Li and Qian (2002); Witkoskie and Cao (2006); Tu (2008); Amann et al. (2010); Jia and Chen (2015) investigate nonequilibrium detection based on a two-state coarse-grained trajectory. Recently, several studies have estimated the values of some macroscopic thermodynamic quantities, such as entropy production and cycle affinities, based on partial observations Skinner and Dunkel (2021); Harunari et al. (2022); Van der Meer et al. (2022); Van Vu and Saito (2023); Ghosal and Bisker (2023). In this study, we develop a sufficient statistics approach and set up a theoretical framework of parameter recovery and nonequilibrium detection for a general Markovian system with a given transition topology. Our results only depend on data available from a sufficient long trajectory between coarse-grained states.

Model — We consider an $N$-state molecular system modeled by a continuous-time Markov chain $(X_{s})_{s\geq 0}$ with microstates $x=1,\cdots,N$ and generator matrix $Q=(q_{xy})$, where $q_{xy}$ denotes the transition rate from state $x$ to state $y$ whenever $x\neq y$ and $q_{xx}=-\sum_{y\neq x}q_{xy}$. Recall that the transition diagram of the Markovian system is a directed graph with vertex set $V=\{1,\cdots,N\}$ and edge set $E=\{(x,y):q_{xy}>0\}$. Experimentally, it often occurs that not all microstates can be observed — what can be observed are often some coarse-grained states, each being composed of multiple microstates. Specifically, we assume that the $N$ microstates can be divided into $n$ coarse-grained states $A_{1},\cdots,A_{n}$ (Fig. 1(a)). We also assume that the observation is an infinitely long trajectory of coarse-grained states (Fig. 1(b)) Flomenbom et al. (2005); Flomenbom and Silbey (2006, 2008).

Based on the coarse-grained clusters, the generator matrix $Q$ can be represented as the block form

$$Q=\begin{pmatrix}Q_{A_{1}}&Q_{A_{1}A_{2}}&\cdots&Q_{A_{1}A_{n}}\\ Q_{A_{2}A_{1}}&Q_{A_{2}}&\cdots&Q_{A_{2}A_{n}}\\ \vdots&\vdots&\ddots&\vdots\\ Q_{A_{n}A_{1}}&Q_{A_{n}A_{2}}&\cdots&Q_{A_{n}}\end{pmatrix}.$$

(1)

The steady-state distribution $\mathbf{\pi}=(\pi_{A_{1}},\cdots,\pi_{A_{n}})$ of the system must satisfy $\mathbf{\pi}Q=\mathbf{0}$, where $\mathbf{0}=(0,\cdots,0)$ is the zero vector. For simplicity, we assume that each diagonal block $Q_{A_{i}}$ has different eigenvalues $-\lambda^{i}_{1},\cdots,-\lambda^{i}_{|A_{i}|}$ (any matrix with repeated eigenvalues can be approximated by matrices with different eigenvalues to any degree of accuracy Horn and Johnson (2012)). Then there exists an invertible matrix $\Phi_{A_{i}}$ such that $Q_{A_{i}}=\Phi^{-1}_{A_{i}}\Lambda_{A_{i}}\Phi_{A_{i}}$, where $\Lambda_{A_{i}}=\mathrm{diag}(-\lambda^{i}_{1},\cdots,-\lambda^{i}_{|A_{i}|})$ is a diagonal matrix. Note that each row of $\Phi_{A_{i}}$ is an eigenvector of $Q_{A_{i}}$. For convenience, the sum of elements of each eigenvector is normalized to one, i.e, $\Phi_{A_{i}}\mathbf{1}^{T}=\mathbf{1}^{T}$, where $\mathbf{1}=(1,\cdots,1)$. Moreover, we set

$$\begin{split}\Lambda_{A_{i}A_{j}}&=\Phi_{A_{i}}Q_{A_{i}A_{j}}\Phi_{A_{j}}^{-1}=(\lambda^{ij}_{kl}),\\ \alpha_{A_{i}}&=\pi_{A_{i}}\Phi_{A_{i}}^{-1}=(\alpha^{i}_{k}).\end{split}$$

(2)

These quantities will play a crucial role in our analysis.

Sufficient statistics — In what follows, we assume that the system has reached a steady state. We next examine which information can be extracted from the infinitely long coarse-grained trajectory (Fig. 1(b)). Clearly, what can be observed from the trajectory are the jump times and the coarse-grained states before and after each jump within any finite period of time. In other words, for any given jump times $0\leq t_{1}<...<t_{m}\leq t$ and any coarse-grained states $A_{i_{1}},\cdots,A_{i_{m+1}}$, we can estimate the following probability (density) since the trajectory is infinitely long Fredkin and Rice (1986):

$$\begin{split}\mathbb{P}(X_{s}\in A_{i_{1}}(s<t_{1}),\cdots,X_{s}\in A_{i_{m+1}}(t_{m}\leq s\leq t)).\end{split}$$

(3)

These probabilities are actually all statistical information that can be extracted from coarse-grained observations.

We first consider the case where there is no jump before time $t$. Based on the notation in Eq. (2), it is clear that

$$\begin{split}&\mathbb{P}\left(X_{s}\in A_{i}(0\leq s\leq t)\right)=\pi_{A_{i}}e^{Q_{A_{i}}t}\mathbf{1}^{T}\\ &=\alpha_{A_{i}}e^{\Lambda_{A_{i}}t}\mathbf{1}^{T}=\sum_{k}\alpha^{i}_{k}e^{-\lambda^{i}_{k}t}.\end{split}$$

(4)

Based on coarse-grained observations, we can estimate the probability on the left-hand side of the above equation for each time $t$. Since time $t$ is arbitrary, we can estimate the values of all $\alpha^{i}_{k}$ and $\lambda^{i}_{k}$, and hence $\alpha_{A_{i}}$ and $\Lambda_{A_{i}}$ can be determined. Similarly, if there is only one jump before time $t$, then for any $i\neq j$, we have

$$\begin{split}&\mathbb{P}\left(X_{s}\in A_{i}(0\leq s<t_{1}),X_{s}\in A_{j}(t_{1}\leq s\leq t)\right)\\ &=\pi_{A_{i}}e^{Q_{A_{i}}t_{1}}Q_{A_{i}A_{j}}e^{Q_{A_{j}}(t-t_{1})}\mathbf{1}^{T}\\ &=\alpha_{A_{i}}e^{\Lambda_{A_{i}}t_{1}}\Lambda_{A_{i}A_{j}}e^{\Lambda_{A_{j}}(t-t_{1})}\mathbf{1}^{T}.\end{split}$$

(5)

Since times $t_{1}$ and $t$ are arbitrary and since we have determined $\alpha_{A_{i}}$ and $\Lambda_{A_{i}}$, we can also determine $\Lambda_{A_{i}A_{j}}$ from coarse-grained observations for any $i\neq j$ sup .

Similarly to Eqs. (4) and (5), it can be proved that sup

	$\displaystyle\mathbb{P}(X_{s}\in A_{i_{1}}(0\leq s<t_{1}),\cdots,X_{s}\in A_{i_{m+1}}(t_{m}\leq s\leq t))$
	$\displaystyle=\pi_{A_{i_{1}}}e^{Q_{A_{i_{1}}}t_{1}}\cdots Q_{A_{i_{m}}A_{i_{m+1}}}e^{Q_{A_{i_{m+1}}}(t-t_{m})}\mathbf{1}^{T}$		(6)
	$\displaystyle=\alpha_{A_{i_{1}}}e^{\Lambda_{A_{i_{1}}}t_{1}}\cdots\Lambda_{A_{i_{m}}A_{i_{m+1}}}e^{\Lambda_{A_{i_{m+1}}}(t-t_{m})}\mathbf{1}^{T}.$

This shows that the probability given in Eq. (3) can be represented by $\alpha_{A_{i}}$, $\Lambda_{A_{i}}$, and $\Lambda_{A_{i}A_{j}}$. Hence these three quantities contain all coarse-grained statistical information. In fact, $\alpha_{A_{i}}$, $\Lambda_{A_{i}}$, and $\Lambda_{A_{i}A_{j}}$ are not independent. Since $Q\mathbf{1}^{T}=\mathbf{0}^{T}$, it is clear that $\Lambda\mathbf{1}^{T}=\mathbf{0}^{T}$, where

$$\Lambda=\begin{pmatrix}\Lambda_{A_{1}}&\Lambda_{A_{1}A_{2}}&\cdots&\Lambda_{A_{1}A_{n}}\\ \vdots&\vdots&\ddots&\vdots\\ \Lambda_{A_{n}A_{1}}&\Lambda_{A_{n}A_{2}}&\cdots&\Lambda_{A_{n}}\end{pmatrix}.$$

(7)

This implies that $\lambda^{i}_{k}=\sum_{j,l}\lambda^{ij}_{kl}$, where $\lambda^{ij}_{kl}$ are the elements of $\Lambda_{A_{i}A_{j}}$ defined in Eq. (2). Hence $\Lambda_{A_{i}}$ can be determined by all $\Lambda_{A_{i}A_{j}}$. Since $\pi Q=\mathbf{0}$, we have $\alpha\Lambda=\mathbf{0}$, where $\alpha=(\alpha_{A_{1}},\cdots,\alpha_{A_{n}})$. This equation, together with the normalization condition

$$\alpha\mathbf{1}^{T}=\sum_{i=1}^{n}\pi_{A_{i}}\Phi_{A_{i}}^{-1}\mathbf{1}^{T}=\sum_{i=1}^{n}\pi_{A_{i}}\mathbf{1}^{T}=1,$$

(8)

shows that $\alpha_{A_{i}}$ can also be determined by all $\Lambda_{A_{i}A_{j}}$.

Recall that a family of statistics that contain all statistical information of the observations are called sufficient statistics Fisher (1922). Since the probability in Eq. (3) can be represented by $\alpha_{A_{i}}$, $\Lambda_{A_{i}}$, and $\Lambda_{A_{i}A_{j}}$ and since $\alpha_{A_{i}}$ and $\Lambda_{A_{i}}$ can be determined by all $\Lambda_{A_{i}A_{j}}$, it is clear that all $\Lambda_{A_{i}A_{j}}$ are the sufficient statistics for infinitely long coarse-grained observations. In particular, the number of these sufficient statistics is given by

$$S=\sum_{1\leq i\neq j\leq n}|A_{i}||A_{j}|=2\sum_{1\leq i<j\leq n}|A_{i}||A_{j}|.$$

(9)

Note that based on coarse-grained observations, two systems with different generator matrices but having the same $\Lambda_{A_{i}A_{j}}$ are statistically indistinguishable.

Parameter inference — In practice, a crucial question is whether all transition rates of a Markovian system with a given transition topology can be inferred from an infinitely long coarse-grained trajectory. Note that each transition rate $q_{xy}$ of the system corresponds to a direct edge in the transition diagram $(V,E)$. Hence the system has $|E|$ unknown parameters, where $|E|$ denotes the number of elements in the edge set $E$. Note that each $\Lambda_{A_{i}A_{j}}$ is uniquely determined by all transition rates and in the following, we rewrite it as $\Lambda_{A_{i}A_{j}}(q_{xy})$. Based on coarse-grained observations, we can obtain estimates of the sufficient statistics $\Lambda_{A_{i}A_{j}}$ and hence all transition rates should satisfy the following set of equations:

$$\Lambda_{A_{i}A_{j}}(q_{xy})=\tilde{\Lambda}_{A_{i}A_{j}},\;\;\;\forall\;1\leq i\neq j\leq n,$$

(10)

where $\tilde{\Lambda}_{A_{i}A_{j}}$ are the estimates of $\Lambda_{A_{i}A_{j}}$. Clearly, Eq. (10) has $|E|$ unknown parameters and $S$ equations, where $S$ is the number of sufficient statistics. Therefore, we obtain the following criteria regarding parameter inference: (i) when $S<|E|$, it is impossible to determine all transition rates of the system; (ii) when $S\geq|E|$, all transition rates can be determined if and only if Eq. (10) has a unique solution. In general, it is very difficult to give a simple criterion for the unique solvability of Eq. (10); however, it can be checked numerically using, e.g., Gröbner basis computation Weispfenning (1992) and can be proved theoretically in some simple examples. Next we focus on three examples.

1) The ladder model (Fig. 1(c)). This model is widely used to model allostery of receptors in living cells with strong cooperativity and high sensitivity Monod et al. (1965). Consider a receptor with two conformational states and $n$ ligand binding sites. According to the conformational state and the number of occupied binding sites, each receptor can be modeled by a Markovian system with $N=2n$ microstates. Due to technical limitations, we are often unable to distinguish between the two conformational states. The $N$ microstates and $n$ coarse-grained states are shown in Fig. 1(c). Clearly, we have $|A_{1}|=\cdots=|A_{n}|=2$ and there are $|E|=3N-4$ unknown parameters for the system. Note that for any $n\geq 2$, we have

$$S=N^{2}-2N\geq 3N-4=|E|.$$

(11)

Hence all parameters of the ladder model can be inferred if and only if Eq. (10) has unique solution. In sup , we show that Eq. (10) is indeed uniquely solvable for the ladder model.

2) The cyclic model (Fig. 1(d)). Many crucial cellular biochemical processes can be modeled as cyclic Markovian systems such as conformational changes of enzymes and ion channels Cornish-Bowden (2013); Sakmann (2013), phosphorylation-dephosphorylation cycle Qian (2007), cell cycle progression Jia and Grima (2021), and gene state switching Jia and Li (2023). An $N$-state cyclic model has $|E|=2N$ unknown parameters. If there are only two coarse-grained states ($n=2$), then it is impossible to infer all unknown parameters in the case of $|A_{1}|=1$ and $|A_{2}|=N-1$ since $S=2(N-1)<|E|$. For any other coarse-grained partitioning, we have $S\geq 4(N-2)\geq|E|$ and hence parameter inference is generally possible. When $n\geq 3$, we also have $S\geq 4N-6\geq|E|$, and hence a complete parameter recovery can be made if Eq. (10) can be uniquely solved. In sup , we show that Eq. (10) is indeed uniquely solvable for the four-state cyclic model under any coarse-grained partitioning whenever $S\geq|E|$.

3) The tree and linear models (Fig. 1(e)). We finally focus on an $N$-state system whose transition diagram is a tree. For any coarse-grained partitioning, it is easy to check that $S\geq 2(N-1)=|E|$. Hence all transition rates can be inferred if Eq. (10) has unique solution. In particular, a system with linear transitions (Fig. 1(f)) can be viewed as a tree. The linear model also widely used in biochemical studies Ullah et al. (2012). In sup , we show that Eq. (10) is indeed uniquely solvable for the linear model if there are only two coarse-grained states ($n=2$) with $|A_{1}|=1$ and $|A_{2}|=N-1$.

Nonequilibrium identification — When the number of sufficient statistics is less than the number of unknown parameters, it is impossible to determine all transition rates from coarse-grained observations. However, we may still identify whether the system is in an NESS. Our Ness criterion is based on the sufficient statistics $\Lambda_{A_{i}A_{j}}$. Recall that once $\Lambda_{A_{i}A_{j}}$ are determined, $\alpha_{A_{i}}$ are automatically determined by solving $\alpha\Lambda=\mathbf{0}$ and $\alpha\mathbf{1}^{T}=1$. In sup , we prove that if the system is in equilibrium, then the entries of $\alpha_{A_{i}}$ and $\Lambda_{A_{i}A_{j}}$ are all real numbers, and the following two conditions must be satisfied:
(i) (coarse-grained probability distribution condition)

$$\alpha^{i}_{k}\geq 0.$$

(12)

(ii) (coarse-grained detailed balance condition)

$$\alpha^{i}_{k}\lambda^{ij}_{kl}=\alpha^{j}_{l}\lambda^{ji}_{lk}.$$

(13)

Recall that $\alpha\mathbf{1}^{T}=1$. When the system is in equilibrium, we have $\alpha^{i}_{k}\geq 0$ and thus $\alpha=(\alpha_{A_{1}},\cdots,\alpha_{A_{n}})$ is a probability distribution. This is why Eq. (12) is called the coarse-grained probability distribution condition. On the other hand, in equilibrium, the system satisfies the detailed balance condition $\pi_{x}q_{xy}=\pi_{y}q_{yx}$. Eq. (13) can be viewed as the coarse-grained version of the detailed balance condition. The above result implies that if any one of Eqs. (12) and (13) is violated, then the system must be in an NESS. This gives a general criterion for detecting nonequilibrium based on coarse-grained observations.

For a three-state cyclic system with two coarse-grained states $A_{1}=\{1\}$ and $A_{2}=\{2,3\}$, we have seen that it is impossible to infer all parameters. In Amann et al. (2010), the authors showed that this system is in an NESS when

$$TL^{2}>LSM-M^{2},$$

(14)

where $L=q_{12}+q_{13}$, $S=q_{21}+q_{23}+q_{31}+q_{32}$, $T=q_{21}q_{31}+q_{21}q_{32}+q_{23}q_{31}$, and $M=q_{12}(q_{23}+q_{31}+q_{32})+q_{13}(q_{21}+q_{23}+q_{32})$ are another set of sufficient statistics for the three-state system that can be determined by coarse-grained observations Amann et al. (2010); Jia and Chen (2015). Our result can be viewed as an extension of Eq. (14) to complex molecular systems with an arbitrary number of microstates and an arbitrary number of coarse-grained states.

In particular, when $N=3$ and $n=2$, our criterion reduces to Eq. (14). This can be seen as follows. First, since $\alpha\Lambda=\mathbf{0}$ and $\Lambda\mathbf{1}^{T}=\mathbf{0}^{T}$, it is easy to see that

$$\alpha^{1}_{1}\lambda^{12}_{12}=\alpha^{2}_{2}\lambda^{2}_{1}=\alpha^{2}_{2}\lambda^{21}_{21},\;\;\;\alpha^{1}_{1}\lambda^{12}_{13}=\alpha^{2}_{3}\lambda^{2}_{2}=\alpha^{2}_{3}\lambda^{21}_{31}.$$

(15)

Hence the coarse-grained detailed balance condition is satisfied. On the other hand, since $\alpha\mathbf{1}^{T}=\alpha^{1}_{1}+\alpha^{2}_{2}+\alpha^{2}_{3}=1$, it follows from Eq. (15) that

$$\begin{split}\alpha^{1}_{1}&=\frac{\lambda^{21}_{21}\lambda^{21}_{31}}{\lambda^{21}_{21}\lambda^{21}_{31}+\lambda^{12}_{12}\lambda^{21}_{31}+\lambda^{12}_{13}\lambda^{21}_{21}},\\ \alpha^{2}_{2}&=\frac{\lambda^{12}_{12}\lambda^{21}_{31}}{\lambda^{21}_{21}\lambda^{21}_{31}+\lambda^{12}_{12}\lambda^{21}_{31}+\lambda^{12}_{13}\lambda^{21}_{21}},\\ \alpha^{2}_{3}&=\frac{\lambda^{12}_{13}\lambda^{21}_{21}}{\lambda^{21}_{21}\lambda^{21}_{31}+\lambda^{12}_{12}\lambda^{21}_{31}+\lambda^{12}_{13}\lambda^{21}_{21}}.\end{split}$$

(16)

Furthermore, it is easy to check that sup

$$\begin{split}&\;\;\lambda^{21}_{21}\lambda^{21}_{31}=T,\;\;\;\lambda^{12}_{12}\lambda^{21}_{31}+\lambda^{12}_{13}\lambda^{21}_{21}=M,\\ &\lambda^{12}_{12}\lambda^{21}_{31}\lambda^{12}_{13}\lambda^{21}_{21}=\frac{T(LSM-M^{2}-TL^{2})}{(\lambda^{12}_{13}-\lambda^{12}_{12})^{2}}.\end{split}$$

(17)

Combining Eqs. (16) and (17), we immediately obtain

$$\begin{split}&\alpha^{1}_{1}=\frac{T}{T+M}\geq 0,\;\;\;\alpha^{2}_{2}+\alpha^{2}_{3}=\frac{M}{T+M}\geq 0,\\ &\quad\quad\alpha^{2}_{2}\alpha^{2}_{3}=\frac{T(LSM-M^{2}-TL^{2})}{(T+M)^{2}(\lambda^{12}_{13}-\lambda^{12}_{12})^{2}}.\end{split}$$

(18)

Since $\alpha^{1}_{1}\geq 0$ and $\alpha^{2}_{2}+\alpha^{2}_{3}\geq 0$, the coarse-grained probability distribution condition is violated if and only if $\alpha^{2}_{2}\alpha^{2}_{3}<0$, which is obviously equivalent to Eq. (14).

We now compare our NESS criterion with the classical criterion based on autocorrelation functions. For any observable $\phi$, recall that the autocorrelation function of the system, $B_{\phi}(t)=\mathrm{Cov}(\phi(X_{0}),\phi(X_{t}))$, is defined as the steady-state covariance between $\phi(X_{0})$ and $\phi(X_{t})$. For coarse-grained observations, the observable $\phi$ is usually chosen as $\phi(x)=i$ for all $x\in A_{i}$. It is well-known Qian et al. (2003) that if the system is in equilibrium, then

$$B_{\phi}(t)=\sum_{m=1}^{N-1}c_{m}e^{-\ell_{m}t},\;\;\;\ell_{m}>0,\;c_{m}\geq 0,$$

(19)

where $-\ell_{m}<0$ are all nonzero eigenvalues of the generator matrix $Q$ and $c_{m}\geq 0$ are the coefficients (note that in the case, the autocorrelation function must be monotonic). Hence if there exists some $1\leq m\leq N-1$ such that any one of $\ell_{m}$ and $c_{m}$ is negative or not real, then the system must be in an NESS. Clearly, this NESS criterion is stronger than the criterion based on oscillatory or non-monotonic autocorrelation functions Qian and Qian (2000).

In sup , we have proved a stronger result — if Eqs. (12) and (13) hold, then we also have $\ell_{m}>0$ and $c_{m}\geq 0$. This suggests that all NESS scenarios that can identified by the autocorrelation criterion can definitely be identified by our criterion; in other words, our NESS criterion is mathematically stronger than the autocorrelation criterion. In particular, if there are only two coarse-grained states ($n=2$) with $|A_{1}|=1$ and $|A_{2}|=N-1$, then the two criteria are equivalent; in this case, the number of sufficient statistics is $S=2(N-1)$ and $\{\ell_{1},\cdots,\ell_{N-1},c_{1},\cdots,c_{N-1}\}$ is exactly a set of sufficient statistics sup . For other coarse-grained partitionings, the two criteria are in general not equivalent and our criterion may extend the NESS region significantly beyond the one identified by the autocorrelation criterion.

We stress that our NESS criterion is only a sufficient condition; there may be some NESS scenarios that fail to be detected by our criterion. However, we prove in the End Matter that for any three-state system with two coarse-grained states ($N=3$ and $n=2$), our criterion is also a necessary condition. In other words, if Eqs. (12) and (13) are both satisfied, then among all three-state systems having the same sufficient statistics $\Lambda_{A_{i}A_{j}}$, there must exist a system which is in equilibrium.

Finally, as an example, we focus on a four-state fully connected system with two different coarse-grained partitionings (Fig. 2(a),(b)). The system has $12$ unknown parameters and we want to determine whether it is in an NESS. We first consider the partitioning of $A_{1}=\{1\}$ and $A_{2}=\{2,3,4\}$ (Fig. 2(a)). In this case, our criterion is equivalent to the autocorrelation criterion. Fig. 2(c) illustrates the NESS region in the parameter space that can be identified by our criterion, or equivalently, the autocorrelation criterion (shown in shaded blue) and the region that fails to be identified by the two criteria (shown in green). The red line shows the region of equilibrium states. In this case, NESS can only be detected in the parameter region far from the red line.

We then consider another partitioning, i.e. $A_{1}=\{1,2\}$ and $A_{2}=\{3,4\}$ (Fig. 2(b)). Fig. 2(c) shows the NESS regions that can be identified by our criterion (shown in blue) and by the autocorrelation criterion (shown in shaded blue). Clearly, the entire NESS region can be captured by our criterion but only a small subregion can be detected by the autocorrelation criterion. To gain deeper insights, note that there are $S=6$ sufficient statistics for the partitioning shown in Fig. 2(a), while for the one shown in Fig. 2(b), there are $S=8$ sufficient statistics. More sufficient statistics means that more information can be extracted from coarse-grained observations, which generally leads to a larger NESS region that can be identified by our criterion.

Estimation of sufficient statistics — We emphasize that our methods of parameter inference and nonequilibrium detection are based on an accurate estimation of all the sufficient statistics. In the End Matter, we have proposed a maximum likelihood approach to accurately inferring the sufficient statistics $\Lambda_{A_{i}A_{j}}$ as well as the transition rates $q_{xy}$ (when $S\geq|E|$). This method is then validated using synthetic time-trace data for the ladder, cyclic, and linear models generated using stochastic simulations.

Conclusions and discussion — Another crucial question beyond this study is whether the transition topology of the system can also be inferred based on coarse-grained observations. In fact, this is impossible in many cases due to the loss of information during coarse-graining; however, when there are two coarse-grained states, Refs. Bruno et al. (2005); Flomenbom et al. (2005); Flomenbom and Silbey (2006, 2008) have developed a canonical form method of finding all possible transition topologies of the underlying Markovian dynamics. Hence in this paper, we always assume that the transition topology of the system is given.

Here we established a framework of parameter inference and nonequilibrium identification based on coarse-grained observations for a general Markovian system with an arbitrary number of microstates and an arbitrary number of coarse-grained states when the underlying transition topology is given. We provided a criterion for evaluating whether the coarse-grained information is enough for estimating all transition rates and also a criterion for detecting whether the system is in an NESS. Our method is based on extracting a set of sufficient statistics that incorporates all statistical information of an infinitely long coarse-grained trajectory. Using these sufficient statistics, the problems of parameter recovery and nonequilibrium detection can be brought into a unified theoretical framework.

Acknowledgements — C. J. acknowledges support from National Natural Science Foundation of China with grant No. U2230402 and No. 12271020.