A statistical approach to topological entanglement:
Boltzmann machine representation of high-order irreducible correlations

TO is characterized by global long-range entanglement. For gapped systems the von-Neumann entropy of a subsystem density matrix $\rho_{A}$ for the ground state scales as

$$S(\rho_{A})=\alpha|\partial A|-\gamma+O(1/|\partial A|)\ .$$

(1)

The TO is reflected in the topological entanglement entropy (TEE) $S_{\rm topo}\equiv-\gamma$ [10, 9, 41]. In the Kitaev-Preskill (KP) construction, $\gamma$ is extracted by a tripartite disk $A\cup B\cup C$ within the 2D lattice according to

$$\begin{split}S_{\rm topo}&=S(\rho_{A})+S(\rho_{B})+S(\rho_{C})\\ &-S(\rho_{AB})-S(\rho_{BC})-S(\rho_{AC})+S(\rho_{ABC})\end{split}$$

(2)

whereby the linear combination is engineered in a way such that the area-law contribution is canceled out. $\gamma$ is related to the quantum dimension of anyonic charges in the medium by a topological quantum field theory (TQFT) whereby $\gamma=\log\mathcal{D},~{}\mathcal{D}=\sqrt{\sum_{a}d_{a}^{2}}$. Besides the relation with TQFT, recently there have been pioneering works in relating the information-theoretic framework, i.e. the quantum (conditional) mutual information with TO [42, 43, 44] resulting in non-trivial bounds and proofs.

We propose to view TO from the information-theoretic point of view as an emergent statistical synergy [22] due to the underlying gauge structure, which provides more direct intuition and possible detection of TO by quantum sampling in relevant computational and experimental platforms such as tensor network and Rydberg atom arrays. The simple interpretation that is central to our argument is that the topological entropy can be written in the form of (quantum) conditional mutual information:

$$S_{\rm topo}=I(A:B)-I(A:B|C)$$

(3)

where $I(A:B)$ and $I(A:B|C)$ are quantum mutual information and quantum conditional mutual information, respectively defined by

	$\displaystyle I(A:B)$	$\displaystyle=S(\rho_{A})+S(\rho_{B})-S(\rho_{AB})$		(4)
	$\displaystyle I(A:B|C)$	$\displaystyle=S(\rho_{AC})+S(\rho_{BC})-S(\rho_{C})-S(\rho_{ABC})$		(5)

This is consistent with the KP construction defined in Eq. 2 if subsystems are properly chosen. To be specific, the mutual information $I(A:B)$ quantifies the amount of shared information between $A$ and $B$; whereas the conditional mutual information $I(A:B|C)$ quantifies the amount of shared information between $A$ and $B$ given that $C$ is known (e.g. by a local projection on a quantum state), and is able to include the irreducible tripartite information which is also known as synergistic information in the field of statistics [22]. While a non-zero $I(A:B|C)$ is capable of detecting the existence of synergy that information shared between two subsystems could be influenced by a third, it should be noted that it could confuse a trivial bipartite mutual information and the intrinsic synergy if there exists some shared information between $A$ and $B$ independent of $C$. A trivial case is where $A$ and $B$ is completely disjoint from $C$ thus $I(A:B|C)=I(A:B)$. Therefore, to remove such trivial cases, one must compare $I(A:B|C)$ against $I(A:B)$, and look into $I_{3}(A:B:C)$ defined in Eq. 3 or Eq. 6. It is only in the simplest case where $I(A:B)=0$ that a non-zero $I(A:B|C)$ alone suffices to determine the synergistic information (as we are to demonstrate in the coming section, this is exactly the case of the LW construction).

If the tripartition of a subsystem is topological non-trivial e.g. shown in Fig. 1(b,c) and the ground state is topologically ordered, the resulting $I_{3}$ will equal to $S_{\rm topo}$ [45, 46]. In contrast, for a topologically trivial geometry e.g. shown in Fig. 1(a), $I_{3}=0$ is always true for gapped systems since $I(A:B)=0$ and $(A:B|C)=0$; and it is true regardless of the order of $A,B$ and $C$ because $I_{3}(A:B:C)$ is symmetric under permutation of variables.

Notably, this interpretation coincides with that of the third order mutual information $I_{3}^{c}$ for classical random variables and is related to the measure of frustration and synergy [1, 22]. In the probabilistic context, given three random variables $A,B,C$ generated from a classical ensemble or sampling of quantum density matrix, we can define $I_{3}^{c}$ for classical random variables as

$$\begin{split}I_{3}^{c}(A:B:C)=I^{c}(A:B)-I^{c}(A:B|C)\end{split}$$

(6)

where the mutual information and the conditional mutual information are defined respectively as

	$\displaystyle I^{c}(A:B)$	$\displaystyle=\sum_{a,b}p(a,b)\log\frac{p(a,b)}{p(a)p(b)}$		(7)
	$\displaystyle I^{c}(A:B|C)$	$\displaystyle=\sum_{a,b,c}p(a,b,c)\log\frac{p(c)p(a,b,c)}{p(a,c)p(b,c)}$		(8)

hence the $I_{3}^{c}(A:B:C)$ in Eq. 6 can be expressed compactly as

$$I_{3}^{c}(A:B:C)=-\sum_{a,b,c}p(a,b,c)\log\frac{p(a,b,c)p(a)p(b)p(c)}{p(a,b)p(b,c)p(a,c)}$$

(9)

Note this definition is symmetric under permutations of indices of $A,B,C$. A negative $I_{3}^{c}$ is intimately related to the so-called synergistic information or interaction information, that is, the intrinsic many-body and non-dyadic correlation that cannot be reduced to pair-wise relations [1, 22]. A negative value of $I_{3}^{c}$ is then interpreted as a statistical situation in which the knowledge of any one of the three variables, $A,B$ and $C$, enhances the correlation between the other two. Such statistical intuition remains valid in the quantum many-body cases, whereas the many-body quantum correlation is attributed to the non-local entanglement in a topologically ordered wave function. Indeed, in pure lattice gauge theories or integrable models where gauge sector is disjoint from matter (such as Kitaev spin liquid), by choosing a set of basis whereby the Wilson loops are explicitly constructed, the quantum samples taken under these basis can be interpreted using the classical $I_{3}^{c}(A:B:C)$, and relevant statistical methods like restricted Boltzmann model can be straightforwardly applied on subsystems thereof.

This formulation of topological entropy can be applied to both classical and quantum context. Indeed, even for classically frustrated spins, the thermal fluctuations exhibit topological entropy and the synergy with negative $I_{3}$ [1, 47]. In the context of topologically ordered quantum matter, the non-local constraint e.g. plaquettes operators and Wilson loops in Toric code and Kitaev spin liquids intuitively resemble the aforesaid synergy. Indeed, as we are to show in detail, the topological entanglement entropy is equivalent to a quantum case of third order mutual information that indicates a synergy of quantum fluctuations which is present at zero temperature. This rethinking of topological entropy allows us to unify classically frustrated systems and the TO of quantum frustrated systems in the same information-theoretic framework.

Indeed the Levin-Wen (LW) construction is equivalent to the KP construction, thus to $I_{3}$; and the aforementioned statistical interpretation of TEE remains valid as well. The LW construction extracts TEE using the partition shown in Fig. 1(d) and the following linear combination of entropies

$$S_{\rm topo}=S(\rho_{ABC})-S(\rho_{AC})-S(\rho_{BC})+S(\rho_{C})$$

(10)

It is equivalent to negative conditional mutual information $-I(A:B|C)$ which measures the shared information between $A,B$ conditioned on $C$, and a non-zero value indicates the existence of the long-range entanglement as a global constraint. It is necessarily non-positive due to the strong subadditivity inequality. Indeed, the LW construction defined above is also equivalent to $I_{3}$. This would be clear if we realize that $I_{3}$ for the geometry of Fig. 1(d) is $I_{3}(A:B:C)=I(A:B)-I(A:B|C)=I(A:B)+S_{\rm topo}$. For a large enough $C$ whose length scale is larger than correlation length i.e. $\xi/|\partial C|\sim 0$, the Hilbert subspaces of $\rho_{A}$ and $\rho_{B}$ are disjoint, hence

	$\displaystyle S(\rho_{AB})$	$\displaystyle=S(\rho_{A}\otimes\rho_{B})=S(\rho_{A})+S(\rho_{B})$
		$\displaystyle\Rightarrow~{}I(A:B)=0$		(11)

for the partition of Fig. 1(d). Combining Eq. 11 with Eq. 3 or Eq. 6 gives exactly the LW construction in Eq. 10. Nevertheless, if $\xi/|\partial C|$ is not negligibly small, it would be more accurate to include the term $I(A:B)$ which removes the residual non-topological information caused by finite range correlation. We would like to mention that there are also other construction of topological entropy that are statistically equivalent to the third order mutual information $I_{3}$ such as the multi-partite entanglement in the context of holographic theory [48, 21, 49, 50, 51], which we do not elaborate in this paper.

In this section we show that, for a TO with emergent gauge theory, the $n$-th order quantum mutual information $I_{n}$ ($n>3$) is equivalent to $I_{3}$. According to the aforementioned idea of higher order mutual information, we can always generate higher order constructions as descriptors of higher order irreducible many-body correlation. The generic $n$-th order quantum mutual information can be expressed recursively as

$$\begin{split}I_{n}(P_{1}:\cdots:P_{n})=&I_{n-1}(P_{1}:\cdots:P_{n-1})\\ &-I_{n-1}(P_{1}:\cdots:P_{n-1}|P_{n})\end{split}$$

(12)

where the conditional mutual information satisfies

$$\begin{split}I_{n-1}(P_{1}:\cdots:P_{n-1}&|P_{n})=I_{n-1}(P_{1}:\cdots:P_{n-1}P_{n})\\ &-I_{n-1}(P_{1}:\cdots:P_{n-2}:P_{n})\end{split}$$

(13)

Similar to $I_{3}$, Eq. 12 can be perceived as an $n$-body irreducible correlation, or $n$-body synergy that cannot be reduced to any correlations between fewer degrees of freedom. Indeed, it is intuitively clear that TEE is a direct consequence of the Gauss law of an emergent gauge theory, thus encodes such synergy between all degrees of freedom that live on a closed Wilson loop. Without loss of generality, let us assume a gapped TO whose correlation length is negligibly small compared to the sizes of subsystems. Note that for $n\geq 3$, the first term on the right-hand side of Eq. 12 and the second term on the right-hand side of Eq. 13 must be zero since the union of partitions in the parenthesis does not fill up an annular region i.e. is not subjected to gauge constraint; and the small correlation length guarantees there is no statistical correlation between subsystems that do not share a common boundary. Hence, we have

$$I_{n}(P_{1}:\cdots:P_{n})=-I_{n-1}(P_{1}:\cdots:P_{n-1}P_{n})$$

(14)

Following such recursion

$$\begin{split}I_{n}(P_{1}:\cdots:P_{n})&=(-1)^{n-1}I_{3}(P_{1}:P_{2}:P_{3}\cdots P_{n})\\ &\equiv(-1)^{n-1}S_{\rm topo}\end{split}$$

(15)

Hence TEE is equivalent to $I_{n}~{}(\forall n\geq 3)$ up to a sign. As a concrete example, we demonstrate that the construction in Eq. 12 with $n=4$, i.e. the $4$th order mutual information with the partition shown in Fig. 1(e), is equivalent to $I_{3}$. For a quadrupartite annular disk $A\cup B\cup C\cup D$, $I_{4}(A:B:C:D)$ is defined as

$$I_{4}(A:B:C:D)=I_{3}(A:B:C)-I_{3}(A:B:C|D)$$

(16)

where $I_{3}(A:B:C)$ is necessary zero since $A\cup B\cup C$ does not form a closed topology and is thus equivalent to the conditional entropy of a quantum Markov state. The third order conditional mutual information $I_{3}(A:B:C|D)$ can be expanded in a non-conditional form as

$$I_{3}(A:B:C|D)=I_{3}(A:C:BD)-I_{3}(A:C:D)$$

(17)

where we have exploited the permutation symmetry of $I_{n}(P_{1}:\cdots:P_{n})$ to switch $C$ and $B$. Note in the given geometry of Fig. 1(e) $I_{3}(A:C:D)$ is necessarily zero [43], in the same way that $I_{3}(A:B:C)=0$. Therefore, for a quadrupartite annular disk $A\cup C\cup B\cup D$, we have

$$I_{4}(A:B:C:D)=-I_{3}(A:C:BD)$$

(18)

the latter is the same as a tripartite annular disk $A\cup C\cup BD$, thus $I_{4}$ is reduced to the LW construction of $I_{3}$ as shown in Fig. 1(d). It is then trivial to extend to proof to $n$th order by induction. Indeed, this is in accordance with the irreducible correlation $C_{\rho}^{(k)}$ which measures the (quantum) information that is contained in $k$ parties yet absent in $(k-1)$ or less [52]; and, in particular, both KP and LW constructions can be understood as the third order irreducible correlation $C_{\rho}^{(3)}$ [53]. Generally, in systems where the correlation length is not negligibly small compared to the distances between subsystem partitions, we can write down the complete form of $I_{4}$ by Eq. 12 and Eq. 13 to extract topological entanglement entropy $S_{\rm topo}^{(4)}$ using the quadripartite disk in Fig. 1(e):

$$\begin{split}-S_{\rm topo}^{(4)}&=S(\rho_{A})+S(\rho_{B})+S(\rho_{C})+S(\rho_{D})\\ &-S(\rho_{AB})-S(\rho_{BC})-S(\rho_{AC})-S(\rho_{BD})\\ &-S(\rho_{AD})-S(\rho_{CD})\\ &+S(\rho_{ABD})+S(\rho_{BCD})+S(\rho_{ACD})+S(\rho_{ABC})\\ &-S(\rho_{ABCD})\end{split}$$

(19)

which is the quadripartite analogue to the tripartite KP construction. It is readily to check that all boundary contributions cancel with each other. Higher order constructions can be represented by the same token. For a generic $n$-partite partition shown in Fig. 1(f), we have $\forall n\geq 3$:

$$S_{\rm topo}^{(n)}=(-1)^{n-1}I_{n}=\sum_{i=1}^{n}(-1)^{i+n}\sum_{k_{1},\cdots,k_{i}}S(\rho_{(P_{k_{1}}\cdots P_{k_{i}})})$$

(20)

as a generic recipe for extracting TEE in an $n$-partite subsystem.

In particular, we focus on the entanglement of “skeletal” regions in lattice models, which in 2D lattices are lines with no volume. The advantage of such partition is that for gapped systems it requires the minimum number of qubit samples in order expose the topological structure of the wave function, such as a Wilson loop operator; and $I_{n}$ can thereby be interpreted as the $n$-th order mutual information between $n$ sampled qubits. By exploiting the inherent probabilistic nature of quantum wavefunction, we can define the probability distribution given a set of projective measurements $\mathcal{P}_{i}$ [27]

$$p_{i}=\Tr(\mathcal{P}_{i}^{\dagger}\mathcal{P}_{i}\rho),~{}~{}\rho^{\prime}=\frac{\mathcal{P}_{i}\rho\mathcal{P}_{i}^{\dagger}}{\Tr(\mathcal{P}_{i}^{\dagger}\mathcal{P}_{i}\rho)}$$

(21)

where $i$ denotes a state in certain complete computational basis $\{\sigma_{i}^{\alpha_{i}}|i=1\cdots n,\alpha\in\{0,x,y,z\}\}$ and $\rho^{\prime}$ is the resultant density matrix after the measurement. Assume each $\mathcal{P}_{i}$ is a local projector, we conduct sampling on skeletal partition of the lattice i.e. regions (lines) that have no volume [54] but with non-trivial topology as closed loops. Then in an n-site subsystem one can consider the probability in the auto-regression form

$$p(\sigma_{1}^{\alpha_{1}},\cdots,\sigma_{n}^{\alpha_{n}})=\prod_{i=1}^{n}p(\sigma_{i}^{\alpha_{i}}|\vec{\sigma}_{<i})$$

(22)

where $\vec{\sigma}_{<i}\equiv\{\sigma_{j}^{\alpha_{j}}|j<i\}$. It should be pointed out that in principle one can simply project the state into a definite many-body basis without doing a series of local projection, however, the local projections easily allow the study on local density matrices and also make it amenable to tenor network methods like density matrix renormalization group and matrix product states. Eq. 22 can be treated as a classical probability distribution for any normalized wavefunction. Note that even though it is formalized in a classical probability, quantum information are nevertheless accessible by shuffling the local computational basis; and since the probability $p(\sigma_{i}^{\alpha_{i}}|\vec{\sigma}_{<i})$ is determined by the projective measurements conditioned on the totality of previous measurements, the sampled data is in general able to reflect the entanglement structure of $\rho$ according to Eq. 21. Assume that $\{\sigma_{i}\}$ are gauge field degrees of freedoms as is required by the TO, where local contractable Wilson loops is constraint to have zero total flux, and any other operators that do not form closed loops, i.e. those that are not gauge-invariant, are not subjected to such constraint. Then the joint probability in Eq. 22 will exhibit a strong synergy if $(\sigma_{1}^{\alpha_{1}},\cdots,\sigma_{n}^{\alpha_{n}})$ forms a Wilson loop whereby strong fluctuations are accompanied by many-body constraints; and weak synergy otherwise.

In this section, we demonstrate the equivalence between the following three key concepts: (i) zero-flux constraint for Wilson loops (ii) topological entanglement entropy and (iii) higher order mutual information $I_{3}$ in the TC model as a pure $Z_{2}$ gauge theory by classical probabilistic treatment.

The simplest macroscopic model that realizes a pure $Z_{2}$ gauge theory and TO is the Toric code model, where the local energy density is given by mutually commuting stabalizers that are responsible for the non-local loop operators. Let us start with a pure $Z_{2}$ gauge theory, which can be realized by macroscopic Hamiltonian by e.g. Toric code model

$$H_{\rm TC}=-J_{\rm TC}\left[\sum_{s}A_{s}+\sum_{p}B_{p}\right]$$

(23)

with $A_{s}$ and $B_{p}$ given by

$$A_{s}=\prod_{i\in+_{s}}\sigma_{i}^{x},~{}~{}B_{p}=\prod_{i\in\square_{p}}\sigma_{i}^{z}$$

(24)

as shown in Fig. 2(a). Its ground state can be exactly constructed by superposing all gauge-equivalent wave functions: $\ket{\Psi}\propto\prod_{s}(1+A_{s})\ket{\Psi_{0}}$, where $\ket{\Psi_{0}}$ can take the form of any $z$-basis state that satisfies $\prod_{i\in\square_{p}}\sigma_{i}^{z}=1$. Let us consider four-site plaquette subsystem consisting of link variable $\sigma_{\square}\equiv\{\sigma_{1},\sigma_{2},\sigma_{3},\sigma_{4}\}$. The topological nature can be reflected in the fact that the product of four $\sigma^{z}$ ($\sigma^{x}$) that reside in a plaquette (star) must be $+1$ for the ground state wavefunction; and that the gauge operator enforces the superposition of all gauge configurations that satisfies the aforesaid constraint. The ground state wave function can hence be written as

$$\ket{\Psi}\propto\sum_{\{\sigma_{\square}\}}\ket{\sigma_{\square}}\otimes\ket{\overline{\sigma_{\square}}}$$

(25)

up to a normalizing factor, where $\ket{\overline{\sigma_{\square}}}$ denotes the gauge configuration of links complementary to the $\sigma_{\square}$. The summation is over the configuration $\{\sigma_{\square}\}$ of subsystem, which determine the set of configurations in the environment. Due to the zero-flux constraint in the ground state, there are three degrees of freedom which fluctuate independently, hence, the normalized reduced density matrix takes the form

$$\rho_{\square}=\Tr_{\overline{\sigma_{\square}}}(\ket{\Psi}\bra{\Psi})=\frac{1}{2^{3}}\ket{\sigma_{\square}}\bra{\sigma_{\square}}$$

(26)

which immediately gives the entropy $S=L\log 2-\log 2$, with $L=4$ and the second term $S_{\rm topo}=-\log 2$. The simplicity of $\rho_{\square}$ of TC makes it an ideal and minimal platform to test the statistical approach. To apply a statistical measure of topological entanglement, we use the projective measurement which produces classical probabilities. Given a mixed state $\{p_{i},\ket{\psi_{i}}\}$, the (reduced) density matrix is defined by $\rho=\sum_{i}p_{i}\ket{\psi_{i}}\bra{\psi_{i}}$. A projetive meaurement by projector $\mathcal{P}_{i}$ gives the outcome $i$ with probability $p_{i}$, and $\rho$ collapses into $\rho^{\prime}$ as discussed in Eq. 21.

Note that the gauge constraint in $Z_{2}$ TO states that any contractable loops must have zero flux in the ground state expectation, hence, under $z$ basis, in each set of samples the four spins must multiply to one. Equivalently, fixing one of the four spins by a projection into $\ket{\uparrow}$, projective measurements on the remaining three $\sigma_{i}$ should give a dependent sample distribution. This allows us to write it in terms of the tripartite form, which immediately gives a non-zero synergistic information as the topological entanglement. To see this, we calculate the (conditional) mutual information of $\{\sigma_{1},\sigma_{2},\sigma_{3}\}$ in the classical information context given a fixed $\sigma_{4}=+1$:

$$I(\sigma_{1}:\sigma_{2}|\sigma_{4}=+1)=\sum_{\sigma_{1},\sigma_{2}}p(\sigma_{1},\sigma_{2})\log\frac{p(\sigma_{1},\sigma_{2})}{p(\sigma_{1})p(\sigma_{2})}$$

(27)

$$\begin{split}I(\sigma_{1}:\sigma_{2}|\sigma_{3};\sigma_{4}=+1)&=\sum_{\sigma_{1}\sigma_{2}\sigma_{3}}p(\sigma_{1},\sigma_{2},\sigma_{3})\\ &\times\log\frac{p(\sigma_{3})p(\sigma_{1},\sigma_{2},\sigma_{3})}{p(\sigma_{1},\sigma_{3})p(\sigma_{2},\sigma_{3})}\end{split}$$

(28)

It is straightforward to evaluate these equations noting that there are only a few choices of $\{\sigma_{1},\sigma_{2},\sigma_{3}\}$ given the zero-flux constraint. Here we can directly write down the marginal probabilities. Due to the gauge symmetry, all valid gauge configurations share the same probability weight, hence we have

	$\displaystyle p(\sigma_{i}=\pm 1)=\frac{1}{2},~{}p(\sigma_{1}=\pm 1,\sigma_{2}=\pm 1)=\frac{1}{4}$		(29)
	$\displaystyle p(\sigma_{1}=\pm 1,\sigma_{2}=\pm 1,\sigma_{3}=\pm 1)=\frac{1}{4}$		(30)

These immediately gives

	$\displaystyle I(\sigma_{1}:\sigma_{2}|\sigma_{4}=+1)=0,$		(31)
	$\displaystyle I(\sigma_{1}:\sigma_{2}|\sigma_{3};\sigma_{4}=+1)=\log 2$		(32)

thus the third order mutual information which coincides with TEE:

$$I_{3}(\sigma_{1}:\sigma_{2}:\sigma_{3}|\sigma_{4}=+1)=-\log 2=S_{\rm topo}(\rm TC)$$

(33)

By the same token we would arrive at the same $\log 2$ with the fixed $\sigma_{4}=-1$. This is also consistent with that obtained by LW construction on a skeletal region [54]. Hence in the chosen basis, the topological entanglement inside the plaquette Wilson loop can be treated as a classical statistical problem where a negative third order mutual information $I_{3}$ is indicative of the existence of TEE; and this holds true for larger loops with negative $I_{n}$. Note that the equal-weight superposition between gauge configuration, thus the presence of the gauge operator $\sum_{s}\prod_{i\in+_{s}}\sigma_{i}^{x}$ in the Hamiltonian, is essential in deriving the above result. Its absence will lead to a product state that trivially satisfies the constraint $\expectationvalue{\prod_{\square}\sigma_{i}^{z}}=1$ at zero temperature without fluctuation in the samples of projective measurements.

This toy example also clearly showcases the gauge constraint as an essential piece that give rise the synergistic information, in the same way it is needed in the conventional derivation of TEE in previous references. In the statistical point of view, the TEE is equivalent to the intrinsic many-body correlation that cannot be reduced to several correlations of pairs of qubits. The key role played by the gauge constraint can also be reflected in another kind of subsystem partition: assume a subsystem whereby the constituent four qubits are colinear, thus the gauge constraint does not interfere. In this case $I_{3}=0$ since no synergy would emerge in absence of gauge constraint; this is also consistent with Ref. [43] where the authors showed the quantum conditional mutual information vanishes for subsystems with colinear topology. Hence, to witness $S_{\rm topo}$ or $I_{n}$, we must partition a subsystem into a topology such that the gauge constraint is present, such as the skeletal loop shown in Fig. 2(b). Nevertheless, we would like to point out that it is still possible to extract TEE in certain fine-tuned scenarios using only two-point correlations if there are particles in additional quantum sectors that are coupled to the emergent gauge field, whereby the information of gauge sector is imprinted into the two-point correlators of the matter sector [55].

In this subsection we explain the intuition that a two-body interacting model like RBM can be used to construct many-body correlation/interaction. It has been proven that with a sufficiently large number of hidden nodes, any probability distribution can be well approximated by RBM [58]. The essence of RBM is the representation of an effective model with many-body interaction using a model with redundant degrees of freedoms which are coupled only by two-body interactions. To make this point explicit, Let $H_{\tau}$ be the Hamiltonian of the hidden nodes, which can simply take the form of Pauli matrices $\tau$ in presence of magnetic fields; and let $H_{\sigma}$ be the Hamiltonian of the visible nodes that are coupled by less than or equal to two-body interactions, and $H_{\sigma\tau}$ the Hamiltonian of the two-body interactions between the visible and the hidden nodes. The generic Green function $G$ of the whole system is given by $(E-H)G=I$, in block matrix form it can be written as

$$\begin{pmatrix}E-\mathcal{H}_{\tau}&-\mathcal{H}_{\sigma\tau}\\ -\mathcal{H}_{\sigma\tau}^{\dagger}&E-\mathcal{H}_{\sigma}\end{pmatrix}\begin{pmatrix}G_{\tau}&G_{\sigma\tau}\\ G_{\tau\sigma}&G_{\sigma}\end{pmatrix}=\begin{pmatrix}1&0\\ 0&1\end{pmatrix}$$

(35)

This immediately gives

$$\left\{E-\left[\mathcal{H}_{\sigma}+\mathcal{H}_{\sigma\tau}(E-H_{\tau})^{-1}H_{\sigma\tau}\right]\right\}G_{\sigma}=I$$

(36)

which means the Green function of the visible nodes is

$$G_{\sigma}=\frac{1}{E-(\mathcal{H}_{\sigma}+\Sigma)}$$

(37)

with the self energy $\Sigma$ given by

$$\Sigma=\mathcal{H}_{\sigma\tau}(E-\mathcal{H}_{\tau})^{-1}\mathcal{H}_{\sigma\tau}$$

(38)

It is $\Sigma$ which involves higher order interaction in the perturbation expansion and gives the representation power of RBM. Therefore we can identify an effective Hamiltonian of the visible nodes under the influence of the hidden nodes. The effective Hamiltonian is simply

$$\mathcal{H}^{\rm eff}_{\sigma}(E)=\mathcal{P}_{\sigma}(\mathcal{H}_{\sigma}+\Sigma)\mathcal{P}_{\sigma}^{\dagger}$$

(39)

where $\mathcal{P}_{\sigma}$ projects states onto the manifold of visible nodes. The Green function formalism makes clear that, even though all the contributing Hamiltonian are local, two-body in nature, the self energy term in $\mathcal{H}^{\rm eff}_{\sigma}$ can contain non-local, many-body interactions encoded in the entanglement Hamiltonian in Eq. 34, as illustrated in Fig. 3, which can be made formally explicit by perturbation or cumulant expansion. Figure 2(c) shows an example of the RBM description for the data obtained from projections on a small Wilson loop of a lattice model.

As we are to discuss in detail in the following subsection, RBM is a specific implementation of the above picture, where $H_{\sigma}(\mathbf{a})$ and $H_{\tau}(\mathbf{b})$ have only one-body energy contribution to the total Hamiltonian with $\mathbf{a},\mathbf{b}$ coupling parameters; and $\mathcal{H}_{\tau\sigma}(\mathbf{w})$ encodes all two-body interactions $w_{ij}$ between the visible and hidden nodes. By tuning the coupling parameters $\mathbf{a},\mathbf{b},\mathbf{w}$, and tracing out the redundant degrees of freedom $\tau$, we arrive at

$$\mathcal{H}^{\rm eff}=\sum_{s=1}^{N}\sum_{k_{1}<\cdots<k_{s}}\mathcal{I}^{(s)}_{k_{1},\cdots,k_{s}}\sigma_{k_{1}}\cdots\sigma_{k_{s}}$$

(40)

where $\mathcal{I}^{(s)}_{k_{1},\cdots,k_{s}}$ is a function of $\mathbf{a},\mathbf{b},\mathbf{w}$; and it gives the effective coupling between $s$ visible nodes $\sigma_{k_{1}},\cdots,\sigma_{k_{s}}$. Indeed, the ability to encode arbitrarily high order interactions makes RBM a universal representation for all statistical distributions.

In general, a large $n$-body mutual information $I_{n}$ indicates a large $\mathcal{I}^{(n)}_{k_{1},\cdots,k_{n}}$ in the effective energy of RBM defined by $p(\sigma)\sim\exp(-\mathcal{H}^{\rm eff})$, which is equivalent to Eq. 34 for the diagonal elements, and will be discussed in detail in the next subsection. A similar idea by the multi-layered deep Boltzmann machine has been used to resolve the non-local entanglement features on the boundary by a local Ising model in the context of holographic duality [59, 60]. One contribution of our work is to make explicit the form of $\mathcal{I}^{(n)}_{k_{1},\cdots,k_{n}}$ of an RBM for $\sigma\in\{+1,-1\}$ relevant for projective samples of spin-$\frac{1}{2}$, and also for other binary bits, without involving perturbation or cumulant expansion, making it easy to extract interaction strength of arbitrary order. This allows us to interrogate the RBM the existence of non-local many-body correlation between sampled degrees of freedoms, where a large $\mathcal{I}^{(n)}$ of the trained network is indicative of $n$-order correlation as TEE encoded in the entanglement Hamiltonian under the basis by which Wilson loop is diagonal. Furthermore, we also present the effective interaction for $\sigma\in\{0,1\}$ in the Appendix, which can be used to represent high-order density-density interaction in fermion models [24]; however, since any presence of zero would lead to a trivial energy contribution in Eq. 40 regardless of the order, it is not capable of representing non-trivial many-body correlation like Wilson loops of spins.

Previous section has established the possibility of representing the (projected) entanglement Hamiltonian by means of an effective many-body Ising model with arbitrary orders of interactions due to RBM. However, we would like to point out the following caveat that needs clarification: High-order interactions and high-order correlations address fundamentally different aspects of a system: the former address the effective Hamiltonian dynamics that stabilizes low-energy states, while the latter focus on emergent statistical properties of these states. Indeed, as pointed out in Ref. [1, 4], high-order mutual information $I_{n\geq 3}$ can arise in frustrated Ising models with only pair-wise interactions. For example, consider a simple frustrated Ising model with three binary spins, whose Hamiltonian is given by

$$\mathcal{H}_{3}=-\mathcal{I}^{(2)}(\sigma_{1}\sigma_{2}+\sigma_{1}\sigma_{3}+\sigma_{2}\sigma_{3})-\mathcal{I}^{(3)}\sigma_{1}\sigma_{2}\sigma_{3}$$

(41)

This three-spin model is pair-wisely frustrated when $\mathcal{I}^{(3)}=0$ and $\mathcal{I}^{(2)}<0$, and straightforward calculation shows the third order mutual information is negative, indicating a frustration-induced irreducible three-body correlation. Therefore, one may ask if an $n$-th order correlation in a closed Wilson loop can also be reflected in lower order interactions $\mathcal{I}^{(n^{\prime}<n)}$, causing a large degeneracy in RBM representation.

Here we show that such confusion due to the potential degeneracy of representation does not happen in representing the non-local correlation of a TO with $S_{\rm topo}=-\log 2$. Indeed, even though the frustrated model $\mathcal{H}_{3}$ gives rise to a negative $I_{3}$ in absence of high-order interaction $\mathcal{I}^{(3)}$, it cannot generate the high-order mutual information as large as $I_{3}=S_{\rm topo}=-\log 2$ (See Eq. 33) without a dominant $\mathcal{I}^{(3)}$. Straightforward calculation shows

$$\lim_{\mathcal{I}^{(2)}\rightarrow-\infty}I_{3}[\mathcal{H}_{3}(\mathcal{I}^{(2)},\mathcal{I}^{(3)}=0)]=-\log(\frac{9}{8})$$

(42)

which obviously deviates from $-\log 2$ for a $Z_{2}$ gauge theory. Therefore, a large value of $\mathcal{I}^{(3)}$ is required in the optimized RBM in order to generate the correct many-body correlation in the projectively measured data. As shown in Fig. 4(a,b), a faithful RBM representation of $S_{\rm topo}=-\log 2$ is only possible with the highest-order interaction $\mathcal{I}^{(3)}$, and parameters in the effective Hamiltonian of the RBM must flow along the direction where $\mathcal{I}^{(3)}$ increases. The same holds in a four-spin model, as shown in Fig. 4(c,d), which we will demonstrate by our RBM implementation in the following sections. Indeed, this is similar to the classical picture of topological order, where the topological entropy can be perceived as thermally mixed classical loops with energetic constraints [47]. By induction one can show that high-order scenarios have the same properties, which we do not enumerate in this paper.

In this section, we discuss how to interrogate a trained RBM to extract effective interactions of arbitrary order. The physical spins are to be obtained by projective measurements with a set of definite projection basis. The network of RBM is given by the energy function

$$\mathcal{H}(\sigma,\tau)=\sum_{i}-a_{i}\sigma_{i}-b_{i}\tau_{i}-\sum_{j}w_{ij}\sigma_{i}\tau_{j}$$

(43)

and the joint probability distribution is $p(\sigma,\tau)=\frac{1}{Z}e^{-\mathcal{H}(\sigma,\tau)}$ with $Z$ the partition function. The probability distribution of the physical spins $\sigma$ can be obtained by marginalization over unphysical spins

$$p(\sigma)=\Tr_{\tau}p(\sigma,\tau)=\frac{e^{-\mathcal{H}(\sigma)}}{Z^{\prime}}$$

(44)

where the associated partition function is

$$Z^{\prime}=\frac{Z}{\Tr_{\tau}\exp(\sum_{i}b_{i}\tau_{i})}$$

(45)

The the effective energy $\mathcal{H}(\sigma)$ are given by

$\displaystyle\mathcal{H}(\sigma)=\sum_{i}a_{i}\sigma_{i}+K\left(\sum_{i}w_{ij}\sigma_{i}\right)$

(46)

where $K\left(\sum_{i}w_{ij}\sigma_{i}\right)$ is the cumulant generating function

$$K\left(\sum_{i}w_{ij}\sigma_{i}\right)=\log\Tr_{\tau}\left[\exp(\sum_{ij}w_{ij}\sigma_{i}\tau_{j})\rho(\tau)\right]$$

(47)

and $\rho(\tau)$ is a probability density function of unphysical spins

$$\rho(\tau)=\exp(\sum_{i}b_{i}\tau_{i})\Big{/}\Tr_{\tau}\exp(\sum_{i}b_{i}\tau_{i})$$

(48)

Here we use the spin states in $\{+1,-1\}$, and we seek to expand the $\mathcal{I}$ in Eq. 40, which usually requires a cumulant expansion of Eq. 47 to arbitrary orders of $\sigma$ such that different orders or correlation become explicit:

$$K(x)=\sum_{n}\frac{1}{n!}\kappa^{(n)}x^{n},~{}~{}x\equiv\sum_{i}w_{ij}\sigma_{i}$$

(49)

where $\kappa^{(n)}$ is the $n$th cumulant function

$$\kappa^{(n)}=\frac{\partial^{n}}{\partial x^{n}}K(x)\Big{|}_{x=0}$$

(50)

Indeed, this is usually the method used to extract or construct effective interactions in RBM [39, 61, 24]. However, the $\kappa^{(n)}$ in Eq. 50 requires high order derivative in presence of many-body interaction, making it inconvenient to analytically track the effective interaction of arbitrary order and evaluate $\mathcal{I}^{(n)}$ for large $n$. Hence, the implementation in previous pioneering works have exploited only the first few orders of interaction.

Here, instead of using conventional cumulant expansion, we propose a projective construction and derive the closed form of effective interaction at arbitrary order without involving derivatives or cumulant functions. For $\sigma\in\{+1,-1\}$ which is anti-symmetric in the $z$ basis of the Pauli matrix, it is intuitively clear that $r$th order interaction should be captured by

$$\mathcal{I}^{(r)}_{k_{1},\cdots,k_{r}}\sim\Tr_{\{\sigma_{1},\cdots,\sigma_{N}\}}\left[\prod_{i}^{r}\sigma_{k_{i}}\mathcal{H}^{\rm eff}\right],~{}\sigma_{i}=\pm 1$$

(51)

since all energy contributions are cancelled due to the anti-symmetry of $\sigma^{z}$, except for those of $\sigma_{k_{1}},\cdots,\sigma_{k_{r}}$ which are made symmetric by $\prod_{i}^{r}\sigma_{k_{i}}$. Indeed, we will show this representation of $\mathcal{I}^{(r)}$ is true in the case of $\sigma\in\{+1,-1\}$ and is suitable for the application in detecting $I_{n}$ of emergent gauge field. Yet, in cases such as $\sigma\in\{0,1\}$ or others, i.e. in absence of anti-symmetry, we need to explicitly construct the effective interaction using a different method. Below we present a general construction that is valid for all binary $\sigma$. The trick is to construct proper resolution of identity which makes all orders of interaction explicit at once. For convenience, we here present the derivation for $\sigma\in\{+1,-1\}$, and use Dirac notation in the $z$ basis of Hilbert space with zero off-diagonal elements. In the appendix we present another example of such construction, where binary spins can be either $0$ or $1$. We first define the projector $\mathcal{P}_{k}$ that selects the state where all but the $k$th visible node $\sigma_{k}$ are $-1$:

$$\mathcal{P}_{k}=\ket{\sigma_{k}=1;\sigma_{k^{\prime}\neq k}=-1}\bra{\sigma_{k}=1;\sigma_{k^{\prime}\neq k}=-1}$$

(52)

Then the global projection on $\vec{\sigma}$ supported on the subspace of visible nodes that selects out states where there exists only one visible active node is defined by

$$\mathcal{P}^{(1)}\equiv\sum_{k=1}^{N}\mathcal{P}_{k}$$

(53)

Similarly for any positive integer $m\leq N$ we can define the projector $\mathcal{P}^{(m)}$ that selects $m$ out of $N$ visible nodes that are active:

$$\mathcal{P}^{(m)}=\sum_{k_{1}<\cdots<k_{m}}\mathcal{P}_{k_{1},\cdots,k_{m}}$$

(54)

where $\mathcal{P}_{k_{1},\cdots,k_{m}}$ is the projector which selects the state with $\sigma_{k\in\{k_{1},\cdots,k_{m}\}}=1$ and others $-1$:

$$\begin{split}\mathcal{P}_{k_{1},\cdots,k_{m}}=&\ket{\sigma_{k\in\{k_{1},\cdots,k_{m}\}}=1;\sigma_{k^{\prime}\notin\{k_{1},\cdots,k_{m}\}}=-1}\\ &\bra{\sigma_{k\in\{k_{1},\cdots,k_{m}\}}=1;\sigma_{k\notin\{k_{1},\cdots,k_{m}\}}=-1}\end{split}$$

(55)

These give a useful resolution of identity

$$\sum_{m=0}^{N}\mathcal{P}^{(m)}=I$$

(56)

by which the Eq.  47 as a diagonal operator can be readily written as combination of different groups:

$$\begin{split}\hat{K}&(\mathbf{w}^{\intercal}\vec{\sigma})=\sum_{\vec{\sigma}}\sum_{m=0}^{N}\sum_{k_{1}<\cdots<k_{m}}K_{m}(\mathbf{w})\\ &\times\delta_{\sigma_{k\in\{k_{1},\cdots,k_{m}\}},1}~{}\delta_{\sigma_{k\notin\{k_{1},\cdots,k_{m}\}},-1}\\ &\times\ket{\sigma_{k\in\{k_{1},\cdots,k_{m}\}}=1;\sigma_{k^{\prime}\notin\{k_{1},\cdots,k_{m}\}}=-1}\bra{\vec{\sigma}}\end{split}$$

(57)

where for convenience we have defined

$$K_{m}(\mathbf{w}^{\intercal}_{k_{j}})\equiv K\left(\sum_{j=1}^{m}\mathbf{w}_{k_{j}}^{\intercal}-\sum_{j=m+1}^{N}\mathbf{w}^{\intercal}_{k_{j}}\right)$$

(58)

Noting that $\sigma_{k}$ is binary and classical, we write the Kronecker delta in Eq. 57 as

$$\begin{split}\delta_{\sigma_{k\in}}&{}_{\{k_{1},\cdots,k_{m}\},1}~{}\delta_{\sigma_{k\notin\{k_{1},\cdots,k_{m}\}},-1}\\ &=\prod_{i=1}^{m}\left(\frac{1+\sigma_{k_{i}}}{2}\right)\prod_{k\notin\{k_{1},\cdots,k_{m}\}}\left(\frac{1-\sigma_{k}}{2}\right)\end{split}$$

(59)

Hence the diagonal terms of $\hat{K}$ given by the trace over $\sigma$ is

$$\begin{split}K(\mathbf{w}^{\intercal}\vec{\sigma})&=\sum_{m=0}^{N}\sum_{k_{1}<\cdots<k_{m}}K_{m}(\mathbf{w}^{\intercal}_{k_{j}})\\ &\times\prod_{i=1}^{m}\left(\frac{1+\sigma_{k_{i}}}{2}\right)\prod_{k\notin\{k_{1},\cdots,k_{m}\}}\left(\frac{1-\sigma_{k}}{2}\right)\end{split}$$

(60)

where we can expand the two products respectively. The first product in Eq. 60 is then written into:

$$\prod_{i=1}^{m}\left(\frac{1+\sigma_{k_{i}}}{2}\right)=\frac{1}{2^{m}}\sum_{q=0}^{m}\sum_{k_{1}<\cdots<k_{q}}\sigma_{k_{1}}\cdots\sigma_{k_{q}}$$

(61)

and the other product into:

$$\begin{split}\prod_{k\notin\{k_{1},\cdots,k_{m}\}}&\left(\frac{1-\sigma_{k}}{2}\right)=\frac{1}{2^{N-m}}\sum_{p=0}^{N-m}(-1)^{p}\\ &\times\left(\sum_{k_{m+1}<\cdots<k_{m+p}}\sigma_{k_{m+1}}\cdots\sigma_{k_{m+p}}\right)\end{split}$$

(62)

Then, combining them together we have the $K$ in the following form:

$$\begin{split}&K(\mathbf{w}^{\intercal}\vec{\sigma})=\sum_{m=0}^{N}\sum_{p=0}^{N-m}\sum_{\begin{subarray}{c}k_{1}<\cdots<k_{m};\\ k_{m+1}<\cdots<k_{m+p}\\ \neq(k_{1},\cdots,k_{m})\end{subarray}}K_{m}(\mathbf{w}^{\intercal}_{k_{j}})\\ &(-1)^{p}\left[\sum_{q=0}^{m}\left(\sum_{k_{1}<\cdots<k_{q}}\sigma_{k_{1}}\cdots\sigma_{k_{q}}\right)\sigma_{k_{m+1}}\cdots\sigma_{k_{m+p}}\right]\end{split}$$

(63)

where we have left out the constant prefactor $\frac{1}{2^{N}}$. Given the size of sample space $N$, each term in the above expression will be determined by a triplet $(m,q,p)$ where $q$ is upper bounded by $m$ and $p$ by $N-m$. This looks complicated and hard to rearrange. In order to write down an explicit energy function for $r$-th order of interaction, the above summation can be grouped according to different doublet $(p,q)$, which determines the order $r=p+q$. we replace the index $q$ by $q=r-p$, we have

$$\begin{split}&K^{(r)}(\mathbf{w}^{\intercal}\vec{\sigma})=\sum_{m=0}^{N}\sum_{p=0}^{N-m}\sum_{\begin{subarray}{c}k_{1}<\cdots<k_{m};\\ k_{m+1}<\cdots<k_{m+p}\\ \neq(k_{1},\cdots,k_{m})\end{subarray}}K_{m}(\mathbf{w}^{\intercal}_{k_{j}})\\ &(-1)^{p}\left[\sum_{\begin{subarray}{c}k_{1}<\cdots<k_{r-p};\\ k_{i}\in\{k_{1},\cdots,k_{m}\}\end{subarray}}\sigma_{k_{1}}\cdots\sigma_{k_{r-p}}\sigma_{k_{m+1}}\cdots\sigma_{k_{m+p}}\right]\end{split}$$

(64)

where the first $r-p$spins $\sigma_{k_{1}}\cdots\sigma_{k_{r-p}}$ in the product are attributed to the projection into $\sigma=+1$, i.e. from Eq. 61, thus are responsible for $+\mathbf{w}^{\intercal}_{k_{s_{i}}}$ in $K$, where the nested label $k_{s_{i}}$ ($1\leq i\leq r-p$) is for the $\sigma_{k_{j_{s}}}$ – the $r-p$ out of $r$spins that are chosen to be projected into $+1$; the spins $\sigma_{k_{m+1}}\cdots\sigma_{k_{m+p}}$ in the product are attributed to the projection into $\sigma=-1$, i.e. Eq. 62, thus are responsible for $-\mathbf{w}^{\intercal}_{k_{j_{i}}}$ in $K$, and we group these spins using labels $k_{j_{i}}$ with $p=|\{k_{j_{i}}\}|$; finally the rest $N-r$spins, though not in the $r$spins of interest, still contribute to energy (in contrast to the case of $\sigma\in\{1,0\}$) and are associated with $+\mathbf{w}^{\intercal}_{k_{l_{i}}}$ for $1\leq i\leq N-r$. Therefore, by rearranging indices it is straightforward to read out the interaction strength $\mathcal{I}^{(r)}_{k_{1},\cdots,k_{r}}$ between $r$spins $\sigma_{k_{1}}\cdots\sigma_{k_{r}}$:

$$\begin{split}\mathcal{I}^{(r)}_{k_{1},\cdots,k_{r}}=&\sum_{\begin{subarray}{c}\{k_{j_{i}}\},\{k_{s_{i}}\}\\ \subseteq\{k_{1},\cdots,k_{r}\}\end{subarray}}\sum_{\eta}(-1)^{p}K\Bigl{(}-\sum_{i=1}^{p}\mathbf{w}^{\intercal}_{k_{j_{i}}}\\ &+\sum_{i=1}^{r-p}\mathbf{w}^{\intercal}_{k_{s_{i}}}+\sum_{i=1}^{N-r}\eta^{k_{l_{i}}}\mathbf{w}^{\intercal}_{k_{l_{i}}}\Bigr{)}\end{split}$$

(65)

where in indeces are grouped according to

	$\displaystyle\{k_{j_{i}}\}\cap\{k_{s_{i}}\}=\emptyset,$		(66)
	$\displaystyle\{k_{j_{i}}\}\cup\{k_{s_{i}}\}=\{k_{1},\cdots,k_{r}\},$		(67)
	$\displaystyle k_{l_{i}}\in\{k_{r+1},\cdots,k_{N}\}$		(68)

and $\sum_{\eta}$ is the summation over all vectors $\eta$ of dimension $|\{k_{j_{l}}\}|=N-r$ whose elements $\eta^{k_{j_{l}}}$ are $\pm 1$ binaries. It is then easy to see the compact equivalent form of Eq. 65:

$$\mathcal{I}^{(r)}_{k_{1},\cdots,k_{r}}=\Tr_{\{\sigma_{1},\cdots\sigma_{N}\}}\left[\sigma_{k_{1}}\cdots\sigma_{k_{r}}K\left(\sum_{i}w_{ij}\sigma_{i}\right)\right]$$

(69)

which is consistent with Eq. 51. This is our central result for the RBM representation. We will test the strength of the formalism in the coming section under the context of $Z_{2}$ TO such as toric code that harbors non-local, many-body correlation in reduced density matrices. The construction for $\sigma\in\{0,1\}$ is presented in Appendix. A for other potential applications.