Observational entropy with general quantum priors

The above discussion suggests one possible generalization of OE, starting from the re-writing of (2) recently noticed by some of us [8]. Consider the measurement channel $\mathcal{M}$ associated to the measurement $\mathsf{M}$, defined as

$\displaystyle\mathcal{M}(\rho)\coloneqq\sum_{y}\mathrm{Tr}[\Pi_{y}\rho]|y\rangle\!\!\!\;\langle y|\;,$

(3)

where $\{\ket{y}\}$ is an arbitrary but fixed orthonormal basis of the system that records the measurement outcome. By further noticing that $V_{y}=d\operatorname{Tr}\!\left[\Pi_{y}u\right]$ with $u=\mathds{1}/d$ the maximally mixed state, one obtains

$\displaystyle\Sigma_{\mathsf{M}}(\rho)=D(\rho\|u)-D(\mathcal{M}(\rho)\|\mathcal{M}(u))\;,$

(4)

where

$\displaystyle D(\rho\|\sigma)\coloneqq\mathrm{Tr}[\rho(\ln\rho-\ln\sigma)]$

(5)

is the Umegaki quantum relative entropy between states $\rho$ and $\sigma$ [22, 23], which generalizes the relative entropy (a.k.a. Kullback-Leibler divergence)

$\displaystyle D(p\|q)\coloneqq\sum_{i}p_{i}\ln\frac{p_{i}}{q_{i}}$

(6)

between probability distributions $\{p_{i}\}$ and $\{q_{i}\}$ [24].

The expression (4) makes it clear that the quantity $\Sigma_{\mathsf{M}}(\rho)$ exactly equals the loss of distinguishability between the signal $\rho$ and the totally uniform background $u$ that occurs when the measurement $\mathsf{M}$ is used instead of the best possible measurement allowed by quantum theory. In statistical jargon, we thus say that $\Sigma_{\mathsf{M}}(\rho)$ measures the statistical deficiency of the measurement $\mathsf{M}$ in distinguishing $\rho$ against $u$.

This observation enlightens something implicit in the original definition (1) of OE: the coarse-graining is captured by the “volumes” $V_{y}=\operatorname{Tr}\!\left[\Pi_{y}\right]$ only because the maximally mixed state is chosen as the reference background. It is thus natural to try to incorporate more general references in the definition of OE. A direct generalization could be obtained, therefore, by replacing $u$ with another reference state $\gamma$ in (4), so that

	$\displaystyle S_{\mathsf{M},\gamma}(\rho)$	$\displaystyle\coloneqq S(\rho)+\Sigma_{\mathsf{M},\gamma}(\rho)$
		$\displaystyle\coloneqq S(\rho)+\mathfrak{D}(\rho\|\gamma)-D(\mathcal{M}(\rho)\|\mathcal{M}(\gamma))\,,$		(7)

where $\mathfrak{D}(\cdot\|\cdot)$ represents some non-commutative generalization of the Kullback–Leibler divergence, not necessarily Umegaki’s one.

There exists another evocative re-writing of (2), which in turn suggests that further structures may play a role in the definition of OE. Specifically, here we exhibit a dynamical interpretation of OE, based on a measurement process defined as follows.

Let $\rho=\sum_{x=1}^{d}\lambda_{x}|\psi_{x}\rangle\!\!\!\;\langle\psi_{x}|$ be a diagonal decomposition of the state of the system. We consider a stochastic map associated to a prepare-and-measure protocol: with probability $\lambda_{x}$, the state $\ket{\psi_{x}}$ is prepared, and it is then measured with the POVM $\mathsf{M}=\{\Pi_{y}\}_{y}$, yielding outcome $y$ with a probability given by the Born rule, that is

	$\displaystyle P_{F}(y|x)$	$\displaystyle\coloneqq\bra{\psi_{x}}\Pi_{y}\ket{\psi_{x}}\,,$		(8)
	$\displaystyle P_{F}(x,y)$	$\displaystyle\coloneqq\lambda_{x}\bra{\psi_{x}}\Pi_{y}\ket{\psi_{x}}\,.$		(9)

The subscript $F$ stands for “forward”. This is because, as we will see in what follows, the quantity $\Sigma_{\mathsf{M}}(\rho)$ in (2) emerges also from a comparison between the forward map defined above and a suitably defined “reverse” map.

Traditionally, the definition of the reverse process, given a forward process, relies on a detailed knowledge of the physical dynamics involved [25, 26, 27]. In the absence of such knowledge, which is typically the case for a system interacting with a complex environment, one must resort to physical intuition, plausibility arguments, or, failing that, arbitrary assumptions. In order to avoid all this, a systematic recipe has recently been found [28, 29], which allows to define the reverse process only from unavoidable rules of logical retrodiction: specifically, Jeffrey’s theory of probability kinematics [30] or, equivalently, Pearl’s virtual evidence method [31, 32].

The idea is as follows: given a forward conditional probability $P_{F}(y|x)$, how should information, obtained at later times about the final outcome and encoded in an arbitrary distribution $q_{y}$, be propagated back to the initial state in a way that ensures logical consistency? Jeffrey’s theory of probability kinematics, which is equivalent to Pearl’s virtual evidence method [31, 32], stipulates that the only logically consistent back-propagation rule is what is now known as Jeffrey’s update: starting from an arbitrarily chosen reference prior on the initial state $x$, say $\gamma_{x}$, one constructs the Bayesian inverse of $P_{F}(y|x)$, i.e.,

$\displaystyle P_{R}^{\gamma}(x|y):=\frac{\gamma_{x}P_{F}(y|x)}{\sum_{x^{\prime}}\gamma_{x^{\prime}}P_{F}(y|{x^{\prime}})}\,,$

(10)

and uses that as a stochastic channel to back-propagate the new information $q_{y}$ from the final outcome $y$ to the initial state $x$, so that as the reverse process we obtain

$\displaystyle P_{R}^{\gamma}(y,x)=q_{y}\,P_{R}^{\gamma}(x|y)\;.$

(11)

Jeffrey’s update constitutes a generalization of Bayes’ theorem, as the latter is recovered as a special case of the former when $q_{y}=\delta_{y,y_{0}}$, i.e., when the information about the final outcome is definite [32].

An important point to emphasize here is that the reference prior $\gamma_{x}$ used to construct the retrodictive channel in (10) is merely a formal device needed to establish a mathematical correspondence between forward and backward process: it need not be related in any way with the “true” distribution $\lambda_{x}$. Likewise, the distribution $q_{y}$ represents new and completely arbitrary information, which need not correspond to any input distribution under $P_{F}$, that is, there may exist no distribution $q^{\prime}_{x}$ such that $q_{y}=\sum_{x}P_{F}(y|x)q^{\prime}_{x}$. Moreover, in principle, $q_{y}$ may also be incompatible with the reference prior $\gamma_{x}$, in the sense that it could happen that, for some $y$, $q_{y}>0$ but $\sum_{x}\gamma_{x}P_{F}(y|x)=0$. In such a situation, one would conclude that the data falsify the inferential model, but for simplicity we will avoid such cases by assuming that all probabilities are strictly greater than zero (though possibly arbitrarily small).

We now go back to our specific forward process (9), i.e., $P_{F}(y|x)=\bra{\psi_{x}}\Pi_{y}\ket{\psi_{x}}$. If we choose as reference the uniform distribution $\gamma=u$, i.e., $\gamma_{x}=1/d$ for all $x$, and as new information information the outcomes’ expected probability of occurrence, i.e., $q_{y}=p_{y}=\mathrm{Tr}[\rho\Pi_{y}]$, by direct substitution in (10) and (11), we obtain

$\displaystyle P^{u}_{R}(y,x)=p_{y}\bra{\psi_{x}}\frac{\Pi_{y}}{\operatorname{Tr}\!\left[\Pi_{y}\right]}\ket{\psi_{x}}\;.$

(12)

The above can also be read as a prepare-and-measure process, in which the state $\sigma_{y}\coloneqq\frac{\Pi_{y}}{\operatorname{Tr}\!\left[\Pi_{y}\right]}$ is prepared with probability $p_{y}$, and later measured in the basis $\{\ket{\psi_{x}}\}$. The process in (12) is the process that a retrodictive agent would infer, knowing only the forward process (9) and the outcome distribution $p_{y}$, but completely ignoring the actual distribution $\lambda_{x}$, so that the latter is replaced by the uniform distribution.

Using (9) and (12), it is straightforward to check that

$\displaystyle\Sigma_{\mathsf{M}}(\rho)=D(P_{F}\|P^{u}_{R})\;.$

(13)

The above relation suggests an alternative interpretation for the difference $\Sigma_{\mathsf{M}}(\rho)$, as the degree of statistical distinguishability between a predictive process, i.e., $P_{F}(x,y)$, and a retrodictive process constructed from a uniform reference, i.e., $P_{R}^{u}(y,x)$. Thus, the larger $\Sigma_{\mathsf{M}}(\rho)$, the more irretrodictable the process becomes [33, 34].

Eq. (13) also offers an alternative way of thinking about generalizations of OE, where the uniform reference is again replaced by an arbitrary state, as was done in Section 2.1, but this time for the purpose of constructing another reverse process. That is, one could also consider generalizations such as

$\displaystyle\widetilde{S}_{\mathsf{M},\gamma}(\rho)\coloneqq S(\rho)+\mathfrak{D}(Q_{F}\|Q_{R}^{\gamma})\;,$

(14)

where $\mathfrak{D}$ is again some quantum relative entropy (not necessarily Umegaki’s), $Q_{F}$ is an input-output description of the quantum process consisting in preparing the state $\rho$ and measuring it with the channel $\mathcal{M}$, and $Q_{R}^{\gamma}$ is the description of the corresponding reverse process computed with respect to the reference prior $\gamma$. All these ingredients will be rigorously defined in Section 4.

Let $A$ and $B$ respectively denote the input and output systems of the measurement channel $\mathcal{M}$ in Eq. (3). The general recipe for the retrodiction of a quantum process $\mathcal{M}$ [28, 8, 35] is defined via its Petz recovery map [36, 37] as

$\displaystyle\widetilde{\mathcal{M}}^{\gamma}(\tau)\coloneqq\gamma^{1/2}\mathcal{M}^{\dagger}(\mathcal{M}(\gamma)^{-1/2}\tau\mathcal{M}(\gamma)^{-1/2})\gamma^{1/2}$

(22)

where $\tau:=\sum_{y}q_{y}|y\rangle\!\!\!\;\langle y|$ encodes the distribution $\{q_{y}\}$, describing the retrodictor’s knowledge, cfr. Eq. (11). We will mainly discuss the case where $\tau=\mathcal{M}(\rho)$, namely $q_{y}=p_{y}=\mathrm{Tr}[\Pi_{y}\rho]$. For the measurement channel given in (3), the Petz recovery map can be written as

$\displaystyle\widetilde{\mathcal{M}}^{\gamma}(\tau)=\sum_{y}\frac{\bra{y}\tau\ket{y}}{\mathrm{Tr}[\Pi_{y}\gamma]}\sqrt{\gamma}\Pi_{y}\sqrt{\gamma}\,.$

(23)

As an ingredient for later constructions, we introduce the Choi operator [38], defined for the process $\mathcal{M}$ from system $A$ to system $B$ as

$\displaystyle C_{\mathcal{M}}$

$\displaystyle\coloneqq\sum_{i,j}\mathcal{M}(\ket{i}\bra{j})\otimes\ket{i}\bra{j}\,,$

(24)

where $\ket{i}$ and $\ket{j}$ belong to an arbitrary but fixed orthonormal basis of the input Hilbert space $\mathcal{H}_{A}$ of system $A$. The reverse process has the following Choi operator

$\displaystyle C_{\widetilde{\mathcal{M}}^{\gamma}}$

$\displaystyle\coloneqq\sum_{k,l}\ket{k}\bra{l}\otimes\widetilde{\mathcal{M}}^{\gamma}(\ket{k}\bra{l})\,,$

(25)

where $\ket{k}$ and $\ket{l}$ belong to an arbitrarily fixed orthonormal basis of the Hilbert space $\mathcal{H}_{B}$. Note that we put system $B$ first and system $A$ second, in order to have the same ordering of systems for both $C_{\mathcal{M}}$ and $C_{\widetilde{\mathcal{M}}^{\gamma}}$.

With such a definition, the Choi operators of the forward and reverse processes are related by the following lemma (proof in Appendix A):

We now want to construct two objects, $Q_{F}$ and $Q_{R}^{\gamma}$, which, analogously to the joint distributions $P_{F}$ and $P^{\gamma}_{R}$, are able to capture both the input and output of the forward and reverse processes. Specifically, the marginals of the operator $Q_{F}$ should recover the input state $\rho$ and the output $\mathcal{M}(\rho)$ respectively, and analogously for $Q_{R}^{\gamma}$.

One choice is to define

$\displaystyle Q_{F}:=(\mathds{1}_{B}\otimes\sqrt{\rho^{T}})C_{\mathcal{M}}(\mathds{1}_{B}\otimes\sqrt{\rho^{T}})\,.$

(27)

Such an operator is indeed able to capture the input and output of the forward process, in the sense that:

$\displaystyle\mathrm{Tr}_{A}[Q_{F}]=\mathcal{M}(\rho),\quad\mathrm{Tr}_{B}[Q_{F}]=\rho^{T}\,.$

(28)

We define the representation for the reverse process $\widetilde{\mathcal{M}}^{\gamma}$ similarly as

$\displaystyle Q_{R}^{\gamma}$

$\displaystyle:=\left(\sqrt{\tau}\otimes\mathds{1}_{A}\right)C_{\widetilde{\mathcal{M}}^{\gamma}}^{T}\left(\sqrt{\tau}\otimes\mathds{1}_{A}\right)\,,$

(29)

where $\tau=\mathcal{M}(\rho)$ is the input of the reverse process. We use the transpose of the Choi operator of the reverse process so that it can be linked to $C_{\mathcal{M}}$ by Lemma 1 in the following way:

	$\displaystyle Q_{R}^{\gamma}=$
	$\displaystyle\left(\sqrt{\tau}\sqrt{\mathcal{M}{(\gamma)}}^{-1}\otimes\sqrt{\gamma^{T}}\right)C_{\mathcal{M}}\left(\sqrt{\mathcal{M}{(\gamma)}}^{-1}\sqrt{\tau}\otimes\sqrt{\gamma^{T}}\right)\,.$

This operator captures the input and output of the reverse process:

$\displaystyle\mathrm{Tr}_{A}[Q_{R}^{\gamma}]=\tau,\quad\mathrm{Tr}_{B}[Q_{R}^{\gamma}]=[\widetilde{\mathcal{M}}^{\gamma}(\tau)]^{T}\,.$

(30)

The operators $Q_{F},Q_{R}^{\gamma}$ just defined are analogous to the state over time proposed by Leifer and Spekkens [39, 40] up to a partial transpose.

Other definitions of input-output operators may satisfy nice properties. An alternative choice is, for example,

$\displaystyle{}^{t}Q_{F}:=\sqrt{C_{\mathcal{M}}}(\mathds{1}_{B}\otimes\rho^{T})\sqrt{C_{\mathcal{M}}}$

(31)

and

		$\displaystyle{}^{t}Q_{R}^{\gamma}:=$		(32)
		$\displaystyle\sqrt{C_{\mathcal{M}}}(\mathcal{M}(\gamma)^{-1/2}\tau\mathcal{M}(\gamma)^{-1/2}\otimes\gamma^{T})\sqrt{C_{\mathcal{M}}}\;.$

The superscript ${}^{t}\bullet$ in (31) and (32) (not to be confused with $\bullet^{T}$) is used because the operators ${}^{t}Q_{F}$ and ${}^{t}Q_{R}^{\gamma}$ are, in a loose sense, a “transposition” of $Q_{F}$ and $Q_{F}^{\gamma}$, respectively. If $\Pi_{y},\rho,\gamma$ do not commute, in general ${}^{t}Q_{F}\neq Q_{F}$ and ${}^{t}Q_{R}^{\gamma}\neq Q_{R}^{\gamma}$. For example, $\mathrm{Tr}_{B}[{}^{t}Q_{F}]=(\sum_{y}\sqrt{\Pi_{y}}\rho\sqrt{\Pi_{y}})^{T}$ which in general differs from $\rho^{T}$. Yet, they are similar, in the sense that $Q_{F}$ and ${}^{t}Q_{F}$ (resp. $Q_{R}^{\gamma}$ and ${}^{t}Q_{R}^{\gamma}$) share the same eigenvalues, and are thus unitarily equivalent, as it happens when doing a proper transposition. Therefore, ${}^{t}Q_{F}$ and ${}^{t}Q_{R}^{\gamma}$ can be viewed as legitimate representations (up to unitaries) of the forward and reverse processes, and they will be useful in the irretrodictability interpretation of OE.

Eqs. (7) and (14) provide two forms of the observational entropy: Eq. (7), arising from the statistical deficiency approach, is the difference between relative entropies evaluated on the input system and the output system; Eq. (14), arising from the irretrodictability approach, is the relative entropy between the forward and reverse processes. In the remainder of this section, we will propose generalizations of OE that take either or both of these forms.

In the presence of a Hamiltonian $H=\sum_{n=0}^{d-1}E_{n}|n\rangle\!\!\!\;\langle n|$, a very natural choice of non-uniform prior is the Gibbs state

$\displaystyle\gamma:=e^{-\beta H}/\mathrm{Tr}[e^{-\beta H}],\leavevmode\nobreak\ \beta>0.$

(45)

We also consider the measurement in the energy eigenbasis $\mathsf{M}:=\{|0\rangle\!\!\!\;\langle 0|,\dots,|d-1\rangle\!\!\!\;\langle d-1|\}$, but to move far away from the classical case we assume that the input state is pure and maximally unbiased with the energy eigenbasis:

$\displaystyle\rho:=|\psi\rangle\!\!\!\;\langle\psi|$

$\displaystyle\textrm{ with }\ket{\psi}:=\frac{1}{\sqrt{d}}\sum_{n=0}^{d-1}\ket{n}.$

(46)

With these assumptions, the first definition yields

$\displaystyle S_{\mathsf{M},\gamma}^{(1)}(\rho)$

$\displaystyle=S_{\mathsf{M}}(\rho)=\ln d\,,$

(47)

which is also the case if $\rho$ is a mixture of maximally unbiased states. Like $S^{\textrm{clax}}_{\mathsf{M},\gamma}$, $S_{\mathsf{M},\gamma}^{(1)}$ reduces to the original $S_{\mathsf{M}}$ when the prior is a convex sum of the measurement elements.

The second definition yields

$\displaystyle S_{\mathsf{M},\gamma}^{(2)}(\rho)$

$\displaystyle=\infty\,,$

(48)

for any pure state, since the support of $Q_{R}^{\gamma}=\frac{1}{d}\sum_{n}|n\rangle\!\!\!\;\langle n|\otimes|n\rangle\!\!\!\;\langle n|$ does not contain the support of $Q_{F}=\frac{1}{d}\mathds{1}\otimes|\psi\rangle\!\!\!\;\langle\psi|$. We shall comment on this result after the next example.

Finally, the third definition yields

	$\displaystyle S_{\mathsf{M},\gamma}^{(3)}(\rho)$	$\displaystyle=\ln\mathrm{Tr}[\gamma^{-1}]+\frac{1}{d}\mathrm{Tr}[\ln\gamma]$		(49)
		$\displaystyle=\ln\frac{1-e^{\beta\omega d}}{1-e^{\beta\omega}}-\frac{\beta\omega(d-1)}{2}\,.$		(50)

where the first line is general, while the second is the expression for equidistant spectrum $E_{n}=n\omega$. Thus $S_{\mathsf{M},\gamma}^{(3)}$ is more sensitive than $S_{\mathsf{M},\gamma}^{(1)}$ to quantum situations.

The following example is inspired by a simple error-correcting code, the three-qubit encoding of a pure qubit:

$\displaystyle\alpha\ket{0}+\beta\ket{1}\mapsto\alpha\ket{000}+\beta\ket{111},\leavevmode\nobreak\ |\alpha|^{2}+|\beta|^{2}=1\,.$

(51)

Suppose $\rho$ is the encoded state

$\displaystyle\rho:=(\alpha\ket{000}+\beta\ket{111})(\alpha^{*}\bra{000}+\beta^{*}\bra{111})\,.$

(52)

We consider the measurement of each qubit in the $\{\ket{+},\ket{-}\}$ basis, i.e. the POVM elements are projectors on the basis vectors

$\displaystyle\{\ket{+++},\ket{++-},\ket{+-+},\dots,\ket{---}|\}\,.$

(53)

As for the prior, we suppose that the observer knows the encoding of the error correction code, and expect $\rho$ to be more probably in the subspace spanned by $\ket{000}$ and $\ket{111}$, possibly with a bias towards one of those product states; whence

	$\displaystyle\gamma$	$\displaystyle:=p_{0}|000\rangle\!\!\!\;\langle 000|+p_{1}|111\rangle\!\!\!\;\langle 111|$
		$\displaystyle\leavevmode\nobreak\ +\frac{1-p_{0}-p_{1}}{6}(\mathds{1}-|000\rangle\!\!\!\;\langle 000|-|111\rangle\!\!\!\;\langle 111|)\,,$		(54)
		$\displaystyle p_{0},p_{1}>0,\leavevmode\nobreak\ p_{0}+p_{1}<1\,.$

In this case, the three definitions proposed here yield

	$\displaystyle S_{\mathsf{M},\gamma}^{(1)}(\rho)$	$\displaystyle=|\alpha|^{2}\ln\frac{1}{p_{0}}+|\beta|^{2}\ln\frac{1}{p_{1}}-D(\mathcal{M}(\rho)\|\mathcal{M}(\gamma))\,,$		(55)
	$\displaystyle S_{\mathsf{M},\gamma}^{(2)}(\rho)$	$\displaystyle=\infty\,,$		(56)
	$\displaystyle S_{\mathsf{M},\gamma}^{(3)}(\rho)$	$\displaystyle=\ln\left(\frac{|\alpha|^{2}}{p_{0}}+\frac{|\beta|^{2}}{p_{1}}\right)-D(\mathcal{M}(\rho)\|\mathcal{M}(\gamma))$		(57)

with

	$\displaystyle D(\mathcal{M}(\rho)\|\mathcal{M}(\gamma))$
	$\displaystyle=|\alpha+\beta|^{2}\ln|\alpha+\beta|+|\alpha-\beta|^{2}\ln|\alpha-\beta|\,.$		(58)

$S_{\mathsf{M},\gamma}^{(1)}$ and $S_{\mathsf{M},\gamma}^{(3)}$ differ in the first term, as long as $p_{0}\neq p_{1}$: for $p_{0}=p_{1}=p$, both yield $\ln(1/p)$. In particular, when $p=\frac{1}{8}$, $\gamma=u$ and therefore $S_{\mathsf{M},\gamma}^{(1)}(\rho)=S_{\mathsf{M},\gamma}^{(3)}(\rho)=S_{\mathsf{M}}(\rho)$. We see that $S_{\mathsf{M},\gamma}^{(2)}$ is still infinite, for the same reason of support mismatch as in the previous example. From the examples, we observe that $S_{\mathsf{M},\gamma}^{(2)}$ is often overly sensitive to the non-commutativity between $\rho$ and $\gamma$. This suggests that, instead of the natural choice of $Q_{F}$ and $Q_{R}^{\gamma}$ as input-output representations, one could opt for representations whose supports are more aligned, such as ${}^{t}Q_{F}$ and ${}^{t}Q_{R}^{\gamma}$, which relate to $S_{\mathsf{M},\gamma}^{(3)}$ via Eq. (37).