Generalized relative entropy

The generalized relative entropy is a distance measure between two quantum states. As the name suggests, it is a generalization of the quantum relative entropy. It shares many properties with the quantum relative entropy. For instance, it is non-negative and obeys strong sub-additivity.

In the study of quantum information theory, one typically assumes that our experiments are independently repeated multiple times. Our measures of information processing are defined in this asymptotic limit. The quintessential entropy measure is the von Neumann entropy. In contrast, the study of one-shot quantum information theory is concerned with information processing when our experiment is only going to be conducted once. There is a need for new measures in this scenario as the von Neumann entropy ceases to give a correct characterization of operational quantities. Generalizations of the von Neumann entropy have been developed over the past few years. One particularly interesting measure is the $\epsilon$-relative entropy.

In the asymptotic scenario, the relative entropy acted as a parent quantity for other measures besides being an important measure itself. Similarly, the $\epsilon$-relative entropy functions as a parent quantity for other measures in the one-shot scenario.

Definitions

To motivate the definition of the $\epsilon$-relative entropy $D^{\epsilon }(\rho ||\sigma )$, we consider the information processing protocol of hypothesis testing. In hypothesis testing, we have to devise a strategy to distinguish between two density operators $\rho$ and $\sigma$. A strategy is a POVM with elements $Q$ and $I-Q$. The probability that the strategy produces a correct guess on input $\rho$ is given by $Tr(\rho Q)$ and the probability that it produces a wrong guess is given by $Tr(\sigma Q)$. The $\epsilon$-relative entropy is the negative logarithm of the minimum probability of failure when the state is $\sigma$, given that the success probability for $\rho$ is at least $\epsilon$.

Definition: For $0\leq \epsilon 1$, the $\epsilon$-relative entropy between two state operators $\rho$ and $\sigma$ is

$D^{\epsilon }(\rho ||\sigma )=-\log {\frac {1}{\epsilon }}\min\{\langle Q,\sigma \rangle |0\leq Q\leq I{\text{ and }}\langle Q,\rho \rangle \geq \epsilon \}$

From the definition, it is clear that $D^{\epsilon }(\rho ||\sigma )\geq 0$. This inequality is saturated if and only if $\rho =\sigma$ (Shown below).

Two relative min and max entropies defined as

$D_{\min }(\rho ||\sigma )=-\log(F(\rho ,\sigma )^{2})$

$D_{\max }(\rho ||\sigma )=\min\{\lambda :2^{\lambda }\sigma \geq \rho \}$

where $F(\rho ,\sigma )$ denotes the fidelity between $\rho$ and $\sigma$. Intuitively, the quantity $D_{\max }$ can be interpreted as follows. For some fixed measurement that we perform on $\rho$ and $\sigma$, the probability of detecting $\sigma$ is $2^{\lambda }$ times more likely than the probability of detecting $\rho$.

The corresponding smooth $\epsilon$-relative entropy is defined as

$D_{\min }^{\epsilon }(\rho ||\sigma )=\max _{{\bar {\rho }}\in {\mathcal {B}}_{\epsilon }(\rho )}D_{\min }({\bar {\rho }}||\sigma )$

$D_{\max }^{\epsilon }(\rho ||\sigma )=\min _{{\bar {\rho }}\in {\mathcal {B}}_{\epsilon }(\rho )}D_{\max }({\bar {\rho }}||\sigma )$

where the set ${\mathcal {B}}_{\epsilon }(\rho )=\{{\bar {\rho }}|P({\bar {\rho }},\rho )\leq \epsilon \}$ where ${\bar {\rho }}$ ranges over the set of all positive semi-definite operators with trace lesser than or equal to 1. These quantities are discussed in detail by Dutta.

Relationship to the trace distance

Suppose the trace distance between two density operators $\rho$ and $\sigma$ is

$||\rho -\sigma ||_{1}=\delta$

For $0<\epsilon <1$, it holds that

a) $\log {\frac {\epsilon }{\epsilon -(1-\epsilon )\delta }}\quad \leq \quad D^{\epsilon }(\rho ||\sigma )\quad \leq \quad \log {\frac {\epsilon }{\epsilon -\delta }}$

In particular, this implies the following analogue of the Pinsker inequality

b) ${\frac {1-\epsilon }{\epsilon }}\cdot ||\rho -\sigma ||_{1}\quad \leq \quad D^{\epsilon }(\rho ||\sigma )$.

Furthermore, the proposition implies that for $0<\epsilon <1$, $D^{\epsilon }(\rho ||\sigma )=0$ if and only if $\rho =\sigma$, inheriting this property from the trace distance. This result and its proof can be found in Dupuis et. al.

Proof of the inequality

Proof of inequality a):

Upper bound: The trace distance can be written as

$||\rho -\sigma ||_{1}=\max _{0\leq Q\leq 1}Tr(Q(\rho -\sigma ))$

This maximum is achieved when $Q$ is the projector onto the positive part of $\rho -\sigma$. For any $Q$ such that $Tr(Q\rho )\geq \epsilon$ we have

$Tr(Q(\rho -\sigma ))\leq \delta$

$Tr(Q\sigma )\geq Tr(Q\rho )-\delta \geq \epsilon -\delta$.

From the definition of the $\epsilon$-relative entropy, we get

$2^{-D^{\epsilon }(\rho ||\sigma )}\geq {\frac {\epsilon -\delta }{\epsilon }}$

Lower bound: Choose ${\bar {Q}}$ such that it is the following convex combination of $I$ and $Q$

${\bar {Q}}=(\epsilon -\mu )I+(1-\epsilon +\mu )Q$

where $\mu \equiv {\frac {(1-\epsilon )Tr(Q\rho )}{1-Tr(Q\rho )}}$

This means $\mu =(1-\epsilon +\mu )Tr(Q\rho )$ and thus

$Tr({\bar {Q}}\rho )=(\epsilon -\mu )+(1-\epsilon +\mu )Tr(Q\rho )=\epsilon$

Moreover,

$Tr({\bar {Q}}\sigma )=\epsilon -\mu +(1-\epsilon +\mu )Tr(Q\sigma )$

We can substitute the above expression for $\mu$ to re-write this as

$\epsilon -{\frac {(1-\epsilon +\mu )Tr(Q\rho )}{1-Tr(Q\rho )}}+(1-\epsilon +\mu )Tr(Q\sigma )$

$\epsilon -{\frac {(1-\epsilon )\delta }{1-Tr(Q\rho )}}\leq \epsilon -(1-\epsilon )\delta$

Hence

$D^{\epsilon }(\rho ||\sigma )\geq \log {\frac {\epsilon }{\epsilon -(1-\epsilon )\delta }}$.

Proof of inequality (b):

Pinsker-like inequality: Observe that $\log {\frac {\epsilon }{\epsilon -(1-\epsilon )\delta }}=-\log \left(1-{\frac {(1-\epsilon )\delta }{\epsilon }}\right)\geq \delta {\frac {1-\epsilon }{\epsilon }}$

Data processing inequality

One of the properties that the $\epsilon$-relative entropy and the von Neumann entropy share is strong subadditivity. Intuitively, this inequality states that the information content in a system cannot increase when one performs only local operations on that system. This inequality is also known as the data processing inequality.

For the case of the von Neumann entropy, the strong subadditivity can be expressed as follows: Let $\rho _{ABC}$ be a density operator on the tensor product Hilbert space ${\mathcal {H}}_{A}\otimes {\mathcal {H}}_{B}\otimes {\mathcal {H}}_{C}$,

$S(\rho _{ABC})+S(\rho _{B})\leq S(\rho _{AB})+S(\rho _{BC})$

where $\rho _{AB},\rho _{BC},\rho _{B}$ refer to the marginals. Using the relative entropy, this inequality is can equivalently be expressed as

$S(\rho ||\sigma )-S({\mathcal {E}}(\rho )||{\mathcal {E}}(\sigma ))\geq 0$

for every CPTP map ${\mathcal {E}}$. In the literature, this inequality is referred to as the monotonicity of the relative entropy under CPTP maps.

The $\epsilon$-relative entropy also obeys strong subadditivity. This can be expressed as

$D^{\epsilon }(\rho ||\sigma )\geq D^{\epsilon }({\mathcal {E}}(\rho )||{\mathcal {E}}(\sigma ))$.

Not only is this interesting in itself, but it also allows us to prove the strong subadditivity of the von Neumann entropy in yet another way.

Monotonicity of $\epsilon$-relative entropy

Wang and Renner showed that the $\epsilon$-relative entropy is monotonic under partial trace. The proof goes as follows.

Suppose we had some POVM $(R,I-R)$ to distinguish between ${\mathcal {E}}(\rho )$ and ${\mathcal {E}}(\sigma )$. We can then construct a new POVM to distinguish between $\rho$ and $\sigma$ by preceeding the given POVM with the CPTP map ${\mathcal {E}}$. However,

$\langle R,{\mathcal {E}}(\sigma )\rangle =\langle {\mathcal {E}}^{\dagger }(R),\sigma \rangle$

$\geq \langle Q,\sigma \rangle$

because by definition, $Q$ was chosen optimally in the definition of the $\epsilon$-relative entropy.

Proof of the data processing inequality for the von Neumann entropy

This proof can be found in . From Stein's lemma , it follows that

$\lim _{n\rightarrow \infty }{\frac {1}{n}}D^{\epsilon }(\rho ^{\otimes n}||\sigma ^{\otimes n})=\lim _{n\rightarrow \infty }{\frac {-1}{n}}\log \min {\frac {1}{\epsilon }}Tr(\sigma ^{\otimes n}Q)$

$=D(\rho ||\sigma )-\lim _{n\rightarrow \infty }{\frac {1}{n}}\left(\log {\frac {1}{\epsilon }}\right)$

$=D(\rho ||\sigma )$

where the minimum is taken over $0\leq Q\leq 1$ such that $Tr(Q\rho )\geq \epsilon$.

Applying the data processing inequality to the states $\rho ^{\otimes n}$ and $\sigma ^{\otimes n}$ with the CPT map ${\mathcal {E}}^{\otimes n}$, we get

$D^{\epsilon }(\rho ^{\otimes n}||\sigma ^{\otimes n})\geq D^{\epsilon }({\mathcal {E}}(\rho )^{\otimes n}||{\mathcal {E}}(\sigma )^{\otimes n})$

Dividing by $n$ on either side and taking the limit, we get the desired result.

See Also

Quantum relative entropy

Strong subadditivity

Classical information theory

Min entropy