Generalized relative entropy

In the study of quantum information theory, one typically assumes that our experiments are independently repeated multiple times. Our measures of information processing are defined in this asymptotic limit. The quintessential entropy measure is the von Neumann entropy. In contrast, the study of one-shot quantum information theory is concerned with information processing when our experiment is only going to be conducted once. There is a need for new measures in this scenario as the von Neumann entropy ceases to give a correct characterization of operational quantities. Generalizations of the von Neumann entropy have been developed over the past few years. One particularly interesting measure is the $\epsilon$-relative entropy.

In the asymptotic scenario, the relative entropy acted as a parent quantity for other measures besides being an important measure itself. Similarly, the $\epsilon$-relative entropy functions as a parent quantity for other measures in the one-shot scenario.

Definitions

From an operational perspective, the $\epsilon$-relative entropy $D^{\epsilon }(\rho ||\sigma )$ is defined in the context of hypothesis testing. Here, we have to devise a strategy to distinguish between two density operators $\rho$ and $\sigma$. A strategy is defined by a POVM with elements $Q$ and $I-Q$. The probability that the strategy produces a correct guess on input $\rho$ is given by $Tr(\rho Q)$ and the probability that it produces a wrong guess is given by $Tr(\sigma Q)$. The $\epsilon$-relative entropy represents the negative logarithm of the probability of failure when the state is $\sigma$, given that the success probability for $\rho$ is at least $\epsilon$.

Definition: The $\epsilon$-relative entropy between two state operators $\rho$ and $\sigma$ is

$D^{\epsilon }(\rho ||\sigma )=-\log {\frac {1}{\epsilon }}\min\{\langle Q,\sigma \rangle |0\leq Q\leq I{\text{ and }}\langle Q,\rho \rangle \geq \epsilon \}$

From the definition, it is clear that $D^{\epsilon }(\rho ||\sigma )\geq 0$. This inequality is saturated if and only if $\rho =\sigma$ (Shown below).

Two relative min and max entropies defined as

$D_{\min }(\rho ||\sigma )=-\log(F(\rho ,\sigma )^{2})$

$D_{\max }(\rho ||\sigma )=\min\{\lambda :2^{\lambda }\sigma \geq \rho \}$

The corresponding smooth $\epsilon$-relative entropy is defined as

$D_{\min }^{\epsilon }(\rho ||\sigma )=\max _{{\bar {\rho }}\in {\mathcal {B}}_{\epsilon }(\rho )}D_{\min }({\bar {\rho }}||\sigma )$

$D_{\max }^{\epsilon }(\rho ||\sigma )=\min _{{\bar {\rho }}\in {\mathcal {B}}_{\epsilon }(\rho )}D_{\max }({\bar {\rho }}||\sigma )$

where the set ${\mathcal {B}}_{\epsilon }(\rho )=\{{\bar {\rho }}|P({\bar {\rho }},\rho )\leq \epsilon \}$ where ${\bar {\rho }}$ ranges over the set of all positive semi-definite operators with trace lesser than or equal to 1.

Relationship to the trace distance

This result and its proof can be found in Dupuis et. al. Suppose the trace distance between two density operators $\rho$ and $\sigma$ is

$||\rho -\sigma ||_{1}=\delta$

For $0<\epsilon <1$, it holds that

$\log {\frac {\epsilon }{\epsilon -(1-\epsilon )\delta }}\leq D^{\epsilon }(\rho ||\sigma )\leq \log {\frac {\epsilon }{\epsilon -\delta }}$.

In particular, this implies the following analogue of the Pinsker inequality

${\frac {1-\epsilon }{\epsilon }}\cdot ||\rho -\sigma ||_{1}\leq D^{\epsilon }(\rho ||\sigma )$.

Furthermore, the proposition implies that for $0<\epsilon <1$, $D^{\epsilon }(\rho ||\sigma )=0$ if and only if $\rho =\sigma$, inheriting this property from the trace distance.

Proof of the inequality

Upper bound: The trace distance can be written as

$||\rho -\sigma ||_{1}=\max _{0\leq Q\leq 1}Tr(Q(\rho -\sigma ))$

This is maximized when $Q$ is the projector along the positive part of $\rho -\sigma$.

$Tr(Q\sigma )=Tr(Q\rho )-\delta \geq \epsilon -\delta$.

This directly implies that

$2^{-D^{\epsilon }(\rho ||\sigma )}\geq {\frac {\epsilon -\delta }{\epsilon }}$

Lower bound: Choose ${\bar {Q}}$ such that it is the following convex combination of $I$ and $Q$

${\bar {Q}}=(\epsilon -\mu )I+(1-\epsilon +\mu )Q$

where $\mu \equiv {\frac {(1-\epsilon )Tr(Q\rho )}{1-Tr(Q\rho )}}$

This means $\mu =(1-\epsilon +\mu )Tr(Q\rho )$ and thus

$Tr({\bar {Q}}\rho )=(\epsilon -\mu )+(1-\epsilon +\mu )Tr(Q\rho )=\epsilon$

Moreover,

$Tr({\bar {Q}}\sigma )=\epsilon -\mu +(1-\epsilon +\mu )Tr(Q\sigma )=\epsilon -{\frac {(1-\epsilon )\delta }{1-Tr(Q\rho )}}\leq \epsilon -(1-\epsilon )\delta$

Hence

$D^{\epsilon }(\rho ||\sigma )\geq \log {\frac {\epsilon }{\epsilon -(1-\epsilon )\delta }}$.

Pinsker-like inequality: Observe that $\log {\frac {\epsilon }{\epsilon -(1-\epsilon )\delta }}=-\log \left(1-{\frac {(1-\epsilon )\delta }{\epsilon }}\right)\geq \delta {\frac {1-\epsilon }{\epsilon }}$

Data processing inequality

One of the properties that the $\epsilon$-relative entropy and the von Neumann entropy share is the strong subadditivity. Intuitively, this inequality states that the information content in a system cannot increase when one performs only local operations on that system. This inequality is also known as the data processing inequality.

For the case of the von Neumann entropy, the strong subadditivity can be expressed as follows: For a tripartite state $\rho _{ABC}$

$S(ABC)+S(B)\leq S(AB)+S(BC)$

Using the relative entropy, this inequality is equivalent to

$S(\rho ||\sigma )-S({\mathcal {E}}(\rho )||{\mathcal {E}}(\sigma ))\geq 0$

for every CPTP map ${\mathcal {E}}$. In the literature, this inequality is referred to as the monotonicity of the relative entropy under CPTP maps.

One of the interesting features of the $\epsilon$-relative entropy is that it too obeys the strong subadditivity. This can be expressed as

$D^{\epsilon }(\rho ||\sigma )-D^{\epsilon }({\mathcal {E}}(\rho )||{\mathcal {E}}(\sigma ))\geq 0$.

Not only is this interesting in itself, but it also allows us to prove the strong subadditivity of the von Neumann entropy in yet another way.

Monotonicity of $\epsilon$-relative entropy

Wang and Renner showed that the $\epsilon$-relative entropy is monotonic under partial trace. The proof goes as follows.

Suppose we had some POVM $(R,I-R)$ to distinguish between ${\mathcal {E}}(\rho )$ and ${\mathcal {E}}(\sigma )$. We can then construct a new POVM to distinguish between $\rho$ and $\sigma$ by preceeding the given POVM with the CPTP map ${\mathcal {E}}$. However,

$\langle R,{\mathcal {E}}(\sigma )\rangle =\langle {\mathcal {E}}^{\dagger }(R),\sigma \rangle$

$\geq \langle Q,\sigma \rangle$

because by definition, $Q$ was chosen optimally in the definition of the $\epsilon$-relative entropy.

Proof of the data processing inequality for the von Neumann entropy

This proof can be found in . From Stein's lemma , it follows that

$\lim _{n\rightarrow \infty }{\frac {1}{n}}D^{\epsilon }(\rho ^{\otimes n}||\sigma ^{\otimes n})=\lim _{n\rightarrow \infty }{\frac {-1}{n}}\log \min {\frac {1}{\epsilon }}Tr(\sigma ^{\otimes n}Q)$

$=D(\rho ||\sigma )-\lim _{n\rightarrow \infty }{\frac {1}{n}}\left(\log {\frac {1}{\epsilon }}\right)$

$=D(\rho ||\sigma )$

where the minimum is taken over $0\leq Q\leq 1$ such that $Tr(Q\rho )\geq \epsilon$.

Applying the data processing inequality to the states $\rho ^{\otimes n}$ and $\sigma ^{\otimes n}$ with the CPT map ${\mathcal {E}}^{\otimes n}$, we get

$D^{\epsilon }(\rho ^{\otimes n}||\sigma ^{\otimes n})\geq D^{\epsilon }({\mathcal {E}}(\rho )^{\otimes n}||{\mathcal {E}}(\sigma )^{\otimes n})$

Dividing by $n$ on either side and taking the limit, we get the desired result.

See Also

Strong subadditivity

Classical information theory

Min entropy