The Exercise

The relative entropy $S(\rho\|\sigma)$ is usually taken between two density operators, but its formula asks only that $\rho$ and $\sigma$ be positive and that the support of $\rho$ lie inside the support of $\sigma$ ; neither operator needs unit trace. For this wider relative entropy, show that subadditivity under sums of operators is equivalent to joint convexity under convex combinations. Find how $S(\rho\|\sigma)$ changes when $\rho$ and $\sigma$ are multiplied by positive numbers. Then bound the relative entropy of a mixture $\sum_i p_i r_i$ of positive operators to a mixture $\sum_i q_i s_i$ , with positive weights $p_i$ and $q_i$ , by the relative entropies of the terms and the weights.

Relative Entropy of Positive Operators

Throughout, $H$ is a finite-dimensional complex Hilbert space and logarithms are natural. The support of a positive operator $\rho$ on $H$ is the subspace of $H$ spanned by the eigenvectors of $\rho$ with nonzero eigenvalue.

The relative entropy of a positive operator $\rho$ on $H$ to a positive operator $\sigma$ on $H$ whose support contains that of $\rho$ is the real number $S(\rho\|\sigma) = \operatorname{tr}(\rho\log\rho) - \operatorname{tr}(\rho\log\sigma).$ (1) The classical relative entropy of two families of positive numbers $p_i$ and $q_i$ , indexed by the same finite set, is the real number $H(p\|q) = \sum_i p_i \log\frac{p_i}{q_i}.$ (2)

The Claim

Theorem 1. Consider positive operators $r_1, r_2, s_1, s_2$ on $H$ with the support of $r_1$ inside that of $s_1$ and the support of $r_2$ inside that of $s_2$ , and real numbers $t, u \ge 0$ with $t + u = 1$ . The relative entropy is subadditive, $S(r_1 + r_2 \,\|\, s_1 + s_2) \le S(r_1\|s_1) + S(r_2\|s_2),$ (3)^✦ for all such operators, if and only if it is jointly convex, $S(t r_1 + u r_2 \,\|\, t s_1 + u s_2) \le t\,S(r_1\|s_1) + u\,S(r_2\|s_2),$ (4)^✦ for all such operators and all such $t$ and $u$ .

Theorem 2. Let $\rho$ and $\sigma$ be positive operators on $H$ with the support of $\rho$ inside that of $\sigma$ , and let $c$ and $d$ be positive real numbers. Then $S(c\rho \,\|\, d\sigma) = c\,S(\rho\|\sigma) + c\operatorname{tr}(\rho)\log\frac{c}{d}.$ (5)^✦

Theorem 3. Let $r_i$ and $s_i$ , for $i$ in a finite nonempty index set, be positive operators on $H$ with the support of each $r_i$ inside that of $s_i$ , and let $p_i$ and $q_i$ be positive real numbers. Then $S\Bigl(\sum_i p_i r_i \mathrel{\Big\|} \sum_i q_i s_i\Bigr) \le \sum_i p_i\,S(r_i\|s_i) + \sum_i p_i \operatorname{tr}(r_i)\log\frac{p_i}{q_i}.$ (6)^✦ If moreover every $r_i$ has unit trace, then $S\Bigl(\sum_i p_i r_i \mathrel{\Big\|} \sum_i q_i s_i\Bigr) \le \sum_i p_i\,S(r_i\|s_i) + H(p\|q).$ (7)^✦

What Is Checked

The claims are stated over positive operators on a finite-dimensional complex Hilbert space, the support of each, real weights, and the relative entropy (1), joined in the last bound by the classical relative entropy (2) of the weights. About these the library proves the scaling law (5), each of subadditivity (3) and joint convexity (4) from the other, and the two bounds (6) and (7) on mixtures.