The Exercise

An experiment has finitely many outcomes $x$, and two probability distributions $p$ and $q$ assign them the probabilities $p_x$ and $q_x$; every outcome that is impossible under $q$ is impossible under $p$ as well. The relative entropy $D(p\|q)$ of $p$ to $q$ is then zero exactly when the two distributions are the same, $p_x = q_x$ for every outcome.^✦

Relative Entropy

The relative entropy $D(p\|q)$ of two families $p = (p_x)$ and $q = (q_x)$ of real numbers on a finite set $X$ is the real number $$D(p\|q) = \sum_{x \in X} p_x \left( \log p_x - \log q_x \right),$$(1)^✦ with $\log$ the natural logarithm.

The Claim

The library states the theorem for families of nonnegative numbers with the same total.

For probability distributions, whose totals are both $1$, this is the exercise.

What Is Checked

The claim is stated over the relative entropy (1) of two real families on a finite set, and what the library proves about it is the equivalence (2) of the theorem above.