Geometry of Rényi Entropy on the Majorization Lattice

Abstract

Majorization is a stochastic ordering relation that compares the relative diversity of probability distributions with numerous applications in econometrics, spectral theory, and ecology. It is well-known that the majorization partial order forms a complete lattice on the set of ordered probability distributions. In this work, we study the properties of Rényi entropy on the majorization lattice. We establish a fundamental relation between the comonotone coupling and the independent coupling associated with a collection of marginal distributions. Consequently, we show that, for every order $\alpha \in [0,\infty ]$, the Rényi entropy is subadditive on the majorization lattice. We further characterize the supermodular regime, showing that Rényi entropy is supermodular on the majorization lattice for $\alpha \in \{0\}\cup [1,\infty ]$. For the Tsallis entropy, we show that it also satisfies subadditivity on the majorization lattice, for every order $\alpha \in [0,\infty )$. Finally, we show that, unlike the Rényi entropy, the Tsallis entropy is supermodular on the majorization lattice for every $\alpha \in [0,\infty )$.

20 pages, 2 figures