Some Properties of Rényi Entropy over Countably Infinite Alphabets

Abstract

In this paper we study certain properties of Rényi entropy functionals $H_{\alpha }(\mathcal{P})$ on the space of probability distributions over ${\mathbb{Z}}_{+}$. Primarily, continuity and convergence issues are addressed. Some properties shown parallel those known in the finite alphabet case, while others illustrate a quite different behaviour of Rényi entropy in the infinite case. In particular, it is shown that, for any distribution $\mathcal{P}$ and any $r\in [0,\infty ]$, there exists a sequence of distributions ${\mathcal{P}}_n$ converging to $\mathcal{P}$ with respect to the total variation distance, such that ${\lim }_{n\to \infty }{\lim }_{\alpha \to 1+}{H}_{\alpha }({\mathcal{P}}_n)={\lim }_{\alpha \to 1+}{\lim }_{n\to \infty }{H}_{\alpha }({\mathcal{P}}_n)+r$.

13 pages (single-column)

References in corpus (2)

• Rényi Divergence and Majorization
• The world according to Renyi: Thermodynamics of multifractal systems

Cited by in corpus (2)

• On the Entropy of Couplings
• Generalizations of Fano's Inequality for Conditional Information Measures via Majorization Theory