Rényi Divergence and Kullback-Leibler Divergence

Tim van Erven, Peter Harremoës

Rényi divergence is related to Rényi entropy much like Kullback-Leibler divergence is related to Shannon's entropy, and comes up in many settings. It was introduced by Rényi as a m…