1.6k citations · 1.9k across the 13 of their papers we have counts for
7 papers · 1 filter
Extendable MDL
Peter Harremoës
In this paper we show that combination of the minimum description length principle and a exchange-ability condition leads directly to the use of Jeffreys prior. This approach works…
Minimum KL-divergence on complements of balls
Daniel Berend, Peter Harremoës, Aryeh Kontorovich
Pinsker's widely used inequality upper-bounds the total variation distance in terms of the Kullback-Leibler divergence . Although in general a bound in the rev…
Rényi Divergence and Kullback-Leibler Divergence
Tim van Erven, Peter Harremoës
Rényi divergence is related to Rényi entropy much like Kullback-Leibler divergence is related to Shannon's entropy, and comes up in many settings. It was introduced by Rényi as a m…
On Pairs of -divergences and their Joint Range
Peter Harremoës, Igor Vajda
We compare two f-divergences and prove that their joint range is the convex hull of the joint range for distributions supported on only two points. Some applications of this result…
Rényi Divergence and Majorization
Tim van Erven, Peter Harremoës
Rényi divergence is related to Rényi entropy much like information divergence (also called Kullback-Leibler divergence or relative entropy) is related to Shannon's entropy, and com…
Joint Range of f-divergences
Peter Harremoës, Igor Vajda
We provide a general method for evaluation of the joint range of f-divergences for two different functions f. Via topological arguments we prove that the joint range for general di…
Maximum Entropy on Compact Groups
Peter Harremoes
On a compact group the Haar probability measure plays the role of uniform distribution. The entropy and rate distortion theory for this uniform distribution is studied. New results…