1.6k citations · 1.9k across the 13 of their papers we have counts for
17 papers
Bounds on the Information Divergence for Hypergeometric Distributions
Peter Harremoës, František Matúš
The hypergeometric distributions have many important applications, but they have not had sufficient attention in information theory. Hypergeometric distributions can be approximate…
Statistical Inference and Exact Saddle Point Approximations
Peter Harremoës
Statistical inference may follow a frequentist approach or it may follow a Bayesian approach or it may use the minimum description length principle (MDL). Our goal is to identify s…
Entropy on Spin Factors
Peter Harremoës
Recently it has been demonstrated that the Shannon entropy or the von Neuman entropy are the only entropy functions that generate a local Bregman divergences as long as the state s…
Proper Scoring and Sufficiency
Peter Harremoës
Logarithmic score and information divergence appear in both information theory, statistics, statistical mechanics, and portfolio theory. We demonstrate that all these topics involv…
Mutual information of Contingency Tables and Related Inequalities
Peter Harremoës
For testing independence it is very popular to use either the -statistic or -statistics (mutual information). Asymptotically both are -distributed so an obviou…
Extendable MDL
Peter Harremoës
In this paper we show that combination of the minimum description length principle and a exchange-ability condition leads directly to the use of Jeffreys prior. This approach works…
Minimum KL-divergence on complements of balls
Daniel Berend, Peter Harremoës, Aryeh Kontorovich
Pinsker's widely used inequality upper-bounds the total variation distance in terms of the Kullback-Leibler divergence . Although in general a bound in the rev…
Rényi Divergence and Kullback-Leibler Divergence
Tim van Erven, Peter Harremoës
Rényi divergence is related to Rényi entropy much like Kullback-Leibler divergence is related to Shannon's entropy, and comes up in many settings. It was introduced by Rényi as a m…
Information Divergence is more chi squared distributed than the chi squared statistics
Peter Harremoës, Gábor Tusnády
For testing goodness of fit it is very popular to use either the chi square statistic or G statistics (information divergence). Asymptotically both are chi square distributed so an…
Is Zero a Natural Number?
Peter Harremoës
It is argued that zero should be considered as a cardinal number but not an ordinal number. One should make a clear distinction between order types that are labels for well-ordered…
On Pairs of -divergences and their Joint Range
Peter Harremoës, Igor Vajda
We compare two f-divergences and prove that their joint range is the convex hull of the joint range for distributions supported on only two points. Some applications of this result…
Rényi Divergence and Majorization
Tim van Erven, Peter Harremoës
Rényi divergence is related to Rényi entropy much like information divergence (also called Kullback-Leibler divergence or relative entropy) is related to Shannon's entropy, and com…
Joint Range of f-divergences
Peter Harremoës, Igor Vajda
We provide a general method for evaluation of the joint range of f-divergences for two different functions f. Via topological arguments we prove that the joint range for general di…
Dutch Books and Combinatorial Games
Peter Harremoes
The theory of combinatorial game (like board games) and the theory of social games (where one looks for Nash equilibria) are normally considered as two separate theories. Here we s…
Maximum Entropy on Compact Groups
Peter Harremoes
On a compact group the Haar probability measure plays the role of uniform distribution. The entropy and rate distortion theory for this uniform distribution is studied. New results…
Properties of Classical and Quantum Jensen-Shannon Divergence
Jop Briët, Peter Harremoës
Jensen-Shannon divergence (JD) is a symmetrized and smoothed version of the most important divergence measure of information theory, Kullback divergence. As opposed to Kullback div…
Interpretations of Renyi Entropies And Divergences
Peter Harremoes
In this paper a new operational definition of Renyi entropy and Renyi divergence is presented. Other operational definitions are mentioned.