When do finite sample effects significantly affect entropy estimates ?

Abstract

An expression is proposed for determining the error caused on entropy estimates by finite sample effects. This expression is based on the Ansatz that the ranked distribution of probabilities tends to follow an empirical Zipf law.

10 pages, 2 figures

Cited by in corpus (3)

• The Organization of Intrinsic Computation: Complexity-Entropy Diagrams and the Diversity of Natural Information Processing
• Entropy estimates of small data sets
• Similarity of symbol frequency distributions with heavy tails