Estimating Functions of Probability Distributions from a Finite Set of Samples, Part 1: Bayes Estimators and the Shannon Entropy

Abstract

We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint distribution, we present finite sample estimators for the mutual information, covariance, and chi-squared functions of that probability distribution.

uuencoded compressed tarfile, submitted

Cited by in corpus (5)

• Pairwise maximum-entropy models and their Glauber dynamics: bimodality, bistability, non-ergodicity problems, and their elimination via inhibition
• Estimating Functions of Distributions Defined over Spaces of Unknown Size
• Mutual Information as a Bayesian Measure of Independence
• Estimating Functions of Distributions from A Finite Set of Samples, Part 2: Bayes Estimators for Mutual Information, Chi-Squared, Covariance and other Statistics
• Finding Structure in Text, Genome and Other Symbolic Sequences