Bayesian Entropy Estimation for Countable Discrete Distributions

Evan Archer, Il Memming Park, Jonathan Pillow

We consider the problem of estimating Shannon's entropy $H$ from discrete data, in cases where the number of possible symbols is unknown or even countably infinite. The Pitman-Yor…