Estimators of Entropy and Information via Inference in Probabilistic Models

Feras A. Saad, Marco Cusumano-Towner, Vikash K. Mansinghka

Estimating information-theoretic quantities such as entropy and mutual information is central to many problems in statistics and machine learning, but challenging in high dimension…