Bias Analysis in Entropy Estimation

Abstract

We consider the problem of finite sample corrections for entropy estimation. New estimates of the Shannon entropy are proposed and their systematic error (the bias) is computed analytically. We find that our results cover correction formulas of current entropy estimates recently discussed in literature. The trade-off between bias reduction and the increase of the corresponding statistical error is analyzed.

5 pages, 3 figures

Cited by in corpus (18)

• Equitability, mutual information, and the maximal information coefficient
• Entropy estimates of small data sets
• Partial Transfer Entropy on Rank Vectors
• Similarity of symbol frequency distributions with heavy tails
• Improved Information Gain Estimates for Decision Tree Induction
• Estimating the Mutual Information between two Discrete, Asymmetric Variables with Limited Samples
• Mixing Bandt-Pompe and Lempel-Ziv approaches: another way to analyze the complexity of continuous-states sequences
• Mapping flows on sparse networks with missing links
• A note on entropy estimation
• Effects of demographic stochasticity on biological community assembly on evolutionary time scales
• Entropy estimators for Markovian sequences: A comparative analysis
• Mathematical Characterization of Private and Public Immune Repertoire Sequences
• Domain Adaptive Decision Trees: Implications for Accuracy and Fairness
• Symbolic partition in chaotic maps
• An improved estimator of Shannon entropy with applications to systems with memory
• ComplexityMeasures.jl: scalable software to unify and accelerate entropy and complexity timeseries analysis
• Fingerprinting Cryptographic Protocols with Key Exchange using an Entropy Measure
• Entropy Analysis of Financial Time Series