Determining the Number of Samples Required to Estimate Entropy in Natural Sequences

Andrew D. Back, Daniel Angus, Janet Wiles

Calculating the Shannon entropy for symbolic sequences has been widely considered in many fields. For descriptive statistical problems such as estimating the N-gram entropy of Engl…