Regularities Unseen, Randomness Observed: Levels of Entropy Convergence

James P. Crutchfield, David P. Feldman

We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applyin…