Empirical entropy in context

Abstract

We trace the history of empirical entropy, touching briefly on its relation to Markov processes, normal numbers, Shannon entropy, the Chomsky hierarchy, Kolmogorov complexity, Ziv-Lempel compression, de Bruijn sequences and stochastic complexity.

A survey of some results related to empirical entropy, written in the spring of 2007 as part of an introduction to a PhD thesis