A statistical mechanical interpretation of instantaneous codes

Abstract

In this paper we develop a statistical mechanical interpretation of the noiseless source coding scheme based on an absolutely optimal instantaneous code. The notions in statistical mechanics such as statistical mechanical entropy, temperature, and thermal equilibrium are translated into the context of noiseless source coding. Especially, it is discovered that the temperature 1 corresponds to the average codeword length of an instantaneous code in this statistical mechanical interpretation of noiseless source coding scheme. This correspondence is also verified by the investigation using box-counting dimension. Using the notion of temperature and statistical mechanical arguments, some information-theoretic relations can be derived in the manner which appeals to intuition.

5 pages, Proceedings of the 2007 IEEE International Symposium on Information Theory, pp.1906 - 1910, Nice, France, June 24 - 29, 2007

Cited by in corpus (8)

• The Generalized Random Energy Model and its Application to the Statistical Physics of Ensembles of Hierarchical Codes
• Phase Transition and Strong Predictability
• The random energy model in a magnetic field and joint source-channel coding
• A statistical mechanical interpretation of algorithmic information theory III: Composite systems and fixed points
• Fixed point theorems on partial randomness
• A Computational Complexity-Theoretic Elaboration of Weak Truth-Table Reducibility
• A new representation of Chaitin Ωnumber based on compressible strings
• Relations between random coding exponents and the statistical physics of random codes