Information Theory and Statistical Physics - Lecture Notes
arXiv:1006.1565
Abstract
This document consists of lecture notes for a graduate course, which focuses on the relations between Information Theory and Statistical Physics. The course is aimed at EE graduate students in the area of Communications and Information Theory, as well as to graduate students in Physics who have basic background in Information Theory. Strong emphasis is given to the analogy and parallelism between Information Theory and Statistical Physics, as well as to the insights, the analysis tools and techniques that can be borrowed from Statistical Physics and `imported' to certain problem areas in Information Theory. This is a research trend that has been very active in the last few decades, and the hope is that by exposing the student to the meeting points between these two disciplines, we will enhance his/her background and perspective to carry out research in the field. A short outline of the course is as follows: Introduction; Elementary Statistical Physics and its Relation to Information Theory; Analysis Tools in Statistical Physics; Systems of Interacting Particles and Phase Transitions; The Random Energy Model (REM) and Random Channel Coding; Additional Topics (optional).
176 pages, 26 figures. Lecture notes of a graduate course delivered at the Technion in the Spring of 2010
References in corpus (6)
- Dissipation: The phase-space perspective
- Universal geometric approach to uncertainty, entropy and information
- Relative Entropy: Free Energy Associated with Equilibrium Fluctuations and Nonequilibrium Deviations
- On the Thermodynamic Temperature of a General Distribution
- The random energy model in a magnetic field and joint source-channel coding
- Some remarks on Rényi relative entropy in a thermostatistical framework