Lectures on Probability, Entropy, and Statistical Physics

Abstract

These lectures deal with the problem of inductive inference, that is, the problem of reasoning under conditions of incomplete information. Is there a general method for handling uncertainty? Or, at least, are there rules that could in principle be followed by an ideally rational mind when discussing scientific matters? What makes one statement more plausible than another? How much more plausible? And then, when new information is acquired how do we change our minds? Or, to put it differently, are there rules for learning? Are there rules for processing information that are objective and consistent? Are they unique? And, come to think of it, what, after all, is information? It is clear that data contains or conveys information, but what does this precisely mean? Can information be conveyed in other ways? Is information physical? Can we measure amounts of information? Do we need to? Our goal is to develop the main tools for inductive inference--probability and entropy--from a thoroughly Bayesian point of view and to illustrate their use in physics with examples borrowed from the foundations of classical statistical physics.

170 pages. Invited lectures at MaxEnt 2008, the 28th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering (July 8-13, 2008, Boraceia Beach, Sao Paulo, Brazil)

References in corpus (9)

• Updating Probabilities
• Maximum Entropy and Bayesian Data Analysis: Entropic Priors
• Entropic Dynamics
• From Information Geometry to Newtonian Dynamics
• Deriving laws from ordering relations
• What is a Question?
• Yet another resolution of the Gibbs paradox: an information theory approach
• Dynamical transitions in the evolution of learning algorithms by selection
• Maximum Entropy and the Variational Method in Statistical Mechanics: an Application to Simple Fluids

Cited by in corpus (10)

• Nonequilibrium Detailed Fluctuation Theorem for Repeated Discrete Feedback
• Using Relative Entropy to Find Optimal Approximations: an Application to Simple Fluids
• For whom will the Bayesian agents vote?
• Generalized Galilean Transformations and the Measurement Problem in the Entropic Dynamics Approach to Quantum Theory
• Classical description of quantum randomness using stochastic gauge systems
• Infinite Distance Limits and Information Theory
• Bayesian quantum thermometry based on thermodynamic length
• The Census and the Second Law: An Entropic Approach to Optimal Apportionment for the U.S. House of Representatives
• Random Switching for High Performance DC-DC Power Converters
• Indeterministic finite-precision physics and intuitionistic mathematics