How to read probability distributions as statements about process

Abstract

Probability distributions can be read as simple expressions of information. Each continuous probability distribution describes how information changes with magnitude. Once one learns to read a probability distribution as a measurement scale of information, opportunities arise to understand the processes that generate the commonly observed patterns. Probability expressions may be parsed into four components: the dissipation of all information, except the preservation of average values, taken over the measurement scale that relates changes in observed values to changes in information, and the transformation from the underlying scale on which information dissipates to alternative scales on which probability pattern may be expressed. Information invariances set the commonly observed measurement scales and the relations between them. In particular, a measurement scale for information is defined by its invariance to specific transformations of underlying values into measurable outputs. Essentially all common distributions can be understood within this simple framework of information invariance and measurement scale.

v2: added table of contents, adjusted section numbers v3: minor editing, updated reference

References in corpus (6)

• The common patterns of nature
• Input-output relations in biological systems: measurement, information and the Hill equation
• Generative models versus underlying symmetries to explain biological pattern
• A simple derivation and classification of common probability distributions based on information symmetry and measurement scale
• Measurement Invariance, Entropy, and Probability
• Measurement scale in maximum entropy models of species abundance

Cited by in corpus (4)

• The Price equation program: simple invariances unify population dynamics, thermodynamics, probability, information and inference
• Universal expressions of population change by the Price equation: natural selection, information, and maximum entropy production
• Common probability patterns arise from simple invariances
• Re-evaluating phoneme frequencies