Description length of canonical and microcanonical models

Abstract

The (non-)equivalence of canonical and microcanonical ensembles is a fundamental question in statistical physics, concerning whether the use of soft and hard constraints in the maximum-entropy construction leads to the same description of a system. Despite the fact that maximum-entropy models are also commonly used in statistical inference, pattern detection, and hypothesis testing, a complete understanding of the effects of ensemble non-equivalence on statistical modeling is still missing. Here, we study this problem from a rigorous model selection perspective by comparing canonical and microcanonical models via the Minimum Description Length (MDL) principle, which yields a trade-off between likelihood, measuring model accuracy, and complexity, measuring model flexibility and its potential to overfit data. We compute the Normalized Maximum Likelihood (NML) of both formulations and find that: (i) microcanonical models always achieve higher likelihood but are always more complex; (ii) the optimal model choice depends on the empirical values of the constraints -- the canonical model performs best when its fit to the observed data exceeds its uniform average fit across all realizations; (iii) in the thermodynamic limit, the difference in description length per node vanishes when ensemble equivalence holds but persists otherwise, showing that non-equivalence implies extensive differences between large canonical and microcanonical models. Finally, we compare the NML approach to Bayesian methods, showing that (iv) the choice of priors, practically irrelevant in equivalent models, becomes crucial when an extensive number of constraints is enforced, possibly leading to very different outcomes.

References in corpus (25)

• The Statistical Physics of Real-World Networks
• Negative Specific Heat in Astronomy, Physics and Chemistry
• Inequivalence of ensembles in a system with long range interactions
• Negative heat capacity in the critical region of nuclear fragmentation: an experimental evidence of the liquid-gas phase transition
• Nonparametric Bayesian inference of the microcanonical stochastic block model
• Classification of phase transitions and ensemble inequivalence, in systems with long range interactions
• Equivalence and nonequivalence of ensembles: Thermodynamic, macrostate, and measure levels
• Inferring monopartite projections of bipartite networks: an entropy-based approach
• Gravitational instability of isothermal and polytropic spheres
• Breaking of ensemble equivalence in networks
• Thermodynamic versus statistical nonequivalence of ensembles for the mean-field Blume-Emery-Griffiths model
• Reconstruction methods for networks: the case of economic and financial systems
• Phase transitions in a complex network
• Unbiased sampling of network ensembles
• Consistencies and inconsistencies between model selection and link prediction in networks
• Minimum Description Length Revisited
• Ensemble nonequivalence in random graphs with modular structure
• The ambiguity of nestedness under soft and hard constraints
• Ensemble inequivalence in random graphs
• Strong ensemble nonequivalence in systems with local constraints
• Covariance structure behind breaking of ensemble equivalence in random graphs
• Maximum Entropy approach to multivariate time series randomization
• Stirling's Series Revisited
• Correspondence between temporal correlations in time series, inverse problems, and the Spherical Model
• Learn your entropy from informative data: an axiom ensuring the consistent identification of generalized entropies