Machine-learning Iterative Calculation of Entropy for Physical Systems

Abstract

Characterizing the entropy of a system is a crucial, and often computationally costly, step in understanding its thermodynamics. It plays a key role in the study of phase transitions, pattern formation, protein folding and more. Current methods for entropy estimation suffer either from a high computational cost, lack of generality or inaccuracy, and inability to treat complex, strongly interacting systems. In this paper, we present a novel method, termed MICE, for calculating the entropy by iteratively dividing the system into smaller subsystems and estimating the mutual information between each pair of halves. The estimation is performed with a recently proposed machine learning algorithm which works with arbitrary network architectures that can be chosen to fit the structure and symmetries of the system at hand. We show that our method can calculate the entropy of various systems, both thermal and athermal, with state-of-the-art accuracy. Specifically, we study various classical spin systems, and identify the jamming point of a bidisperse mixture of soft disks. Lastly, we suggest that besides its role in estimating the entropy, the mutual information itself can provide an insightful diagnostic tool in the study of physical systems.

arXiv comment8 pages, 4 figures + Supplementary information: 10 pages, 6 figures

References in corpus (19)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• A nonequilibrium equality for free energy differences
• Jamming at Zero Temperature and Zero Applied Stress: the Epitome of Disorder
• Supercooled Liquids for Pedestrians
• Area laws in quantum systems: mutual information and correlations
• The structural glass transition and the entropy of the metastable states
• Multiresolution community detection for megascale networks by information-based replica correlations
• Athermal Shear-Transformation-Zone Theory of Amorphous Plastic Deformation I: Basic Principles
• Tensor renormalization group study of classical XY model on the square lattice
• Quantifying hidden order out of equilibrium
• Finite-Size-Scaling at the Jamming Transition: Corrections to Scaling and the Correlation Length Critical Exponent
• Universal and accessible entropy estimation using a compression algorithm
• Detecting Classical Phase Transitions with Renyi Mutual Information
• Mutual information in classical spin models
• Mapping the jamming transition of bidisperse mixtures
• Cluster representations and the Wolff algorithm in arbitrary external fields
• Information density, structure and entropy in equilibrium and non-equilibrium systems
• Inferring entropy from structure
• Inference of hidden structures in complex physical systems by multi-scale clustering

Cited by in corpus (14)

• Symmetric inseparability and number entanglement in charge conserving mixed states
• Relevance in the Renormalization Group and in Information Theory
• Statistical physics through the lens of real-space mutual information
• Symmetries and phase diagrams with real-space mutual information neural estimation
• Mutual information, quantum phase transition, and phase coherence in Kondo systems
• Vibrational entropy of crystalline solids from covariance of atomic displacements
• Non-parametric learning critical behavior in Ising partition functions: PCA entropy and intrinsic dimension
• Universal Relation between Entropy and Kinetics
• Resolving entropy contributions in nonequilibrium transitions
• Superconducting fluctuations and charge-4$e$ plaquette state at strong coupling
• Detecting and characterizing phase transitions in active matter using entropy
• Quantifying spatio-temporal patterns in classical and quantum systems out of equilibrium
• First-Passage Approach to Optimizing Perturbations for Improved Training of Machine Learning Models
• An Information-theoretic Collective Variable for Configurational Entropy