Optimal high-level descriptions of dynamical systems

Abstract

To analyze high-dimensional systems, many fields in science and engineering rely on high-level descriptions, sometimes called "macrostates," "coarse-grainings," or "effective theories". Examples of such descriptions include the thermodynamic properties of a large collection of point particles undergoing reversible dynamics, the variables in a macroeconomic model describing the individuals that participate in an economy, and the summary state of a cell composed of a large set of biochemical networks. Often these high-level descriptions are constructed without considering the ultimate reason for needing them in the first place. Here, we formalize and quantify one such purpose: the need to predict observables of interest concerning the high-dimensional system with as high accuracy as possible, while minimizing the computational cost of doing so. The resulting State Space Compression (SSC) framework provides a guide for how to solve for the {optimal} high-level description of a given dynamical system, rather than constructing it based on human intuition alone. In this preliminary report, we introduce SSC, and illustrate it with several information-theoretic quantifications of "accuracy", all with different implications for the optimal compression. We also discuss some other possible applications of SSC beyond the goal of accurate prediction. These include SSC as a measure of the complexity of a dynamical system, and as a way to quantify information flow between the scales of a system.

33 pages. Updated discussion and references

References in corpus (22)

• The structure and function of complex networks
• Measuring Information Transfer
• The information bottleneck method
• Self-similarity of complex networks
• Statistical mechanics for natural flocks of birds
• Minimal Energy Cost for Thermodynamic Information Processing: Measurement and Information Erasure
• Superfluid transport of information in turning flocks of starlings
• The thermodynamics of prediction
• Skeleton and fractal scaling in complex networks
• Coarse-Graining and Self-Dissimilarity of Complex Networks
• Self-similar scaling of density in complex real-world networks
• Generalising Landauer's Principle
• On computational irreducibility and the predictability of complex physical systems
• Coarse-graining of cellular automata, emergence, and the predictability of complex systems
• Complex networks renormalization: flows and fixed points
• What Is a Macrostate? Subjective Observations and Objective Dynamics
• Information Bottlenecks, Causal States, and Statistical Relevance Bases: How to Represent Relevant Information in Memoryless Transduction
• Dynamics and Processing in Finite Self-Similar Networks
• Information-theoretic bound on the energy cost of stochastic simulation
• Major Transitions in Political Order
• A dual eigenvector condition for strong lumpability of Markov chains
• Circumventing the Curse of Dimensionality in Prediction: Causal Rate-Distortion for Infinite-Order Markov Processes

Cited by in corpus (7)

• Black-boxing and cause-effect power
• The evolution of lossy compression
• Maximally Informative Hierarchical Representations of High-Dimensional Data
• Major Transitions in Political Order
• Fate of Duplicated Neural Structures
• On the predictability of infectious disease outbreaks
• A Non-equilibrium Thermodynamic Framework of Consciousness