Energy--Information Trade-off Induces Continuous and Discontinuous Phase Transitions in Lateral Predictive Coding

Abstract

Lateral predictive coding is a recurrent neural network which creates energy-efficient internal representations by exploiting statistical regularity in sensory inputs. Here we investigate the trade-off between information robustness and energy in a linear model of lateral predictive coding analytically and by numerical minimization of a free energy. We observe several phase transitions in the synaptic weight matrix, especially a continuous transition which breaks reciprocity and permutation symmetry and builds cyclic dominance and a discontinuous transition with the associated sudden emergence of tight balance between excitatory and inhibitory interactions. The optimal network follows an ideal-gas law in an extended temperature range and saturates the efficiency upper-bound of energy utilization. These results bring theoretical insights on the emergence and evolution of complex internal models in predictive processing systems.

6 pages main text, supplementary text combined. This is an extensively revised version, containing new analytical results and numerical results

References in corpus (12)

• Weak pairwise correlations imply strongly correlated network states in a neural population
• Recurrent computations for visual pattern completion
• Random numbers for large scale distributed Monte Carlo simulations
• Less is More: Wiring-Economical Modular Networks Support Self-Sustained Firing-Economical Neural Avalanches for Efficient Processing
• The Optimal Size of Stochastic Hodgkin-Huxley Neuronal Systems for Maximal Energy Efficiency in Coding of Pulse Signals
• Inferring Global Dynamics of a Black-Box System Using Machine Learning
• Copy the dynamics using a learning machine
• Inferring Global Dynamics Using a Learning Machine
• The information content of brain states is explained by structural constraints on state energetics
• Bayesian mechanics of perceptual inference and motor control in the brain
• Moderate Point: Balanced Entropy and Enthalpy Contributions in Soft Matter
• Lateral predictive coding revisited: Internal model, symmetry breaking, and response time

Cited by in corpus (2)

• Discontinuous phase transition of feature detection in lateral predictive coding
• Network Dynamics-Based Framework for Understanding Deep Neural Networks