Toward a statistical mechanics of four letter words

Abstract

We consider words as a network of interacting letters, and approximate the probability distribution of states taken on by this network. Despite the intuition that the rules of English spelling are highly combinatorial (and arbitrary), we find that maximum entropy models consistent with pairwise correlations among letters provide a surprisingly good approximation to the full statistics of four letter words, capturing ~92% of the multi-information among letters and even "discovering" real words that were not represented in the data from which the pairwise correlations were estimated. The maximum entropy model defines an energy landscape on the space of possible words, and local minima in this landscape account for nearly two-thirds of words used in written English.

References in corpus (6)

• Weak pairwise correlations imply strongly correlated network states in a neural population
• Network information and connected correlations
• Ising models for networks of real neurons
• Faster solutions of the inverse pairwise Ising problem
• Rediscovering the power of pairwise interactions
• Estimating mutual information and multi--information in large networks

Cited by in corpus (24)

• Statistical mechanics for natural flocks of birds
• Searching for collective behavior in a network of real neurons
• Social interactions dominate speed control in driving natural flocks toward criticality
• The simplest maximum entropy model for collective behavior in a neural network
• Searching for simplicity: Approaches to the analysis of neurons and behavior
• The distinct flavors of Zipf's law in the rank-size and in the size-distribution representations, and its maximum-likelihood fitting
• The brevity law as a scaling law, and a possible origin of Zipf's law for word frequencies
• Random versus maximum entropy models of neural population activity
• Identifying trends in word frequency dynamics
• Style Imitation and Chord Invention in Polyphonic Music with Exponential Families
• Systems poised to criticality through Pareto selective forces
• Pairwise Network Information and Nonlinear Correlations
• Lognormals, Power Laws and Double Power Laws in the Distribution of Frequencies of Harmonic Codewords from Classical Music
• Inferring Cultural Landscapes with the Inverse Ising Model
• When are correlations strong?
• From Boltzmann to Zipf through Shannon and Jaynes
• Maximum Entropy Principle Analysis in Network Systems with Short-time Recordings
• Improving predictability of time series using maximum entropy methods
• CoHSI V: Identical multiple scale-independent systems within genomes and computer software
• Nonparametric Maximum Entropy Estimation on Information Diagrams
• Hamiltonian Modeling of Macro-Economic Urban Dynamics
• Surveying the space of descriptions of a composite system with machine learning
• Convenient Interface to Inverse Ising (ConIII): A Python 3 Package for Solving Ising-Type Maximum Entropy Models
• Thermodynamics of Information Retrieval