On the origin of long-range correlations in texts
arXiv:1207.0658 · doi:10.1073/pnas.1117723109
Abstract
The complexity of human interactions with social and natural phenomena is mirrored in the way we describe our experiences through natural language. In order to retain and convey such a high dimensional information, the statistical properties of our linguistic output has to be highly correlated in time. An example are the robust observations, still largely not understood, of correlations on arbitrary long scales in literary texts. In this paper we explain how long-range correlations flow from highly structured linguistic levels down to the building blocks of a text (words, letters, etc..). By combining calculations and data analysis we show that correlations take form of a bursty sequence of events once we approach the semantically relevant topics of the text. The mechanisms we identify are fairly general and can be equally applied to other hierarchical settings.
Full paper (8 pages) and Supporting Information (19 pages)
References in corpus (8)
- Scaling laws of human interaction activity
- Beyond word frequency: Bursts, lulls, and scaling in the temporal distributions of words
- Entropy and Long range correlations in literary English
- Entropy and Long range correlations in literary English
- Statistical keyword detection in literary corpora
- Towards the quantification of the semantic information encoded in written language
- Intermittency and scale-free networks: a dynamical model for human language complexity
- Competition between Two Kinds of Correlations in Literary Texts
Cited by in corpus (27)
- Languages cool as they expand: Allometric scaling and the decreasing need for new words
- Natural Language Statistical Features of LSTM-generated Texts
- Statistical laws in linguistics
- Probing the statistical properties of unknown texts: application to the Voynich Manuscript
- Testing reanalysis datasets in Antarctica: Trends, persistence properties and trend significance
- Criticality in Formal Languages and Statistical Physics
- A joint text mining-rank size investigation of the rhetoric structures of the US Presidents' speeches
- DNA viewed as an out-of-equilibrium structure
- The scarcity of crossing dependencies: a direct outcome of a specific constraint?
- Language Design as Information Renormalization
- Do Neural Nets Learn Statistical Laws behind Natural Language?
- Constant conditional entropy and related hypotheses
- Complexity-entropy analysis at different levels of organization in written language
- Heaps' law, statistics of shared components and temporal patterns from a sample-space-reducing process
- Memory and long-range correlations in chess games
- Tensor network language model
- Are queries and keys always relevant? A case study on Transformer wave functions
- Dynamic Natural Language Processing with Recurrence Quantification Analysis
- Characterization of DNA methylation as a function of biological complexity via dinucleotide inter-distances
- Entropy and type-token ratio in gigaword corpora
- Ordinal analysis of lexical patterns
- Two halves of a meaningful text are statistically different
- Assessing Language Models with Scaling Properties
- Exploring language relations through syntactic distances and geographic proximity
- A short-ranged memory model with preferential growth
- Correlation Dimension of Natural Language in a Statistical Manifold
- A Zipf-preserving, long-range correlated surrogate for written language and other symbolic sequences