498 citations · 1.6k across the 46 of their papers we have counts for
49 papers
Statistical Dark Matter: Synergy is everywhere but is hard to capture
Alberto Liardi, Fernando E. Rosas, Daniele Marinazzo +3
The information-theoretic construct of synergy refers to the statistical structure that is contained in three or more variables, but not in any subset of them. Here we leverage rec…
Information Dynamics of Language Communication
Leonardo S. Goodall, Andrea I. Luppi, Pedro A. M. Mediano
Quantifying how meaning propagates through communicative exchanges remains underdeveloped in computational linguistics. Here we introduce an information-theoretic framework that qu…
A Brain-like Synergistic Core in LLMs Drives Behaviour and Learning
Pedro Urbina-Rodriguez, Zafeirios Fountas, Fernando E. Rosas +5
The independent evolution of intelligence in biological and artificial systems offers a unique opportunity to identify its fundamental computational principles. Here we show that l…
On the Identifiability of Regime-Switching Models with Multi-Lag Dependencies
Carles Balsells-Rodas, Toshiko Matsui, Pedro A. M. Mediano +2
Identifiability is central to the interpretability of deep latent variable models, ensuring parameterisations are uniquely determined by the data-generating distribution. However,…
Improved estimators of causal emergence for large systems
Madalina I. Sas, Fernando E. Rosas, Hardik Rajpal +3
A central challenge in the study of complex systems is the quantification of emergence -- understood as the ability of the system to exhibit collective behaviours that cannot be tr…
A scalable estimator of higher-order information in complex dynamical systems
Alberto Liardi, George Blackburne, Hardik Rajpal +2
Our understanding of complex systems rests on our ability to characterise how they perform distributed computation and integrate information. Advances in information theory have in…
Synergistic Motifs in Gaussian Systems
Enrico Caprioglio, Pedro A. M. Mediano, Luc Berthouze
High-order interdependencies are central features of complex systems, yet a mechanistic explanation for their emergence remains elusive. Currently, it is unknown under what conditi…
Simple physical systems as a reference for multivariate information dynamics
Alberto Liardi, Madalina I. Sas, George Blackburne +3
Understanding a complex system entails capturing the non-trivial collective phenomena that arise from interactions between its different parts. Information theory is a flexible and…
The topology of synergy: linking topological and information-theoretic approaches to higher-order interactions in complex systems
Thomas F. Varley, Pedro A. M. Mediano, Alice Patania +1
The study of irreducible higher-order interactions has become a core topic of study in complex systems. Two of the most well-developed frameworks, topological data analysis and mul…
Grokking at the Edge of Numerical Stability
Lucas Prieto, Melih Barsbey, Pedro A. M. Mediano +1
Grokking, the sudden generalization that occurs after prolonged overfitting, is a surprising phenomenon challenging our understanding of deep learning. Although significant progres…
Null models for comparing information decomposition across complex systems
Alberto Liardi, Fernando E. Rosas, Robin L. Carhart-Harris +3
A key feature of information theory is its universality, as it can be applied to study a broad variety of complex systems. However, many information-theoretic measures can vary sig…
Characterising high-order interdependence via entropic conjugation
Fernando E. Rosas, Aaron Gutknecht, Pedro A. M. Mediano +1
High-order phenomena play crucial roles in many systems of interest, but their analysis is often highly nontrivial. There is a rich literature providing a number of alternative inf…
The Fast Möbius Transform: An algebraic approach to information decomposition
Abel Jansma, Pedro A. M. Mediano, Fernando E. Rosas
The partial information decomposition (PID) and its extension integrated information decomposition (ID) are promising frameworks to investigate information phenomena involving m…
Algebraic Representations of Entropy and Fixed-Parity Information Quantities
Keenan J. A. Down, Pedro A. M. Mediano
Many information-theoretic quantities have corresponding representations in terms of sets. The prevailing signed measure space for characterising entropy, the -measure of Yeung,…
From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks
Clémentine C. J. Dominé, Nicolas Anguita, Alexandra M. Proca +4
Biological and artificial neural networks develop internal representations that enable them to perform complex tasks. In artificial networks, the effectiveness of these models reli…
A Logarithmic Decomposition and a Signed Measure Space for Entropy
Keenan J. A. Down, Pedro A. M. Mediano
The Shannon entropy of a random variable has much behaviour analogous to a signed measure. Previous work has explored this connection by defining a signed measure on abstract sets,…
Evolving reservoir computers reveals bidirectional coupling between predictive power and emergent dynamics
Hanna M. Tolle, Andrea I Luppi, Anil K. Seth +1
Biological neural networks can perform complex computations to predict their environment, far above the limited predictive capabilities of individual neurons. While conventional ap…
Identifying Nonstationary Causal Structures with High-Order Markov Switching Models
Carles Balsells-Rodas, Yixin Wang, Pedro A. M. Mediano +1
Causal discovery in time series is a rapidly evolving field with a wide variety of applications in other areas such as climate science and neuroscience. Traditional approaches assu…
Characterising directed and undirected metrics of high-order interdependence
Fernando E. Rosas, Pedro A. M. Mediano, Michael Gastpar
Systems of interest for theoretical or experimental work often exhibit high-order interactions, corresponding to statistical interdependencies in groups of variables that cannot be…
Software in the natural world: A computational approach to hierarchical emergence
Fernando E. Rosas, Bernhard C. Geiger, Andrea I Luppi +4
Understanding the functional architecture of complex systems is crucial to illuminate their inner workings and enable effective methods for their prediction and control. Recent adv…
Quantifying Hierarchical Selection
Hardik Rajpal, Clem von Stengel, Pedro A. M. Mediano +4
At what level does selective pressure effectively act? When considering the reproductive dynamics of interacting and mutating agents, it has long been debated whether selection is…
Information decomposition reveals hidden high-order contributions to temporal irreversibility
Andrea I Luppi, Fernando E. Rosas, Gustavo Deco +2
Temporal irreversibility, often referred to as the arrow of time, is a fundamental concept in statistical mechanics. Markers of irreversibility also provide a powerful characterisa…
Quantifying synergy and redundancy in multiplex networks
Andrea I. Luppi, Eckehard Olbrich, Conor Finn +4
Understanding how different networks relate to each other is key for obtaining a greater insight into complex systems. Here, we introduce an intuitive yet powerful framework to cha…
Interaction Measures, Partition Lattices and Kernel Tests for High-Order Interactions
Zhaolu Liu, Robert L. Peach, Pedro A. M. Mediano +1
Models that rely solely on pairwise relationships often fail to capture the complete statistical structure of the complex multivariate data found in diverse domains, such as socio-…