5.7k citations · 8.6k across the 42 of their papers we have counts for
43 papers
Unsupervised detection of semantic correlations in big data
Santiago Acevedo, Alex Rodriguez, Alessandro Laio
In real-world data, information is stored in extremely large feature vectors. These variables are typically correlated due to complex interactions involving many features simultane…
A distributional simplicity bias in the learning dynamics of transformers
Riccardo Rende, Federica Gerace, Alessandro Laio +1
The remarkable capability of over-parameterised neural networks to generalise effectively has been explained by invoking a ``simplicity bias'': neural networks prevent overfitting…
Density Estimation via Binless Multidimensional Integration
Matteo Carli, Alex Rodriguez, Alessandro Laio +1
We introduce the Binless Multidimensional Thermodynamic Integration (BMTI) method for nonparametric, robust, and data-efficient density estimation. BMTI estimates the logarithm of…
Emergence of a High-Dimensional Abstraction Phase in Language Transformers
Emily Cheng, Diego Doimo, Corentin Kervadec +4
A language model (LM) is a mapping from a linguistic context to an output token. However, much remains to be known about this mapping, including how its geometric properties relate…
Scale adaptive and robust intrinsic dimension estimation via optimal neighbourhood identification
Antonio Di Noia, Iuri Macocco, Aldo Glielmo +2
The Intrinsic Dimension (ID) is a key concept in unsupervised learning and feature selection, as it is a lower bound to the number of variables which are necessary to describe a sy…
Mapping of attention mechanisms to a generalized Potts model
Riccardo Rende, Federica Gerace, Alessandro Laio +1
Transformers are neural networks that revolutionized natural language processing and machine learning. They process sequences of inputs, like words, using a mechanism called self-a…