5 papers
Density Estimation via Binless Multidimensional Integration
Matteo Carli, Alex Rodriguez, Alessandro Laio +1
We introduce the Binless Multidimensional Thermodynamic Integration (BMTI) method for nonparametric, robust, and data-efficient density estimation. BMTI estimates the logarithm of…
Scale adaptive and robust intrinsic dimension estimation via optimal neighbourhood identification
Antonio Di Noia, Iuri Macocco, Aldo Glielmo +2
The Intrinsic Dimension (ID) is a key concept in unsupervised learning and feature selection, as it is a lower bound to the number of variables which are necessary to describe a sy…
Unsupervised detection of semantic correlations in big data
Santiago Acevedo, Alex Rodriguez, Alessandro Laio
In real-world data, information is stored in extremely large feature vectors. These variables are typically correlated due to complex interactions involving many features simultane…
Emergence of a High-Dimensional Abstraction Phase in Language Transformers
Emily Cheng, Diego Doimo, Corentin Kervadec +4
A language model (LM) is a mapping from a linguistic context to an output token. However, much remains to be known about this mapping, including how its geometric properties relate…
A distributional simplicity bias in the learning dynamics of transformers
Riccardo Rende, Federica Gerace, Alessandro Laio +1
The remarkable capability of over-parameterised neural networks to generalise effectively has been explained by invoking a ``simplicity bias'': neural networks prevent overfitting…