6 papers
The Interplay of Data Structure and Imbalance in the Learning Dynamics of Diffusion Models
Flavio Nicoletti, Chenxiao Ma, Enrico Ventura +2
Real-world datasets are inherently heterogeneous, yet how per-class structural differences and sampling imbalance shape the training dynamics of diffusion models-and potentially ex…
Biased Generalization in Diffusion Models
Jerome Garnier-Brun, Luca Biggio, Davide Beltrame +2
Generalization in generative modeling is defined as the ability to learn an underlying distribution from a finite dataset and produce novel samples, with evaluation largely driven…
Spontaneous and Collective Learning in a Minimal Agent-Environment Model
Sebastiano Ariosto, Jerome Garnier-Brun, Luca Saglietti +1
How can information be extracted from data without explicit guidance or rewards? We investigate this question in a minimal setting where a classifying agent is exposed to a stream…
On the Bias of Next-Token Predictors Toward Systematically Inefficient Reasoning: A Shortest-Path Case Study
Riccardo Alberghi, Elizaveta Demyanenko, Luca Biggio +1
Recent advances in natural language processing highlight two key factors for improving reasoning in large language models (LLMs): (i) allocating more test-time compute tends to hel…
Isolating the hard core of phaseless inference: the Phase selection formulation
Davide Straziota, Luca Saglietti
Real-valued Phase retrieval is a non-convex continuous inference problem, where a high-dimensional signal is to be reconstructed from a dataset of signless linear measurements. Foc…
How transformers learn structured data: insights from hierarchical filtering
Jerome Garnier-Brun, Marc Mézard, Emanuele Moscato +1
Understanding the learning process and the embedded computation in transformers is becoming a central goal for the development of interpretable AI. In the present study, we introdu…