5 citations · 10 across the 18 of their papers we have counts for
19 papers
Embedding Geometry Shapes LLM Routing: Tracing Failure Modes and Improving Generalization
Leonardo Torti, Emanuele Rimoldi, Tomer Galanti +2
With the growing number of LLMs available, selecting which one to use for a prompt has become a key question. A large part of the routing literature relies on semantically embeddin…
Distribution-Aware Programming: Learning Specialized Solvers from Experience
Saharsh Koganti, Priyadarsi Mishra, Pierfrancesco Beneventano +1
Many optimization problems are solved repeatedly on instances drawn from the same underlying distribution. In this setting, a system can begin with a general-purpose solver and use…
Edge of Stability Selectively Shapes Learning Across the Data Distribution
Shauna Kwag, Anakha Ganesh, Tomaso Poggio +1
Existing analyses of the edge of stability (EoS) treat it as a global property of optimization. We show that it is also selective: the stability constraint redistributes learning a…
The Spectral Dynamics and Noise Geometry of Muon
Pierfrancesco Beneventano, Mahmoud Abdelmoneum, Tomaso Poggio
Muon replaces a matrix gradient by its polar factor . This keeps the singular directions selected by the gradient, but makes the update spectrum flat. We stud…
Do Deep Networks Forget Initialization? A Forgetting-Time View of Practical Inductive Bias
Mohua Das, Pierfrancesco Beneventano, Shibshankar Dey +2
Randomly initialized neural networks induce a prior over functions, but the predictor used in practice is produced only after training. We ask how much of this initial bias survive…
Retrieval Is Not Enough: Why Organizational AI Needs Epistemic Infrastructure
Federico Bottino, Carlo Ferrero, Nicholas Dosio +1
Organizational knowledge used by AI agents typically lacks epistemic structure: retrieval systems surface semantically relevant content without distinguishing binding decisions fro…
Does Weight Decay Enhance Training Stability?
Marius Saether, Amir Kolic, Tomaso Poggio +1
In modern deep learning, weight decay is often credited with "stabilizing" training dynamics, diverging from its classical role as a static regularization penalty. We investigate a…
Inference-Time Amplification of Weak Reasoning Models
Varun Sunkaraneni, Pierfrancesco Beneventano, Riccardo Neumarker +2
How much of the capability of a reasoning model exposed by repeated sampling can an imperfect selector recover? We study this problem as {\em inference-time amplification}, separat…
pAI/MSc: ML Theory Research with Humans on the Loop
Mahmoud Abdelmoneum, Pierfrancesco Beneventano, Tomaso Poggio
We present pAI/MSc, an open-source, customizable, modular multi-agent system for academic research workflows. Our goal is not autonomous scientific ideation, nor fully automated re…
Too Sharp, Too Sure: When Calibration Follows Curvature
Alessandro Morosini, Matea Gjika, Tomaso Poggio +1
Modern neural networks can achieve high accuracy while remaining poorly calibrated, producing confidence estimates that do not match empirical correctness. Yet calibration is often…
Momentum Further Constrains Sharpness at the Edge of Stochastic Stability
Arseniy Andreyev, Advikar Ananthkumar, Marc Walden +2
Recent work suggests that (stochastic) gradient descent self-organizes near an instability boundary, shaping both optimization and the solutions found. Momentum and mini-batch grad…
Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Series
Federico Vittorio Cortesi, Giuseppe Iannone, Giulia Crippa +2
Neural networks applied to financial time series operate in a regime of underspecification, where model predictors achieve indistinguishable out-of-sample error. Using large-scale…
Does SGD Seek Flatness or Sharpness? An Exactly Solvable Model
Yizhou Xu, Pierfrancesco Beneventano, Isaac Chuang +1
A large body of theory and empirical work hypothesizes a connection between the flatness of a neural network's loss landscape during training and its performance. However, there ha…
Gradient Descent Converges Linearly to Flatter Minima than Gradient Flow in Shallow Linear Networks
Pierfrancesco Beneventano, Blake Woodworth
We study the gradient descent (GD) dynamics of a depth-2 linear neural network with a single input and output. We show that GD converges at an explicit linear rate to a global mini…
Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD
Arseniy Andreyev, Pierfrancesco Beneventano
Recent findings by Cohen et al., 2021, demonstrate that when training neural networks using full-batch gradient descent with a step size of , the largest eigenvalue o…
How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD
Pierfrancesco Beneventano, Andrea Pinto, Tomaso Poggio
We investigate the ability of deep neural networks to identify the support of the target function. Our findings reveal that mini-batch SGD effectively learns the support in the fir…
On the Trajectories of SGD Without Replacement
Pierfrancesco Beneventano
This article examines the implicit regularization effect of Stochastic Gradient Descent (SGD). We consider the case of SGD without replacement, the variant typically used to optimi…
Deep neural network approximation theory for high-dimensional functions
Pierfrancesco Beneventano, Patrick Cheridito, Robin Graeber +2
The purpose of this article is to develop a machinery to study the capacity of deep neural networks (DNNs) to approximate high-dimensional functions. In particular, we show that DN…
High-dimensional approximation spaces of artificial neural networks and applications to partial differential equations
Pierfrancesco Beneventano, Patrick Cheridito, Arnulf Jentzen +1
In this paper we develop a new machinery to study the capacity of artificial neural networks (ANNs) to approximate high-dimensional functions without suffering from the curse of di…