1 citations · 2 across the 25 of their papers we have counts for
27 papers
Embedding Geometry Shapes LLM Routing: Tracing Failure Modes and Improving Generalization
Leonardo Torti, Emanuele Rimoldi, Tomer Galanti +2
With the growing number of LLMs available, selecting which one to use for a prompt has become a key question. A large part of the routing literature relies on semantically embeddin…
Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws
Liu Ziyin, Yizhou Xu, Tomaso Poggio +1
Neural networks trained by gradient descent on a smooth cost function can nevertheless learn in steps: the cost holds on long plateaus and then drops abruptly. Meanwhile, training…
Edge of Stability Selectively Shapes Learning Across the Data Distribution
Shauna Kwag, Anakha Ganesh, Tomaso Poggio +1
Existing analyses of the edge of stability (EoS) treat it as a global property of optimization. We show that it is also selective: the stability constraint redistributes learning a…
Do Deep Networks Forget Initialization? A Forgetting-Time View of Practical Inductive Bias
Mohua Das, Pierfrancesco Beneventano, Shibshankar Dey +2
Randomly initialized neural networks induce a prior over functions, but the predictor used in practice is produced only after training. We ask how much of this initial bias survive…
Does Weight Decay Enhance Training Stability?
Marius Saether, Amir Kolic, Tomaso Poggio +1
In modern deep learning, weight decay is often credited with "stabilizing" training dynamics, diverging from its classical role as a static regularization penalty. We investigate a…
Inference-Time Amplification of Weak Reasoning Models
Varun Sunkaraneni, Pierfrancesco Beneventano, Riccardo Neumarker +2
How much of the capability of a reasoning model exposed by repeated sampling can an imperfect selector recover? We study this problem as {\em inference-time amplification}, separat…
Generalizing the Turing Test to Interactive Agents
Daniel Mitropolsky, Susan S. Hong, Riccardo Neumarker +2
We initiate the study of the Generalized Turing Test (GTT), a formal generalization of Turing's imitation game from humans to arbitrary interactive agents. For agents and ,…
pAI/MSc: ML Theory Research with Humans on the Loop
Mahmoud Abdelmoneum, Pierfrancesco Beneventano, Tomaso Poggio
We present pAI/MSc, an open-source, customizable, modular multi-agent system for academic research workflows. Our goal is not autonomous scientific ideation, nor fully automated re…
Too Sharp, Too Sure: When Calibration Follows Curvature
Alessandro Morosini, Matea Gjika, Tomaso Poggio +1
Modern neural networks can achieve high accuracy while remaining poorly calibrated, producing confidence estimates that do not match empirical correctness. Yet calibration is often…
Momentum Further Constrains Sharpness at the Edge of Stochastic Stability
Arseniy Andreyev, Advikar Ananthkumar, Marc Walden +2
Recent work suggests that (stochastic) gradient descent self-organizes near an instability boundary, shaping both optimization and the solutions found. Momentum and mini-batch grad…
Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Series
Federico Vittorio Cortesi, Giuseppe Iannone, Giulia Crippa +2
Neural networks applied to financial time series operate in a regime of underspecification, where model predictors achieve indistinguishable out-of-sample error. Using large-scale…
Tool Use Reduces Depth-Induced Collapse in OOD Reasoning
David Koplow, Tomer Galanti, Tomaso Poggio
Humans can apply ideas learned in one context to substantially different situations. We call this process of searching for and constructing novel recombinations of learned relation…
On efficiently computable functions, deep networks and sparse compositionality
Tomaso Poggio
We show that \emph{efficient Turing computability} at any fixed input/output precision implies the existence of \emph{compositionally sparse} (bounded-fan-in, polynomial-size) DAG…
Topological Invariance and Breakdown in Learning
Yongyi Yang, Tomaso Poggio, Isaac Chuang +1
We prove that for a broad class of permutation-equivariant learning rules (including SGD, Adam, and others), the training process induces a bi-Lipschitz mapping between neurons and…
Does Feedback Alignment Work at Biological Timescales?
Marc Gong Bacvanski, Liu Ziyin, Tomaso Poggio
Feedback alignment and related weight-transport-free algorithms are often proposed as biologically plausible alternatives to backpropagation, yet they are typically formulated in d…
Learning Multi-Index Models with Hyper-Kernel Ridge Regression
Shuo Huang, Hippolyte Labarrière, Ernesto De Vito +2
Deep neural networks excel in high-dimensional problems, outperforming models such as kernel methods, which suffer from the curse of dimensionality. However, the theoretical founda…
Unraveling Syntax: Language Modeling and the Substructure of Grammars
Laura Ying Schulz, Daniel Mitropolsky, Tomaso Poggio
While language models achieve impressive results, their learning dynamics are far from understood. Many domains of interest -- such as natural language syntax, coding languages, ar…
Position: A Theory of Deep Learning Must Include Compositional Sparsity
David A. Danhofer, Davide D'Ascenzo, Rafael Dubach +1
Overparametrized Deep Neural Networks (DNNs) have demonstrated remarkable success in a wide variety of domains too high-dimensional for classical shallow networks subject to the cu…
Heterosynaptic Circuits Are Universal Gradient Machines
Liu Ziyin, Isaac Chuang, Tomaso Poggio
We propose a design principle for the learning circuits of the biological brain. The principle states that almost any dendritic weights updated via heterosynaptic plasticity can im…
Ubiquity of Emergent Hebbian Dynamics in Regularized Learning
David Koplow, Tomaso Poggio, Liu Ziyin
Hebbian and anti-Hebbian plasticity are widely observed in the brain and are classically modeled as mechanistic, local homosynaptic rules stabilized by homeostatic constraints. Thi…
What if Eye...? Computationally Recreating Vision Evolution
Kushagra Tiwary, Aaron Young, Zaid Tasneem +6
Vision systems in nature show remarkable diversity, from simple light-sensitive patches to complex camera eyes with lenses. While natural selection has produced these eyes through…
Parameter Symmetry Potentially Unifies Deep Learning Theory
Liu Ziyin, Yizhou Xu, Tomaso Poggio +1
The dynamics of learning in modern large AI systems is hierarchical, often characterized by abrupt, qualitative shifts akin to phase transitions observed in physical systems. While…
On Generalization Bounds for Neural Networks with Low Rank Layers
Andrea Pinto, Akshay Rangamani, Tomaso Poggio
While previous optimization results have suggested that deep neural networks tend to favour low-rank weight matrices, the implications of this inductive bias on generalization boun…
Training the Untrainable: Introducing Inductive Bias via Representational Alignment
Vighnesh Subramaniam, David Mayo, Colin Conwell +4
We demonstrate that architectures which traditionally are considered to be ill-suited for a task can be trained using inductive biases from another architecture. We call a network…