9 papers
Post-Training with Policy Gradients: Optimality and the Base Model Barrier
Alireza Mousavi-Hosseini, Murat A. Erdogdu
We study post-training linear autoregressive models with outcome and process rewards. Given a context , the model must predict the response …
Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws
Gérard Ben Arous, Murat A. Erdogdu, Nuri Mert Vural +1
We study the optimization and sample complexity of gradient-based training of a two-layer neural network with quadratic activation function in the high-dimensional regime, where th…
Distributional Training Data Attribution: What do Influence Functions Sample?
Bruno Mlodozeniec, Isaac Reid, Sam Power +4
Randomness is an unavoidable part of training deep learning models, yet something that traditional training data attribution algorithms fail to rigorously account for. They ignore…
From Information to Generative Exponent: Learning Rate Induces Phase Transitions in SGD
Konstantinos Christopher Tsiolis, Alireza Mousavi-Hosseini, Murat A. Erdogdu
To understand feature learning dynamics in neural networks, recent theoretical works have focused on gradient-based learning of Gaussian single-index models, where the label is a n…
A Geometric Analysis of PCA
Ayoub El Hanchi, Murat Erdogdu, Chris Maddison
What property of the data distribution determines the excess risk of principal component analysis? In this paper, we provide a precise answer to this question. We establish a centr…
Robust Feature Learning for Multi-Index Models in High Dimensions
Alireza Mousavi-Hosseini, Adel Javanmard, Murat A. Erdogdu
Recently, there have been numerous studies on feature learning with neural networks, specifically on learning single- and multi-index models where the target is a function of a low…