Feature Learning in Linear-Width Two-Layer Networks: Two vs. One Step of Gradient Descent

Behrad Moniri, Hamed Hassani

We study feature learning in two-layer neural networks within the linear-width regime, where the number of hidden neurons, sample size, and input dimension scale proportionally. Wh…