2 citations · 4 across the 6 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023★ 2 cited
One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention
Arvind Mahankali, Tatsunori B. Hashimoto, Tengyu Ma
Recent works have empirically analyzed in-context learning and shown that transformers trained on synthetic linear regression tasks can learn to implement ridge regression, which i…
cs.LG2023★ 1 cited
Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time
Arvind Mahankali, Jeff Z. Haochen, Kefan Dong +2
Despite recent theoretical progress on the non-convex optimization of two-layer neural networks, it is still an open question whether gradient descent on neural networks without un…