3 citations · 4 across the 6 of their papers we have counts for
4 papers · 1 filter
Learning to Recall with Transformers Beyond Orthogonal Embeddings
Nuri Mert Vural, Alberto Bietti, Mahdi Soltanolkotabi +1
Modern large language models (LLMs) excel at tasks that require storing and retrieving knowledge, such as factual recall and question answering. Transformers are central to this ca…
Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws
Gérard Ben Arous, Murat A. Erdogdu, Nuri Mert Vural +1
We study the optimization and sample complexity of gradient-based training of a two-layer neural network with quadratic activation function in the high-dimensional regime, where th…
Pruning is Optimal for Learning Sparse Features in High-Dimensions
Nuri Mert Vural, Murat A. Erdogdu
While it is commonly observed in practice that pruning networks to a certain level of sparsity can improve the quality of the features, a theoretical explanation of this phenomenon…
Mirror Descent Strikes Again: Optimal Stochastic Convex Optimization under Infinite Noise Variance
Nuri Mert Vural, Lu Yu, Krishnakumar Balasubramanian +2
We study stochastic convex optimization under infinite noise variance. Specifically, when the stochastic gradient is unbiased and has uniformly bounded -th moment, for some…