3 citations · 4 across the 6 of their papers we have counts for
7 papers
Learning to Recall with Transformers Beyond Orthogonal Embeddings
Nuri Mert Vural, Alberto Bietti, Mahdi Soltanolkotabi +1
Modern large language models (LLMs) excel at tasks that require storing and retrieving knowledge, such as factual recall and question answering. Transformers are central to this ca…
Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws
Gérard Ben Arous, Murat A. Erdogdu, Nuri Mert Vural +1
We study the optimization and sample complexity of gradient-based training of a two-layer neural network with quadratic activation function in the high-dimensional regime, where th…
Pruning is Optimal for Learning Sparse Features in High-Dimensions
Nuri Mert Vural, Murat A. Erdogdu
While it is commonly observed in practice that pruning networks to a certain level of sparsity can improve the quality of the features, a theoretical explanation of this phenomenon…
Mirror Descent Strikes Again: Optimal Stochastic Convex Optimization under Infinite Noise Variance
Nuri Mert Vural, Lu Yu, Krishnakumar Balasubramanian +2
We study stochastic convex optimization under infinite noise variance. Specifically, when the stochastic gradient is unbiased and has uniformly bounded -th moment, for some…
Achieving Online Regression Performance of LSTMs with Simple RNNs
N. Mert Vural, Fatih Ilhan, Selim F. Yilmaz +2
Recurrent Neural Networks (RNNs) are widely used for online regression due to their ability to generalize nonlinear temporal dependencies. As an RNN model, Long-Short-Term-Memory N…
Minimax Optimal Algorithms for Adversarial Bandit Problem with Multiple Plays
N. Mert Vural, Hakan Gokcesu, Kaan Gokcesu +1
We investigate the adversarial bandit problem with multiple plays under semi-bandit feedback. We introduce a highly efficient algorithm that asymptotically achieves the performance…
An Efficient and Effective Second-Order Training Algorithm for LSTM-based Adaptive Learning
N. Mert Vural, Salih Ergüt, Suleyman S. Kozat
We study adaptive (or online) nonlinear regression with Long-Short-Term-Memory (LSTM) based networks, i.e., LSTM-based adaptive learning. In this context, we introduce an efficient…