13 citations · 18 across the 5 of their papers we have counts for
5 papers
Next-token prediction capacity: general upper bounds and a lower bound for transformers
Liam Madden, Curtis Fox, Christos Thrampoulidis
Given a sequence of tokens, such as words, the task of next-token prediction is to predict the next-token conditional probability distribution. Decoder-only transformers have becom…
Memory capacity of two layer neural networks with smooth activations
Liam Madden, Christos Thrampoulidis
Determining the memory capacity of two layer neural networks with hidden neurons and input dimension (i.e., total trainable parameters), which refers to the largest…
Sharp global convergence guarantees for iterative nonconvex optimization: A Gaussian process perspective
Kabir Aladin Chandrasekher, Ashwin Pananjady, Christos Thrampoulidis
We consider a general class of regression models with normally distributed covariates, and the associated nonconvex problem of fitting these models from data. We develop a general…
Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolation
Ke Wang, Vidya Muthukumar, Christos Thrampoulidis
The literature on "benign overfitting" in overparameterized models has been mostly restricted to regression or binary classification; however, modern machine learning operates in t…
Binary Classification of Gaussian Mixtures: Abundance of Support Vectors, Benign Overfitting and Regularization
Ke Wang, Christos Thrampoulidis
Deep neural networks generalize well despite being exceedingly overparameterized and being trained without explicit regularization. This curious phenomenon has inspired extensive r…