1.3k citations · 4.9k across the 150 of their papers we have counts for
47 papers · 1 filter
A Primal-Dual Framework for Transformers and Neural Networks
Tan M. Nguyen, Tam Nguyen, Nhat Ho +3
Self-attention is key to the remarkable success of transformers in sequence modeling tasks including many applications in natural language processing and computer vision. Like neur…
Self-Consuming Generative Models Go MAD
Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi +5
Seismic advances in generative AI algorithms for imagery, text, and other data types has led to the temptation to use synthetic data to train next-generation models. Repeating this…
Frozen Overparameterization: A Double Descent Perspective on Transfer Learning of Deep Neural Networks
Yehuda Dar, Lorenzo Luzi, Richard G. Baraniuk
We study the generalization behavior of transfer learning of deep neural networks (DNNs). We adopt the overparameterization perspective -- featuring interpolation of the training d…
Batch Normalization Explained
Randall Balestriero, Richard G. Baraniuk
A critically important, ubiquitous, and yet poorly understood ingredient in modern deep networks (DNs) is batch normalization (BN), which centers and normalizes the feature maps. T…
Momentum Transformer: Closing the Performance Gap Between Self-attention and Its Linearization
Tan Nguyen, Richard G. Baraniuk, Robert M. Kirby +2
Transformers have achieved remarkable success in sequence modeling and beyond but suffer from quadratic computational and memory complexities with respect to the length of the inpu…
A Blessing of Dimensionality in Membership Inference through Regularization
Jasper Tan, Daniel LeJeune, Blake Mason +2
Is overparameterization a privacy liability? In this work, we study the effect that the number of parameters has on a classifier's vulnerability to membership inference attacks. We…