33 citations · 129 across the 17 of their papers we have counts for
4 papers · 1 filter
Adaptive Memory Replay for Continual Learning
James Seale Smith, Lazar Valkov, Shaunak Halbe +4
Foundation Models (FMs) have become the hallmark of modern AI, however, these models are trained on massive data, leading to financially expensive training. Updating FMs as new dat…
Learning to Grow Pretrained Models for Efficient Transformer Training
Peihao Wang, Rameswar Panda, Lucas Torroba Hennigen +6
Scaling transformers has led to significant breakthroughs in many domains, leading to a paradigm in which larger versions of existing models are trained and released on a periodic…
S3Pool: Pooling with Stochastic Spatial Sampling
Shuangfei Zhai, Hui Wu, Abhishek Kumar +4
Feature pooling layers (e.g., max pooling) in convolutional neural networks (CNNs) serve the dual purpose of providing increasingly abstract representations as well as yielding com…
Generative Adversarial Networks as Variational Training of Energy Based Models
Shuangfei Zhai, Yu Cheng, Rogerio Feris +1
In this paper, we study deep generative models for effective unsupervised learning. We propose VGAN, which works by minimizing a variational lower bound of the negative log likelih…