101 citations · 219 across the 15 of their papers we have counts for
5 papers · 1 filter
Neural Priming for Sample-Efficient Adaptation
Matthew Wallingford, Vivek Ramanujan, Alex Fang +5
We propose Neural Priming, a technique for adapting large pretrained models to distribution shifts and downstream tasks given few or no labeled examples. Presented with class names…
Matryoshka Representation Learning
Aditya Kusupati, Gantavya Bhatt, Aniket Rege +8
Learned representations are a central component in modern ML systems, serving a multitude of downstream tasks. When training such representations, it is often the case that computa…
LLC: Accurate, Multi-purpose Learnt Low-dimensional Binary Codes
Aditya Kusupati, Matthew Wallingford, Vivek Ramanujan +6
Learning binary representations of instances and classes is a classical problem with several high potential applications. In modern settings, the compression of high-dimensional ne…
Supermasks in Superposition
Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu +4
We present the Supermasks in Superposition (SupSup) model, capable of sequentially learning thousands of tasks without catastrophic forgetting. Our approach uses a randomly initial…
Soft Threshold Weight Reparameterization for Learnable Sparsity
Aditya Kusupati, Vivek Ramanujan, Raghav Somani +4
Sparsity in Deep Neural Networks (DNNs) is studied extensively with the focus of maximizing prediction accuracy given an overall parameter budget. Existing methods rely on uniform…