2 papers
cs.LG2021
Learning Neural Network Subspaces
Mitchell Wortsman, Maxwell Horton, Carlos Guestrin +2
Recent observations have advanced our understanding of the neural network optimization landscape, revealing the existence of (1) paths of high accuracy containing diverse solutions…
cs.LG2020
Supermasks in Superposition
Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu +4
We present the Supermasks in Superposition (SupSup) model, capable of sequentially learning thousands of tasks without catastrophic forgetting. Our approach uses a randomly initial…