3.5k citations · 7.7k across the 28 of their papers we have counts for
6 papers · 1 filter
Plug and Play Language Models: A Simple Approach to Controlled Text Generation
Sumanth Dathathri, Andrea Madotto, Janice Lan +5
Large transformer-based language models (LMs) trained on huge text corpora have shown unparalleled generation capabilities. However, controlling attributes of the generated languag…
First-Order Preconditioning via Hypergradient Descent
Ted Moskovitz, Rui Wang, Janice Lan +4
Standard gradient descent methods are susceptible to a range of issues that can impede training, such as high correlations and different scaling in parameter space.These difficulti…
LCA: Loss Change Allocation for Neural Network Training
Janice Lan, Rosanne Liu, Hattie Zhou +1
Neural networks enjoy widespread use, but many aspects of their training, representation, and operation are poorly understood. In particular, our view into the training process is…
Hamiltonian Neural Networks
Sam Greydanus, Misko Dzamba, Jason Yosinski
Even though neural networks enjoy widespread use, they still struggle to learn the basic laws of physics. How might we endow them with better inductive biases? In this paper, we dr…
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask
Hattie Zhou, Janice Lan, Rosanne Liu +1
The recent "Lottery Ticket Hypothesis" paper by Frankle & Carbin showed that a simple approach to creating sparse networks (keeping the large weights) results in models that are tr…
Understanding Neural Networks via Feature Visualization: A survey
Anh Nguyen, Jason Yosinski, Jeff Clune
A neuroscience method to understanding the brain is to find and study the preferred stimuli that highly activate an individual cell or groups of cells. Recent advances in machine l…