3.5k citations · 7.7k across the 28 of their papers we have counts for
14 papers · 1 filter
Training LLMs for Honesty via Confessions
Manas Joglekar, Jeremy Chen, Gabriel Wu +4
Large language models (LLMs) can be dishonest when reporting on their actions and beliefs -- for example, they may overstate their confidence in factual claims or cover up evidence…
When does loss-based prioritization fail?
Niel Teng Hu, Xinyu Hu, Rosanne Liu +2
Not all examples are created equal, but standard deep neural network training protocols treat each training point uniformly. Each example is propagated forward and backward through…
Supermasks in Superposition
Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu +4
We present the Supermasks in Superposition (SupSup) model, capable of sequentially learning thousands of tasks without catastrophic forgetting. Our approach uses a randomly initial…
Estimating Q(s,s') with Deep Deterministic Dynamics Gradients
Ashley D. Edwards, Himanshu Sahni, Rosanne Liu +7
In this paper, we introduce a novel form of value function, , that expresses the utility of transitioning from a state to a neighboring state and then acting opt…
First-Order Preconditioning via Hypergradient Descent
Ted Moskovitz, Rui Wang, Janice Lan +4
Standard gradient descent methods are susceptible to a range of issues that can impede training, such as high correlations and different scaling in parameter space.These difficulti…
LCA: Loss Change Allocation for Neural Network Training
Janice Lan, Rosanne Liu, Hattie Zhou +1
Neural networks enjoy widespread use, but many aspects of their training, representation, and operation are poorly understood. In particular, our view into the training process is…
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask
Hattie Zhou, Janice Lan, Rosanne Liu +1
The recent "Lottery Ticket Hypothesis" paper by Frankle & Carbin showed that a simple approach to creating sparse networks (keeping the large weights) results in models that are tr…
Understanding Neural Networks via Feature Visualization: A survey
Anh Nguyen, Jason Yosinski, Jeff Clune
A neuroscience method to understanding the brain is to find and study the preferred stimuli that highly activate an individual cell or groups of cells. Recent advances in machine l…
Measuring the Intrinsic Dimension of Objective Landscapes
Chunyuan Li, Heerad Farkhoor, Rosanne Liu +1
Many recently trained neural networks employ large numbers of parameters to achieve good performance. One may intuitively use the number of parameters required as a rough gauge of…
Convergent Learning: Do different neural networks learn the same representations?
Yixuan Li, Jason Yosinski, Jeff Clune +2
Recent success in training deep neural networks have prompted active investigation into the features learned on their intermediate layers. Such research is difficult because it req…
Can deep learning help you find the perfect match?
Harm de Vries, Jason Yosinski
Is he/she my type or not? The answer to this question depends on the personal preferences of the one asking it. The individual process of obtaining a full answer may generally be d…
GSNs : Generative Stochastic Networks
Guillaume Alain, Yoshua Bengio, Li Yao +4
We introduce a novel training principle for probabilistic models that is an alternative to maximum likelihood. The proposed Generative Stochastic Networks (GSN) framework is based…
How transferable are features in deep neural networks?
Jason Yosinski, Jeff Clune, Yoshua Bengio +1
Many deep neural networks trained on natural images exhibit a curious phenomenon in common: on the first layer they learn features similar to Gabor filters and color blobs. Such fi…
Deep Generative Stochastic Networks Trainable by Backprop
Yoshua Bengio, Éric Thibodeau-Laufer, Guillaume Alain +1
We introduce a novel training principle for probabilistic models that is an alternative to maximum likelihood. The proposed Generative Stochastic Networks (GSN) framework is based…