95 citations · 201 across the 14 of their papers we have counts for
15 papers
There Will Be a Scientific Theory of Deep Learning
Jamie Simon, Daniel Kunin, Alexander Atanasov +11
In this paper, we make the case that a scientific theory of deep learning is emerging. By this we mean a theory which characterizes important properties and statistics of the train…
On the creation of narrow AI: hierarchy and nonlocality of neural network skills
Eric J. Michaud, Asher Parker-Sartori, Max Tegmark
We study the problem of creating strong, yet narrow, AI systems. While recent AI progress has been driven by the training of large general-purpose foundation models, the creation o…
Physics of Skill Learning
Ziming Liu, Yizhou Liu, Eric J. Michaud +2
We aim to understand physics of skill learning, i.e., how skills are learned in neural networks during training. We start by observing the Domino effect, i.e., skills are learned s…
The Geometry of Concepts: Sparse Autoencoder Feature Structure
Yuxiao Li, Eric J. Michaud, David D. Baek +3
Sparse autoencoders have recently produced dictionaries of high-dimensional vectors corresponding to the universe of concepts represented by large language models. We find that thi…
Efficient Dictionary Learning with Switch Sparse Autoencoders
Anish Mudide, Joshua Engels, Eric J. Michaud +2
Sparse autoencoders (SAEs) are a recent technique for decomposing neural network activations into human-interpretable features. However, in order for SAEs to identify all features…
Survival of the Fittest Representation: A Case Study with Modular Addition
Xiaoman Delores Ding, Zifan Carl Guo, Eric J. Michaud +2
When a neural network can learn multiple distinct algorithms to solve a task, how does it "choose" between them during training? To approach this question, we take inspiration from…
Not All Language Model Features Are One-Dimensionally Linear
Joshua Engels, Eric J. Michaud, Isaac Liao +2
Recent work has proposed that language models perform computation by manipulating one-dimensional representations of concepts ("features") in activation space. In contrast, we expl…
Opening the AI black box: program synthesis via mechanistic interpretability
Eric J. Michaud, Isaac Liao, Vedang Lad +7
We present MIPS, a novel method for program synthesis based on automated mechanistic interpretability of neural networks trained to perform the desired task, auto-distilling the le…
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Stephen Casper, Xander Davies, Claudia Shi +29
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals. RLHF has emerged as the central method used to finetune state-of…
The Quantization Model of Neural Scaling
Eric J. Michaud, Ziming Liu, Uzay Girit +1
We propose the Quantization Model of neural scaling laws, explaining both the observed power law dropoff of loss with model and data size, and also the sudden emergence of new capa…
Precision Machine Learning
Eric J. Michaud, Ziming Liu, Max Tegmark
We explore unique considerations involved in fitting ML models to data with very high precision, as is often required for science applications. We empirically compare various funct…
Omnigrok: Grokking Beyond Algorithmic Data
Ziming Liu, Eric J. Michaud, Max Tegmark
Grokking, the unusual phenomenon for algorithmic datasets where generalization happens long after overfitting the training data, has remained elusive. We aim to understand grokking…
Towards Understanding Grokking: An Effective Theory of Representation Learning
Ziming Liu, Ouail Kitouni, Niklas Nolte +3
We aim to understand grokking, a phenomenon where models generalize long after overfitting their training set. We present both a microscopic analysis anchored by an effective theor…
Understanding Learned Reward Functions
Eric J. Michaud, Adam Gleave, Stuart Russell
In many real-world tasks, it is not possible to procedurally specify an RL agent's reward function. In such cases, a reward function must instead be learned from interacting with a…
Examining the causal structures of deep neural networks using information theory
Simon Mattsson, Eric J. Michaud, Erik Hoel
Deep Neural Networks (DNNs) are often examined at the level of their response to input, such as analyzing the mutual information between nodes and data sets. Yet DNNs can also be e…