4 papers
Interpreting Language Model Parameters
Lucius Bushnaq, Dan Braun, Oliver Clive-Griffin +5
Mechanistic interpretability aims to uncover how neural networks use their parameters to implement their impressive neural algorithms. Although previous work has uncovered substant…
LLMs Can Annotate Attribution Graphs
Ameen Patel, Max Zhang, Nathan Hu
Circuit tracing is an exciting technique for revealing the internal computation of language models, but it requires a time-intensive manual step of grouping individual features or…
Measuring Sparse Autoencoder Feature Sensitivity
Claire Tian, Katherine Tian, Nathan Hu
Sparse Autoencoder (SAE) features have become essential tools for mechanistic interpretability research. SAE features are typically characterized by examining their activating exam…
Meta-Learning Online Adaptation of Language Models
Nathan Hu, Eric Mitchell, Christopher D. Manning +1
Large language models encode impressively broad world knowledge in their parameters. However, the knowledge in static language models falls out of date, limiting the model's effect…