4 papers
Large Language Model Reasoning Failures
Peiyang Song, Pengrui Han, Noah Goodman
Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, achieving impressive results across a wide range of tasks. Despite these advances, significant reason…
Bayesian scaling laws for in-context learning
Aryaman Arora, Dan Jurafsky, Christopher Potts +1
In-context learning (ICL) is a powerful technique for getting language models to perform complex tasks with no training updates. Prior work has established strong correlations betw…
Emergent Symbol-like Number Variables in Artificial Neural Networks
Satchel Grant, Noah D. Goodman, James L. McClelland
What types of numeric representations emerge in neural systems, and what would a satisfying answer to this question look like? In this work, we interpret Neural Network (NN) soluti…
Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability
Atticus Geiger, Duligur Ibeling, Amir Zur +8
Causal abstraction provides a theoretical foundation for mechanistic interpretability, the field concerned with providing intelligible algorithms that are faithful simplifications…