6 citations · 6 across the 16 of their papers we have counts for
11 papers · 1 filter
Causal and Interpretable Structures in LLM Compositional Tasks
Gurbir Arora, Toni J. B. Liu, Jiajun Bao +2
Large language models are able to solve tasks whose answers depend on not only individual input tokens, but also on relations among them. How is such relational information represe…
The Geometry of Ignorance: LLMs Know When to Temper Bayesian Priors
Toni J. B. Liu, Jiajun Bao, Yizhou Liu +4
What does a language model predict when it has few clues? The answer lurks in its unembedding geometry: a single direction of the unembedding matrix encodes the unigram distributio…
A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In-Context Learning
Jiajun Bao, Zihao Qi, Toni J. B. Liu +4
Pretrained large language models (LLMs) have demonstrated in-context learning (ICL) capabilities for numerical inference over sequences serialized as text. Prior work has identifie…
What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness
Raphaël Sarfati, Pratyush Ranjan Tiwari, Siddharth Boppana +3
Large language models fine-tuned for forecasting can be accurate yet poorly calibrated, and their chain-of-thought (CoT) reasoning may not faithfully reflect the evidence behind a…
Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signal
Leon Bergen, Usha Bhalla, Sidharth Baskaran +14
Language-model post-training is the main stage at which model behavior is shaped, yet it still largely involves optimization of scalar rewards that summarize diverse desiderata. Th…
Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space
Eric Bigelow, Raphaël Sarfati, Daniel Wurgaft +5
Large Language Models (LLMs) update their behavior in context, which can be viewed as a form of Bayesian inference. However, the structure of the latent hypothesis space over which…
Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Daniel Wurgaft, Can Rager, Matthew Kowal +13
Neural representations carry rich geometric structure; but does that structure causally shape behavior? To address this question, we intervene along paths through activation space…
Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
Sheridan Feucht, Tal Haklay, Usha Bhalla +9
Does structure in representations imply structure in computation? We study how Llama-3.1-8B reasons over cyclic concepts (e.g., "what month is six months after August?"). Even thou…
Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought
Siddharth Boppana, Annabel Ma, Max Loeffler +5
We provide evidence of performative chain-of-thought (CoT) in reasoning models, where a model becomes strongly confident in its final answer, but continues generating tokens withou…
The Shape of Beliefs: Geometry, Dynamics, and Interventions along Representation Manifolds of Language Models' Posteriors
Raphaël Sarfati, Eric Bigelow, Daniel Wurgaft +6
Large language models (LLMs) form implicit beliefs (posteriors over latent variables) from prompts, but we lack a mechanistic account of how these beliefs are encoded in representa…
Jacobian Scopes: token-level causal attributions in LLMs
Toni J. B. Liu, Baran Zadeoğlu, Nicolas Boullé +3
Large language models (LLMs) make next-token predictions based on clues present in their context, such as semantic descriptions and in-context examples. Yet, elucidating which prio…