2 citations · 4 across the 10 of their papers we have counts for
10 papers
On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness
Matthew Nguyen, Kyle Cox, Austin Meek +1
Model capabilities have improved in large part due to scaling chain of thought. This has been a promising development for AI safety--where models verbalize their reasoning, it is p…
Automatically Finding Reward Model Biases
Atticus Wang, Iván Arcuschin, Arthur Conmy
Reward models are central to large language model (LLM) post-training. However, past work has shown that they can reward spurious or undesirable attributes such as length, format,…
Mind the Performance Gap: Capability-Behavior Trade-offs in Feature Steering
Eitan Sprejer, Oscar Agustín Stanchi, María Victoria Carro +2
Feature steering has emerged as a promising approach for controlling LLM behavior through direct manipulation of internal representations, offering advantages over prompt engineeri…
Biases in the Blind Spot: Detecting What LLMs Fail to Mention
Iván Arcuschin, David Chanin, Adrià Garriga-Alonso +1
Large Language Models (LLMs) often provide chain-of-thought (CoT) reasoning traces that appear plausible, but may hide internal biases. We call these unverbalized biases. Monitorin…
Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity
Austin Meek, Eitan Sprejer, Iván Arcuschin +2
Chain-of-thought (CoT) outputs let us read a model's step-by-step reasoning. Since any long, serial reasoning process must pass through this textual trace, the quality of the CoT i…
Base Models Know How to Reason, Thinking Models Learn When
Constantin Venhoff, Iván Arcuschin, Philip Torr +2
What do thinking language models learn during training that their base models lack? We first present an unsupervised method that discovers a model's reasoning behaviors by training…
Understanding Reasoning in Thinking Language Models via Steering Vectors
Constantin Venhoff, Iván Arcuschin, Philip Torr +2
Recent advances in large language models (LLMs) have led to the development of thinking language models that generate extensive internal reasoning chains before producing responses…
MIB: A Mechanistic Interpretability Benchmark
Aaron Mueller, Atticus Geiger, Sarah Wiegreffe +20
How can we know whether new mechanistic interpretability methods achieve real improvements? In pursuit of lasting evaluation standards, we propose MIB, a Mechanistic Interpretabili…
Chain-of-Thought Reasoning In The Wild Is Not Always Faithful
Iván Arcuschin, Jett Janiak, Robert Krzyzanowski +3
Recent studies indicate that when faced with explicit biases in prompts, models often omit mentioning these biases in their Chain-of-Thought (CoT) output, revealing that verbalized…
InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniques
Rohan Gupta, Iván Arcuschin, Thomas Kwa +1
Mechanistic interpretability methods aim to identify the algorithm a neural network implements, but it is difficult to validate such methods when the true algorithm is unknown. Thi…