4 citations · 4 across the 13 of their papers we have counts for
4 papers · 1 filter
Would this change your answer? Evaluating Explanations of LLM Behavior In The Wild with Counterfactual Experiments
Adam Karvonen, Euan Ong, Subhash Kantamneni +1
Many areas of AI research, such as language model interpretability and chain of thought faithfulness, seek to explain model behaviors. But what constitutes a "good" explanation? In…
Building Better Activation Oracles
Jan Bauer, Celeste De Schamphelaere, Adam Karvonen +2
Activation Oracles (AOs) are promising methods for interpreting residual stream activations. However, current AOs face important issues, such as hallucinations and vagueness. Addit…
Negation Neglect: When models fail to learn negations in training
Harry Mayne, Lev McKinney, Jan Dubiński +3
We introduce Negation Neglect, where finetuning LLMs on documents that flag a claim as false makes them believe the claim is true. For example, models are finetuned on documents th…
Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
Adam Karvonen, James Chua, Clément Dumas +8
Large language model (LLM) activations are notoriously difficult to understand, with most existing techniques using complex, specialized methods for interpreting them. Recent work…