2 citations · 2 across the 7 of their papers we have counts for
4 papers · 1 filter
How Transparent is DiffusionGemma?
Joshua Engels, Callum McDougall, Bilal Chughtai +11
LLM reasoning transparency is a critical affordance for understanding model decisions, mitigating misuse and misalignment, and debugging surprising model behaviors. However, Diffus…
What's the plan? Metrics for implicit planning in LLMs and their application to rhyme generation and question answering
Jim Maar, Denis Paperno, Callum Stuart McDougall +1
Prior work suggests that language models, while trained on next token prediction, show implicit planning behavior: they may select the next token in preparation to a predicted futu…
SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
Adam Karvonen, Can Rager, Johnny Lin +12
Sparse autoencoders (SAEs) are a popular technique for interpreting language model activations, and there is extensive recent work on improving SAE effectiveness. However, most pri…
Copy Suppression: Comprehensively Understanding an Attention Head
Callum McDougall, Arthur Conmy, Cody Rushing +2
We present a single attention head in GPT-2 Small that has one main role across the entire training distribution. If components in earlier layers predict a certain token, and this…