From the 1 of 8 linked papers with an AI index.
8 papers
Verbalizable Representations Form a Global Workspace in Language Models
Wes Gurnee, Nicholas Sofroniew, Adam Pearce +13
Out of everything the human brain processes, only a small fraction is consciously accessible, in the sense of being available for verbal report, deliberate control, and flexible re…
A Shared Subcircuit Lets LLMs Count Down Across Tasks
Jacob Dunefsky, Wes Gurnee, Emmanuel Ameisen
The paper identifies a "countdown subcircuit" in Llama-3.1-70B-Instruct that compares the current token position to a target length, enabling the model to track remaining tokens fo…
Mechanisms of Introspective Awareness
Uzay Macar, Li Yang, Atticus Wang +3
Recent work has shown that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept -- a phenomenon termed "introsp…
SAE-StatSteer: Statistical Consensus Feature Selection for Optimization-Free Activation Steering of Large Language Models
Oshayer Siddique, J. M Areeb Uzair Alam, Md Jobayer Rahman Rafy +3
Activation steering adds a residual-stream direction at inference time, providing lightweight behavioral control without fine-tuning. Sparse autoencoders (SAEs) can make such inter…
Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
Adly Templeton, Tom Conerly, Jonathan Marcus +23
We demonstrate that sparse autoencoders can extract interpretable features from Claude 3 Sonnet, a production-scale language model, addressing the open question of whether dictiona…
Latent Planning Emerges with Scale
Michael Hanna, Emmanuel Ameisen
LLMs can perform seemingly planning-intensive tasks, like writing coherent stories or functioning code, without explicitly verbalizing a plan; however, the extent to which they imp…