7 papers
Verbalizable Representations Form a Global Workspace in Language Models
Wes Gurnee, Nicholas Sofroniew, Adam Pearce +13
Out of everything the human brain processes, only a small fraction is consciously accessible, in the sense of being available for verbal report, deliberate control, and flexible re…
ReM-MoA: Reasoning Memory Sustains Mixture-of-Agents Scaling
Heng Ping, Arijit Bhattacharjee, Peiyu Zhang +5
Mixture-of-Agents (MoA) architectures improve inference-time scaling by organizing multiple LLM agents into layered reasoning pipelines. However, existing MoA variants fail to sust…
RaMem: Contextual Reinstatement for Long-term Agentic Memory
Wei Yang, Bryce Kan, Shixuan Li +5
Long-term memory has become increasingly important for LLM agents that operate across extended interactions and evolving task contexts. Recent memory systems have made past experie…
Slot Machines: How LLMs Keep Track of Multiple Entities
Paul C. Bogdan, Jack Lindsey
Language models must bind entities to the attributes they possess and maintain several such binding relationships within a context. We study how multiple entities are represented a…
Thought Branches: Interpreting LLM Reasoning Requires Resampling
Uzay Macar, Paul C. Bogdan, Senthooran Rajamanoharan +1
Most work interpreting reasoning models studies only a single chain-of-thought (CoT), yet these models define distributions over many possible CoTs. We argue that studying a single…
Thought Anchors: Which LLM Reasoning Steps Matter?
Paul C. Bogdan, Uzay Macar, Neel Nanda +1
Current frontier large-language models rely on reasoning to achieve state-of-the-art performance. Many existing interpretability are limited in this area, as standard methods have…