277 citations · 353 across the 16 of their papers we have counts for
10 papers · 1 filter
Metropolis-Hastings Dominates Importance Resampling for Policy Composition
Alexey Kurennoy, Ramil Yarullin, Fergal Reid
Post-training a large language model (LLM) often requires exploring trade-offs between multiple rewards, but retraining for each trade-off is expensive. Decoding-time policy compos…
Residual Streams Read, Recurrent States Remember: The Global Workspace in Mamba Models
Wenlong Wang, Fergal Reid
Can the global-workspace account of transformer representations extend to state-space language models? We fit Jacobian lenses to the residual streams and recurrent states of Mamba-…
Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?
Wenlong Wang, Fergal Reid
Recent work identifies a mid-depth band of verbalisable, causally potent representations in a standard feedforward transformer --- a functional analogue of a global workspace. Whet…
Protoreasoning in Tiny Transformers
Eduardo Valle, Fergal Reid
We show that tiny transformers can profitably employ a simple form of Chain of Thought, which we call protoreasoning, allowing us to study step-by-step reasoning on ~1M-parameter m…
Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers
James O' Neill, Fergal Reid
Looped, weight-tied Transformers reduce parameters by reusing a single block, but decoding still stores a separate K/V cache for every recurrence step. We show that this loop-index…
Does Verbose Chain-of-Thought Really Help? In-Distribution Evidence that Content, Not Length, Matters
Wenlong Wang, Fergal Reid
Chain-of-thought (CoT) prompting improves LLM reasoning, but the source is contested: do the intermediate steps help because they carry useful semantic content, or because conditio…
When RL Suppresses Its Own Vocabulary: Recovering Reasoning Diversity in Puzzle-to-Math Transfer
Mayug Maniparambil, Arjun Karuvally, Terrence Sejnowski +1
Reinforcement learning using verifiable rewards (RLVR) improves LLM reasoning, but the conditions under which it transfers across domains -- and why it does so -- remain under-expl…
Tiny Recursive Reasoning with Mamba-2 Attention Hybrid
Wenlong Wang, Fergal Reid
Recent work on recursive reasoning models like TRM demonstrates that tiny networks (7M parameters) can achieve strong performance on abstract reasoning tasks through latent recursi…
TopoBench: Benchmarking LLMs on Hard Topological Reasoning
Mayug Maniparambil, Nils Hoehing, Janak Kapuriya +5
Solving topological grid puzzles requires reasoning over global spatial invariants such as connectivity, loop closure, and region symmetry and remains challenging for even the most…
Low-Rank Key Value Attention
James O'Neill, Robert Clancy, Mariia Matskevichus +1
The key-value (KV) cache is a primary memory bottleneck in Transformers. We propose Low-Rank Key-Value (LRKV) attention, which reduces KV cache memory by exploiting redundancy acro…