What Attention Recalls and Recurrence Controls in Hybrid Language Models

Kirill Afendulev, Alexey Dontsov, Elena Tutubalina +1

Hybrid language models combine attention with a fixed-size recurrent state, but the role of each channel remains unclear. We introduce two cache-level interventions. Split-prefill…