Some Attention is All You Need for Retrieval

Felix Michalak, Steven Abreu

We demonstrate complete functional segregation in hybrid SSM-Transformer architectures: retrieval depends exclusively on self-attention layers. Across RecurrentGemma-2B/9B and Jamb…