2 papers
cs.LG2025
Some Attention is All You Need for Retrieval
Felix Michalak, Steven Abreu
We demonstrate complete functional segregation in hybrid SSM-Transformer architectures: retrieval depends exclusively on self-attention layers. Across RecurrentGemma-2B/9B and Jamb…
cs.NE2025
Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineering
Joris Postmus, Steven Abreu
Large language models have transformed AI, yet reliably controlling their outputs remains a challenge. This paper explores activation engineering, where outputs of pre-trained LLMs…