Self-Pruned Key-Value Attention: Learning When to Write by Predicting Future Utility

Gergely Szilvasy, Manuel Faysse, Maria Lomeli +5

Under modern test-time compute and agentic paradigms, language models process ever-longer sequences. Efficient text generation with transformer architectures is increasingly constr…