4 papers
Blurry Window Attention
Axel Laborieux, Christos Sourmpis, Juan Gabriel Kostelec +1
The Softmax Attention operation in Transformer language models has a quadratic complexity in the sequence length and a growing state size in the form of KV cache, which becomes a b…
IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning
Yinhan He, Yaochen Zhu, Mingjia Shi +5
Large language models increasingly rely on long chains of thought to improve accuracy, yet such gains come with substantial inference-time costs. We revisit token-efficient post-tr…
AllMem: A Memory-centric Recipe for Efficient Long-context Modeling
Ziming Wang, Xiang Wang, Kailong Peng +5
Large Language Models (LLMs) encounter significant performance bottlenecks in long-sequence tasks due to the computational complexity and memory overhead inherent in the self-atten…
FlashEVA: Accelerating LLM inference via Efficient Attention
Juan Gabriel Kostelec, Qinghai Guo
Transformer models have revolutionized natural language processing, achieving state-of-the-art performance and demonstrating remarkable scalability. However, their memory demands,…