2 papers
cs.CL2026
SPLA: Block Sparse Plus Linear Attention for Long Context Modeling
Bailin Wang, Dan Friedman, Tao Lei +1
Block-wise sparse attention offers significant efficiency gains for long-context modeling, yet existing methods often suffer from low selection fidelity and cumulative contextual l…
cs.CL2025
RATTENTION: Towards the Minimal Sliding Window Size in Local-Global Attention Models
Bailin Wang, Chang Lan, Chong Wang +1
Local-global attention models have recently emerged as compelling alternatives to standard Transformers, promising improvements in both training and inference efficiency. However,…