ConSA: Controllable Sparsity in Hybrid Attention via Learnable Allocation

Yao Chen, Yinqi Yang, Junyuan Shang +6

Hybrid architectures combining full attention (FA) and sliding-window attention (SWA) are a promising paradigm for efficient LLM inference. However, existing methods typically rely…