HLA: Expressive Hybrid Linear Attention via Chunk-Wise Dynamic Mixing

Zhuokun Chen, Xi Lin, Xiyu Wu +3

Linear attention enables efficient long-context autoregressive decoding by compressing history into recurrent states, but this compression can make selective access to sparse and d…