Bole: Efficient Tree Speculation for Hybrid-Attention Language Models

Li Wang, Yi Su, Xiabao Wu +9

Hybrid-attention large language models combine full attention with recurrent linear attention to reduce long-context inference costs, yet their autoregressive decoding remains memo…