HyLRA: Hybrid Layer Reuse Attention for Efficient Long-Context Inference

Xuan Ai, Qingqing Yang, Peng Wang +4

Long-context inference in Large Language Models (LLMs) is bottlenecked by the quadratic computation complexity of attention and the substantial memory footprint of Key-Value (KV) c…