Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation

Shaoyuan Chen, Wencong Xiao, Yutong Lin +5

Transformer-based large language models (LLMs) exhibit impressive performance in generative tasks but also introduce significant challenges in real-world serving due to inefficient…