Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs

Xing Chen, Rong Shi, Lu Zhao +4

LLM-based applications have been widely used in various industries, but with the increasing of models size, an efficient large language model (LLM) inference system is an urgent pr…