P/D-Device: Disaggregated Large Language Model between Cloud and Devices

Yibo Jin, Yixu Xu, Yue Chen +27

Serving disaggregated large language models has been widely adopted in industrial practice for enhanced performance. However, too many tokens generated in decoding phase, i.e., occ…