MoSE: Mode-Switching Expander for Mixed LLM Training and Inference

Fan Yang, Ying Zhou, Binglei Wang +2

AI clusters increasingly run large language model (LLM) inference and training on the same fabric. Prefill-decode (P-D) disaggregation creates key-value (KV) cache transfers betwee…