32 citations · 44 across the 4 of their papers we have counts for
5 papers
ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
Sangjin Choi, Sukmin Cho, Yifan Xiong +3
In prefill-decode (PD) disaggregated LLM serving, each request is assigned to a decode worker after prefill. Existing decode routers balance only load; for mixture-of-experts (MoE)…
An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Context
Xiaoyu Chen, Xiangming Zhu, Yufeng Zheng +8
One of the key challenges in deploying RL to real-world applications is to adapt to variations of unknown environment contexts, such as changing terrains in robotic tasks and fluct…
Tutel: Adaptive Mixture-of-Experts at Scale
Changho Hwang, Wei Cui, Yifan Xiong +12
Sparsely-gated mixture-of-experts (MoE) has been widely adopted to scale deep learning models to trillion-plus parameters with fixed computational cost. The algorithmic performance…
NetKernel: Making Network Stack Part of the Virtualized Infrastructure
Zhixiong Niu, Hong Xu, Peng Cheng +4
This paper presents a system called NetKernel that decouples the network stack from the guest virtual machine and offers it as an independent module. NetKernel represents a new par…
Micro-burst in Data Centers: Observations, Implications, and Applications
Danfeng Shan, Fengyuan Ren, Peng Cheng +1
Micro-burst traffic is not uncommon in data centers. It can cause packet dropping, which results in serious performance degradation (e.g., Incast problem). However, current solutio…