1 citations · 2 across the 3 of their papers we have counts for
5 papers
HiNa-MoE: High-Performance, Non-Intrusive MoE Inference on CPUs with Matrix Engines
Weiling Yang, Junwen Zhang, Dezun Dong +4
Mixture-of-Experts (MoE) inference is increasingly deployed in local and on-premise environments, where expert parameters often exceed GPU memory capacity. In latency-sensitive, lo…
Physically Partitioned KVCache Format for CPU--GPU Load Balancing in MoE Inference
Enda Yu, Dezun Dong, Xiangke Liao
Single-GPU long-context inference with Mixture-of-Experts (MoE) models requires spilling the key-value cache (KVCache) to CPU memory. The spilled KV serves two complementary purpos…
LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Servers
Enda Yu, Dezun Dong, Zhaoning Zhang +6
Mixture-of-Experts (MoE) models face memory and PCIe latency bottlenecks when deployed on commodity hardware. Offloading expert weights to CPU memory results in PCIe transfer laten…
Nezha: Breaking Multi-Rail Network Barriers for Distributed DNN Training
Enda Yu, Dezun Dong, Xiangke Liao
In distributed deep learning, communication remains a critical bottleneck. While modern hardware advances rapidly, over 60 percent of production HPC systems still rely on legacy in…
CD-SGD: Distributed Stochastic Gradient Descent with Compression and Delay Compensation
Enda Yu, Dezun Dong, Yemao Xu +2
Communication overhead is the key challenge for distributed training. Gradient compression is a widely used approach to reduce communication traffic. When combining with parallel c…