67 citations · 79 across the 6 of their papers we have counts for
6 papers
LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Servers
Enda Yu, Dezun Dong, Zhaoning Zhang +6
Mixture-of-Experts (MoE) models face memory and PCIe latency bottlenecks when deployed on commodity hardware. Offloading expert weights to CPU memory results in PCIe transfer laten…
Nezha: Breaking Multi-Rail Network Barriers for Distributed DNN Training
Enda Yu, Dezun Dong, Xiangke Liao
In distributed deep learning, communication remains a critical bottleneck. While modern hardware advances rapidly, over 60 percent of production HPC systems still rely on legacy in…
Large Language Models are Few-Shot Summarizers: Multi-Intent Comment Generation via In-Context Learning
Mingyang Geng, Shangwen Wang, Dezun Dong +5
Code comment generation aims at generating natural language descriptions for a code snippet to facilitate developers' program comprehension activities. Despite being studied for a…
CD-SGD: Distributed Stochastic Gradient Descent with Compression and Delay Compensation
Enda Yu, Dezun Dong, Yemao Xu +2
Communication overhead is the key challenge for distributed training. Gradient compression is a widely used approach to reduce communication traffic. When combining with parallel c…
OD-SGD: One-step Delay Stochastic Gradient Descent for Distributed Training
Yemao Xu, Dezun Dong, Weixia Xu +1
The training of modern deep learning neural network calls for large amounts of computation, which is often provided by GPUs or other specific accelerators. To scale out to achieve…
Communication optimization strategies for distributed deep neural network training: A survey
Shuo Ouyang, Dezun Dong, Yemao Xu +1
Recent trends in high-performance computing and deep learning have led to the proliferation of studies on large-scale deep neural network training. However, the frequent communicat…