11 citations · 16 across the 13 of their papers we have counts for
6 papers · 1 filter
Epoch: Compiling Diffusion Blocks for Sparse MoE Serving
Jianian Zhu, Hang Wu, Yinghui Li +3
Diffusion language models generate text by refining a fixed-size block of token positions through many forward passes, a loop that does not match the per-forward execution unit use…
RailS: Load Balancing for All-to-All Communication in Distributed Mixture-of-Experts Training
Heng Xu, Zhiwei Yu, Chengze Du +5
Training Mixture-of-Experts (MoE) models introduces sparse and highly imbalanced all-to-all communication that dominates iteration time. Conventional load-balancing methods fail to…
NineToothed: A Triton-Based High-Level Domain-Specific Language for Machine Learning
Jiacheng Huang, Zimin Li, Yinghui Li +1
The emergence of deep learning domain-specific languages (DSLs) has substantially reduced the obstacles in developing high-performance, cross-platform compute kernels. However, cur…
Temporal-Aware GPU Resource Allocation for Distributed LLM Inference via Reinforcement Learning
Chengze Du, Zhiwei Yu, Heng Xu +3
The rapid growth of large language model (LLM) services imposes increasing demands on distributed GPU inference infrastructure. Most existing scheduling systems follow a reactive p…
FlexPie: Accelerate Distributed Inference on Edge Devices with Flexible Combinatorial Optimization[Technical Report]
Runhua Zhang, Hongxu Jiang, Jinkun Geng +3
The rapid advancement of deep learning has catalyzed the development of novel IoT applications, which often deploy pre-trained deep neural network (DNN) models across multiple edge…
Xenos: Dataflow-Centric Optimization to Accelerate Model Inference on Edge Devices
Zhang Runhua, Jiang Hongxu, Tian Fangzheng +7
Edge computing has been emerging as a popular scenario for model inference. However, the inference performance on edge devices (e.g., Multi-Core DSP, FGPA, etc.) suffers from ineff…