4 papers · 1 filter
FreeBalance: Pre-Routing Online Moe Load Balancing via Residual Workload Prediction
Pengfei Chen, Yize Wu, Shouxu Kuang +2
Load imbalance poses a major bottleneck to the efficiency of expert parallelism in distributed inference of Mixture-of-Experts (MoE) models. The most heavily loaded rank stalls glo…
EasyBalance: Cross-Layer Load Balancing in Distributed MoE Inference
Yize Wu, Ke Gao, Ling Li +1
Load Balancing has emerged as a critical problem in expert-parallel distributed inference of Mixture-of-Experts (MoE) models. As routing distributions are typically skewed across e…
Director: Instance-aware Gaussian Splatting for Dynamic Scene Modeling and Understanding
Yuheng Jiang, Yiwen Cai, Zihao Wang +5
Volumetric video seeks to model dynamic scenes as temporally coherent 4D representations. While recent Gaussian-based approaches achieve impressive rendering fidelity, they primari…
Stable-LoRA: Stabilizing Feature Learning of Low-Rank Adaptation
Yize Wu, Ke Gao, Ling Li +1
Low-Rank Adaptation (LoRA) is a widely adopted parameter-efficient method for fine-tuning Large Langauge Models. It updates the weight matrix as , where is the ori…