Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
EasyBalance: Cross-Layer Load Balancing in Distributed MoE Inference
Yize Wu, Ke Gao, Ling Li +1
Load Balancing has emerged as a critical problem in expert-parallel distributed inference of Mixture-of-Experts (MoE) models. As routing distributions are typically skewed across e…
cs.LG2026
Stable-LoRA: Stabilizing Feature Learning of Low-Rank Adaptation
Yize Wu, Ke Gao, Ling Li +1
Low-Rank Adaptation (LoRA) is a widely adopted parameter-efficient method for fine-tuning Large Langauge Models. It updates the weight matrix as , where is the ori…
cs.LG2025
QiMeng-TensorOp: Automatically Generating High-Performance Tensor Operators with Hardware Primitives
Xuzhi Zhang, Shaohui Peng, Qirui Zhou +12
Computation-intensive tensor operators constitute over 90\% of the computations in Large Language Models (LLMs) and Deep Neural Networks.Automatically and efficiently generating hi…