6 papers
Scaling LLM Inference Beyond Amdahl`s Limits via Eliminating Non-Scalable Overheads
Alan Zhao, Cyril Y. He, Wei Xu
Deployers of online LLM services usually seek to maximize cluster-wide performance given a fixed number of GPUs. Tensor parallelism (TP) is necessary to fit modern models but scale…
SiDP: Memory-Efficient Data Parallelism for Offline LLM Inference
Alan Zhao, Cyril Y. He
The rapid adoption of large language models (LLMs) has shifted a substantial portion of inference workloads into throughput-oriented offline regimes, where fully utilizing GPU comp…
FluxMoE: Decoupling Expert Residency for High-Performance MoE Serving
Qingxiu Liu, Cyril Y. He, Hanser Jiang +3
Mixture-of-Experts (MoE) models have become a dominant paradigm for scaling large language models, but their rapidly growing parameter sizes introduce a fundamental inefficiency du…
Micro-Macro Retrieval: Reducing Long-Form Hallucination in Large Language Models
Yujie Feng, Jian Li, Zhihan Zhou +7
Large Language Models (LLMs) achieve impressive performance across many tasks but remain prone to hallucination, especially in long-form generation where redundant retrieved contex…
Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimization
Jian Li, Shenglin Yin, Yujia Zhang +4
Direct Preference Optimization (DPO) is a widely used reinforcement learning from human feedback (RLHF) method across various domains. Recent research has increasingly focused on t…
AIMMerging: Adaptive Iterative Model Merging Using Training Trajectories for Language Model Continual Learning
Yujie Feng, Jian Li, Xiaoyu Dong +8
Continual learning (CL) is essential for deploying large language models (LLMs) in dynamic real-world environments without the need for costly retraining. Recent model merging-base…