6 papers
Demystifying Data Organization for Enhanced LLM Training
Yalun Dai, Yangyu Huang, Tongshen Yang +8
Large Language Models (LLMs) have revolutionized various fields, yet their training efficiency is heavily reliant on effective data curation. While data selection has been widely s…
ESPO: Early-Stopping Proximal Policy Optimization
Zihang Li, Rui Zhou, Yingcheng Shi +8
When a large language model under reinforcement learning commits a wrong reasoning step early in a trajectory, standard algorithms force it to keep generating until the maximum hor…
RTP-LLM: High-Performance Alibaba LLM Inference Engine
Boyu Tan, Jiarui Guo, Zongwei Lv +26
Large Language Models (LLMs) have revolutionized AI applications, but deploying them at scale presents significant challenges. We present RTP-LLM, a high-performance inference engi…
ConMoE: Expert-Pool Consolidation via Prototype Reassignment for MoE Compression
Yilun Yao, Jiaming Pan, Elsie Dai +3
Mixture-of-Experts (MoE) language models reduce per-token computation but still require storing and serving all experts, making deployment memory-intensive. Existing post-training…
Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning
Peizhuang Cong, Wenpu Liu, Wenhan Yu +2
Large language models (LLMs) have demonstrated remarkable success across various tasks, accompanied by a continuous increase in their parameter size. Parameter-efficient fine-tunin…
BATON: Enhancing Batch-wise Inference Efficiency for Large Language Models via Dynamic Re-batching
Peizhuang Cong, Qizhi Chen, Haochen Zhao +1
The advanced capabilities of Large Language Models (LLMs) have inspired the development of various interactive web services or applications, such as ChatGPT, which offer query infe…