14 papers
Libra: Efficient Resource Management for Agentic RL Post-Training
Kaiwen Chen, Xin Tan, Jingzong Li +1
Reinforcement learning (RL) has emerged as a standard post-training paradigm for shaping large language models (LLMs) into capable agents. In agentic RL, the rollout stage generate…
ReLoRA: Knowledge-Reusing Adaptation for Fast Rollout of Evolving LLM Services
Yang Xu, Zihuai Xu, Hongli Xu +3
Large Language Models (LLMs) are increasingly deployed as continuously evolving services, where frequent base-model updates may invalidate previously deployed task-specific Low-Ran…
DySTop
Yizhou Shi, Qianpiao Ma, Yan Xu +4
Federated Learning (FL) has emerged as a potential distributed learning paradigm that enables model training on edge devices (i.e., workers) while preserving data privacy. However,…
Collaborative Speculative Inference for Efficient LLM Inference Serving
Luyao Gao, Jianchun Liu, Hongli Xu +3
Speculative inference is a promising paradigm employing small speculative models (SSMs) as drafters to generate draft tokens, which are subsequently verified in parallel by the tar…
Resource-Efficient Federated Fine-Tuning Large Language Models for Heterogeneous Data
Jun Liu, Yunming Liao, Hongli Xu +1
Fine-tuning large language models (LLMs) via federated learning, i.e., FedLLM, has been proposed to adapt LLMs for various downstream applications in a privacy-preserving way. To r…
A Novel Hat-Shaped Device-Cloud Collaborative Inference Framework for Large Language Models
Zuan Xie, Yang Xu, Hongli Xu +2
Recent advancements in large language models (LLMs) have catalyzed a substantial surge in demand for LLM services. While traditional cloud-based LLM services satisfy high-accuracy…