6 papers
Cordon: Semantic Transactions for Tool-Using LLM Agents
Zheng Chen, Hanqing Liu, Duling Xu +4
Tool-using LLM agents are shifting the unit of computation from explicit human-issued commands to model-driven tasks with stateful consequences. Yet today's agent runtimes still ex…
SkillSmith: Compiling Agent Skills into Boundary-Guided Runtime Interfaces
Duling Xu, Zheng Chen, Zaifeng Pan +4
Recently, skills have been widely adopted in large language model (LLM)-based agent systems across various domains. In existing frameworks, skills are typically injected into the a…
AuroraRL: Fast, Fault-Tolerant, and Cost-Efficient Reinforcement Learning over Decentralized Network
Chaoyi Ruan, Geng Luo, Xinyi Wan +12
LLM reinforcement learning (RL) requires frequent synchronization of large model parameters between the trainer and distributed rollout actors. High-throughput RL post-training the…
Cortex: Achieving Low-Latency, Cost-Efficient Remote Data Access For LLM via Semantic-Aware Knowledge Caching
Chaoyi Ruan, Chao Bi, Kaiwen Zheng +3
Large Language Model (LLM) agents tackle data-intensive tasks such as deep research and code generation. However, their effectiveness depends on frequent interactions with knowledg…
Revisiting Parameter Server in LLM Post-Training
Xinyi Wan, Penghui Qi, Guangxing Huang +3
Modern data parallel (DP) training favors collective communication over parameter servers (PS) for its simplicity and efficiency under balanced workloads. However, the balanced wor…
PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization
Xinyi Wan, Penghui Qi, Guangxing Huang +2
Pipeline parallelism (PP) is widely used for training large language models (LLMs), yet its scalability is often constrained by high activation memory consumption as the number of…