8 papers
FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications
Krish Agarwal, Zhuoming Chen, Yanyuan Qin +3
Real-time multimodal applications, including voice agents and interactive video generation, compose heterogeneous models into pipelines whose efficient deployment requires applicat…
Vortex: Efficient and Programmable Sparse Attention Serving for AI Agents
Zhuoming Chen, Xinrui Zhong, Qilong Feng +5
Sparse attention is becoming increasingly important for serving large language models (LLMs) as generation lengths continue to grow. However, deploying and evaluating new sparse at…
WWW.Serve: Interconnecting Global LLM Services through Decentralization
Huanyu Wang, Ziyu Xia, Zhuoming Chen +1
Large language model (LLM) services are mostly centralized, leading to scalability bottlenecks and underutilization of substantial scattered GPU resources. While decentralization o…
MonarchRT: Efficient Attention for Real-Time Video Generation
Krish Agarwal, Zhuoming Chen, Cheng Luo +5
Real-time video generation with Diffusion Transformers is bottlenecked by the quadratic cost of 3D self-attention, especially in real-time regimes that are both few-step and autore…
Jackpot: Optimal Budgeted Rejection Sampling for Extreme Actor-Policy Mismatch Reinforcement Learning
Zhuoming Chen, Hongyi Liu, Yang Zhou +2
Reinforcement learning (RL) for large language models (LLMs) remains expensive, particularly because the rollout is expensive. Decoupling rollout generation from policy optimizatio…
Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
Haizhong Zheng, Yang Zhou, Brian R. Bartoldson +4
Reinforcement learning, such as PPO and GRPO, has powered recent breakthroughs in LLM reasoning. Scaling rollout to sample more prompts enables models to selectively use higher-qua…