5 papers
TomasuLLM: Out-of-Order Speculative Execution for LLM Agents
Jiangnan Yu, Ceyu Xu, Mengming Li +6
Long-running tools can dominate coding-agent latency: compilers, test suites, and repository commands take seconds to minutes while the agent idles. This observation stall presents…
Memory Compression for High-Fanout Agent Sandboxes
Mengming Li, Ceyu XU, Qijun Zhang +4
High-fanout agent workloads create a growing memory bottleneck because a single task may spawn many concurrent sandbox sessions. Yet these sandboxes are far from independent: they…
Ave: Guiding Agentic GPU Optimization Using Data-Flow Invariants
Haohui Mai, Xiaoyan Guo, Xiangyun Ding +7
LLM coding agents can generate correct GPU kernels, but their performance still trails expert libraries. Reaching peak throughput requires coordinating low-level optimizations such…
AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUs
Ran Yan, Youhe Jiang, Tianyuan Wu +7
Maximizing training throughput and cost-efficiency of RL for LLMs is essential to democratize this advanced technique. One promising but challenging approach is to deploy such a co…
FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel
Ran Yan, Youhe Jiang, Zhuoming Chen +3
Recent advances in sparse attention mechanisms have demonstrated strong potential for reducing the computational cost of long-context training and inference in large language model…