2 papers
cs.DC2026
BatchGen: An Architecture for Scalable and Efficient Batch Inference
Tairan Xu, Leyang Xue, Zhan Lu +7
Batch inference has become a central mode of AI computation, yet existing inference engines still rely on execution models designed for interactive serving. When scaled to millions…
cs.DC2025
MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching
Tairan Xu, Leyang Xue, Zhan Lu +2
This paper presents MoE-Gen, a high-throughput MoE inference system optimized for single-GPU execution. Existing inference systems rely on model-based or continuous batching strate…