4 papers
Staggered Batch Scheduling: Co-optimizing Time-to-First-Token and Throughput for High-Efficiency LLM Inference
Jian Tian, Shuailong Li, Yang Cao +8
The evolution of Large Language Model (LLM) serving towards complex, distributed architectures--specifically the P/D-separated, large-scale DP+EP paradigm--introduces distinct sche…
ESS: An Offload-Centric Latent-Cache Management Architecture for DeepSeek-V3.2-Exp
Xinhang Chen, Chao Zhang, Jiahuan He +9
DeepSeek-V3.2-Exp introduces a sparse attention mechanism that significantly reduces inference latency in long-context scenarios. Although the overall throughput has improved great…
KnapFormer: An Online Load Balancer for Efficient Diffusion Transformers Training
Kai Zhang, Peng Wang, Sai Bi +2
We present KnapFormer, an efficient and versatile framework to combine workload balancing and sequence parallelism in distributed training of Diffusion Transformers (DiT). KnapForm…
EP-CFG: Energy-Preserving Classifier-Free Guidance
Kai Zhang, Fujun Luan, Sai Bi +1
Classifier-free guidance (CFG) is widely used in diffusion models but often introduces over-contrast and over-saturation artifacts at higher guidance strengths. We present EP-CFG (…