240 citations · 241 across the 4 of their papers we have counts for
3 papers · 2 filters
ForeSpark: Joint Draft and Verification Budget Scheduling for Parallel Speculative Decoding
Jaeyong Song, Daniel Wang, Pratyush Patel +3
Parallel drafters accelerate LLM decoding by proposing a whole block of draft tokens in a single forward pass, making them attractive for LLM serving. Recent systems optimize goodp…
Architecting for Hardware Diversity: Meta's End-to-End Co-Design for High-Performance AI Training on AMD GPUs
Haoyu Zhang, Arshad Rahman, Adnan Aziz +55
Meta deploys a heterogeneous fleet comprising NVIDIA GPUs, AMD GPUs, and proprietary MTIA accelerators to reduce costs and mitigate vendor lock-in. However, the extensive history o…
Crossflow: Prefill-Decode Elasticity for Agentic LLM Serving
Yi Xu, Ehsan K. Ardestani, Wenyin Fu +7
As serving capacity demand surpasses that of training, serving efficiency becomes increasingly important. Prefill-decode (P/D) disaggregation improves serving efficiency through sp…