240 citations · 241 across the 4 of their papers we have counts for
5 papers · 1 filter
ForeSpark: Joint Draft and Verification Budget Scheduling for Parallel Speculative Decoding
Jaeyong Song, Daniel Wang, Pratyush Patel +3
Parallel drafters accelerate LLM decoding by proposing a whole block of draft tokens in a single forward pass, making them attractive for LLM serving. Recent systems optimize goodp…
Architecting for Hardware Diversity: Meta's End-to-End Co-Design for High-Performance AI Training on AMD GPUs
Haoyu Zhang, Arshad Rahman, Adnan Aziz +55
Meta deploys a heterogeneous fleet comprising NVIDIA GPUs, AMD GPUs, and proprietary MTIA accelerators to reduce costs and mitigate vendor lock-in. However, the extensive history o…
Crossflow: Prefill-Decode Elasticity for Agentic LLM Serving
Yi Xu, Ehsan K. Ardestani, Wenyin Fu +7
As serving capacity demand surpasses that of training, serving efficiency becomes increasingly important. Prefill-decode (P/D) disaggregation improves serving efficiency through sp…
TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory
Hasan Al Maruf, Hao Wang, Abhishek Dhanotia +7
The increasing demand for memory in hyperscale applications has led to memory becoming a large portion of the overall datacenter spend. The emergence of coherent interfaces like CX…
Live Recovery of Bit Corruptions in Datacenter Storage Systems
Amy Tai, Andrew Kryczka, Shobhit Kanaujia +6
Due to its high performance and decreasing cost per bit, flash is becoming the main storage medium in datacenters for hot data. However, flash endurance is a perpetual problem, and…