4 papers
FlashDiff: Efficient Regional Execution and Scheduling for Diffusion Model Serving
Yaqi Qiao, Ping He, Songrun Xie +4
Diffusion models have become the central backbone for modern image, video, and audio generation, but their efficient service remains a challenge. Unlike autoregressive decoding, di…
SoundWeaver: Compositional Warm-Starting for Text-to-Audio Diffusion Serving
Ayush Barik, Sofia Stoica, Nikhil Sarda +4
Text-to-audio (T2A) diffusion models generate high-quality audio but require tens of neural function evaluations (NFEs), resulting in substantial inference latency. Existing accele…
HyGen: Efficient LLM Serving via Elastic Online-Offline Request Co-location
Ting Sun, Penghan Wang, Fan Lai
Large language models (LLMs) have facilitated a wide range of applications with distinct service-level objectives (SLOs), from latency-sensitive online tasks like interactive chatb…
IC-Cache: Efficient Large Language Model Serving via In-context Caching
Yifan Yu, Yu Gan, Nikhil Sarda +7
Large language models (LLMs) have excelled in various applications, yet serving them at scale is challenging due to their substantial resource demands and high latency. Our real-wo…