From the 1 of 4 linked papers with an AI index.
4 papers
FlashDiff: Efficient Regional Execution and Scheduling for Diffusion Model Serving
Yaqi Qiao, Ping He, Songrun Xie +4
FlashDiff is a system that speeds up diffusion model inference by dynamically selecting which latent regions need further processing and efficiently scheduling those regions across…
SoundWeaver: Semantic Warm-Starting for Text-to-Audio Diffusion Serving
Ayush Barik, Sofia Stoica, Nikhil Sarda +4
Text-to-audio diffusion models produce high-fidelity audio but require tens of function evaluations (NFEs), incurring multi-second latency and limited throughput. We present SoundW…
HyGen: Efficient LLM Serving via Elastic Online-Offline Request Co-location
Ting Sun, Penghan Wang, Fan Lai
Large language models (LLMs) have facilitated a wide range of applications with distinct service-level objectives (SLOs), from latency-sensitive online tasks like interactive chatb…
IC-Cache: Efficient Large Language Model Serving via In-context Caching
Yifan Yu, Yu Gan, Nikhil Sarda +7
Large language models (LLMs) have excelled in various applications, yet serving them at scale is challenging due to their substantial resource demands and high latency. Our real-wo…