12 citations · 15 across the 5 of their papers we have counts for
5 papers
SoundWeaver: Compositional Warm-Starting for Text-to-Audio Diffusion Serving
Ayush Barik, Sofia Stoica, Nikhil Sarda +4
Text-to-audio (T2A) diffusion models generate high-quality audio but require tens of neural function evaluations (NFEs), resulting in substantial inference latency. Existing accele…
JITServe: SLO-aware LLM Serving with Imprecise Request Information
Wei Zhang, Zhiyu Wu, Yi Mu +5
The integration of Large Language Models (LLMs) into applications ranging from interactive chatbots to multi-agent systems has introduced a wide spectrum of service-level objective…
IC-Cache: Efficient Large Language Model Serving via In-context Caching
Yifan Yu, Yu Gan, Nikhil Sarda +7
Large language models (LLMs) have excelled in various applications, yet serving them at scale is challenging due to their substantial resource demands and high latency. Our real-wo…
PAIGE: Examining Learning Outcomes and Experiences with Personalized AI-Generated Educational Podcasts
Tiffany D. Do, Usama Bin Shafqat, Elsie Ling +1
Generative AI is revolutionizing content creation and has the potential to enable real-time, personalized educational experiences. We investigated the effectiveness of converting t…
Prompt Cache: Modular Attention Reuse for Low-Latency Inference
In Gim, Guojun Chen, Seung-seob Lee +3
We present Prompt Cache, an approach for accelerating inference for large language models (LLM) by reusing attention states across different LLM prompts. Many input prompts have ov…