24 citations · 47 across the 8 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
cs.DC2023★ 4 cited
Punica: Multi-Tenant LoRA Serving
Lequn Chen, Zihao Ye, Yongji Wu +3
Low-rank adaptation (LoRA) has become an important and popular method to adapt pre-trained models to specific domains. We present Punica, a system to serve multiple LoRA models in…
cs.LG2023★ 24 cited
Atom: Low-bit Quantization for Efficient and Accurate LLM Serving
Yilong Zhao, Chien-Yu Lin, Kan Zhu +7
The growing demand for Large Language Models (LLMs) in applications such as content generation, intelligent chatbots, and sentiment analysis poses considerable challenges for LLM s…
cs.DC2023★ 1 cited
Symphony: Optimized DNN Model Serving using Deferred Batch Scheduling
Lequn Chen, Weixin Deng, Anirudh Canumalla +4
Having large batch sizes is one of the most critical aspects of increasing the accelerator efficiency and the performance of DNN model inference. However, existing model serving sy…