Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling

Jingfeng Wu, Yiyuan He, Minxian Xu +3

The rise of large language models (LLMs) has created new opportunities across various fields but has also introduced significant challenges in resource management. Current LLM serv…