Hierarchical Prediction-based Management for LMaaS Systems

Zhihan Jiang, Yujie Huang, Guangba Yu +3

Large Language Models (LLMs) have revolutionized numerous domains, driving the rise of Language-Model-as-a-Service (LMaaS) platforms that process millions of queries daily. These p…