Equinox: Holistic Fair Scheduling in Serving Large Language Models

Zhixiang Wei, James Yen, Jingyi Chen +11

We address the limitations of current LLM serving with a dual-counter framework separating user and operator perspectives. The User Fairness Counter measures quality of service via…