LLM Inference Under Bursty Workload Distribution: Modifying the WAIT Algorithm

Anjali Gangadhar Katageria, Shobha Rani, Raghu Nandan Sengupta

Large Language Models (LLMs) such as ChatGPT and Claude are widely used for information retrieval and problem-solving. Recent work has focused on improving scheduling algorithms to…