Resource Management for GPT-based Model Deployed on Clouds: Challenges, Solutions, and Future Directions

Yongkang Dang, Minxian Xu, Kejiang Ye

The widespread adoption of the large language model (LLM), e.g. Generative Pre-trained Transformer (GPT), deployed on cloud computing environment (e.g. Azure) has led to a huge inc…