GUIDE: A Global Unified Inference Engine for Deploying Large Language Models in Heterogeneous Environments

Yanyu Chen, Ganhong Huang

Efficiently deploying large language models (LLMs) in real-world scenarios remains a critical challenge, primarily due to hardware heterogeneity, inference framework limitations, a…