Optimizing Resource Allocation for Geographically-Distributed Inference by Large Language Models

Tingyang Sun, Ting He, Bo Ji +1

Large language models have demonstrated extraordinary performance in many AI tasks but are expensive to use, even after training, due to their requirement of high-end GPUs. Recentl…