Distributed Generative Inference of LLM at Internet Scales with Multi-Dimensional Communication Optimization

Jiu Chen, Shuangyan Yang, Xu Xiong +4

Decentralized LLM inference distributes computation among heterogeneous nodes across the internet, offering a performant and cost-efficient solution, alternative to traditional cen…