Understanding and Improving Communication Performance in Multi-node LLM Inference

Prajwal Singhania, Siddharth Singh, Lannie Dalton Hough +4

As large language models (LLMs) continue to grow in size, distributed inference has become increasingly important. Model-parallel strategies must now efficiently scale not only acr…