Avoiding Cross-Datacenter Collective Congestion via Disaggregated Buffering

Mariano Scazzariello, Noga H. Rotman, Dima Gavrilenko +6

LLM training at the scale of tens of thousands of GPUs now spans multiple datacenters (DC), making cross-DC collectives over long-haul links unavoidable. A critical and overlooked…