SP-Allreduce: Symmetric Tree Schedules for Tori and Beyond

Mikhail Cherniavskii, Andrey Kupavskii, Georgy Mironov +1

Allreduce is a fundamental communication primitive in large-scale distributed training, and torus interconnects remain widely deployed in high-performance systems. We introduce SP-…