Trivance: Latency-Optimal AllReduce by Shortcutting Multiport Networks

Anton Juerss, Vamsi Addanki, Stefan Schmid

AllReduce is a fundamental collective communication operation in distributed computing and a key performance bottleneck for large-scale training and inference. Its completion time…