Canary: Congestion-Aware In-Network Allreduce Using Dynamic Trees

Daniele De Sensi, Edgar Costa Molero, Salvatore Di Girolamo +2

The allreduce operation is an essential building block for many distributed applications, ranging from the training of deep learning models to scientific computing. In an allreduce…