NetReduce: RDMA-Compatible In-Network Reduction for Distributed DNN Training Acceleration

Shuo Liu, Qiaoling Wang, Junyi Zhang +5

We present NetReduce, a novel RDMA-compatible in-network reduction architecture to accelerate distributed DNN training. Compared to existing designs, NetReduce maintains a reliable…