Rina: Enhancing Ring-AllReduce with In-network Aggregation in Distributed Model Training

Zixuan Chen, Xuandong Liu, Minglin Li +7

Parameter Server (PS) and Ring-AllReduce (RAR) are two widely utilized synchronization architectures in multi-worker Deep Learning (DL), also referred to as Distributed Deep Learni…