AllReduce Scheduling with Hierarchical Deep Reinforcement Learning

Yufan Wei, Mickel Liu, Wenfei Wu

AllReduce is a technique in distributed computing which saw use in many critical applications of deep learning. Existing methods of AllReduce scheduling oftentimes lack flexibility…