Expediting Distributed DNN Training with Device Topology-Aware Graph Deployment

Shiwei Zhang, Xiaodong Yi, Lansong Diao +3

This paper presents TAG, an automatic system to derive optimized DNN training graph and its deployment onto any device topology, for expedited training in device- and topology- het…