NEST: Network- and Memory-Aware Device Placement For Distributed Deep Learning

Irene Wang, Vishnu Varma Venkata, Arvind Krishnamurthy +1

The growing scale of deep learning demands distributed training frameworks that jointly reason about parallelism, memory, and network topology. Prior works often rely on heuristic…