Placeto: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning

Abstract

We present Placeto, a reinforcement learning (RL) approach to efficiently find device placements for distributed neural network training. Unlike prior approaches that only find a device placement for a specific computation graph, Placeto can learn generalizable device placement policies that can be applied to any graph. We propose two key ideas in our approach: (1) we represent the policy as performing iterative placement improvements, rather than outputting a placement in one shot; (2) we use graph embeddings to capture relevant information about the structure of the computation graph, without relying on node labels for indexing. These ideas allow Placeto to train efficiently and generalize to unseen graphs. Our experiments show that Placeto requires up to 6.1x fewer training steps to find placements that are on par with or better than the best placements found by prior approaches. Moreover, Placeto is able to learn a generalizable placement policy for any given family of graphs, which can then be used without any retraining to predict optimized placements for unseen graphs from the same family. This eliminates the large overhead incurred by prior RL approaches whose lack of generalizability necessitates re-training from scratch every time a new graph is to be placed.

References in corpus (7)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Inductive Representation Learning on Large Graphs
• Geometric deep learning: going beyond Euclidean data
• Efficient Neural Architecture Search via Parameter Sharing
• Massively Parallel Methods for Deep Reinforcement Learning
• Hindsight Experience Replay
• Learning Scheduling Algorithms for Data Processing Clusters

Cited by in corpus (9)

• Chip Placement with Deep Reinforcement Learning
• GDDR: GNN-based Data-Driven Routing
• GDP: Generalized Device Placement for Dataflow Graphs
• A View on Deep Reinforcement Learning in System Optimization
• Extract-Transform-Load for Video Streams
• Synthesizing Optimal Parallelism Placement and Reduction Strategies on Hierarchical Systems for Deep Learning
• Resource Allocation in Disaggregated Data Centre Systems with Reinforcement Learning
• Transferable Graph Optimizers for ML Compilers
• Wield: Systematic Reinforcement Learning With Progressive Randomization