Auto-MAP: A DQN Framework for Exploring Distributed Execution Plans for DNN Workloads
arXiv:2007.04069
Abstract
The last decade has witnessed growth in the computational requirements for training deep neural networks. Current approaches (e.g., data/model parallelism, pipeline parallelism) parallelize training tasks onto multiple devices. However, these approaches always rely on specific deep learning frameworks and requires elaborate manual design, which make it difficult to maintain and share between different type of models. In this paper, we propose Auto-MAP, a framework for exploring distributed execution plans for DNN workloads, which can automatically discovering fast parallelization strategies through reinforcement learning on IR level of deep learning models. Efficient exploration remains a major challenge for reinforcement learning. We leverage DQN with task-specific pruning strategies to help efficiently explore the search space including optimized strategies. Our evaluation shows that Auto-MAP can find the optimal solution in two hours, while achieving better throughput on several NLP and convolution models.
References in corpus (17)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Playing Atari with Deep Reinforcement Learning
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Prioritized Experience Replay
- MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
- Dueling Network Architectures for Deep Reinforcement Learning
- One weird trick for parallelizing convolutional neural networks
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Noisy Networks for Exploration
- Rainbow: Combining Improvements in Deep Reinforcement Learning
- GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- Beyond Data and Model Parallelism for Deep Neural Networks
- PipeDream: Fast and Efficient Pipeline Parallel DNN Training
- ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
- DAPPLE: A Pipelined Data Parallel Approach for Training Large Models
- PipeMare: Asynchronous Pipeline Parallel DNN Training