Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning
arXiv:2007.07298
Abstract
For deep neural network accelerators, memory movement is both energetically expensive and can bound computation. Therefore, optimal mapping of tensors to memory hierarchies is critical to performance. The growing complexity of neural networks calls for automated memory mapping instead of manual heuristic approaches; yet the search space of neural network computational graphs have previously been prohibitively large. We introduce Evolutionary Graph Reinforcement Learning (EGRL), a method designed for large search spaces, that combines graph neural networks, reinforcement learning, and evolutionary search. A set of fast, stateless policies guide the evolutionary search to improve its sample-efficiency. We train and validate our approach directly on the Intel NNP-I chip for inference. EGRL outperforms policy-gradient, evolutionary search and dynamic programming baselines on BERT, ResNet-101 and ResNet-50. We additionally achieve 28-78\% speed-up compared to the native NNP-I compiler on all three workloads.
arXiv commentUpdated manuscript
References in corpus (14)
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Addressing Function Approximation Error in Actor-Critic Methods
- Empirical Evaluation of Rectified Activations in Convolutional Network
- Learning Combinatorial Optimization Algorithms over Graphs
- Device Placement Optimization with Reinforcement Learning
- DocBERT: BERT for Document Classification
- Learning to Optimize Tensor Programs
- Chip Placement with Deep Reinforcement Learning
- Simple Applications of BERT for Ad Hoc Document Retrieval
- Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning
- Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues
- Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications
- Learning Heuristics over Large Graphs via Deep Reinforcement Learning
- Evolution-Guided Policy Gradient in Reinforcement Learning