Scheduling Computation Graphs of Deep Learning Models on Manycore CPUs

Abstract

For a deep learning model, efficient execution of its computation graph is key to achieving high performance. Previous work has focused on improving the performance for individual nodes of the computation graph, while ignoring the parallelization of the graph as a whole. However, we observe that running multiple operations simultaneously without interference is critical to efficiently perform parallelizable small operations. The attempt of executing the computation graph in parallel in deep learning frameworks usually involves much resource contention among concurrent operations, leading to inferior performance on manycore CPUs. To address these issues, in this paper, we propose Graphi, a generic and high-performance execution engine to efficiently execute a computation graph in parallel on manycore CPUs. Specifically, Graphi minimizes the interference on both software/hardware resources, discovers the best parallel setting with a profiler, and further optimizes graph execution with the critical-path first scheduling. Our experiments show that the parallel execution consistently outperforms the sequential one. The training times on four different neural networks with Graphi are 2.1x to 9.5x faster than those with TensorFlow on a 68-core Intel Xeon Phi processor.

References in corpus (14)

• Sequence to Sequence Learning with Neural Networks
• TensorFlow: A system for large-scale machine learning
• Recurrent Neural Network Regularization
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Wide Residual Networks
• cuDNN: Efficient Primitives for Deep Learning
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Grid Long Short-Term Memory
• Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences
• Factorization tricks for LSTM networks
• Recurrent Dropout without Memory Loss
• Optimizing Performance of Recurrent Neural Networks on GPUs
• Omnivore: An Optimizer for Multi-device Deep Learning on CPUs and GPUs
• ZNN - A Fast and Scalable Algorithm for Training 3D Convolutional Networks on Multi-Core and Many-Core Shared Memory Machines

Cited by in corpus (4)

• High-Throughput CNN Inference on Embedded ARM big.LITTLE Multi-Core Processors
• Optimizing CNN Model Inference on CPUs
• IOS: Inter-Operator Scheduler for CNN Acceleration
• Brief Announcement: On the Limits of Parallelizing Convolutional Neural Networks on GPUs