DeepProf: Performance Analysis for Deep Learning Applications via Mining GPU Execution Patterns
arXiv:1707.03750
Abstract
Deep learning applications are computation-intensive and often employ GPU as the underlying computing devices. Deep learning frameworks provide powerful programming interfaces, but the gap between source codes and practical GPU operations make it difficult to analyze the performance of deep learning applications. In this paper, through examing the features of GPU traces and deep learning applications, we use the suffix tree structure to extract the repeated patten in GPU traces. Performance analysis graphs can be generated from the preprocessed GPU traces. We further present \texttt{DeepProf}, a novel tool to automatically process GPU traces and generate performance analysis reports for deep learning applications. Empirical study verifies the effectiveness of \texttt{DeepProf} in performance analysis and diagnosis. We also find out some interesting properties of Tensorflow, which can be used to guide the deep learning system setup.
Cited by in corpus (6)
- Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads
- Semi-Dynamic Load Balancing: Efficient Distributed Learning in Non-Dedicated Environments
- tf-Darshan: Understanding Fine-grained I/O Performance in Machine Learning Workloads
- Topology-based Representative Datasets to Reduce Neural Network Training Resources
- Characterizing Deep Learning Training Workloads on Alibaba-PAI
- HPCClusterScape: Increasing Transparency and Efficiency of Shared High-Performance Computing Clusters for Large-scale AI Models