HPC AI500: A Benchmark Suite for HPC AI Systems
arXiv:1908.02607
Abstract
In recent years, with the trend of applying deep learning (DL) in high performance scientific computing, the unique characteristics of emerging DL workloads in HPC raise great challenges in designing, implementing HPC AI systems. The community needs a new yard stick for evaluating the future HPC systems. In this paper, we propose HPC AI500 --- a benchmark suite for evaluating HPC systems that running scientific DL workloads. Covering the most representative scientific fields, each workload from HPC AI500 is based on real-world scientific DL applications. Currently, we choose 14 scientific DL benchmarks from perspectives of application scenarios, data sets, and software stack. We propose a set of metrics for comprehensively evaluating the HPC AI systems, considering both accuracy, performance as well as power and cost. We provide a scalable reference implementation of HPC AI500. HPC AI500 is a part of the open-source AIBench project, the specification and source code are publicly available from \url{http://www.benchcouncil.org/AIBench/index.html}.
References in corpus (11)
- PYTHIA 6.4 Physics and Manual
- PYTHIA 6.2 Physics and Manual
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Horovod: fast and easy distributed deep learning in TensorFlow
- Application of Deep Convolutional Neural Networks for Detecting Extreme Weather in Climate Datasets
- Fathom: Reference Workloads for Modern Deep Learning Methods
- TBD: Benchmarking and Analyzing Deep Neural Network Training
- AIBench: An Industry Standard Internet Service AI Benchmark Suite
- GlobeNet: Convolutional Neural Networks for Typhoon Eye Tracking from Remote Sensing Imagery
- Cosmological model discrimination with Deep Learning
- Scaling GRPC Tensorflow on 512 nodes of Cori Supercomputer