Convergence of Artificial Intelligence and High Performance Computing on NSF-supported Cyberinfrastructure

Abstract

Significant investments to upgrade and construct large-scale scientific facilities demand commensurate investments in R&D to design algorithms and computing approaches to enable scientific and engineering breakthroughs in the big data era. Innovative Artificial Intelligence (AI) applications have powered transformational solutions for big data challenges in industry and technology that now drive a multi-billion dollar industry, and which play an ever increasing role shaping human social patterns. As AI continues to evolve into a computing paradigm endowed with statistical and mathematical rigor, it has become apparent that single-GPU solutions for training, validation, and testing are no longer sufficient for computational grand challenges brought about by scientific facilities that produce data at a rate and volume that outstrip the computing capabilities of available cyberinfrastructure platforms. This realization has been driving the confluence of AI and high performance computing (HPC) to reduce time-to-insight, and to enable a systematic study of domain-inspired AI architectures and optimization schemes to enable data-driven discovery. In this article we present a summary of recent developments in this field, and describe specific advances that authors in this article are spearheading to accelerate and streamline the use of HPC platforms to design and apply accelerated AI algorithms in academia and industry.

arXiv commentWhite paper accepted to the NSF Workshop on Smart Cyberinfrastructure, February 25-27, 2020 http://smartci.sci.utah.edu/. v2: Survey paper accepted to Journal of Big Data

References in corpus (19)

• Adam: A Method for Stochastic Optimization
• Deep Learning in Neural Networks: An Overview
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• The Sloan Digital Sky Survey: Technical Summary
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• Learning phase transitions by confusion
• Horovod: fast and easy distributed deep learning in TensorFlow
• Deep Learning and its Application to LHC Physics
• The Unreasonable Effectiveness of Deep Learning in Artificial Intelligence
• Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes
• Topology optimization of 2D structures with nonlinearities using deep learning
• A Data Ecosystem to Support Machine Learning in Materials Science
• Regularization for Deep Learning: A Taxonomy
• Enabling real-time multi-messenger astrophysics discoveries with deep learning
• Deep Learning at Scale for the Construction of Galaxy Catalogs in the Dark Energy Survey
• Physics-inspired deep learning to characterize the signal manifold of quasi-circular, spinning, non-precessing binary black hole mergers
• Artificial neural network subgrid models of 2-D compressible magnetohydrodynamic turbulence
• Training Distributed Deep Recurrent Neural Networks with Mixed Precision on GPU Clusters
• Review and Examination of Input Feature Preparation Methods and Machine Learning Models for Turbulence Modeling

Cited by in corpus (15)

• FAIR for AI: An interdisciplinary and international community building perspective
• Containerisation for High Performance Computing Systems: Survey and Prospects
• Applications of physics informed neural operators
• Inference-optimized AI and high performance computing for gravitational wave detection at scale
• Physics-inspired deep learning to characterize the signal manifold of quasi-circular, spinning, non-precessing binary black hole mergers
• Advances in Machine and Deep Learning for Modeling and Real-time Detection of Multi-Messenger Sources
• End-to-end AI framework for interpretable prediction of molecular and crystal properties
• AI and extreme scale computing to learn and infer the physics of higher order gravitational wave modes of quasi-circular, spinning, non-precessing binary black hole mergers
• Shifting sands of hardware and software in exascale quantum mechanical simulations
• Physics-inspired spatiotemporal-graph AI ensemble for the detection of higher order wave mode signals of spinning binary black hole mergers
• Efficient Data-Parallel Continual Learning with Asynchronous Distributed Rehearsal Buffers
• A task-based data-flow methodology for programming heterogeneous systems with multiple accelerator APIs
• AIPerf: Automated machine learning as an AI-HPC benchmark
• Rethinking Thread Scheduling under Oversubscription: A User-Space Framework for Coordinating Multi-runtime and Multi-process Workloads
• The MIT Supercloud Dataset