Arithmetic-Intensity-Guided Fault Tolerance for Neural Network Inference on GPUs

Abstract

Neural networks (NNs) are increasingly employed in safety-critical domains and in environments prone to unreliability (e.g., soft errors), such as on spacecraft. Therefore, it is critical to impart fault tolerance to NN inference. Algorithm-based fault tolerance (ABFT) is emerging as an efficient approach for fault tolerance in NNs. We propose an adaptive approach to ABFT for NN inference that exploits untapped opportunities in emerging deployment scenarios. GPUs have high compute-to-memory-bandwidth ratios, while NN layers have a wide range of arithmetic intensities. This leaves some layers compute bound and others memory-bandwidth bound, but current approaches to ABFT do not consider these differences. We first investigate ABFT schemes best suited for each of these scenarios. We then propose intensity-guided ABFT, an adaptive, arithmetic-intensity-guided approach that selects the most efficient ABFT scheme for each NN layer. Intensity-guided ABFT reduces execution-time overhead by 1.09--5.3$\times$ across many NNs compared to traditional approaches to ABFT.

arXiv commentAppeared in The International Conference for High Performance Computing, Networking, Storage and Analysis (SC '21), November 14--19, 2021, St. Louis, MO, USA

References in corpus (12)

• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Neural Architecture Search with Reinforcement Learning
• Deep Learning Recommendation Model for Personalization and Recommendation Systems
• MLPerf Training Benchmark
• FT-CNN: Algorithm-Based Fault Tolerance for Convolutional Neural Networks
• Dissecting the NVidia Turing T4 GPU via Microbenchmarking
• Silent Data Corruptions at Scale
• HarDNN: Feature Map Vulnerability Evaluation in CNNs
• TensorFI: A Flexible Fault Injection Framework for TensorFlow Applications
• CodeNet: Training Large Scale Neural Networks in Presence of Soft-Errors
• Efficient Soft-Error Detection for Low-precision Deep Learning Recommendation Models
• Enabling Software Resilience in GPGPU Applications via Partial Thread Protection