Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety

Abstract

The use of deep neural networks (DNNs) in safety-critical applications like mobile health and autonomous driving is challenging due to numerous model-inherent shortcomings. These shortcomings are diverse and range from a lack of generalization over insufficient interpretability to problems with malicious inputs. Cyber-physical systems employing DNNs are therefore likely to suffer from safety concerns. In recent years, a zoo of state-of-the-art techniques aiming to address these safety concerns has emerged. This work provides a structured and broad overview of them. We first identify categories of insufficiencies to then describe research activities aiming at their detection, quantification, or mitigation. Our paper addresses both machine learning experts and safety engineers: The former ones might profit from the broad range of machine learning topics covered and discussions on limitations of recent methods. The latter ones might gain insights into the specifics of modern ML methods. We moreover hope that our contribution fuels discussions on desiderata for ML systems and strategies on how to propel existing approaches accordingly.

arXiv comment94 pages

References in corpus (153)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Auto-Encoding Variational Bayes
• Distilling the Knowledge in a Neural Network
• Explaining and Harnessing Adversarial Examples
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
• Intriguing properties of neural networks
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Axiomatic Attribution for Deep Networks
• Fully Convolutional Networks for Semantic Segmentation
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Striving for Simplicity: The All Convolutional Net
• Conditional Random Fields as Recurrent Neural Networks
• Learning Important Features Through Propagating Activation Differences
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Deep Visual Domain Adaptation: A Survey
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Neural Architecture Search: A Survey
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• On Calibration of Modern Neural Networks
• One pixel attack for fooling deep neural networks
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• Conditional Adversarial Domain Adaptation
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Tutorial on Variational Autoencoders
• DARTS: Differentiable Architecture Search
• Training Very Deep Networks
• Pruning Convolutional Neural Networks for Resource Efficient Inference
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Weight Uncertainty in Neural Networks
• Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization
• Compressing Deep Convolutional Networks using Vector Quantization
• Interpretable Explanations of Black Boxes by Meaningful Perturbation
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Generalizing from a Few Examples: A Survey on Few-Shot Learning
• Opening the Black Box of Deep Neural Networks via Information
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• SmoothGrad: removing noise by adding noise
• Random Erasing Data Augmentation
• The Loss Surfaces of Multilayer Networks
• Synthesizing Robust Adversarial Examples
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Deep Bayesian Active Learning with Image Data
• Domain Generalization via Invariant Feature Representation
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Visualizing and Understanding Convolutional Networks
• Domain Generalization via Model-Agnostic Learning of Semantic Features
• Deep Anomaly Detection with Outlier Exposure
• Regularizing Neural Networks by Penalizing Confident Output Distributions
• Detecting Adversarial Samples from Artifacts
• Learning Confidence for Out-of-Distribution Detection in Neural Networks
• GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks
• Accurate Uncertainties for Deep Learning Using Calibrated Regression
• Predictive Uncertainty Estimation via Prior Networks
• Data Augmentation by Pairing Samples for Images Classification
• Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
• Adversarial Autoencoders
• DLFuzz: Differential Fuzzing Testing of Deep Learning Systems
• Domain Adaptation for Visual Applications: A Comprehensive Survey
• Deep Gaussian Processes
• Understanding disentangling in $\beta$-VAE
• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
• Fully Connected Deep Structured Networks
• Neural Architecture Search with Bayesian Optimisation and Optimal Transport
• On the importance of single directions for generalization
• Adversarial Transformation Networks: Learning to Generate Adversarial Examples
• Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation
• DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars
• Robust PCA via Outlier Pursuit
• A Survey on Methods and Theories of Quantized Neural Networks
• The State of the Art in Integrating Machine Learning into Visual Analytics
• Wasserstein Auto-Encoders
• Regularization for Deep Learning: A Taxonomy
• Why do deep convolutional networks generalize so poorly to small image transformations?
• Simple Black-Box Adversarial Perturbations for Deep Networks
• Object Detection in Videos with Tubelet Proposal Networks
• Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
• Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers
• DPatch: An Adversarial Patch Attack on Object Detectors
• LaVAN: Localized and Visible Adversarial Noise
• TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
• Adversarial Patch
• Coherence Pursuit: Fast, Simple, and Robust Principal Component Analysis
• Faster gaze prediction with dense networks and Fisher pruning
• Deep Gaussian Processes for Regression using Approximate Expectation Propagation
• Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks
• Variational Autoencoders and Nonlinear ICA: A Unifying Framework
• Are Disentangled Representations Helpful for Abstract Visual Reasoning?
• On Physical Adversarial Patches for Object Detection
• Snapshot Ensembles: Train 1, get M for free
• Multivariate Confidence Calibration for Object Detection
• Thieves on Sesame Street! Model Extraction of BERT-based APIs
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation
• Learning Robust Representations by Projecting Superficial Statistics Out
• UPSET and ANGRI : Breaking High Performance Image Classifiers
• Rule Extraction Algorithm for Deep Neural Networks: A Review
• Adversarial Robustness as a Prior for Learned Representations
• Model Compression with Adversarial Robustness: A Unified Optimization Framework
• Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning
• Frustratingly Simple Domain Generalization via Image Stylization
• STFCN: Spatio-Temporal FCN for Semantic Video Segmentation
• Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)
• Causal Learning and Explanation of Deep Neural Networks via Autoencoded Activations
• Semantically-Guided Representation Learning for Self-Supervised Monocular Depth
• Discriminative out-of-distribution detection for semantic segmentation
• DeepBillboard: Systematic Physical-World Testing of Autonomous Driving Systems
• Adversarial T-shirt! Evading Person Detectors in A Physical World
• Conditional Gaussian Distribution Learning for Open Set Recognition
• Hyp-RL : Hyperparameter Optimization by Reinforcement Learning
• Max-Mahalanobis Linear Discriminant Analysis Networks
• Grid Saliency for Context Explanations of Semantic Segmentation
• SCSP: Spectral Clustering Filter Pruning with Soft Self-adaption Manners
• Time-Dynamic Estimates of the Reliability of Deep Semantic Segmentation Networks
• On Sampling from the Gibbs Distribution with Random Maximum A-Posteriori Perturbations
• Robust PCA in High-dimension: A Deterministic Approach
• LiSHT: Non-Parametric Linearly Scaled Hyperbolic Tangent Activation Function for Neural Networks
• Towards Unified INT8 Training for Convolutional Neural Network
• Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles
• Learning Autoencoders with Relational Regularization
• Learning More Robust Features with Adversarial Training
• Confidence Calibration for Object Detection and Segmentation
• Complement Objective Training
• Transferable Universal Adversarial Perturbations Using Generative Models
• Strategy to Increase the Safety of a DNN-based Perception for HAD Systems
• Learning representations in Bayesian Confidence Propagation neural networks
• Disentangled Representation Learning with Wasserstein Total Correlation
• Studying Invariances of Trained Convolutional Neural Networks
• Supervised Dimensionality Reduction and Visualization using Centroid-encoder
• Effects of Loss Functions And Target Representations on Adversarial Robustness
• Efficacy of Pixel-Level OOD Detection for Semantic Segmentation
• Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities
• Learning Implicit Generative Models Using Differentiable Graph Tests
• A Self-Supervised Feature Map Augmentation (FMA) Loss and Combined Augmentations Finetuning to Efficiently Improve the Robustness of CNNs
• L 1-norm double backpropagation adversarial defense
• Learning Robust Low-Rank Representations
• PhysGAN: Generating Physical-World-Resilient Adversarial Examples for Autonomous Driving
• Bayesian Optimization for Iterative Learning
• Non-Parametric Calibration of Probabilistic Regression
• A machine learning environment for evaluating autonomous driving software
• $\beta$-VAEs can retain label information even at high compression
• Outlier Detection and Data Clustering via Innovation Search

Cited by in corpus (7)

• A Review of Product Safety Regulations in the European Union
• Yes We Care! -- Certification for Machine Learning Methods through the Care Label Framework
• SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for Autonomous Driving
• Conditioning Latent-Space Clusters for Real-World Anomaly Classification
• Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spaces
• The Care Label Concept: A Certification Suite for Trustworthy and Resource-Aware Machine Learning
• Detecting underdetermination in parameterized quantum circuits