Ablation Studies in Artificial Neural Networks

Abstract

Ablation studies have been widely used in the field of neuroscience to tackle complex biological systems such as the extensively studied Drosophila central nervous system, the vertebrate brain and more interestingly and most delicately, the human brain. In the past, these kinds of studies were utilized to uncover structure and organization in the brain, i.e. a mapping of features inherent to external stimuli onto different areas of the neocortex. considering the growth in size and complexity of state-of-the-art artificial neural networks (ANNs) and the corresponding growth in complexity of the tasks that are tackled by these networks, the question arises whether ablation studies may be used to investigate these networks for a similar organization of their inner representations. In this paper, we address this question and performed two ablation studies in two fundamentally different ANNs to investigate their inner representations of two well-known benchmark datasets from the computer vision domain. We found that features distinct to the local and global structure of the data are selectively represented in specific parts of the network. Furthermore, some of these representations are redundant, awarding the network a certain robustness to structural damages. We further determined the importance of specific parts of the network for the classification task solely based on the weight structure of single units. Finally, we examined the ability of damaged networks to recover from the consequences of ablations by means of recovery training. We argue that ablations studies are a feasible method to investigate knowledge representations in ANNs and are especially helpful to examine a networks robustness to structural damages, a feature of ANNs that will become increasingly important for future safety-critical applications.

19 pages, 23 figures, intention to submit as a conference publication

References in corpus (17)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Continuous control with deep reinforcement learning
• Playing Atari with Deep Reinforcement Learning
• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Pruning Convolutional Neural Networks for Resource Efficient Inference
• Recent Trends in Deep Learning Based Natural Language Processing
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Pruning Filters for Efficient ConvNets
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• Hindsight Experience Replay
• Speaker-independent Speech Separation with Deep Attractor Network
• Learning Dexterous In-Hand Manipulation
• On the importance of single directions for generalization
• Learning Sparse Neural Networks through $L_0$ Regularization
• Faster gaze prediction with dense networks and Fisher pruning
• Observe and Look Further: Achieving Consistent Performance on Atari

Cited by in corpus (15)

• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• How to avoid machine learning pitfalls: a guide for academic researchers
• Sociotechnical Audits: Broadening the Algorithm Auditing Lens to Investigate Targeted Advertising
• Towards falsifiable interpretability research
• Understanding Neural Networks and Individual Neuron Importance via Information-Ordered Cumulative Ablation
• RatLesNetv2: A Fully Convolutional Network for Rodent Brain Lesion Segmentation
• What Do You See? Evaluation of Explainable Artificial Intelligence (XAI) Interpretability through Neural Backdoors
• Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs
• Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations
• A Hybrid Adaptive Controller for Soft Robot Interchangeability
• Heterogeneous Graph Neural Networks with Post-hoc Explanations for Multi-modal and Explainable Land Use Inference
• Real-Time Forecasting of Dockless Scooter-Sharing Demand: A Spatio-Temporal Multi-Graph Transformer Approach
• daVinciNet: Joint Prediction of Motion and Surgical State in Robot-Assisted Surgery
• Democratizing Artificial Intelligence in Healthcare: A Study of Model Development Across Two Institutions Incorporating Transfer Learning
• Linking average- and worst-case perturbation robustness via class selectivity and dimensionality