Ablation of a Robot's Brain: Neural Networks Under a Knife

Abstract

It is still not fully understood exactly how neural networks are able to solve the complex tasks that have recently pushed AI research forward. We present a novel method for determining how information is structured inside a neural network. Using ablation (a neuroscience technique for cutting away parts of a brain to determine their function), we approach several neural network architectures from a biological perspective. Through an analysis of this method's results, we examine important similarities between biological and artificial neural networks to search for the implicit knowledge locked away in the network's weights.

References in corpus (6)

• Continuous control with deep reinforcement learning
• Playing Atari with Deep Reinforcement Learning
• Opening the Black Box of Deep Neural Networks via Information
• Fractional Max-Pooling
• On the importance of single directions for generalization
• Stacked What-Where Auto-encoders

Cited by in corpus (5)

• Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs
• On the relationship between class selectivity, dimensionality, and robustness
• AnoNet: Weakly Supervised Anomaly Detection in Textured Surfaces
• Fine-grained Interpretation and Causation Analysis in Deep NLP Models
• Linking average- and worst-case perturbation robustness via class selectivity and dimensionality