Interpreting Deep Learning: The Machine Learning Rorschach Test?

Abstract

Theoretical understanding of deep learning is one of the most important tasks facing the statistics and machine learning communities. While deep neural networks (DNNs) originated as engineering methods and models of biological networks in neuroscience and psychology, they have quickly become a centerpiece of the machine learning toolbox. Unfortunately, DNN adoption powered by recent successes combined with the open-source nature of the machine learning community, has outpaced our theoretical understanding. We cannot reliably identify when and why DNNs will make mistakes. In some applications like text translation these mistakes may be comical and provide for fun fodder in research talks, a single error can be very costly in tasks like medical imaging. As we utilize DNNs in increasingly sensitive applications, a better understanding of their properties is thus imperative. Recent advances in DNN theory are numerous and include many different sources of intuition, such as learning theory, sparse signal analysis, physics, chemistry, and psychology. An interesting pattern begins to emerge in the breadth of possible interpretations. The seemingly limitless approaches are mostly constrained by the lens with which the mathematical operations are viewed. Ultimately, the interpretation of DNNs appears to mimic a type of Rorschach test --- a psychological test wherein subjects interpret a series of seemingly ambiguous ink-blots. Validation for DNN theory requires a convergence of the literature. We must distinguish between universal results that are invariant to the analysis perspective and those that are specific to a particular network configuration. Simultaneously we must deal with the fact that many standard statistical tools for quantifying generalization or empirically assessing important network features are difficult to apply to DNNs.

arXiv comment13 pages, 1 figure. Preprint is related to an upcoming Society for Industrial and Applied Mathematics (SIAM) News article

References in corpus (37)

• Adam: A Method for Stochastic Optimization
• Understanding Black-box Predictions via Influence Functions
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Understanding Deep Convolutional Networks
• Residual Networks Behave Like Ensembles of Relatively Shallow Networks
• Deep Neural Networks as Gaussian Processes
• The Marginal Value of Adaptive Gradient Methods in Machine Learning
• Recurrent Network Models Of Sequence Generation And Memory
• Provable Bounds for Learning Some Deep Representations
• Quantum Entanglement in Deep Learning Architectures
• Provable approximation properties for deep neural networks
• Spectrally-normalized margin bounds for neural networks
• An exact mapping between the Variational Renormalization Group and Deep Learning
• Empirical Analysis of the Hessian of Over-Parametrized Neural Networks
• Spurious Local Minima are Common in Two-Layer ReLU Neural Networks
• full-FORCE: A Target-Based Method for Training Recurrent Networks
• Do GANs actually learn the distribution? An empirical study
• On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
• Maximal Sparsity with Deep Networks?
• Adversarial Attacks Against Medical Deep Learning Systems
• Perspective: Energy Landscapes for Machine Learning
• Global Optimality in Tensor Factorization, Deep Learning, and Beyond
• The Emergence of Spectral Universality in Deep Networks
• Stronger generalization bounds for deep nets via a compression approach
• Local Dynamics in Trained Recurrent Neural Networks
• Interpretation of Neural Networks is Fragile
• On the saddle point problem for non-convex optimization
• Failures of Gradient-Based Deep Learning
• On Characterizing the Capacity of Neural Networks using Algebraic Topology
• Theory of Deep Learning IIb: Optimization Properties of SGD
• Why are deep nets reversible: A simple theory, with implications for training
• Deep Convolutional Networks are Hierarchical Kernel Machines
• Analysis and Design of Convolutional Networks via Hierarchical Tensor Decompositions
• Representing smooth functions as compositions of near-identity functions with implications for deep network optimization
• Expressive power of recurrent neural networks
• Solving Equations of Random Convex Functions via Anchored Regression
• Provable learning of Noisy-or Networks

Cited by in corpus (3)

• Explainable Machine Learning for Scientific Insights and Discoveries
• Neural Architecture Search by Estimation of Network Structure Distributions
• Edge of chaos as a guiding principle for modern neural network training