Neural network interpretation using descrambler groups

Abstract

The lack of interpretability and trust is a much-criticised feature of deep neural networks. In fully connected nets, the signalling between inner layers is scrambled because backpropagation training does not require perceptrons to be arranged in any particular order. The result is a black box; this problem is particularly severe in scientific computing and digital signal processing (DSP), where neutral nets perform abstract mathematical transformations that do not reduce to features or concepts. We present here a group-theoretical procedure that attempts to bring inner layer signalling into a human-readable form, the assumption being that this form exists and has identifiable and quantifiable features - for example, smoothness or locality. We applied the proposed method to DEERNet (a DSP network used in electron spin resonance) and managed to descramble it. We found considerable internal sophistication: the network spontaneously invents a bandpass filter, a notch filter, a frequency axis rectifier, frequency division multiplexing, group embedding, spectral filtering regularisation, and a Fourier-like transform that maps harmonic functions into Chebyshev polynomials - in ten minutes of unattended training from a random initial guess.

References in corpus (6)

• How transferable are features in deep neural networks?
• Axiomatic Attribution for Deep Networks
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
• Common Voice: A Massively-Multilingual Speech Corpus
• Measuring the Intrinsic Dimension of Objective Landscapes

Cited by in corpus (6)

• Neural networks in pulsed dipolar spectroscopy: a practical guide
• NMR spectrum reconstruction as a pattern recognition problem
• Discovering conservation laws from trajectories via machine learning
• Model for the curvature response of the CDF II drift chamber
• Review and Prospect: Deep Learning in Nuclear Magnetic Resonance Spectroscopy
• Network Dynamics-Based Framework for Understanding Deep Neural Networks