Exploring explicit coarse-grained structure in artificial neural networks

Abstract

We propose to employ the hierarchical coarse-grained structure in the artificial neural networks explicitly to improve the interpretability without degrading performance. The idea has been applied in two situations. One is a neural network called TaylorNet, which aims to approximate the general mapping from input data to output result in terms of Taylor series directly, without resorting to any magic nonlinear activations. The other is a new setup for data distillation, which can perform multi-level abstraction of the input dataset and generate new data that possesses the relevant features of the original dataset and can be used as references for classification. In both cases, the coarse-grained structure plays an important role in simplifying the network and improving both the interpretability and efficiency. The validity has been demonstrated on MNIST and CIFAR-10 datasets. Further improvement and some open questions related are also discussed.

References in corpus (41)

• Machine learning and the physical sciences
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Exact evolution equation for the effective potential
• Machine learning phases of matter
• MLP-Mixer: An all-MLP Architecture for Vision
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Matrix Product Density Operators: Simulation of finite-T and dissipative systems
• Learning phase transitions by confusion
• Unsupervised learning of phase transitions: from principal component analysis to variational autoencoders
• Deep neural network solution of the electronic Schrödinger equation
• Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks
• Machine learning modeling of superconducting critical temperature
• Why does deep and cheap learning work so well?
• Matrix product operator representations
• Efficient Representation of Quantum Many-body States with Deep Neural Networks
• Unsupervised Generative Modeling Using Matrix Product States
• Second Renormalization of Tensor-Network States
• Solving Statistical Mechanics Using Variational Autoregressive Networks
• Approximating quantum many-body wave-functions using artificial neural networks
• Mutual Information, Neural Networks and the Renormalization Group
• Renormalization of tensor-network states
• Neural Network Renormalization Group
• Dirac-type nodal spin liquid revealed by refined quantum many-body solver using neural-network wave function, correlation ratio, and level spectroscopy
• Tree Tensor Networks for Generative Modeling
• Real Space Renormalization in Statistical Mechanics
• Deep Learning Model for Finding New Superconductors
• Solving frustrated quantum many-particle models with convolutional neural networks
• Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure
• Exponential Thermal Tensor Network Approach for Quantum Lattice Models
• Machine Learning for Condensed Matter Physics
• Variational Neural Annealing
• Tensor lattice field theory with applications to the renormalization group and quantum computing
• Supervised Learning with Projected Entangled Pair States
• Symplectic Neural Networks in Taylor Series Form for Hamiltonian Systems
• Deep learning of the spanwise-averaged Navier-Stokes equations
• The fine structure of the entanglement entropy in the classical XY model
• Automatic Differentiation for Second Renormalization of Tensor Networks
• Is Deep Learning a Renormalization Group Flow?
• Compressing deep neural networks by matrix product operators
• Deep Learning the Functional Renormalization Group
• Deep tensor networks with matrix product operators