Hessian-based toolbox for reliable and interpretable machine learning in physics
arXiv:2108.02154 · doi:10.1088/2632-2153/ac338d
Abstract
Machine learning (ML) techniques applied to quantum many-body physics have emerged as a new research field. While the numerical power of this approach is undeniable, the most expressive ML algorithms, such as neural networks, are black boxes: The user does neither know the logic behind the model predictions nor the uncertainty of the model predictions. In this work, we present a toolbox for interpretability and reliability, agnostic of the model architecture. In particular, it provides a notion of the influence of the input data on the prediction at a given test point, an estimation of the uncertainty of the model predictions, and an extrapolation score for the model predictions. Such a toolbox only requires a single computation of the Hessian of the training loss function. Our work opens the road to the systematic use of interpretability and reliability methods in ML applied to physics and, more generally, science.
arXiv comment17 pages, 7 figures, example code is available at https://github.com/Shmoo137/Hessian-Based-Toolbox
References in corpus (52)
- SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
- A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
- Interpretable machine learning: definitions, methods, and applications
- Machine learning phases of matter
- Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
- Understanding Black-box Predictions via Influence Functions
- Learning phase transitions by confusion
- The Loss Surfaces of Multilayer Networks
- Discovering Phase Transitions with Unsupervised Learning
- Unsupervised learning of phase transitions: from principal component analysis to variational autoencoders
- Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
- Optimizing Neural Networks with Kronecker-factored Approximate Curvature
- QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems part I: spin chains
- Non-standard Hubbard models in optical lattices: a review
- Machine learning quantum phases of matter beyond the fermion sign problem
- Machine Learning Topological Invariants with Neural Networks
- Identifying Quantum Phase Transitions using Artificial Neural Networks on Experimental Data
- Unsupervised machine learning account of magnetic transitions in the Hubbard model
- Estimating the gradient and higher-order derivatives on quantum hardware
- Machine Learning of Explicit Order Parameters: From the Ising Model to SU(2) Lattice Gauge Theory
- Empirical Analysis of the Hessian of Over-Parametrized Neural Networks
- Kernel methods for interpretable machine learning of order parameters
- Identifying Quantum Phase Transitions with Adversarial Neural Networks
- Unsupervised phase discovery with deep anomaly detection
- Discriminative Cooperative Networks for Detecting Phase Transitions
- Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond
- An Investigation into Neural Net Optimization via Hessian Eigenvalue Density
- Machine learning of quantum phase transitions
- Bayesian machine learning for quantum molecular dynamics
- On the Accuracy of Influence Functions for Measuring Group Effects
- Unsupervised machine learning of topological phase transitions from experimental data
- Extrapolating quantum observables with machine learning: Inferring multiple phase transitions from properties of a single phase
- Unsupervised identification of topological order using predictive models
- Discovering Symmetry Invariants and Conserved Quantities by Interpreting Siamese Neural Networks
- Phase diagram of the half-filled one-dimensional t-V-V' model
- Characterizing the loss landscape of variational quantum circuits
- Applications of neural networks to the studies of phase transitions of two-dimensional Potts models
- Automated discovery of characteristic features of phase transitions in many-body localization
- Towards Novel Insights in Lattice Field Theory with Explainable Machine Learning
- Interpreting machine learning of topological quantum phase transitions
- Quantitative and Interpretable Order Parameters for Phase Transitions from Persistent Homology
- Interpretable Machine Learning Study of Many-Body Localization Transition in Disordered Quantum Ising Spin Chains
- Divergence of predictive model output as indication of phase transitions
- Visualizing Strange Metallic Correlations in the 2D Fermi-Hubbard Model with AI
- Neural network setups for a precise detection of the many-body localization transition: finite-size scaling and limitations
- Revealing the Phase Diagram of Kitaev Materials by Machine Learning: Cooperation and Competition between Spin Liquids
- Deep learning of topological phase transitions from entanglement aspects
- Phase Detection with Neural Networks: Interpreting the Black Box
- Interpretable and unsupervised phase classification
- Negative eigenvalues of the Hessian in deep neural networks
- Identification of hidden order and emergent constraints in frustrated magnets using tensorial kernel methods
- Unsupervised interpretable learning of topological indices invariant under permutations of atomic bands