Hessian-based toolbox for reliable and interpretable machine learning in physics
arXiv:2108.02154 · doi:10.1088/2632-2153/ac338d
Abstract
Machine learning (ML) techniques applied to quantum many-body physics have emerged as a new research field. While the numerical power of this approach is undeniable, the most expressive ML algorithms, such as neural networks, are black boxes: The user does neither know the logic behind the model predictions nor the uncertainty of the model predictions. In this work, we present a toolbox for interpretability and reliability, agnostic of the model architecture. In particular, it provides a notion of the influence of the input data on the prediction at a given test point, an estimation of the uncertainty of the model predictions, and an extrapolation score for the model predictions. Such a toolbox only requires a single computation of the Hessian of the training loss function. Our work opens the road to the systematic use of interpretability and reliability methods in ML applied to physics and, more generally, science.
17 pages, 7 figures, example code is available at https://github.com/Shmoo137/Hessian-Based-Toolbox
References in corpus (11)
- Learning phase transitions by confusion
- The Loss Surfaces of Multilayer Networks
- Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
- QuSpin: a Python Package for Dynamics and Exact Diagonalisation of Quantum Many Body Systems part I: spin chains
- Non-standard Hubbard models in optical lattices: a review
- Unsupervised machine learning account of magnetic transitions in the Hubbard model
- Machine Learning of Explicit Order Parameters: From the Ising Model to SU(2) Lattice Gauge Theory
- An Investigation into Neural Net Optimization via Hessian Eigenvalue Density
- Bayesian machine learning for quantum molecular dynamics
- Unsupervised machine learning of topological phase transitions from experimental data
- Negative eigenvalues of the Hessian in deep neural networks