Second-order optimisation strategies for neural network quantum states

Abstract

The Variational Monte Carlo method has recently seen important advances through the use of neural network quantum states. While more and more sophisticated ansätze have been designed to tackle a wide variety of quantum many-body problems, modest progress has been made on the associated optimisation algorithms. In this work, we revisit the Kronecker-Factored Approximate Curvature, an optimiser that has been used extensively in a variety of simulations. We suggest improvements on the scaling and the direction of this optimiser, and find that they substantially increase its performance at a negligible additional cost. We also reformulate the Variational Monte Carlo approach in a game theory framework, to propose a novel optimiser based on decision geometry. We find that, on a practical test case for continuous systems, this new optimiser consistently outperforms any of the KFAC improvements in terms of stability, accuracy and speed of convergence. Beyond Variational Monte Carlo, the versatility of this approach suggests that decision geometry could provide a solid foundation for accelerating a broad class of machine learning algorithms.

35 pages, 9 figures, 4 tables. Accepted in Phil. Trans. R. Soc. A

References in corpus (39)

• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Quantum Monte Carlo methods for nuclear physics
• Coupled-cluster computations of atomic nuclei
• Deep neural network solution of the electronic Schrödinger equation
• Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks
• The In-Medium Similarity Renormalization Group: A Novel Ab Initio Method for Nuclei
• Quantum Natural Gradient
• Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory
• Optimizing Neural Networks with Kronecker-factored Approximate Curvature
• Generalized Lanczos Algorithm for Variational Quantum Monte Carlo
• New insights and perspectives on the natural gradient method
• Wave function optimization in the variational Monte Carlo method
• Energy and variance optimization of many body wave functions
• Backflow Transformations via Neural Networks for Quantum Many-Body Wave-Functions
• NetKet 3: Machine Learning Toolbox for Many-Body Quantum Systems
• Tuning the interactions of spin-polarized fermions using quasi-one-dimensional confinement
• Ab-initio quantum chemistry with neural-network wavefunctions
• Revisiting Natural Gradient for Deep Networks
• Variational Monte Carlo calculations of $\mathbf{A}\le 4$ nuclei with an artificial neural-network correlator ansatz
• Self-consistent Green's function theory for atomic nuclei
• Machine learning the deuteron
• A simple linear algebra identity to optimize Large-Scale Neural Network Quantum States
• Wave function Ansatz (but Periodic) Networks and the Homogeneous Electron Gas
• Message-Passing Neural Quantum States for the Homogeneous Electron Gas
• Bose-Fermi dualities for arbitrary one-dimensional quantum systems in the universal low energy regime
• Nuclei with up to $\mathbf{A}=6$ nucleons with artificial neural network wave functions
• A Self-Attention Ansatz for Ab-initio Quantum Chemistry
• Local proper scoring rules of order two
• Static and dynamic properties of a few spin $1/2$ interacting fermions trapped in an harmonic potential
• A Kaczmarz-inspired approach to accelerate the optimization of neural network wavefunctions
• Solving the nuclear pairing model with neural network quantum states
• Neural Wave Functions for Superfluids
• Simulations of state-of-the-art fermionic neural network wave functions with diffusion Monte Carlo
• Second-order optimisation strategies for neural network quantum states
• Machine learning one-dimensional spinless trapped fermionic systems with neural-network quantum states
• Neural-network quantum states for ultra-cold Fermi gases
• Quantum natural gradient without monotonicity
• Fermi-Bose mapping and N-particle ground state of spin-polarized fermions in tight atom waveguides
• Distilling the essential elements of nuclear binding via neural-network quantum states

Cited by in corpus (5)

• Second-order optimisation strategies for neural network quantum states
• Efficient Optimization of Variational Autoregressive Networks with Natural Gradient
• Modal Backflow Neural Quantum States for Anharmonic Vibrational Calculations
• Digitized Counter-Diabatic Quantum Optimization for Bin Packing Problem
• Generalized Lanczos method for systematic optimization of neural-network quantum states