A simple linear algebra identity to optimize Large-Scale Neural Network Quantum States

Abstract

Neural-network architectures have been increasingly used to represent quantum many-body wave functions. These networks require a large number of variational parameters and are challenging to optimize using traditional methods, as gradient descent. Stochastic Reconfiguration (SR) has been effective with a limited number of parameters, but becomes impractical beyond a few thousand parameters. Here, we leverage a simple linear algebra identity to show that SR can be employed even in the deep learning scenario. We demonstrate the effectiveness of our method by optimizing a Deep Transformer architecture with $3\times 10^5$ parameters, achieving state-of-the-art ground-state energy in the $J_1$-$J_2$ Heisenberg model at $J_2/J_1=0.5$ on the $10\times 10$ square lattice, a challenging benchmark in highly-frustrated magnetism. This work marks a significant step forward in the scalability and efficiency of SR for Neural-Network Quantum States, making them a promising method to investigate unknown quantum phases of matter, where other methods struggle.

7 pages, 4 figure and 1 table

References in corpus (33)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Restricted-Boltzmann-Machine Learning for Solving Strongly Correlated Quantum Systems
• Plaquette Ordered Phase and Quantum Phase Diagram in the Spin-1/2 J1-J2 Square Heisenberg Model
• Green Function Monte Carlo with Stochastic Reconfiguration
• Wave function optimization in the variational Monte Carlo method
• Study of the Two-Dimensional Frustrated J1-J2 Model with Neural Network Quantum States
• Quantum many-body dynamics in two dimensions with artificial neural networks
• Direct evidence for a gapless $Z_2$ spin liquid by frustrating Néel antiferromagnetism
• Variational description of Mott insulators
• NetKet 3: Machine Learning Toolbox for Many-Body Quantum Systems
• Solving frustrated quantum many-particle models with convolutional neural networks
• Neural Gutzwiller-projected variational wave functions
• Transformer variational wave functions for frustrated quantum spin systems
• Neural network wave functions and the sign problem
• Empowering deep neural quantum states through efficient optimization
• Helping restricted Boltzmann machines with quantum-state representation by restoring symmetry
• High-accuracy variational Monte Carlo for frustrated magnets with deep neural networks
• Gauge Invariant and Anyonic Symmetric Transformer and RNN Quantum States for Quantum Lattice Models
• Optimizing Design Choices for Neural Quantum States
• Hybrid convolutional neural network and PEPS wave functions for quantum many-particle states
• Variational Monte Carlo with Large Patched Transformers
• Investigating Topological Order using Recurrent Neural Networks
• Mapping of attention mechanisms to a generalized Potts model
• Expressive power of complex-valued restricted Boltzmann machines for solving non-stoquastic Hamiltonians
• Data-driven emergence of convolutional structure in neural networks
• Iterative Retraining of Quantum Spin Models Using Recurrent Neural Networks
• Highly resolved spectral functions of two-dimensional systems with neural quantum states
• $2^{1296}$ Exponentially Complex Quantum Many-Body Simulation via Scalable Deep Learning Method
• Variational optimization of the amplitude of neural-network quantum many-body ground states
• Learning ground states of gapped quantum Hamiltonians with Kernel Methods
• Scalable Imaginary Time Evolution with Neural Network Quantum States
• Boltzmann machines and quantum many-body problems

Cited by in corpus (34)

• Beyond-classical computation in quantum simulation
• Empowering deep neural quantum states through efficient optimization
• Transformer Wave Function for two dimensional frustrated magnets: emergence of a Spin-Liquid Phase in the Shastry-Sutherland Model
• Neural network approach to quasiparticle dispersions in doped antiferromagnets
• Foundation Neural-Networks Quantum States as a Unified Ansatz for Multiple Hamiltonians
• Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems
• Neural Projected Quantum Dynamics: a systematic study
• Transformer neural networks and quantum simulators: a hybrid approach for simulating strongly correlated systems
• Fine-tuning Neural Network Quantum States
• Autoregressive neural quantum states of Fermi Hubbard models
• Deep learning lattice gauge theories
• Are queries and keys always relevant? A case study on Transformer wave functions
• Accurate neural quantum states for interacting lattice bosons
• Equivariant Variational Quantum Eigensolver to detect Phase Transitions through Energy Level Crossings
• Second-order optimisation strategies for neural network quantum states
• Recurrent neural network wave functions for Rydberg atom arrays on kagome lattice
• Non-stabilizerness of Neural Quantum States
• Quantum many-body solver using artificial neural networks and its applications to strongly correlated electron systems
• Time-dependent Neural Galerkin Method for Quantum Dynamics
• Efficient Optimization of Variational Autoregressive Networks with Natural Gradient
• Neural Quantum State Study of Fracton Models
• Design principles of deep translationally-symmetric neural quantum states for frustrated magnets
• Efficiency of the hidden fermion determinant states Ansatz in the light of different complexity measures
• Simple Fermionic backflow states via a systematically improvable tensor decomposition
• Improving neural network performance for solving quantum sign structure
• Determinant- and Derivative-Free Quantum Monte Carlo Within the Stochastic Representation of Wavefunctions
• Partial suppression of magnetism in the square lattice SU(3) Hubbard model
• Physics-informed Transformers for Electronic Quantum States
• Phase diagram of the J1-J2 Heisenberg second-order topological quantum magnet
• Bound on entanglement in neural quantum states
• Accelerating two-dimensional tensor network contractions using QR decompositions
• Generalized Lanczos method for systematic optimization of neural-network quantum states
• Comparing Symmetrized Determinant Neural Quantum States for the Hubbard Model
• Magnetic correlations in the $SU(3)$ triangular-lattice $t$-$J$ model at finite doping