Is attention all you need to solve the correlated electron problem?

Abstract

The attention mechanism has transformed artificial intelligence research by its ability to learn relations between objects. In this work, we explore how a many-body wavefunction ansatz constructed from a large-parameter self-attention neural network can be used to solve the interacting electron problem in solids. By a systematic neural-network variational Monte Carlo study on a moiré quantum material, we demonstrate that the self-attention ansatz provides an accurate and efficient solution without human bias. Moreover, our numerical study finds that the required number of variational parameters scales roughly as $N^2$ with the number of electrons, which opens a path towards efficient large-scale simulations.

10+5 pages, comments welcome; v2: update refs, extend ED results; v3: minor updates

References in corpus (23)

• Quantum critical scaling of the geometric tensors
• Neural-Network Approach to Dissipative Quantum Many-Body Dynamics
• Variational Quantum Monte Carlo Method with a Neural-Network Ansatz for Open Quantum Systems
• Variational neural network ansatz for steady states in open quantum systems
• Constructing neural stationary states for open quantum many-body systems
• Gate-tunable heavy fermions in a moiré Kondo lattice
• Interplay of Fractional Chern Insulator and Charge-Density-Wave Phases in Twisted Bilayer Graphene
• Fermionic Wave Functions from Neural-Network Constrained Hidden States
• Transformer variational wave functions for frustrated quantum spin systems
• Discovering Quantum Phase Transitions with Fermionic Neural Networks
• Empowering deep neural quantum states through efficient optimization
• Ab initio calculation of real solids via neural network ansatz
• Wave function Ansatz (but Periodic) Networks and the Homogeneous Electron Gas
• Accurate Computation of Quantum Excited States with Neural Networks
• Message-Passing Neural Quantum States for the Homogeneous Electron Gas
• Density Matrix Renormalization Group with Tensor Processing Units
• Metal-insulator transition in transition metal dichalcogenide heterobilayer: accurate treatment of interaction
• Ground state phases of the two-dimension electron gas with a unified variational approach
• Two-Dimensional Moiré Polaronic Electron Crystals
• Solving and visualizing fractional quantum Hall wavefunctions with neural network
• Taming Landau level mixing in fractional quantum Hall states with deep learning
• Deep Neural Networks as Variational Solutions for Correlated Open Quantum Systems
• Variational Neural and Tensor Network Approximations of Thermal States

Cited by in corpus (5)

• Topological Order in Neural Wavefunctions
• Bound on entanglement in neural quantum states
• Variational Monte Carlo Optimization of Topological Chiral Superconductors
• Comparing Symmetrized Determinant Neural Quantum States for the Hubbard Model
• Quantum melting a Wigner crystal into Hall liquids