Is attention all you need to solve the correlated electron problem?
arXiv:2502.05383 · doi:10.1103/qxc3-bkc7
Abstract
The attention mechanism has transformed artificial intelligence research by its ability to learn relations between objects. In this work, we explore how a many-body wavefunction ansatz constructed from a large-parameter self-attention neural network can be used to solve the interacting electron problem in solids. By a systematic neural-network variational Monte Carlo study on a moiré quantum material, we demonstrate that the self-attention ansatz provides an accurate and efficient solution without human bias. Moreover, our numerical study finds that the required number of variational parameters scales roughly as with the number of electrons, which opens a path towards efficient large-scale simulations.
10+5 pages, comments welcome; v2: update refs, extend ED results; v3: minor updates
References in corpus (37)
- Solving the Quantum Many-Body Problem with Artificial Neural Networks
- Optical detection of Mott and generalized Wigner crystal states in WSe2/WS2 moiré superlattices
- Hubbard model physics in transition metal dichalcogenide moiré bands
- Fidelity approach to quantum phase transitions
- Quantum critical scaling of the geometric tensors
- Deep neural network solution of the electronic Schrödinger equation
- Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks
- Neural-Network Approach to Dissipative Quantum Many-Body Dynamics
- Fermionic neural-network states for ab-initio electronic structure
- Variational Quantum Monte Carlo Method with a Neural-Network Ansatz for Open Quantum Systems
- Variational neural network ansatz for steady states in open quantum systems
- Constructing neural stationary states for open quantum many-body systems
- Backflow Transformations via Neural Networks for Quantum Many-Body Wave-Functions
- Moiré quantum chemistry: charge transfer in transition metal dichalcogenide superlattices
- Gate-tunable heavy fermions in a moiré Kondo lattice
- NetKet: A Machine Learning Toolkit for Many-Body Quantum Systems
- Interplay of Fractional Chern Insulator and Charge-Density-Wave Phases in Twisted Bilayer Graphene
- Fermionic Wave Functions from Neural-Network Constrained Hidden States
- Transformer variational wave functions for frustrated quantum spin systems
- Discovering Quantum Phase Transitions with Fermionic Neural Networks
- Empowering deep neural quantum states through efficient optimization
- Ab initio calculation of real solids via neural network ansatz
- Metal-insulator transition in transition metal dichalcogenide heterobilayer moiré superlattices
- Wave function Ansatz (but Periodic) Networks and the Homogeneous Electron Gas
- Accurate Computation of Quantum Excited States with Neural Networks
- Gauge Invariant and Anyonic Symmetric Transformer and RNN Quantum States for Quantum Lattice Models
- Message-Passing Neural Quantum States for the Homogeneous Electron Gas
- Density Matrix Renormalization Group with Tensor Processing Units
- Neural-network quantum states at finite temperature
- Metal-insulator transition in transition metal dichalcogenide heterobilayer: accurate treatment of interaction
- Ground state phases of the two-dimension electron gas with a unified variational approach
- Chebyshev expansion of spectral functions using restricted Boltzmann machines
- Two-Dimensional Moiré Polaronic Electron Crystals
- Solving and visualizing fractional quantum Hall wavefunctions with neural network
- Taming Landau level mixing in fractional quantum Hall states with deep learning
- Deep Neural Networks as Variational Solutions for Correlated Open Quantum Systems
- Variational Neural and Tensor Network Approximations of Thermal States