Physics-informed Transformers for Electronic Quantum States
arXiv:2412.12248 · doi:10.1038/s41467-025-66844-z
Abstract
Neural-network-based variational quantum states in general, and more recently autoregressive models in particular, have proven to be powerful tools to describe complex many-body wave functions. However, their performance crucially depends on the computational basis chosen and they often lack physical interpretability. To mitigate these issues, we here propose a modified variational Monte-Carlo framework which leverages prior physical information to construct a computational second-quantized basis containing a reference state that serves as a rough approximation to the true ground state. In this basis, a Transformer is used to parametrize and autoregressively sample the corrections to the reference state, giving rise to a more interpretable and computationally efficient representation of the ground state. We demonstrate this approach using a non-sparse fermionic model featuring a metal-insulator transition and employing Hartree-Fock and a strong-coupling limit to define physics-informed bases. We also show that the Transformer's hidden representation captures the natural energetic order of the different basis states. This work paves the way for more efficient and interpretable neural quantum-state representations.
9+10 pages, 4+5 figures
References in corpus (37)
- Solving the Quantum Many-Body Problem with Artificial Neural Networks
- A Critical Review of Recurrent Neural Networks for Sequence Learning
- Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks
- Deep autoregressive models for the efficient variational simulation of many-body quantum systems
- Symmetries and many-body excited states with neural-network quantum states
- Fermionic neural-network states for ab-initio electronic structure
- Twisted Bilayer Graphene IV. Exact Insulator Ground States and Phase Diagram
- Backflow Transformations via Neural Networks for Quantum Many-Body Wave-Functions
- Twisted bilayer graphene. II. Stable symmetry anomaly
- Ab-initio quantum chemistry with neural-network wavefunctions
- Fermionic Wave Functions from Neural-Network Constrained Hidden States
- Transformer variational wave functions for frustrated quantum spin systems
- Empowering deep neural quantum states through efficient optimization
- Transformer Quantum State: A Multi-Purpose Model for Quantum Many-Body Problems
- A simple linear algebra identity to optimize Large-Scale Neural Network Quantum States
- Accurate Computation of Quantum Excited States with Neural Networks
- Gauge Invariant and Anyonic Symmetric Transformer and RNN Quantum States for Quantum Lattice Models
- Shampoo: Preconditioned Stochastic Tensor Optimization
- Message-Passing Neural Quantum States for the Homogeneous Electron Gas
- Variational Monte Carlo with Large Patched Transformers
- Mapping of attention mechanisms to a generalized Potts model
- Transformer Wave Function for two dimensional frustrated magnets: emergence of a Spin-Liquid Phase in the Shastry-Sutherland Model
- Correlation-Enhanced Neural Networks as Interpretable Variational Quantum States
- Data-Enhanced Variational Monte Carlo Simulations for Rydberg Atom Arrays
- Neural network approach to quasiparticle dispersions in doped antiferromagnets
- Enhancing variational Monte Carlo using a programmable quantum simulator
- Unifying view of fermionic neural network quantum states: From neural network backflow to hidden fermion determinant states
- Representational Strengths and Limitations of Transformers
- Transformer neural networks and quantum simulators: a hybrid approach for simulating strongly correlated systems
- Autoregressive neural quantum states of Fermi Hubbard models
- Enhancing the Expressivity of Variational Neural, and Hardware-Efficient Quantum States Through Orbital Rotations
- Autoregressive Neural Quantum States with Quantum Number Symmetries
- Neural Quantum States and Peaked Molecular Wave Functions: Curse or Blessing?
- SOAP: Improving and Stabilizing Shampoo using Adam
- AdvNF: Reducing Mode Collapse in Conditional Normalising Flows using Adversarial Learning
- Variational Embeddings for Many Body Quantum Systems
- Simulating continuous-space systems with quantum-classical wave functions