Transformer variational wave functions for frustrated quantum spin systems

Abstract

The Transformer architecture has become the state-of-art model for natural language processing tasks and, more recently, also for computer vision tasks, thus defining the Vision Transformer (ViT) architecture. The key feature is the ability to describe long-range correlations among the elements of the input sequences, through the so-called self-attention mechanism. Here, we propose an adaptation of the ViT architecture with complex parameters to define a new class of variational neural-network states for quantum many-body systems, the ViT wave function. We apply this idea to the one-dimensional $J_1$-$J_2$ Heisenberg model, demonstrating that a relatively simple parametrization gets excellent results for both gapped and gapless phases. In this case, excellent accuracies are obtained by a relatively shallow architecture, with a single layer of self-attention, thus largely simplifying the original architecture. Still, the optimization of a deeper structure is possible and can be used for more challenging models, most notably highly-frustrated systems in two dimensions. The success of the ViT wave function relies on mixing both local and global operations, thus enabling the study of large systems with high accuracy.

arXiv comment7 pages (including the Supplemental Material), 6 figures

References in corpus (25)

• The density-matrix renormalization group in the age of matrix product states
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• The ITensor Software Library for Tensor Network Calculations
• Dimerization and Incommensurate Spiral Spin Correlations in the Zigzag Spin Chain: Analogies to the Kondo Lattice
• Computational Studies of Quantum Spin Systems
• Restricted-Boltzmann-Machine Learning for Solving Strongly Correlated Quantum Systems
• Wave function optimization in the variational Monte Carlo method
• Numerical Evidence for Multiplicative Logarithmic Corrections from Marginal Operators
• Recurrent Neural Network Wave Functions
• Deep autoregressive models for the efficient variational simulation of many-body quantum systems
• Study of the Two-Dimensional Frustrated J1-J2 Model with Neural Network Quantum States
• Neural-Network Quantum States, String-Bond States, and Chiral Topological States
• Neural Quantum States of frustrated magnets: generalization and sign structure
• Entanglement entropy of critical spin liquids
• Solving frustrated quantum many-particle models with convolutional neural networks
• Neural Gutzwiller-projected variational wave functions
• Efficient Matrix Product State Method for periodic boundary conditions
• Neural network wave functions and the sign problem
• Autoregressive Transformer Neural Network for Simulating Open Quantum Systems via a Probabilistic Formulation
• Attention-based Quantum Tomography
• Learning the ground state of a non-stoquastic quantum Hamiltonian in a rugged neural network landscape
• Dynamical structure factor of the $J_1-J_2$ Heisenberg model in one dimension: the variational Monte Carlo approach
• Gauge Invariant and Anyonic Symmetric Transformer and RNN Quantum States for Quantum Lattice Models
• Suppression of Dimer Correlations in the Two-Dimensional $J_1$-$J_2$ Heisenberg Model: an Exact Diagonalization Study
• Neural Network Evolution Strategy for Solving Quantum Sign Structures

Cited by in corpus (41)

• A simple linear algebra identity to optimize Large-Scale Neural Network Quantum States
• Variational Monte Carlo with Large Patched Transformers
• Mapping of attention mechanisms to a generalized Potts model
• Transformer Wave Function for two dimensional frustrated magnets: emergence of a Spin-Liquid Phase in the Shastry-Sutherland Model
• Phase diagram of the $J$-$J_d$ Heisenberg Model on the Maple-Leaf Lattice: Neural networks and density matrix renormalization group
• Foundation Neural-Networks Quantum States as a Unified Ansatz for Multiple Hamiltonians
• Neural Projected Quantum Dynamics: a systematic study
• Is attention all you need to solve the correlated electron problem?
• Variational optimization of the amplitude of neural-network quantum many-body ground states
• Transformer neural networks and quantum simulators: a hybrid approach for simulating strongly correlated systems
• Fine-tuning Neural Network Quantum States
• Scalable Imaginary Time Evolution with Neural Network Quantum States
• Accurate neural quantum states for interacting lattice bosons
• Autoregressive neural quantum states of Fermi Hubbard models
• Impact of conditional modelling for a universal autoregressive quantum state
• Policy-guided Monte Carlo on general state spaces: Application to glass-forming mixtures
• Are queries and keys always relevant? A case study on Transformer wave functions
• Hybrid quantum-classical unsupervised data clustering based on the self-organizing feature map
• Spin Liquid and Superconductivity emerging from Steady States and Measurements
• Transformer Wave Function for Quantum Long-Range models
• Equivariant Variational Quantum Eigensolver to detect Phase Transitions through Energy Level Crossings
• Efficiency of neural quantum states in light of the quantum geometric tensor
• Non-stabilizerness of Neural Quantum States
• Quantum many-body solver using artificial neural networks and its applications to strongly correlated electron systems
• Beyond single-reference fixed-node approximation in ab initio Diffusion Monte Carlo using antisymmetrized geminal power applied to systems with hundreds of electrons
• Time-dependent Neural Galerkin Method for Quantum Dynamics
• First- and second-order quantum phase transitions in the long-range unfrustrated antiferromagnetic Ising chain
• Quantum-assisted variational Monte Carlo
• Fluctuation based interpretable analysis scheme for quantum many-body snapshots
• Design principles of deep translationally-symmetric neural quantum states for frustrated magnets
• Physics-informed Transformers for Electronic Quantum States
• Improving neural network performance for solving quantum sign structure
• Fermi Machine -- Quantum Many-Body Solver Derived from Correspondence between Noninteracting and Strongly Correlated Fermions
• Bound on entanglement in neural quantum states
• Variational Transformer Ansatz for the Density Operator of Steady States in Dissipative Quantum Many-Body Systems
• Emergence of global receptive fields capturing multipartite quantum correlations
• Brute-force positivization of $J_1-J_2$ model ground states
• Basis dependence of Neural Quantum States for the Transverse Field Ising Model
• Comparing Symmetrized Determinant Neural Quantum States for the Hubbard Model
• Group Convolutional Neural Network for the Low-Energy Spectrum in the Quantum Dimer Model
• Hyperbolic recurrent neural network as the first type of non-Euclidean neural quantum state ansatz