Transformer variational wave functions for frustrated quantum spin systems

Abstract

The Transformer architecture has become the state-of-art model for natural language processing tasks and, more recently, also for computer vision tasks, thus defining the Vision Transformer (ViT) architecture. The key feature is the ability to describe long-range correlations among the elements of the input sequences, through the so-called self-attention mechanism. Here, we propose an adaptation of the ViT architecture with complex parameters to define a new class of variational neural-network states for quantum many-body systems, the ViT wave function. We apply this idea to the one-dimensional $J_1$-$J_2$ Heisenberg model, demonstrating that a relatively simple parametrization gets excellent results for both gapped and gapless phases. In this case, excellent accuracies are obtained by a relatively shallow architecture, with a single layer of self-attention, thus largely simplifying the original architecture. Still, the optimization of a deeper structure is possible and can be used for more challenging models, most notably highly-frustrated systems in two dimensions. The success of the ViT wave function relies on mixing both local and global operations, thus enabling the study of large systems with high accuracy.

7 pages (including the Supplemental Material), 6 figures

References in corpus (3)

• The density-matrix renormalization group in the age of matrix product states
• Restricted-Boltzmann-Machine Learning for Solving Strongly Correlated Quantum Systems
• Entanglement entropy of critical spin liquids

Cited by in corpus (40)

• A simple linear algebra identity to optimize Large-Scale Neural Network Quantum States
• Variational Monte Carlo with Large Patched Transformers
• Mapping of attention mechanisms to a generalized Potts model
• Transformer Wave Function for two dimensional frustrated magnets: emergence of a Spin-Liquid Phase in the Shastry-Sutherland Model
• Phase diagram of the $J$-$J_d$ Heisenberg Model on the Maple-Leaf Lattice: Neural networks and density matrix renormalization group
• Foundation Neural-Networks Quantum States as a Unified Ansatz for Multiple Hamiltonians
• Neural Projected Quantum Dynamics: a systematic study
• Is attention all you need to solve the correlated electron problem?
• Variational optimization of the amplitude of neural-network quantum many-body ground states
• Transformer neural networks and quantum simulators: a hybrid approach for simulating strongly correlated systems
• Fine-tuning Neural Network Quantum States
• Scalable Imaginary Time Evolution with Neural Network Quantum States
• Impact of conditional modelling for a universal autoregressive quantum state
• Autoregressive neural quantum states of Fermi Hubbard models
• Policy-guided Monte Carlo on general state spaces: Application to glass-forming mixtures
• Are queries and keys always relevant? A case study on Transformer wave functions
• Accurate neural quantum states for interacting lattice bosons
• Spin Liquid and Superconductivity emerging from Steady States and Measurements
• Equivariant Variational Quantum Eigensolver to detect Phase Transitions through Energy Level Crossings
• Transformer Wave Function for Quantum Long-Range models
• Efficiency of neural quantum states in light of the quantum geometric tensor
• Time-dependent Neural Galerkin Method for Quantum Dynamics
• Non-stabilizerness of Neural Quantum States
• Beyond single-reference fixed-node approximation in ab initio Diffusion Monte Carlo using antisymmetrized geminal power applied to systems with hundreds of electrons
• Quantum many-body solver using artificial neural networks and its applications to strongly correlated electron systems
• First- and second-order quantum phase transitions in the long-range unfrustrated antiferromagnetic Ising chain
• Quantum-assisted variational Monte Carlo
• Design principles of deep translationally-symmetric neural quantum states for frustrated magnets
• Fluctuation based interpretable analysis scheme for quantum many-body snapshots
• Fermi Machine -- Quantum Many-Body Solver Derived from Correspondence between Noninteracting and Strongly Correlated Fermions
• Physics-informed Transformers for Electronic Quantum States
• Improving neural network performance for solving quantum sign structure
• Bound on entanglement in neural quantum states
• Variational Transformer Ansatz for the Density Operator of Steady States in Dissipative Quantum Many-Body Systems
• Emergence of global receptive fields capturing multipartite quantum correlations
• Brute-force positivization of $J_1-J_2$ model ground states
• Basis dependence of Neural Quantum States for the Transverse Field Ising Model
• Comparing Symmetrized Determinant Neural Quantum States for the Hubbard Model
• Group Convolutional Neural Network for the Low-Energy Spectrum in the Quantum Dimer Model
• Hyperbolic recurrent neural network as the first type of non-Euclidean neural quantum state ansatz