Variational Monte Carlo with Large Patched Transformers

Abstract

Large language models, like transformers, have recently demonstrated immense powers in text and image generation. This success is driven by the ability to capture long-range correlations between elements in a sequence. The same feature makes the transformer a powerful wavefunction ansatz that addresses the challenge of describing correlations in simulations of qubit systems. Here we consider two-dimensional Rydberg atom arrays to demonstrate that transformers reach higher accuracies than conventional recurrent neural networks for variational ground state searches. We further introduce large, patched transformer models, which consider a sequence of large atom patches, and show that this architecture significantly accelerates the simulations. The proposed architectures reconstruct ground states with accuracies beyond state-of-the-art quantum Monte Carlo methods, allowing for the study of large Rydberg systems in different phases of matter and at phase transitions. Our high-accuracy ground state representations at reasonable computational costs promise new insights into general large-scale quantum many-body systems.

15 pages, 5 figures

References in corpus (36)

• Array Programming with NumPy
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Dipole Blockade and Quantum Information Processing in Mesoscopic Atomic Ensembles
• Fast quantum gates for neutral atoms
• Quantum Phases of Matter on a 256-Atom Programmable Quantum Simulator
• Programmable quantum simulation of 2D antiferromagnets with hundreds of Rydberg atoms
• Synthetic three-dimensional atomic structures assembled atom by atom
• Reconstructing quantum states with generative models
• Neural-Network Approach to Dissipative Quantum Many-Body Dynamics
• Deep autoregressive models for the efficient variational simulation of many-body quantum systems
• Quantum many-body dynamics in two dimensions with artificial neural networks
• Variational Quantum Monte Carlo Method with a Neural-Network Ansatz for Open Quantum Systems
• Variational neural network ansatz for steady states in open quantum systems
• Quantum phases of Rydberg atoms on a kagome lattice
• Integrating Neural Networks with a Quantum Simulator for State Reconstruction
• Latent Space Purification via Neural Density Operators
• Complex density wave orders and quantum phase transitions in a model of square-lattice Rydberg atom arrays
• Transformer variational wave functions for frustrated quantum spin systems
• Quenches near Ising quantum criticality as a challenge for artificial neural networks
• Neural network quantum state tomography in a two-qubit experiment
• Efficient quantum state tomography with convolutional neural networks
• Precise measurement of quantum observables with neural-network estimators
• Fast Preparation and Detection of a Rydberg Qubit using Atomic Ensembles
• Neural Error Mitigation of Near-Term Quantum Simulations
• Transformer Quantum State: A Multi-Purpose Model for Quantum Many-Body Problems
• Gauge Invariant and Anyonic Symmetric Transformer and RNN Quantum States for Quantum Lattice Models
• Probabilistic Simulation of Quantum Circuits with the Transformer
• Machine learning discovery of new phases in programmable quantum simulator snapshots
• U(1) symmetric recurrent neural networks for quantum state reconstruction
• Investigating Topological Order using Recurrent Neural Networks
• A Self-Attention Ansatz for Ab-initio Quantum Chemistry
• Data-Enhanced Variational Monte Carlo Simulations for Rydberg Atom Arrays
• Correlation-Enhanced Neural Networks as Interpretable Variational Quantum States
• Scaling Vision Transformers
• Unified Quantum State Tomography and Hamiltonian Learning Using Transformer Models: A Language-Translation-Like Approach for Quantum Systems
• Tomography of Quantum States from Structured Measurements via quantum-aware transformer

Cited by in corpus (19)

• Beyond-classical computation in quantum simulation
• A simple linear algebra identity to optimize Large-Scale Neural Network Quantum States
• Transformer Wave Function for two dimensional frustrated magnets: emergence of a Spin-Liquid Phase in the Shastry-Sutherland Model
• Fault-tolerant quantum algorithms for quantum molecular systems: A survey
• Foundation Neural-Networks Quantum States as a Unified Ansatz for Multiple Hamiltonians
• Transformer neural networks and quantum simulators: a hybrid approach for simulating strongly correlated systems
• Fine-tuning Neural Network Quantum States
• Autoregressive neural quantum states of Fermi Hubbard models
• Are queries and keys always relevant? A case study on Transformer wave functions
• Transformer Wave Function for Quantum Long-Range models
• Approximately-symmetric neural networks for quantum spin liquids
• Recurrent neural network wave functions for Rydberg atom arrays on kagome lattice
• Time-dependent Neural Galerkin Method for Quantum Dynamics
• Design principles of deep translationally-symmetric neural quantum states for frustrated magnets
• Simple Fermionic backflow states via a systematically improvable tensor decomposition
• Physics-informed Transformers for Electronic Quantum States
• Mapping Phase Diagrams of Quantum Spin Systems through Semidefinite-Programming Relaxations
• Hyperbolic recurrent neural network as the first type of non-Euclidean neural quantum state ansatz
• Comparing Symmetrized Determinant Neural Quantum States for the Hubbard Model