Self-learning Monte Carlo with equivariant Transformer

Abstract

Machine learning and deep learning have revolutionized computational physics, particularly the simulation of complex systems. Equivariance is essential for simulating physical systems because it imposes a strong inductive bias on the probability distribution described by a machine learning model. However, imposing symmetry on the model can sometimes lead to poor acceptance rates in self-learning Monte Carlo (SLMC). Here, we introduce a symmetry equivariant attention mechanism for SLMC, which can be systematically improved. We evaluate our architecture on a spin-fermion model (\textit{i.e.}, double exchange model) on a two-dimensional lattice. Our results show that the proposed method overcomes the poor acceptance rates of linear models and exhibits a similar scaling law to large language models, with model quality monotonically increasing with the number of layers. Our work paves the way for the development of more accurate and efficient Monte Carlo algorithms with machine learning for simulating complex physical systems.

10 pates, 6 figures, Full paper version

References in corpus (23)

• E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
• Self-Learning Monte Carlo Method
• Flow-based generative models for Markov chain Monte Carlo in lattice field theory
• Equivariant flow-based sampling for lattice gauge theory
• Absence of spontaneous magnetic order at non-zero temperature in one- and two-dimensional Heisenberg and XY systems with long-range interactions
• Sampling using $SU(N)$ gauge equivariant flows
• Hybrid Monte Carlo algorithm for the Double Exchange Model
• Efficient Langevin Simulation of Coupled Classical Fields and Fermions
• Self-Learning Monte Carlo Method in Fermion Systems
• Self-learning Monte Carlo with Deep Neural Networks
• Self-Learning Monte Carlo Method: Continuous-Time Algorithm
• Efficient Modelling of Trivializing Maps for Lattice ${\varphi }^4$ Theory Using Normalizing Flows: A First Look at Scalability
• Gauge-equivariant flow models for sampling in lattice field theories with pseudofermions
• Absence of spontaneous magnetic order of lattice spins coupled to itinerant interacting electrons in one and two dimensions
• Self-learning Hybrid Monte Carlo: A First-principles Approach
• Order N Monte Carlo Algorithm for Fermion Systems Coupled with Fluctuating Adiabatical Fields
• Flow-based sampling in the lattice Schwinger model at criticality
• The Truncated Polynomial Expansion Monte Carlo Method for Fermion Systems Coupled to Classical Fields: A Model Independent Implementation
• Self-learning Monte Carlo method with Behler-Parrinello neural networks
• Monte Carlo Algorithm for the Double Exchange Model Optimized for Parallel Computations
• Self-learning hybrid Monte Carlo method for isothermal-isobaric ensemble: Application to liquid silica
• Atomic diffusion due to hyperatomic fluctuation for quasicrystals
• Effective Ruderman-Kittel-Kasuya-Yosida-like interaction in diluted double-exchange model: self-learning Monte Carlo approach

Cited by in corpus (2)

• Neural-network-assisted Monte Carlo sampling trained by Quantum Approximate Optimization Algorithm
• Extended framework for the hybrid Monte Carlo in lattice gauge theory