A Self-Attention Ansatz for Ab-initio Quantum Chemistry

Abstract

We present a novel neural network architecture using self-attention, the Wavefunction Transformer (Psiformer), which can be used as an approximation (or Ansatz) for solving the many-electron Schrödinger equation, the fundamental equation for quantum chemistry and material science. This equation can be solved from first principles, requiring no external training data. In recent years, deep neural networks like the FermiNet and PauliNet have been used to significantly improve the accuracy of these first-principle calculations, but they lack an attention-like mechanism for gating interactions between electrons. Here we show that the Psiformer can be used as a drop-in replacement for these other neural networks, often dramatically improving the accuracy of the calculations. On larger molecules especially, the ground state energy can be improved by dozens of kcal/mol, a qualitative leap over previous methods. This demonstrates that self-attention networks can learn complex quantum mechanical correlations between electrons, and are a promising route to reaching unprecedented accuracy in chemical calculations on larger systems.

Cited by in corpus (13)

• Variational Monte Carlo with Large Patched Transformers
• DeepQMC: an open-source software suite for variational optimization of deep-learning molecular wave functions
• Improved Optimization for the Neural-network Quantum States and Tests on the Chromium Dimer
• Highly Accurate Real-space Electron Densities with Neural Networks
• Are queries and keys always relevant? A case study on Transformer wave functions
• An improved penalty-based excited-state variational Monte Carlo approach with deep-learning ansatzes
• Second-order optimisation strategies for neural network quantum states
• Deep learning quantum Monte Carlo for solids
• Performance assessment of the effective core potentials under the Fermionic neural network: first and second row elements
• Synergy between deep neural networks and the variational Monte Carlo method for small ${}^{4}H{e}_N$ clusters
• Deep quantum Monte Carlo approach for polaritonic chemistry
• Determinant- and Derivative-Free Quantum Monte Carlo Within the Stochastic Representation of Wavefunctions
• Simple Fermionic backflow states via a systematically improvable tensor decomposition