Differentiable Programming Tensor Networks

Abstract

Differentiable programming is a fresh programming paradigm which composes parameterized algorithmic components and trains them using automatic differentiation (AD). The concept emerges from deep learning but is not only limited to training neural networks. We present theory and practice of programming tensor network algorithms in a fully differentiable way. By formulating the tensor network algorithm as a computation graph, one can compute higher order derivatives of the program accurately and efficiently using AD. We present essential techniques to differentiate through the tensor networks contractions, including stable AD for tensor decomposition and efficient backpropagation through fixed point iterations. As a demonstration, we compute the specific heat of the Ising model directly by taking the second order derivative of the free energy obtained in the tensor renormalization group calculation. Next, we perform gradient based variational optimization of infinite projected entangled pair states for quantum antiferromagnetic Heisenberg model and obtain start-of-the-art variational energy and magnetization with moderate efforts. Differentiable programming removes laborious human efforts in deriving and implementing analytical gradients for tensor network programs, which opens the door to more innovations in tensor network algorithms and applications.

Typos corrected, discussion and refs added; revised version accepted for publication in PRX. Source code available at https://github.com/wangleiphy/tensorgrad

References in corpus (66)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• The density-matrix renormalization group in the age of matrix product states
• A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States
• Entanglement spectrum of a topological phase in one dimension
• Tensor-Entanglement-Filtering Renormalization Approach and Symmetry Protected Topological Order
• Classical simulation of infinite-size quantum lattice systems in one spatial dimension
• Entanglement Renormalization and Holography
• Classical simulation of infinite-size quantum lattice systems in two spatial dimensions
• Tensor renormalization group approach to 2D classical lattice models
• Tensorizing Neural Networks
• Computational Studies of Quantum Spin Systems
• Training Deep Nets with Sublinear Memory Cost
• Holographic duality from random tensor networks
• Competing states in the t-J model: uniform d-wave state versus stripe state
• Accurate determination of tensor network state of quantum lattice models in two dimensions
• The iTEBD algorithm beyond unitary evolution
• Simulating quantum computation by contracting tensor networks
• Corner Transfer Matrix Renormalization Group Method
• Coarse-graining renormalization by higher-order singular value decomposition
• Studying Two Dimensional Systems With the Density Matrix Renormalization Group
• Gapless spin-liquid ground state in the $S=1/2$ kagome antiferromagnet
• Tensor Network Renormalization
• Simulation of strongly correlated fermions in two spatial dimensions with fermionic Projected Entangled-Pair States
• Towards Quantum Machine Learning with Tensor Networks
• Continuous Matrix Product States for Quantum Fields
• Variational optimization algorithms for uniform matrix product states
• Unsupervised Generative Modeling Using Matrix Product States
• Second Renormalization of Tensor-Network States
• Entanglement renormalization for quantum fields
• Variational optimization with infinite projected entangled-pair states
• The iPEPS algorithm, improved: fast full update and gauge fixing
• Efficient Algorithms for Maximum Likelihood Decoding in the Surface Code
• Stripes in the two-dimensional t-J model with infinite projected entangled-pair states
• Gradient methods for variational optimization of projected entangled-pair states
• Renormalization of tensor-network states
• Loop optimization for tensor network renormalization
• Tensor renormalization of quantum many-body systems using projected entangled simplex states
• Speedup for quantum optimal control from automatic differentiation based on graphics processing units
• Faster Methods for Contracting Infinite 2D Tensor Networks
• Tree Tensor Networks for Generative Modeling
• Exploring corner transfer matrices and corner tensors for the classical simulation of quantum lattice systems
• Diagonalizing transfer matrices and matrix product operators: a medley of exact and computational methods
• Algorithmic differentiation and the calculation of forces by quantum Monte Carlo
• Real Space Renormalization in Statistical Mechanics
• Excitations and the tangent space of projected entangled-pair states
• Learning Relevant Features of Data with Multi-scale Tensor Networks
• Automatic differentiation in quantum chemistry with an application to fully variational Hartree-Fock
• Efficient simulation of infinite tree tensor network states on the Bethe lattice
• Tensor Networks and Quantum Error Correction
• Non-Abelian chiral spin liquid in a quantum antiferromagnet revealed by an iPEPS study
• Quantum critical phase with infinite projected entangled paired states
• Don't Unroll Adjoint: Differentiating SSA-Form Programs
• Residual entropies for three-dimensional frustrated spin systems with tensor networks
• Optimized contraction scheme for tensor-network states
• Training Deep Networks with Structured Layers by Matrix Backpropagation
• Gapless spin liquid in the kagome Heisenberg antiferromagnet with Dzyaloshinskii-Moriya interactions
• Phase transition of the q-state clock model: duality and tensor renormalization
• Continuous Tensor Network States for Quantum Fields
• Construction of Hamiltonians by supervised learning of energy and entanglement spectra
• Probabilistic Modeling with Matrix Product States
• Single-layer tensor network study of the Heisenberg model with chiral interactions on a kagome lattice
• Plaquette Renormalization Scheme for Tensor Network States
• Robust entanglement renormalization on a noisy quantum computer
• Auto-Differentiating Linear Algebra
• Continuous tensor network renormalization for quantum fields
• Tensor network language model

Cited by in corpus (163)

• The ITensor Software Library for Tensor Network Calculations
• Kohn-Sham equations as regularizer: building prior knowledge into machine-learned physics
• Beyond-classical computation in quantum simulation
• Developments in the Tensor Network -- from Statistical Mechanics to Quantum Entanglement
• Gapless quantum spin liquid and global phase diagram of the spin-1/2 $J_1$-$J_2$ square antiferromagnetic Heisenberg model
• Gapless spin liquid and valence-bond solid in the $J_1-J_2$ Heisenberg model on the square lattice: insights from singlet and triplet excitations
• Classically optimized Hamiltonian simulation
• Contracting Arbitrary Tensor Networks: General Approximate Algorithm and Applications in Graphical Models and Quantum Circuit Simulations
• Supervised Learning with Projected Entangled Pair States
• Investigation of the Néel phase of the frustrated Heisenberg antiferromagnet by differentiable symmetric tensor networks
• Deep-learning density functional perturbation theory
• Automatic differentiation applied to excitations with Projected Entangled Pair States
• DQC: a Python program package for Differentiable Quantum Chemistry
• Tropical Tensor Network for Ground States of Spin Glasses
• A differentiable programming method for quantum control
• Spin Excitation Spectra of Anisotropic Spin-$1/2$ Triangular Lattice Heisenberg Antiferromagnets
• Drawing Phase Diagrams of Random Quantum Systems by Deep Learning the Wave Functions
• Riemannian geometry and automatic differentiation for optimization problems of quantum physics and quantum technologies
• Simulation of three-dimensional quantum systems with projected entangled-pair states
• Differentiable quantum chemistry with PySCF for molecules and materials at the mean-field level and beyond
• Tensor network representations of parton wave functions
• Gradient Ascent Pulse Engineering with Feedback
• Accurate simulation for finite projected entangled pair states in two dimensions
• Bond-weighted Tensor Renormalization Group
• Optimal compression of quantum many-body time evolution operators into brickwall circuits
• Direct sampling of projected entangled-pair states
• Learning Pair Potentials using Differentiable Simulations
• Efficient Simulation of Dynamics in Two-Dimensional Quantum Spin Systems with Isometric Tensor Networks
• Automatic Differentiation for Second Renormalization of Tensor Networks
• Differentiable Molecular Simulations for Control and Learning
• Automatic differentiation of dominant eigensolver and its applications in quantum physics
• Sparse sampling and tensor network representation of two-particle Green's functions
• Gauging tensor networks with belief propagation
• Variational methods for contracting projected entangled-pair states
• Superconducting stripes in the hole-doped three-band Hubbard model
• Quantum Skyrmion Lattices in Heisenberg Ferromagnets
• First-order transition between the plaquette valence bond solid and antiferromagnetic phases of the Shastry-Sutherland model
• Detecting a $Z_2$ topologically ordered phase from unbiased infinite projected entangled-pair state simulations
• Efficient tensor network representation for Gutzwiller projected states of paired fermions
• Revealing the Phase Diagram of Kitaev Materials by Machine Learning: Cooperation and Competition between Spin Liquids
• Computing the renormalization group flow of two-dimensional ${\varphi }^4$ theory with tensor networks
• Wavefunction positivization via automatic differentiation
• A scaling hypothesis for projected entangled-pair states
• A beginner's guide to non-abelian iPEPS for correlated fermions
• Static and dynamical signatures of Dzyaloshinskii-Moriya interactions in the Heisenberg model on the kagome lattice
• Protocol Discovery for the Quantum Control of Majoranas by Differentiable Programming and Natural Evolution Strategies
• Topological fracton quantum phase transitions by tuning exact tensor network states
• A scheme for automatic differentiation of complex loss functions
• Competing quantum phases of hard-core boson with tilted dipole-dipole interaction
• Automatically Differentiable Quantum Circuit for Many-qubit State Preparation
• Automatic Differentiable Monte Carlo: Theory and Application
• Continuous matrix product operator approach to finite temperature quantum states
• Magnetization of the spin-1/2 Heisenberg antiferromagnet on the triangular lattice
• Generating Function for Tensor Network Diagrammatic Summation
• Augmenting Density Matrix Renormalization Group with Disentanglers
• Exact constraints and appropriate norms in machine learned exchange-correlation functionals
• Gapless spin liquids in disguise
• Fermionic tensor network methods
• Machine-Learned Phase Diagrams of Generalized Kitaev Honeycomb Magnets
• Fractional quantum Hall states with variational Projected Entangled-Pair States: a study of the bosonic Harper-Hofstadter model
• Chiral spin liquids on the kagome lattice with projected entangled simplex states
• An introduction to infinite projected entangled-pair state methods for variational ground state simulations using automatic differentiation
• Accurate Simulation of the Hubbard Model with Finite Fermionic Projected Entangled Pair States
• Variational optimization of tensor-network states with the honeycomb-lattice corner transfer matrix
• Control of Stochastic Quantum Dynamics by Differentiable Programming
• Mitigating the fermion sign problem by automatic differentiation
• Block belief propagation algorithm for two-dimensional tensor networks
• Automatic Differentiable Numerical Renormalization Group
• Scalably learning quantum many-body Hamiltonians from dynamical data
• Generating function for projected entangled-pair states
• Phase diagram of the chiral SU(3) antiferromagnet on the kagome lattice
• Ground state search by local and sequential updates of neural network quantum states
• Optimal Control in Disordered Quantum Systems
• Scaling dimensions from linearized tensor renormalization group transformations
• Simulating Spin Dynamics of Supersolid States in a Quantum Ising Magnet
• Matrix Product States with Backflow correlations
• Differentiable Programming of Isometric Tensor Networks
• Ground-state phase diagram of a spin-1/2 frustrated XXZ ladder
• Learning Effective Spin Hamiltonian of Quantum Magnet
• Fractionalization Signatures in the Dynamics of Quantum Spin Liquids
• Corner transfer matrix renormalization group approach in the zoo of Archimedean lattices
• Critical line of the triangular Ising antiferromagnet in a field from a $C_3$-symmetric corner transfer matrix algorithm
• Functional Tensor Network Solving Many-body Schrödinger Equation
• Automatic design of Hamiltonians
• Stable and efficient differentiation of tensor network algorithms
• Unsupervised mapping of phase diagrams of 2D systems from infinite projected entangled-pair states via deep anomaly detection
• Differentiable programming tensor networks for Kitaev magnets
• Efficient tensor network algorithm for layered systems
• A simplified and improved approach to tensor network operators in two dimensions
• Continuous ferromagnetic quantum phase transition on an anisotropic Kondo lattice
• Tree tensor network hierarchical equations of motion based on time-dependent variational principle for efficient open quantum dynamics in structured thermal environments
• Scalable simulation of non-equilibrium quantum dynamics via classically optimised unitary circuits
• U(1)-symmetric Gaussian fermionic projected entangled paired states and their Gutzwiller projection
• Spectral functions with infinite projected entangled-pair states
• Possible Observation of Quadrupole Waves in Spin Nematics
• The Hubbard model on the Bethe lattice via variational uniform tree states: metal-insulator transition and a Fermi liquid
• Frustration-Induced Superconductivity in the $t$-$t^{\prime }$ Hubbard Model
• Higher-order tensor renormalization group approach to lattice glass model
• Competing States in the Two-Dimensional Frustrated Kondo-Necklace Model
• Tensor network study of the Shastry-Sutherland model with weak interlayer coupling
• Variationally optimizing infinite projected entangled-pair states at large bond dimensions: A split corner transfer matrix renormalization group approach
• Calculation of critical exponents on fractal lattice Ising model by higher-order tensor renormalization group method
• QR and LQ Decomposition Matrix Backpropagation Algorithms for Square, Wide, and Deep -- Real or Complex -- Matrices and Their Software Implementation
• The Green's function Monte Carlo combined with projected entangled pair state approach to the frustrated $J_1$-$J_2$ Heisenberg model
• Quantum spin liquid phase in the Shastry-Sutherland model revealed by high-precision infinite projected entangled-pair states
• Probabilistic Inference in the Era of Tensor Networks and Differential Programming
• Differentiable matrix product states for simulating variational quantum computational chemistry
• Unveiling Correlated Two-dimensional Topological Insulators through Fermionic Tensor Network States -- Classification, Edge Theories and Variational Wavefunctions
• Gauge symmetry of excited states in projected entangled-pair state simulations
• Tensor Networks for Probabilistic Sequence Modeling
• Use and implementation of autodifferentiation in tensor network methods with complex scalars
• Variational Tensor Network Operator
• Symmetric projected entangled-pair states analysis of a phase transition in coupled spin-1/2 ladders
• Controlling quantum many-body systems using reduced-order modelling
• Cubic ferromagnet and emergent $U(1)$ symmetry on its phase boundary
• Simulating chiral spin liquids with projected entangled-pair states
• Measurements of magnetization on the Sierpiński carpet
• Differentiable master equation solver for quantum device characterisation
• A Matrix Product State Model for Simultaneous Classification and Generation
• Efficient conversion from fermionic Gaussian states to matrix product states
• Parametrized multiqubit gate design for neutral-atom based quantum platforms
• Numerical continuum tensor networks in two dimensions
• Inferring Hidden Symmetries of Exotic Magnets from Detecting Explicit Order Parameters
• Efficient optimization and conceptual barriers in variational finite Projected Entangled-Pair States
• Long-range spin-orbital order in the spin-orbital SU(2)$\times$SU(2)$\times$U(1) model
• Continuous topological phase transition between ${\mathbb{Z}}_2$ topologically ordered phases
• Two Dimensional Isometric Tensor Networks on an Infinite Strip
• Theory of magnetism for rare-earth magnets on the Shastry-Sutherland lattice with non-Kramers ions
• Continuous monitoring for noisy intermediate-scale quantum processors
• Efficient and systematic calculation of arbitrary observables for the matrix product state excitation ansatz
• Automatic Structural Search of Tensor Network States including Entanglement Renormalization
• Effective classical correspondence of the Mott transition
• Simulating quantum circuits using the multi-scale entanglement renormalization ansatz
• Finite-size scaling on the torus with periodic projected entangled-pair states
• Frustration-Induced Supersolid Phases of Extended Bose-Hubbard Model in the Hard-Core Limit
• The Cytnx Library for Tensor Networks
• Effective temperature in approximate quantum many-body states
• Approximate Contraction of Arbitrary Tensor Networks with a Flexible and Efficient Density Matrix Algorithm
• Inverse Hamiltonian design of highly-entangled quantum systems
• Efficient iPEPS Simulation on the Honeycomb Lattice via QR-based CTMRG
• Accelerating two-dimensional tensor network optimization by preconditioning
• Tensor renormalization group approach to critical phenomena via symmetry-twisted partition functions
• SANM: A Symbolic Asymptotic Numerical Solver with Applications in Mesh Deformation
• Finite Correlation Length Scaling of Disorder Parameter at Quantum Criticality
• Differentiate Everything with a Reversible Embeded Domain-Specific Language
• Forward-mode automatic differentiation for the tensor renormalization group and its relation to the impurity method
• Accelerating two-dimensional tensor network contractions using QR decompositions
• A hybrid method integrating Green's function Monte Carlo and projected entangled pair states
• Tensor Network Efficiently Representing Schmidt Decomposition of Quantum Many-Body States
• Dynamic Hologram Generation with Automatic Differentiation
• Efficient Simulation of Quantum Many-body Thermodynamics by Tailoring Zero-temperature Tensor Network
• Single-layer framework of variational tensor network states
• Variational optimization of projected entangled-pair states on the triangular lattice
• Kac-Moody symmetries in one-dimensional bosonic systems
• Les Houches Lecture Notes on Tensor Networks
• Dynamical Spectral Function of the Kagome Quantum Spin Liquid
• Itinerant magnetism in the triangular lattice Hubbard model at half-doping: Application to twisted transition-metal dichalcogenides
• Preparation of Many-body Ground States by Time Evolution with Variational Microscopic Magnetic Fields and Incomplete Interactions
• Programmable order by disorder effect and underlying phases through dipolar quantum simulators
• Truncating loopy tensor networks by zero-mode gauge fixing
• Accurate computation of the energy variance and $⟨⟨{\mathcal{L}}^{†}\mathcal{L}⟩⟩$ using iPEPS
• Triangular lattice models of the Kalmeyer-Laughlin spin liquid from coupled wires
• Inverse design and flexible parameterization of meta-optics using algorithmic differentiation