Solving Lattice QCD systems of equations using mixed precision solvers on GPUs

Abstract

Modern graphics hardware is designed for highly parallel numerical tasks and promises significant cost and performance benefits for many scientific applications. One such application is lattice quantum chromodyamics (lattice QCD), where the main computational challenge is to efficiently solve the discretized Dirac equation in the presence of an SU(3) gauge field. Using NVIDIA's CUDA platform we have implemented a Wilson-Dirac sparse matrix-vector product that performs at up to 40 Gflops, 135 Gflops and 212 Gflops for double, single and half precision respectively on NVIDIA's GeForce GTX 280 GPU. We have developed a new mixed precision approach for Krylov solvers using reliable updates which allows for full double precision accuracy while using only single or half precision arithmetic for the bulk of the computation. The resulting BiCGstab and CG solvers run in excess of 100 Gflops and, in terms of iterations until convergence, perform better than the usual defect-correction approach for mixed precision.

30 pages, 7 figures

References in corpus (3)

• Lattice QCD as a video game
• Adaptive Multigrid Algorithm for Lattice QCD
• Excited State Nucleon Spectrum with Two Flavors of Dynamical Fermions

Cited by in corpus (259)

• Excited state baryon spectroscopy from lattice QCD
• Excited and exotic charmonium spectroscopy from lattice QCD
• Energy dependence of the ρ resonance in ππ elastic scattering from lattice QCD
• Coupled $\pi \pi ,K\overline{K}$ scattering in $P$-wave and the $\rho$ resonance from lattice QCD
• A percent-level determination of the nucleon axial coupling from Quantum Chromodynamics
• S and D-wave phase shifts in isospin-2 pi pi scattering from lattice QCD
• Isoscalar $\pi \pi$ scattering and the $\sigma$ meson resonance from QCD
• Toward the excited isoscalar meson spectrum from lattice QCD
• Lattice calculation of the short and intermediate time-distance hadronic vacuum polarization contributions to the muon magnetic moment using twisted-mass fermions
• The Flavor Structure of the Excited Baryon Spectra from Lattice QCD
• Isoscalar meson spectroscopy from lattice QCD
• Highly excited and exotic meson spectrum from dynamical lattice QCD
• Resonances in coupled $\pi K,\eta K$ scattering from lattice QCD
• An $a_0$ resonance in strongly coupled $\pi \eta$, $K\overline{K}$ scattering from lattice QCD
• Coupled-Channel $D\pi$, $D\eta$ and $D_s\bar{K}$ Scattering from Lattice QCD
• Iso-vector and Iso-scalar Tensor Charges of the Nucleon from Lattice QCD
• Two Nucleon Systems at $m_{\pi }\sim 450\mathrm{MeV}$ from Lattice QCD
• Light composite scalar in eight-flavor QCD on the lattice
• Isoscalar $\pi \pi ,K\overline{K},\eta \eta$ scattering and the $\sigma ,f_0,f_2$ mesons from QCD
• Speedup for quantum optimal control from automatic differentiation based on graphics processing units
• Two-Nucleon Higher Partial-Wave Scattering from Lattice QCD
• Valence parton distribution function of pion from fine lattice
• Scaling Lattice QCD beyond 100 GPUs
• Quantum Numbers of Recently Discovered ${\Omega }_c^0$ Baryons from Lattice QCD
• Hybrid Baryons in QCD
• Excited and exotic charmonium, $D_s$ and $D$ meson spectra for two light quark masses from lattice QCD
• The art of simulating the early Universe -- Part I
• Valence parton distribution of pion from lattice QCD: Approaching continuum
• Microscopic Origin of \boldmath{$U_A(1)$} Symmetry Violation in the High Temperature Phase of QCD
• Helicity operators for mesons in flight on the lattice
• Excited spectroscopy of charmed mesons from lattice QCD
• Pion valence quark distribution from current-current correlation in lattice QCD
• Up, down, and strange nucleon axial form factors from lattice QCD
• Precision Nucleon Charges and Form Factors Using 2+1-flavor Lattice QCD
• Baryon-Baryon Interactions and Spin-Flavor Symmetry from Lattice Quantum Chromodynamics
• Spectroscopy of doubly-charmed baryons from lattice QCD
• Gluon gravitational structure of hadrons of different spin
• Gravitational form factors of the proton from lattice QCD
• The $\pi \pi \to \pi {\gamma }^{\star }$ amplitude and the resonant $\rho \to \pi {\gamma }^{\star }$ transition from lattice QCD
• QIRAL: A High Level Language for Lattice QCD Code Generation
• Spectroscopy of triply-charmed baryons from lattice QCD
• Tetraquark operators in lattice QCD and exotic flavour states in the charm sector
• Parton Distribution Functions from Ioffe Time Pseudodistributions from Lattice Calculations: Approaching the Physical Point
• Pion and Kaon Distribution Amplitudes from Lattice QCD
• Decays of an exotic $1^{-+}$ hybrid meson resonance in QCD
• Towards grounding nuclear physics in QCD
• Lattice QCD calculation of the Collins-Soper kernel from quasi TMDPDFs
• The phase-shift of isospin-2 pi-pi scattering from lattice QCD
• A look inside charmed-strange baryons from lattice QCD
• Heavy physics contributions to neutrinoless double beta decay from QCD
• Massively parallel lattice-Boltzmann codes on large GPU clusters
• The resonant ${\pi }^{+}\gamma \to {\pi }^{+}{\pi }^0$ amplitude from Quantum Chromodynamics
• High-precision calculation of the strange nucleon electromagnetic form factors
• Nuclear matrix elements from lattice QCD for electroweak and beyond-Standard-Model processes
• On the Feynman-Hellmann Theorem in Quantum Field Theory and the Calculation of Matrix Elements
• Self-Renormalization of Quasi-Light-Front Correlators on the Lattice
• $T_{cc}^{+}(3875)$ relevant $D{D}^{\ast }$ scattering from $N_f=2$ lattice QCD
• Quark-gluon plasma in an external magnetic field
• Two-nucleon S-wave interactions at the $SU(3)$ flavor-symmetric point with $m_{ud}\simeq m_s^{\mathrm{phys}}$: a first lattice QCD calculation with the stochastic Laplacian Heaviside method
• Exploring strange nucleon form factors on the lattice
• Double-$\beta$ Decay Matrix Elements from Lattice Quantum Chromodynamics
• The energy-dependent ${\pi }^{+}{\pi }^{+}{\pi }^{+}$ scattering amplitude from QCD
• The quark-mass dependence of elastic $\pi K$ scattering from QCD
• Digitizing Gauge Fields: Lattice Monte Carlo Results for Future Quantum Computers
• The isotensor axial polarisability and lattice QCD input for nuclear double-$\beta$ decay phenomenology
• Continuum-extrapolated NNLO Valence PDF of Pion at the Physical Point
• The $b_1$ resonance in coupled $\pi \omega$, $\pi \varphi$ scattering from lattice QCD
• Scattering of two and three physical pions at maximal isospin from lattice QCD
• $DK$ $I=0,$ $D\bar{K}$ $I=0,1$ scattering and the $D_{s0}^{\ast }(2317)$ from lattice QCD
• Sigma terms and strangeness content of the nucleon with $N_f=2+1+1$ twisted mass fermions
• Status of Lattice QCD Determination of Nucleon Form Factors and their Relevance for the Few-GeV Neutrino Program
• Charmed baryon spectrum from lattice QCD near the physical point
• Hadron-Hadron Interactions from $N_f=2+1+1$ Lattice QCD: isospin-2 $\pi \pi$ scattering length
• Distribution Amplitudes of $K^{\ast }$ and $\varphi$ at Physical Pion Mass from Lattice QCD
• Dynamically-coupled partial-waves in $\rho \pi$ isospin-2 scattering from lattice QCD
• Multi-mass solvers for lattice QCD on GPUs
• Neural-network analysis of Parton Distribution Functions from Ioffe-time pseudodistributions
• A variational study of two-nucleon systems with lattice QCD
• $⟨x⟩$ and $⟨x^2⟩$ of the pion PDF from Lattice QCD with $N_f=2+1+1$ dynamical quark flavours
• Lattice computation of the electromagnetic contributions to kaon and pion masses
• First Lattice QCD determination of semileptonic decays of charmed-strange baryons ${\Xi }_c$
• Isospin-1/2 $D\pi$ scattering and the lightest $D_0^{\ast }$ resonance from lattice QCD
• Lattice Calculation of the Intrinsic Soft Function and the Collins-Soper Kernel
• A Framework for Lattice QCD Calculations on GPUs
• Status and Future Perspectives for Lattice Gauge Theory Calculations to the Exascale and Beyond
• Pseudo-random number generators for Monte Carlo simulations on Graphics Processing Units
• First lattice QCD study of the gluonic structure of light nuclei
• Probing the energy-smeared R-ratio on the lattice
• Eigenvalue spectra of QCD and the fate of $U_A(1)$ breaking towards the chiral limit
• $F_K/F_{\pi }$ from Möbius domain-wall fermions solved on gradient-flowed HISQ ensembles
• ${\Xi }_c\gamma \to {\Xi }_c^{\prime }$ transition in lattice QCD
• Low-energy Scattering and Effective Interactions of Two Baryons at $m_{\pi }\sim 450$ MeV from Lattice Quantum Chromodynamics
• S-wave scattering of strangeness -3 baryons
• Radiative transitions of doubly charmed baryons in lattice QCD
• Unpolarized proton PDF at NNLO from lattice QCD with physical quark masses
• Evaluation of disconnected quark loops for hadron structure using GPUs
• Study of the pseudoscalar glueball in $J/\psi$ radiative decays
• Lattice QCD based on OpenCL
• Hadron-Hadron Interactions from $N_f=2+1+1$ Lattice QCD: The $\rho$-resonance
• Searching for the rules that govern hadron construction
• An accurate calculation of the nucleon axial charge with lattice QCD
• QCD constraints on isospin-dense matter and the nuclear equation of state
• Towards the determination of the gluon helicity distribution in the nucleon from lattice quantum chromodynamics
• Renormalization of transverse-momentum-dependent parton distribution on the lattice
• Coulomb, Landau and Maximally Abelian Gauge Fixing in Lattice QCD with Multi-GPUs
• The $\rho$-resonance with physical pion mass from $N_f=2$ lattice QCD
• An estimate of the hadronic vacuum polarization disconnected contribution to the anomalous magnetic moment of the muon from lattice QCD
• Learning Trivializing Gradient Flows for Lattice Gauge Theories
• q-State Potts model metastability study using optimized GPU-based Monte Carlo algorithms
• Moments of nucleon isovector structure functions in $2+1+1$-flavor QCD
• Topological susceptibility from twisted mass fermions using spectral projectors and the gradient flow
• $\omega$ meson from lattice QCD
• QCD simulations with staggered fermions on GPUs
• On the generalised eigenvalue method and its relation to Prony and generalised pencil of function methods
• The Möbius Domain Wall Fermion Algorithm
• Efficient solution of the multi-channel Lüscher determinant condition through eigenvalue decomposition
• The chirally improved quark propagator and restoration of chiral symmetry
• Determination of Delta resonance parameters from lattice QCD
• SIMULATeQCD: A simple multi-GPU lattice code for QCD calculations
• Multigrid for Staggered Lattice Fermions
• An update on the BQCD Hybrid Monte Carlo program
• Uncertainty Quantification in Lattice QCD Calculations for Nuclear Physics
• Calculation of fermion loops for ${\eta }^{\prime }$ and nucleon scalar and electromagnetic form factors
• RI/MOM renormalization of the quasi-PDF in lattice regularization
• Ruling Out the Massless Up-Quark Solution to the Strong CP Problem by Computing the Topological Mass Contribution with Lattice QCD
• Quark mass dependence of $\pi \pi$ scattering in isospin 0, 1, and 2 from lattice QCD
• Excited and exotic bottomonium spectroscopy from lattice QCD
• The Antiferromagnetic Character of the Quantum Phase Transition in the Hubbard Model on the Honeycomb Lattice
• Möbius domain-wall fermions on gradient-flowed dynamical HISQ ensembles
• Scale setting the Möbius Domain Wall Fermion on gradient-flowed HISQ action using the omega baryon mass and the gradient-flow scales $t_0$ and $w_0$
• $1^{-+}$ Hybrid in $J/\psi$ Radiative Decays from Lattice QCD
• A Survey of Numerical Methods Utilizing Mixed Precision Arithmetic
• Study of decuplet baryon resonances from lattice QCD
• Design of an energy aware petaflops class high performance cluster based on power architecture
• Scalar and tensor charmonium resonances in coupled-channel scattering from QCD
• Towards Unpolarized GPDs from Pseudo-Distributions
• Near-threshold states in coupled $D{D}^{\ast }-D^{\ast }{D}^{\ast }$ scattering from lattice QCD
• Design and optimization of a portable LQCD Monte Carlo code using OpenACC
• Distillation at High-Momentum
• Generating SU(Nc) pure gauge lattice QCD configurations on GPUs with CUDA
• ${\Xi }_c-{\Xi }_c^{\prime }$ mixing From Lattice QCD
• QCD Predictions for Meson Electromagnetic Form Factors at High Momenta: Testing Factorization in Exclusive Processes
• SU(2) Lattice Gauge Theory Simulations on Fermi GPUs
• Radiative decay of the resonant $K^{\ast }$ and the $\gamma K\to K\pi$ amplitude from lattice QCD
• The axial charge of the triton from lattice QCD
• The topological objects near the chiral crossover transition in QCD
• Determination of crossing-symmetric $\pi \pi$ scattering amplitudes and the quark mass evolution of the $\sigma$ constrained by lattice QCD
• Towards a full solution of relativistic Boltzmann equation for quark-gluon matter on GPUs
• Hadron-Hadron Interactions from $N_f=2+1+1$ Lattice QCD: $I=3/2$ $\pi K$ Scattering Length
• Hadron-Hadron Interactions from $N_f=2+1+1$ Lattice QCD: isospin-1 $KK$ scattering length
• Excited $J^{--}$ meson resonances at the SU(3) flavor point from lattice QCD
• Lattice QCD constraints on the parton distribution functions of ${}^3\text{He}$
• Lattice QCD Calculation of $x$-dependent Meson Distribution Amplitudes at Physical Pion Mass with Threshold Logarithm Resummation
• The Positive-Parity Baryon Spectrum and the Role of Hybrid Baryons
• Non-singlet quark helicity PDFs of the nucleon from pseudo-distributions
• The Glueball content of ${\eta }_c$
• Charmonium ${\chi }_{c0}$ and ${\chi }_{c2}$ resonances in coupled-channel scattering from lattice QCD
• Pushing Memory Bandwidth Limitations Through Efficient Implementations of Block-Krylov Space Solvers on GPUs
• Parallelizing the QUDA Library for Multi-GPU Calculations in Lattice Quantum Chromodynamics
• Non-Hertz-Millis scaling of the antiferromagnetic quantum critical metal via scalable Hybrid Monte Carlo
• $DK/D\pi$ scattering and an exotic virtual bound state at the $SU(3)$ flavour symmetric point from lattice QCD
• Landau Gauge Fixing on GPUs
• Towards a semi-classical description of QCD vacuum around $T_c$
• Three-dimensional Imaging of Pion using Lattice QCD: Generalized Parton Distributions
• The topological structures in strongly coupled QGP with chiral fermions on the lattice
• Pion Transition Form Factor from Twisted-Mass Lattice QCD and the Hadronic Light-by-Light ${\pi }^0$-pole Contribution to the Muon $g-2$
• Radiative Decay Width of $J/\psi \to \gamma {\eta }_{(2)}$ from $N_f=2$ Lattice QCD
• Exploratory investigation of nucleon-nucleon interactions using Euclidean Monte Carlo simulations
• $\eta$-glueball mixing from $N_f=2$ lattice QCD
• Axial-vector $D_1$ hadrons in $D^{\ast }\pi$ scattering from QCD
• Light Cone Distribution Amplitude for the $\Lambda$ Baryon from Lattice QCD
• Simulating the weak death of the neutron in a femtoscale universe with near-Exascale computing
• Improved method to determine the ${\Xi }_c-{\Xi }_c^{\prime }$ mixing
• ${\Lambda }_c{\Sigma }_c\pi$ coupling and ${\Sigma }_c\to {\Lambda }_c\pi$ decay in lattice QCD
• Quark and gluon momentum fractions in the pion from $N_f=2+1+1$ lattice QCD
• New general approach in few-body scattering calculations: Solving discretized Faddeev equations on a graphics processing unit
• Accelerating the solution of families of shifted linear systems with CUDA
• First lattice QCD calculation of $J/\psi$ semileptonic decay containing $D$ and $D_s$ particles
• Application-Driven Exascale: The JUPITER Benchmark Suite
• Annihilation diagram contribution to charmonium masses
• SVE-enabling Lattice QCD Codes
• Detailed analysis of excited state systematics in a lattice QCD calculation of $g_A$
• Lattice QCD on new chips: a community summary
• Electromagnetic contributions to pseudoscalar masses
• QCDGPU: open-source package for Monte Carlo lattice simulations on OpenCL-compatible multi-GPU systems
• Radiative Transitions in Charmonium from Lattice QCD
• Neutrinoless Double Beta Decay from Lattice QCD: The Short-Distance ${\pi }^{-}\to {\pi }^{+}{e}^{-}{e}^{-}$ Amplitude
• Towards Lattice Quantum Chromodynamics on FPGA devices
• Nucleon Momentum Fraction, Helicity and Transversity from 2+1-flavor Lattice QCD
• Lattice Wess-Zumino model with Ginsparg-Wilson fermions: One-loop results and GPU benchmarks
• Distance between various discretized fermion actions
• Quantum mean estimation for lattice field theory
• Object-oriented implementation of algebraic multi-grid solver for lattice QCD on SIMD architectures and GPU clusters
• Highly excited $B$, $B_s$ and $B_c$ meson spectroscopy from lattice QCD
• On Parallel Solution of Sparse Triangular Linear Systems in CUDA
• Multi GPU Performance of Conjugate Gradient Algorithm with Staggered Fermions
• Fast GPU-based calculations in few-body quantum scattering
• Calm Multi-Baryon Operators
• Energy Efficiency trends in HPC: what high-energy and astrophysicists need to know
• GPU acceleration of ab initio simulations of large-scale identical particles based on path integral molecular dynamics
• Efficient implementation of the overlap operator on multi-GPUs
• Accelerating QDP++/Chroma on GPUs
• Kaon radiative leptonic decay rates from lattice QCD simulations at the physical point
• QCD on GPUs: cost effective supercomputing
• Electromagnetic Structure of Spin-$\frac{1}{2}$ Doubly Charmed Baryons in Lattice QCD
• Di-nucleons do not form bound states at heavy pion mass
• Maximizing the Bang Per Bit
• Correlating confinement to topological fluctuations near the crossover transition in QCD
• Disconnected quark loop contributions to nucleon observables using $N_f=2$ twisted clover fermions at the physical value of the light quark mass
• Staggered Dslash Performance on Intel Xeon Phi Architecture
• Accelerating lattice QCD simulations with 2 flavours of staggered fermions on multiple GPUs using OpenACC - a first attempt
• Evaluation of disconnected contributions using GPUs
• The radiative decay of scalar glueball from lattice QCD
• Solving DWF Dirac Equation Using Multi-splitting Preconditioned Conjugate Gradient with Tensor Cores on NVIDIA GPUs
• Study of quarkonium in QGP from unquenched lattice QCD
• Decay properties of light $1^{-+}$ hybrids
• Topological observables and $\theta$ dependence in high temperature QCD from lattice simulations
• Lattice QCD study of ${\Lambda }_c{\Lambda }_c$ scattering
• LatticeQCD using OpenCL
• Electric polarizability of hadrons with overlap fermions on multi-GPUs
• The role of multigrid algorithms for LQCD
• Exact calculation of disconnected loops
• Lattice Analysis of $SU(2)$ with 1 Adjoint Dirac Flavor
• Collins-Soper kernel from lattice QCD at the physical pion mass
• $D_1$ and $D_2$ resonances in coupled-channel scattering amplitudes from lattice QCD
• Investigating the Dirac operator evaluation with FPGAs
• Finite Temperature Quarkonia Spectral Functions in the Pseudoscalar Channel
• Understanding the approach to thermalization from the eigenspectrum of non-Abelian gauge theories
• Coupled-channel scattering of $DD,D{D}^{\ast }$ and $D^{\ast }{D}^{\ast }$ in isospin-$1$ from lattice QCD
• Computation of disconnected contributions to nucleon observables
• Implementation of the twisted mass fermion operator in the QUDA library
• Emergence of the $\pi (1300)$ Resonance from Lattice QCD
• ${\eta }_c{\eta }_c$ and $J/\psi J/\psi$ scatterings from lattice QCD
• Calculation of the decay width of decuplet baryons
• Lattice QCD package GWU-code and QUDA with HIP
• An anisotropic preconditioning for the Wilson fermion matrix on the lattice
• The Continuum and Leading Twist Limits of Parton Distribution Functions in Lattice QCD
• Study of charm and beauty in QGP from unquenched lattice QCD
• Gauge fixing using overrelaxation and simulated annealing on GPUs
• Excited $D$ and $D_s$ meson spectroscopy from lattice QCD
• A QUDA-branch to compute disconnected diagrams in GPUs
• Towards High-Precision Parton Distributions From Lattice QCD via Distillation
• Unpolarized gluon distribution in the nucleon from lattice quantum chromodynamics
• Excited state baryon spectroscopy from lattice QCD with spin identification
• Mixed precision solvers with half-precision floating point numbers for Lattice QCD on A64FX processor
• Impact of Dynamical Charm Quark and Mixed Action Effect on Light Hadron Masses and Decay Constants
• Implementation of the conjugate gradient algorithm on FPGA devices
• Gauge Field Generation on Large-Scale GPU-Enabled Systems
• Lattice Calculation of the Proton Charge Radius
• Accessing the Gluon Momentum Fraction of Nucleons through the Gradient Flow
• The lightest $D_0^{\ast }$ resonance from lattice QCD
• The transversity parton distribution function of the nucleon using the pseudo-distribution approach
• Exotics in the $\pi D$ system
• Low-energy $DD$ scattering in lattice QCD
• Mixed Precision Solver Scalable to 16000 MPI Processes for Lattice Quantum Chromodynamics Simulations on the Oakforest-PACS System
• Kaon-kaon scattering at maximal isospin from $N_f=2+1+1$ twisted mass lattice QCD
• Imprints of $U_A(1)$ chiral anomaly and disorder in the Dirac eigenspectrum of QCD at finite temperature
• A heterogeneous many-core platform for experiments on scalable custom interconnects and management of fault and critical events, applied to many-process applications: Vol. II, 2012 technical report
• Glueball relevant study on isoscalars from $N_f=2$ lattice QCD