DFT-FE 1.0: A massively parallel hybrid CPU-GPU density functional theory code using finite-element discretization

Abstract

We present DFT-FE 1.0, building on DFT-FE 0.6 [Comput. Phys. Commun. 246, 106853 (2020)], to conduct fast and accurate large-scale density functional theory (DFT) calculations (reaching ~ $100,000$ electrons) on both many-core CPU and hybrid CPU-GPU computing architectures. This work involves improvements in the real-space formulation -- via an improved treatment of the electrostatic interactions that substantially enhances the computational efficiency -- as well high-performance computing aspects, including the GPU acceleration of all the key compute kernels in DFT-FE. We demonstrate the accuracy by comparing the ground-state energies, ionic forces and cell stresses on a wide-range of benchmark systems against those obtained from widely used DFT codes. Further, we demonstrate the numerical efficiency of our implementation, which yields $\sim 20\times$ CPU-GPU speed-up by using GPU acceleration on hybrid CPU-GPU nodes. Notably, owing to the parallel-scaling of the GPU implementation, we obtain wall-times of $80-140$ seconds for full ground-state calculations, with stringent accuracy, on benchmark systems containing ~ $6,000-15,000$ electrons.

55 pages, 7 figures, 7 Tables

References in corpus (7)

• Quantum ESPRESSO: a modular and open-source software project for quantum simulations of materials
• Advanced capabilities for materials modelling with Quantum ESPRESSO
• Dislocation Core Energies and Core Fields from First Principles
• DFT Modelling of Explicit Solid-Solid Interfaces in Batteries: Methods and Challenges
• Large-scale all-electron density functional theory calculations using an enriched finite element basis
• Electronic-structure study of an edge dislocation in Aluminum and the role of macroscopic deformations on its energetics
• Spectrum-splitting approach for Fermi-operator expansion in all-electron Kohn-Sham DFT calculations

Cited by in corpus (7)

• Roadmap on Electronic Structure Codes in the Exascale Era
• Large scale quantum chemistry with Tensor Processing Units
• Accelerating self-consistent field iterations in Kohn-Sham density functional theory using a low rank approximation of the dielectric matrix
• Ionic forces and stress tensor in all-electron DFT calculations using enriched finite element basis
• Subspace recursive Fermi-operator expansion strategies for large-scale DFT eigenvalue problems on HPC architectures
• Fast hardware-aware matrix-free algorithm for higher-order finite-element discretized matrix multivector products on distributed systems
• Chemical bonding in large systems using projected population analysis from real-space density functional theory calculations