DFT-FE 1.0: A massively parallel hybrid CPU-GPU density functional theory code using finite-element discretization

Abstract

We present DFT-FE 1.0, building on DFT-FE 0.6 [Comput. Phys. Commun. 246, 106853 (2020)], to conduct fast and accurate large-scale density functional theory (DFT) calculations (reaching ~ $100,000$ electrons) on both many-core CPU and hybrid CPU-GPU computing architectures. This work involves improvements in the real-space formulation -- via an improved treatment of the electrostatic interactions that substantially enhances the computational efficiency -- as well high-performance computing aspects, including the GPU acceleration of all the key compute kernels in DFT-FE. We demonstrate the accuracy by comparing the ground-state energies, ionic forces and cell stresses on a wide-range of benchmark systems against those obtained from widely used DFT codes. Further, we demonstrate the numerical efficiency of our implementation, which yields $\sim 20\times$ CPU-GPU speed-up by using GPU acceleration on hybrid CPU-GPU nodes. Notably, owing to the parallel-scaling of the GPU implementation, we obtain wall-times of $80-140$ seconds for full ground-state calculations, with stringent accuracy, on benchmark systems containing ~ $6,000-15,000$ electrons.

arXiv comment55 pages, 7 figures, 7 Tables

References in corpus (26)

• Quantum ESPRESSO: a modular and open-source software project for quantum simulations of materials
• Advanced capabilities for materials modelling with Quantum ESPRESSO
• The PseudoDojo: Training and grading a 85 element optimized norm-conserving pseudopotential table
• Optimization Algorithm for the Generation of ONCV Pseudopotentials
• O(N) methods in electronic structure calculations
• Real-space grids and the Octopus code as tools for the development of new simulation approaches for electronic systems
• Ab Initio Study of Screw Dislocations in Mo and Ta: A new picture of plasticity in bcc transition metals
• DFT-FE -- A massively parallel adaptive finite-element code for large-scale density functional theory calculations
• Dislocation Core Energies and Core Fields from First Principles
• Higher-order adaptive finite-element methods for Kohn-Sham density functional theory
• RESCU: a Real Space Electronic Structure Method
• Real-space local polynomial basis for solid-state electronic-structure calculations: A finite-element approach
• SPARC: Accurate and efficient finite-difference formulation and parallel implementation of Density Functional Theory: Isolated clusters
• DGDFT: A Massively Parallel Method for Large Scale Density Functional Theory Calculations
• Real-space density functional theory on graphical processing units: computational approach and comparison to Gaussian basis set methods
• DFT Modelling of Explicit Solid-Solid Interfaces in Batteries: Methods and Challenges
• Large-scale all-electron density functional theory calculations using an enriched finite element basis
• A subquadratic-scaling subspace projection method for large-scale Kohn-Sham density functional theory calculations using spectral finite-element discretization
• Optimizations of the Eigensolvers in the ELPA Library
• Electronic-structure study of an edge dislocation in Aluminum and the role of macroscopic deformations on its energetics
• Configurational forces in electronic structure calculations using Kohn-Sham density functional theory
• Electronic structure study of screw dislocation core energetics in Aluminum and core energetics informed forces in a dislocation aggregate
• Real-time time-dependent density functional theory using higher order finite element methods
• Orbital-free DFT study of the energetics of vacancy clustering and prismatic dislocation loop nucleation in aluminum
• Fast and robust all-electron density functional theory calculations in solids using orthogonalized enriched finite elements
• Spectrum-splitting approach for Fermi-operator expansion in all-electron Kohn-Sham DFT calculations

Cited by in corpus (7)

• Roadmap on Electronic Structure Codes in the Exascale Era
• Large scale quantum chemistry with Tensor Processing Units
• Accelerating self-consistent field iterations in Kohn-Sham density functional theory using a low rank approximation of the dielectric matrix
• Ionic forces and stress tensor in all-electron DFT calculations using enriched finite element basis
• Subspace recursive Fermi-operator expansion strategies for large-scale DFT eigenvalue problems on HPC architectures
• Fast hardware-aware matrix-free algorithm for higher-order finite-element discretized matrix multivector products on distributed systems
• Chemical bonding in large systems using projected population analysis from real-space density functional theory calculations