Multi-GPU thermal lattice Boltzmann simulations using OpenACC and MPI

Abstract

We assess the performance of the hybrid Open Accelerator (OpenACC) and Message Passing Interface (MPI) approach for multi-graphics processing units (GPUs) accelerated thermal lattice Boltzmann (LB) simulation. The OpenACC accelerates computation on a single GPU, and the MPI synchronizes the information between multiple GPUs. With a single GPU, the two-dimension (2D) simulation achieved 1.93 billion lattice updates per second (GLUPS) with a grid number of $8193^2$, and the three-dimension (3D) simulation achieved 1.04 GLUPS with a grid number of $385^3$, which is more than 76% of the theoretical maximum performance. On multi-GPUs, we adopt block partitioning, overlapping communications with computations, and concurrent computation to optimize parallel efficiency. We show that in the strong scaling test, using 16 GPUs, the 2D simulation achieved 30.42 GLUPS and the 3D simulation achieved 14.52 GLUPS. In the weak scaling test, the parallel efficiency remains above 99% up to 16 GPUs. Our results demonstrated that, with improved data and task management, the hybrid OpenACC and MPI technique is promising for thermal LB simulation on multi-GPUs.

arXiv comment31 pages, 12 figures

References in corpus (8)

• Lattice Boltzmann methods for multiphase flow and phase-change heat transfer
• Extreme flow simulations reveal skeletal adaptations of deep-sea sponges
• Sailfish: a flexible multi-GPU implementation of the lattice Boltzmann method
• Lattice Boltzmann simulations of three-dimensional thermal convective flows at high Rayleigh number
• Recent progress and challenges in exploiting graphics processors in computational fluid dynamics
• Massively parallel lattice-Boltzmann codes on large GPU clusters
• Performance and Portability of Accelerated Lattice Boltzmann Applications with OpenACC
• Accelerating CFD simulation with high order finite difference method on curvilinear coordinates for modern GPU clusters

Cited by in corpus (11)

• Particle-resolved thermal lattice Boltzmann simulation using OpenACC on multi-GPUs
• Wall-sheared thermal convection: heat transfer enhancement and turbulence relaminarization
• Particle transport and deposition in wall-sheared thermal turbulence
• Temporal modulation on mixed convection in turbulent channels
• Long-distance migration with minimal energy consumption in a thermal turbulent environment
• Pore-scale statistics of temperature and thermal energy dissipation rate in turbulent porous convection
• Super-resolution reconstruction of turbulent flows from a single Lagrangian trajectory
• Interpolation-supplemented lattice Boltzmann simulation of thermal convection on non-uniform meshes
• Triggering of extreme events and coherent-structure modulation in wall-turbulence under cyclostationary forces
• Performance of a high-order MPI-Kokkos accelerated fluid solver
• GPU-native Embedding of Complex Geometries in Adaptive Octree Grids Applied to the Lattice Boltzmann Method