Lightweight Task Offloading Exploiting MPI Wait Times for Parallel Adaptive Mesh Refinement

Abstract

Balancing the workload of sophisticated simulations is inherently difficult, since we have to balance both computational workload and memory footprint over meshes that can change any time or yield unpredictable cost per mesh entity, while modern supercomputers and their interconnects start to exhibit fluctuating performance. We propose a novel lightweight balancing technique for MPI+X to accompany traditional, prediction-based load balancing. It is a reactive diffusion approach that uses online measurements of MPI idle time to migrate tasks temporarily from overloaded to underemployed ranks. Tasks are deployed to ranks which otherwise would wait, processed with high priority, and made available to the overloaded ranks again. This migration is non-persistent. Our approach hijacks idle time to do meaningful work and is totally non-blocking, asynchronous and distributed without a global data view. Tests with a seismic simulation code developed in the ExaHyPE engine uncover the method's potential. We found speed-ups of up to 2-3 for ill-balanced scenarios without logical modifications of the code base and show that the strategy is capable to react quickly to temporarily changing workload or node performance.

References in corpus (7)

• Space-time adaptive ADER discontinuous Galerkin finite element schemes with a posteriori sub-cell finite volume limiting
• ExaHyPE: An Engine for Parallel Dynamically Adaptive Simulations of Wave Problems
• The Peano software - parallel, automaton-based, dynamically adaptive grid traversals
• Studies on the energy and deep memory behaviour of a cache-oblivious, task-based hyperbolic PDE solver
• Enclave Tasking for Discontinuous Galerkin Methods on Dynamically Adaptive Meshes
• Two Particle-in-Grid Realisations on Spacetrees
• TeaMPI -- Replication-based Resilience without the (Performance) Pain

Cited by in corpus (4)

• Callback-based Completion Notification using MPI Continuations
• Space-time adaptive ADER-DG finite element method with LST-DG predictor and a posteriori sub-cell WENO finite-volume limiting for simulation of non-stationary compressible multicomponent reactive flows
• Space-time adaptive ADER-DG finite element method with LST-DG predictor and a posteriori sub-cell ADER-WENO finite-volume limiting for multidimensional detonation waves simulation
• TeaMPI -- Replication-based Resilience without the (Performance) Pain