A generalized massively parallel ultra-high order FFT-based Maxwell solver

Abstract

Dispersion-free ultra-high order FFT-based Maxwell solvers have recently proven to be paramount to a large range of applications, including the high-fidelity modeling of high-intensity laser-matter interactions with Particle-In-Cell (PIC) codes. To enable a massively parallel scaling of these solvers, a novel parallelization technique was recently proposed, which consists in splitting the simulation domain into several processor sub-domains, with guard regions appended at each sub-domain boundaries. Maxwell's equations are advanced independently on each sub-domain using local shared-memory FFTs (instead of a single distributed global FFT). This implies small truncation errors at sub-domain boundaries, the amplitude of which depends on guard regions sizes and order of the Maxwell solver. For a moderate number of guard cells (<10) , this 'local' technique proved to be highly scalable on up to a million cores. Yet, depending on the targeted applications, the number of guard cells required to mitigate truncations errors might be very large, which would severely limit the parallel efficiency of this technique due to the large volume of guard cells to be exchanged between sub-domains. In this context, we propose a novel 'hybrid' parallelization technique that ensures very good scaling of FFT-based solvers with an arbitrarily high number of guard cells. It consists in performing distributed FFTs on local groups of processors with guard regions now appended to boundaries of each group of processors. A dual domain decomposition method is used for the Maxwell solver and other parts of the PIC cycle to keep the simulation load-balanced. This 'hybrid' technique was implemented in the open source exascale library PICSAR. Benchmarks show that for a large number of guard cells ($>16$), the 'hybrid' technique offers a $\times 3$ speed-up and $\times 8$ memory savings compared to the 'local' one.

arXiv comment11 pages, 9 figures

References in corpus (11)

• SMILEI: a collaborative, open-source, multi-purpose particle-in-cell code for plasma simulation
• P3DFFT: a framework for parallel computations of Fourier transforms in three dimensions
• Warp-X: a new exascale computing platform for beam-plasma simulations
• Exploiting multi-scale parallelism for large scale numerical modelling of laser wakefield accelerators
• Achieving extreme light intensities using relativistic plasma mirrors
• Suppressing the Numerical Cherenkov Instability in FDTD PIC Codes
• Detailed analysis of the effects of stencil spatial variations with arbitrary high-order finite-difference Maxwell solver
• An efficient and portable SIMD algorithm for charge/current deposition in Particle-In-Cell codes
• Ultrahigh-order Maxwell solver with extreme scalability for electromagnetic PIC simulations of plasmas
• Accurate modeling of plasma acceleration with arbitrary order pseudo-spectral particle-in-cell methods
• Pseudo-spectral Maxwell solvers for an accurate modeling of Doppler harmonic generation on plasma mirrors with Particle-In-Cell codes

Cited by in corpus (3)

• A parallel-in-time approach for accelerating direct-adjoint studies
• How to generate intense isolated attosecond pulses from relativistic plasma mirrors?
• Modeling of Advanced Accelerator Concepts