ParaLog: Consistent Host-side Logging for Parallel Checkpoints

Abstract

Output-intensive scientific applications are highly sensitive to low storage throughput. While existing scientific application stacks are optimized for traditional High-Performance Computing (HPC) environments with high remote storage and network bandwidth, these assumptions often fail in modern settings like cloud deployment. This is because the existing scientific application I/O stack fails to leverage the available resources. At the same time, scientific applications exhibit special synchronization and data output requirements that are difficult to satisfy using traditional approaches such as block-level or filesystem-level caching. We introduce ParaLog, a distributed host-side logging approach designed to accelerate scientific applications transparently. ParaLog emphasizes deployability, enabling support for unmodified message passing interface (MPI) applications and implementations while preserving crash consistency semantics. We evaluate ParaLog across traditional HPC, cloud HPC, local clusters, and hybrid environments, demonstrating its capability to reduce end-to-end execution time by 13-26% for popular scientific applications in cloud settings.

Accepted to SoCC 2025

References in corpus (3)

• GROMACS in the cloud: A global supercomputer to speed up alchemical drug design
• On the role of solar wind expansion as a source of whistler waves: scattering of suprathermal electrons and heat flux regulation in the inner heliosphere
• Characterizing magnetic reconnection regions using Gaussian mixture models on particle velocity distributions