On the Scalability of Data Reduction Techniques in Current and Upcoming HPC Systems from an Application Perspective
arXiv:1706.00522 · doi:10.1007/978-3-319-67630-2_2
Abstract
We implement and benchmark parallel I/O methods for the fully-manycore driven particle-in-cell code PIConGPU. Identifying throughput and overall I/O size as a major challenge for applications on today's and future HPC systems, we present a scaling law characterizing performance bottlenecks in state-of-the-art approaches for data reduction. Consequently, we propose, implement and verify multi-threaded data-transformations for the I/O library ADIOS as a feasible way to trade underutilized host-side compute potential on heterogeneous systems for reduced I/O latency.
arXiv comment15 pages, 5 figures, accepted for DRBSD-1 in conjunction with ISC'17
References in corpus (4)
- Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization
- Alpaka - An Abstraction Library for Parallel Kernel Acceleration
- DC magnetic field generation in unmagnetized shear flows
- Performance-Portable Many-Core Plasma Simulations: Porting PIConGPU to OpenPower and Beyond
Cited by in corpus (4)
- Improving I/O Performance for Exascale Applications through Online Data Layout Reorganization
- Transitioning from file-based HPC workflows to streaming data pipelines with openPMD and ADIOS2
- Enabling High-Throughput Parallel I/O in Particle-in-Cell Monte Carlo Simulations with openPMD and Darshan I/O Monitoring
- STZ: A High Quality and High Speed Streaming Lossy Compression Framework for Scientific Data