Compression challenges in large scale PDE solvers

Abstract

Solvers for partial differential equations (PDE) are one of the cornerstones of computational science. For large problems, they involve huge amounts of data that needs to be stored and transmitted on all levels of the memory hierarchy. Often, bandwidth is the limiting factor due to relatively small arithmetic intensity, and increasingly so due to the growing disparity between computing power and bandwidth. Consequently, data compression techniques have been investigated and tailored towards the specific requirements of PDE solvers during the last decades. This paper surveys data compression challenges and corresponding solution approaches for PDE problems, covering all levels of the memory hierarchy from mass storage up to main memory. Exemplarily, we illustrate concepts at particular methods, and give references to alternatives.

arXiv commentrevised version

References in corpus (8)

• Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization
• Accelerating Scientific Computations with Mixed Precision Algorithms
• TTHRESH: Tensor Compression for Multidimensional Visual Data
• Parallel Tensor Compression for Large-Scale Scientific Data
• Error Analysis of ZFP Compression for Floating-Point Data
• TuckerMPI: A Parallel C++/MPI Software Package for Large-scale Data Compression via the Tucker Tensor Decomposition
• Improving Performance of Iterative Methods by Lossy Checkponting
• HexaShrink, an exact scalable framework for hexahedral meshes with attributes and discontinuities: multiresolution rendering and storage of geoscience models