Exploring Scientific Application Performance Using Large Scale Object Storage
arXiv:1807.02562 · doi:10.1007/978-3-030-02465-9_8
Abstract
One of the major performance and scalability bottlenecks in large scientific applications is parallel reading and writing to supercomputer I/O systems. The usage of parallel file systems and consistency requirements of POSIX, that all the traditional HPC parallel I/O interfaces adhere to, pose limitations to the scalability of scientific applications. Object storage is a widely used storage technology in cloud computing and is more frequently proposed for HPC workload to address and improve the current scalability and performance of I/O in scientific applications. While object storage is a promising technology, it is still unclear how scientific applications will use object storage and what the main performance benefits will be. This work addresses these questions, by emulating an object storage used by a traditional scientific application and evaluating potential performance benefits. We show that scientific applications can benefit from the usage of object storage on large scales.
arXiv commentPreprint submitted to WOPSSS workshop at ISC 2018
References in corpus (5)
- Exploring the Performance Benefit of Hybrid Memory System on HPC Environments
- DAOS for Extreme-scale Systems in Scientific Applications
- SAGE: Percipient Storage for Exascale Data Centric Computing
- The SAGE Project: a Storage Centric Approach for Exascale Computing
- Exploring Application Performance on Emerging Hybrid-Memory Supercomputers