24 citations · 34 across the 4 of their papers we have counts for
4 papers
Running Alchemist on Cray XC and CS Series Supercomputers: Dask and PySpark Interfaces, Deployment Options, and Data Transfer Times
Kai Rothauge, Haripriya Ayyalasomayajula, Kristyn J. Maschhoff +2
Alchemist is a system that allows Apache Spark to achieve better performance by interfacing with HPC libraries for large-scale distributed computations. In this paper, we highlight…
CosmoFlow: Using Deep Learning to Learn the Universe at Scale
Amrita Mathuriya, Deborah Bard, Peter Mendygral +14
Deep learning is a promising tool to determine the physical model that describes our universe. To handle the considerable computational cost of this problem, we present CosmoFlow:…
Alchemist: An Apache Spark <=> MPI Interface
Alex Gittens, Kai Rothauge, Shusen Wang +6
The Apache Spark framework for distributed computation is popular in the data analytics community due to its ease of use, but its MapReduce-style programming model can incur signif…
Accelerating Large-Scale Data Analysis by Offloading to High-Performance Computing Libraries using Alchemist
Alex Gittens, Kai Rothauge, Shusen Wang +6
Apache Spark is a popular system aimed at the analysis of large data sets, but recent studies have shown that certain computations---in particular, many linear algebra computations…