42 citations · 54 across the 7 of their papers we have counts for
3 papers · 1 filter
Running Alchemist on Cray XC and CS Series Supercomputers: Dask and PySpark Interfaces, Deployment Options, and Data Transfer Times
Kai Rothauge, Haripriya Ayyalasomayajula, Kristyn J. Maschhoff +2
Alchemist is a system that allows Apache Spark to achieve better performance by interfacing with HPC libraries for large-scale distributed computations. In this paper, we highlight…
Alchemist: An Apache Spark <=> MPI Interface
Alex Gittens, Kai Rothauge, Shusen Wang +6
The Apache Spark framework for distributed computation is popular in the data analytics community due to its ease of use, but its MapReduce-style programming model can incur signif…
Accelerating Large-Scale Data Analysis by Offloading to High-Performance Computing Libraries using Alchemist
Alex Gittens, Kai Rothauge, Shusen Wang +6
Apache Spark is a popular system aimed at the analysis of large data sets, but recent studies have shown that certain computations---in particular, many linear algebra computations…