papers
Publications (3)
cs.DC2018
Accelerating Large-Scale Data Analysis by Offloading to High-Performance Computing Libraries using Alchemist
Alex Gittens, Kai Rothauge, Shusen Wang +6
Apache Spark is a popular system aimed at the analysis of large data sets, but recent studies have shown that certain computations---in particular, many linear algebra computations…
cs.DC2016
Matrix Factorization at Scale: a Comparison of Scientific Data Analytics in Spark and C+MPI Using Three Case Studies
Alex Gittens, Aditya Devarakonda, Evan Racah +14
We explore the trade-offs of performing linear algebra using Apache Spark, compared to traditional C and MPI implementations on HPC platforms. Spark is designed for data analytics…
cs.DC2018
Alchemist: An Apache Spark <=> MPI Interface
Alex Gittens, Kai Rothauge, Shusen Wang +6
The Apache Spark framework for distributed computation is popular in the data analytics community due to its ease of use, but its MapReduce-style programming model can incur signif…