Understanding and Optimizing the Performance of Distributed Machine Learning Applications on Apache Spark

Celestine Dünner, Thomas Parnell, Kubilay Atasu +2

In this paper we explore the performance limits of Apache Spark for machine learning applications. We begin by analyzing the characteristics of a state-of-the-art distributed machi…