252 citations · 267 across the 7 of their papers we have counts for
6 papers · 1 filter
Ray Data: A dynamic and scalable system for heterogeneous data processing
Frank Sifei Luan, Ron Yifeng Wang, Yile Gu +12
While ML model training and inference are both GPU-intensive, CPU-based data processing is often the bottleneck. Existing distributed data processing systems assume homogeneous res…
Exoshuffle-CloudSort
Frank Sifei Luan, Stephanie Wang, Samyukta Yagati +7
We present Exoshuffle-CloudSort, a sorting application running on top of Ray using the Exoshuffle architecture. Exoshuffle-CloudSort runs on Amazon EC2, with input and output data…
Exoshuffle: An Extensible Shuffle Architecture
Frank Sifei Luan, Stephanie Wang, Samyukta Yagati +7
Shuffle is one of the most expensive communication primitives in distributed data processing and is difficult to scale. Prior work addresses the scalability challenges of shuffle b…
Hoplite: Efficient and Fault-Tolerant Collective Communication for Task-Based Distributed Systems
Siyuan Zhuang, Zhuohan Li, Danyang Zhuo +5
Task-based distributed frameworks (e.g., Ray, Dask, Hydro) have become increasingly popular for distributed applications that contain asynchronous and dynamic workloads, including…
Ray: A Distributed Framework for Emerging AI Applications
Philipp Moritz, Robert Nishihara, Stephanie Wang +8
The next generation of AI applications will continuously interact with the environment and learn from these interactions. These applications impose new and demanding systems requir…
Real-Time Machine Learning: The Missing Pieces
Robert Nishihara, Philipp Moritz, Stephanie Wang +7
Machine learning applications are increasingly deployed not only to serve predictions using static models, but also as tightly-integrated components of feedback loops involving dyn…