116 citations · 143 across the 9 of their papers we have counts for
3 papers · 1 filter
MSCCL++: Rethinking GPU Communication Abstractions for AI Inference
Changho Hwang, Peng Cheng, Roshan Dathathri +12
AI applications increasingly run on fast-evolving, heterogeneous hardware to maximize performance, but general-purpose libraries lag in supporting these features. Performance-minde…
A Framework for Fine-Grained Synchronization of Dependent GPU Kernels
Abhinav Jangda, Saeed Maleki, Maryam Mehri Dehnavi +2
Machine Learning (ML) models execute several parallel computations including Generalized Matrix Multiplication, Convolution, Dropout, etc. These computations are commonly executed…
TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches
Aashaka Shah, Vijay Chidambaram, Meghan Cowan +6
Machine learning models are increasingly being trained across multiple GPUs and servers. In this setting, data is transferred between GPUs using communication collectives such as A…