45 citations · 144 across the 35 of their papers we have counts for
4 papers · 1 filter
SpCCL: A Sparsity-Aware Collective Communication Library for GPU Platforms
Lannie Dalton Hough, Emir Gencer, Hoffmann Muki +1
Collective communication is essential to high performance computing and machine learning workloads, yet libraries such as NCCL do not exploit sparsity in message payloads. Sending…
KEET: Explaining Performance of GPU Kernels Using LLM Agents
Joshua H. Davis, Klaudiusz Rydzy, Srinivasan Ramesh +5
Performance profiles of GPU kernels generated by tools such as Nsight Compute are rich in detail but are often challenging to interpret. To achieve the best performance possible on…
Communication-free Sampling and 4D Hybrid Parallelism for Scalable Mini-batch GNN Training
Cunyang Wei, Siddharth Singh, Aishwarya Sarkar +7
Graph neural networks (GNNs) are widely used for learning on graph datasets derived from various real-world scenarios. Learning from extremely large graphs requires distributed tra…
Speculating Experts Accelerates Inference for Mixture-of-Experts
Vivan Madan, Prajwal Singhania, Abhinav Bhatele +2
Mixture-of-Experts (MoE) models have gained popularity as a means of scaling the capacity of large language models (LLMs) while maintaining sparse activations and reduced per-token…