1 citations · 1 across the 2 of their papers we have counts for
3 papers
Mixture-of-Kittens: MoE Megakernel for NVL72s
Stuart H. Sul, Nash Brown, Henry Wildermuth +3
AI accelerator systems are rapidly consolidating into scale-up architectures, where tens to thousands of GPUs communicate over high-bandwidth, single-hop fabrics. We find that exis…
ParallelKittens: Systematic and Practical Simplification of Multi-GPU AI Kernels
Stuart H. Sul, Simran Arora, Benjamin F. Spector +1
Inter-GPU communication has become a major bottleneck for modern AI workloads as models scale and improvements in hardware compute throughput outpace improvements in interconnect b…
HipKittens: Fast and Furious AMD Kernels
William Hu, Drew Wadsworth, Sean Siddens +6
AMD GPUs offer state-of-the-art compute and memory bandwidth; however, peak performance AMD kernels are written in raw assembly. To address the difficulty of mapping AI algorithms…