14 papers
Opus: Photonic Rail-Optimized Fabric in ML Datacenters
Eric Ding, Barry Lyu, Bhaskar Kataria +1
Rail-optimized network fabrics have become the de facto datacenter scale-out fabric for large-scale ML training. However, the use of high-radix electrical switches to provide all-t…
CCL-Bench 1.0: A Trace-Based Benchmark for LLM Infrastructure
Eric Ding, Byungsoo Oh, Bhaskar Kataria +8
Evaluative claims about LLM infrastructure -- ``workload X is fastest on hardware Y with software Z'' -- depend on a complex configuration space spanning hardware accelerators, int…
Eliminating Hidden Serialization in Multi-Node Megakernel Communication
Byungsoo Oh, Rachee Singh
Recent megakernel designs for Mixture-of-Experts (MoE) inference fuse expert computation with fine-grained, GPU-initiated communication into a single persistent GPU kernel, and out…
TVCACHE: A Stateful Tool-Value Cache for Post-Training LLM Agents
Abhishek Vijaya Kumar, Bhaskar Kataria, Byungsoo Oh +2
In RL post-training of LLM agents, calls to external tools take several seconds or even minutes, leaving allocated GPUs idle and inflating post-training time and cost. While many t…
FlashMoE: Fast Distributed MoE in a Single Kernel
Osayamen Jonathan Aimuyo, Byungsoo Oh, Rachee Singh
The computational sparsity of Mixture-of-Experts (MoE) models enables sub-linear growth in compute cost as model size increases, thus offering a scalable path to training massive n…
Stable and Fault-Tolerant Decentralized Traffic Engineering
Arjun Devraj, Umesh Krishnaswamy, Ying Zhang +4
Cloud providers have recently decentralized their wide-area network traffic engineering (TE) systems to contain the impact of TE controller failures. In the decentralized design, a…