5 papers · 1 filter
Understanding the Synchronization Tax in GPU Scale-Up Domains
Arjun Devraj, Lindsey Bowen, Rachee Singh
GPU scale-up domains have become the building block of modern machine learning infrastructure, and their design follows a clear trajectory of exponential growth in both interconnec…
CCL-Bench 1.0: A Trace-Based Benchmark for LLM Infrastructure
Eric Ding, Byungsoo Oh, Bhaskar Kataria +8
Evaluative claims about LLM infrastructure -- ``workload X is fastest on hardware Y with software Z'' -- depend on a complex configuration space spanning hardware accelerators, int…
Eliminating Hidden Serialization in Multi-Node Megakernel Communication
Byungsoo Oh, Rachee Singh
Recent megakernel designs for Mixture-of-Experts (MoE) inference fuse expert computation with fine-grained, GPU-initiated communication into a single persistent GPU kernel, and out…
TVCACHE: A Stateful Tool-Value Cache for Post-Training LLM Agents
Abhishek Vijaya Kumar, Bhaskar Kataria, Byungsoo Oh +2
In RL post-training of LLM agents, calls to external tools take several seconds or even minutes, leaving allocated GPUs idle and inflating post-training time and cost. While many t…
Opus: Photonic Rail-Optimized Fabric in ML Datacenters
Eric Ding, Barry Lyu, Bhaskar Kataria +1
Rail-optimized network fabrics have become the de facto datacenter scale-out fabric for large-scale ML training. However, the use of high-radix electrical switches to provide all-t…