1 citations · 1 across the 3 of their papers we have counts for
5 papers
TRANSIT: Transparent Scale-in for Multi-Node LLM Training
Hyungyo Kim, Nicholas Satchanov, Hrishi Shah +9
TRANSIT is a transparent scale-in framework to enable multi-node model training on fewer GPUs while maintaining training efficiency by transparently leveraging CPU DRAM as an exten…
Towards Continuous Profiling and Optimization of Quantum-Classical Pipelines
Ayush Bansal, Owen Cochell, Santiago Núñez-Corrales +4
Quantum applications increasingly execute as multi-stage quantum-classical pipelines, interleaving QPU computation with classical stages like circuit generation, transpilation, lay…
Tuning Collective Patterns to Alleviate Congestion in Shared AI Clusters
Eashan Gupta, Yongzhou Chen, Apoorve Mohan +3
Distributed AI training involves recurring rounds of data exchange between multiple pairs of GPU nodes. Slowdown in even one flow due to congestion can cause the entire communicati…
Cost-Efficient LLM Training with Lifetime-Aware Tensor Offloading via GPUDirect Storage
Ziqi Yuan, Haoyang Zhang, Yirui Eric Zhou +4
We present the design and implementation of a new lifetime-aware tensor offloading framework for GPU memory expansion using low-cost PCIe-based solid-state drives (SSDs). Our frame…
The infrastructure powering IBM's Gen AI model development
Talia Gershon, Seetharami Seelam, Brian Belgodere +143
AI Infrastructure plays a key role in the speed and cost-competitiveness of developing and deploying advanced AI models. The current demand for powerful AI infrastructure for model…