4 papers · 1 filter
CoCoDiff: Optimizing Collective Communications for Distributed Diffusion Transformer Inference Under Ulysses Sequence Parallelism
Bin Ma, Xingjian Ding, Tekin Bicer +2
Diffusion Transformers (DiTs) are increasingly adopted in scientific computing, yet growing model sizes and resolutions make distributed multi-GPU inference essential. Ulysses sequ…
cMPI: Using CXL Memory Sharing for MPI One-Sided and Two-Sided Inter-Node Communications
Xi Wang, Bin Ma, Jongryool Kim +3
Message Passing Interface (MPI) is a foundational programming model for high-performance computing. MPI libraries traditionally employ network interconnects (e.g., Ethernet and Inf…
mLR: Scalable Laminography Reconstruction based on Memoization
Bin Ma, Viktor Nikitin, Xi Wang +2
ADMM-FFT is an iterative method with high reconstruction accuracy for laminography but suffers from excessive computation time and large memory consumption. We introduce mLR, which…
ZenFlow: Enabling Stall-Free Offloading Training via Asynchronous Updates
Tingfeng Lan, Yusen Wu, Bin Ma +7
Fine-tuning large language models (LLMs) often exceeds GPU memory limits, prompting systems to offload model states to CPU memory. However, existing offloaded training frameworks l…