6 papers
CoCoDiff: Optimizing Collective Communications for Distributed Diffusion Transformer Inference Under Ulysses Sequence Parallelism
Bin Ma, Xingjian Ding, Tekin Bicer +2
Diffusion Transformers (DiTs) are increasingly adopted in scientific computing, yet growing model sizes and resolutions make distributed multi-GPU inference essential. Ulysses sequ…
TierBPF: Page Migration Admission Control for Tiered Memory via eBPF
Xi Wang, Tal Zussman, Yuang Xu +3
Existing software-based memory tiering systems decide which pages to place on the slower or faster tier. However, they do not take into account two important factors that greatly i…
Hybrid Adaptive Tuning for Tiered Memory Systems
Xi Wang, Jie Liu, Shuangyan Yang +3
Memory tiering provides a cost-effective solution to increase memory capacity, utilization, and even bandwidth. Memory tiering relies on system software for memory profiling, detec…
cMPI: Using CXL Memory Sharing for MPI One-Sided and Two-Sided Inter-Node Communications
Xi Wang, Bin Ma, Jongryool Kim +3
Message Passing Interface (MPI) is a foundational programming model for high-performance computing. MPI libraries traditionally employ network interconnects (e.g., Ethernet and Inf…
mLR: Scalable Laminography Reconstruction based on Memoization
Bin Ma, Viktor Nikitin, Xi Wang +2
ADMM-FFT is an iterative method with high reconstruction accuracy for laminography but suffers from excessive computation time and large memory consumption. We introduce mLR, which…
ZenFlow: Enabling Stall-Free Offloading Training via Asynchronous Updates
Tingfeng Lan, Yusen Wu, Bin Ma +7
Fine-tuning large language models (LLMs) often exceeds GPU memory limits, prompting systems to offload model states to CPU memory. However, existing offloaded training frameworks l…