4 papers
VTC: DNN Compilation with Virtual Tensors for Data Movement Elimination
Muyan Hu, Ahan Gupta, Jiachen Yuan +7
With the widening gap between compute and memory operation latencies, data movement optimizations have become increasingly important for DNN compilation. Current optimizations such…
MegaFold: Efficient Training of Next-Generation 3D Attention Protein Models on Cross-Platform GPUs
Hoa La, Ahan Gupta, Alex Morehead +2
Recent advances in biomolecular modeling have been catalyzed by models such as AlphaFold3 (AF3), which introduce science-informed changes to the transformer architecture. Unlike tr…
AutoSP: Unlocking Long-Context LLM Training Via Compiler-Based Sequence Parallelism
Ahan Gupta, Zhihao Wang, Neel Dani +3
Large-language-models (LLMs) demonstrate enormous utility in long-context tasks which require processing prompts that consist of tens to hundreds of thousands of tokens. However, e…
X-MoE: Enabling Scalable Training for Emerging Mixture-of-Experts Architectures on HPC Platforms
Yueming Yuan, Ahan Gupta, Jianping Li +3
Emerging expert-specialized Mixture-of-Experts (MoE) architectures, such as DeepSeek-MoE, deliver strong model quality through fine-grained expert segmentation and large top-k rout…