Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
AutoSP: Unlocking Long-Context LLM Training Via Compiler-Based Sequence Parallelism
Ahan Gupta, Zhihao Wang, Neel Dani +3
Large-language-models (LLMs) demonstrate enormous utility in long-context tasks which require processing prompts that consist of tens to hundreds of thousands of tokens. However, e…
cs.LG2025
X-MoE: Enabling Scalable Training for Emerging Mixture-of-Experts Architectures on HPC Platforms
Yueming Yuan, Ahan Gupta, Jianping Li +3
Emerging expert-specialized Mixture-of-Experts (MoE) architectures, such as DeepSeek-MoE, deliver strong model quality through fine-grained expert segmentation and large top-k rout…
cs.LG2024
FLuRKA: Fast and accurate unified Low-Rank & Kernel Attention
Ahan Gupta, Hao Guo, Yueming Yuan +2
Many efficient self-attention techniques have become prevalent since the inception of the transformer architecture. Two popular classes of these techniques a…