54 citations · 78 across the 17 of their papers we have counts for
4 papers · 1 filter
Efficient Long-context Language Model Training by Core Attention Disaggregation
Yonghao Zhuang, Junda Chen, Bo Pang +6
We present core attention disaggregation (CAD), a technique that improves long-context large language model training by decoupling the core attention computation, softmax(QK^T)V, f…
Frontier: Simulating the Next Generation of LLM Inference Systems
Yicheng Feng, Xin Tan, Kin Hang Sew +3
Large Language Model (LLM) inference is growing increasingly complex with the rise of Mixture-of-Experts (MoE) models and disaggregated architectures that decouple components like…
StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation
Yinmin Zhong, Zili Zhang, Xiaoniu Song +11
Reinforcement learning (RL) has become the core post-training technique for large language models (LLMs). RL for LLMs involves two stages: generation and training. The LLM first ge…
CDMPP: A Device-Model Agnostic Framework for Latency Prediction of Tensor Programs
Hanpeng Hu, Junwei Su, Juntao Zhao +4
Deep Neural Networks (DNNs) have shown excellent performance in a wide range of machine learning applications. Knowing the latency of running a DNN model or tensor program on a spe…