2 papers
cs.DC2024
MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Services
Dianhai Yu, Liang Shen, Hongxiang Hao +5
While modern internet services, such as chatbots, search engines, and online advertising, demand the use of large-scale deep neural networks (DNNs), distributed training and infere…
cs.CL2024
RecycleGPT: An Autoregressive Language Model with Recyclable Module
Yufan Jiang, Qiaozhi He, Xiaomin Zhuang +4
Existing large language models have to run K times to generate a sequence of K tokens. In this paper, we present RecycleGPT, a generative language model with fast decoding speed by…