35 citations · 50 across the 3 of their papers we have counts for
3 papers
cs.CL2022★ 6 cited
WeLM: A Well-Read Pre-trained Language Model for Chinese
Hui Su, Xiao Zhou, Houjin Yu +5
Large Language Models pre-trained with self-supervised learning have demonstrated impressive zero-shot generalization capabilities on a wide spectrum of tasks. In this work, we pre…
cs.LG2021★ 35 cited
PatrickStar: Parallel Training of Pre-trained Models via Chunk-based Memory Management
Jiarui Fang, Zilin Zhu, Shenggui Li +4
The pre-trained model (PTM) is revolutionizing Artificial Intelligence (AI) technology. However, the hardware requirement of PTM training is prohibitively high, making it a game fo…
cs.DC2020★ 9 cited
TurboTransformers: An Efficient GPU Serving System For Transformer Models
Jiarui Fang, Yang Yu, Chengduo Zhao +1
The transformer is the most critical algorithm innovation of the Nature Language Processing (NLP) field in recent years. Unlike the Recurrent Neural Network (RNN) models, Transform…