22 citations · 38 across the 3 of their papers we have counts for
3 papers
cs.CL2021★ 15 cited
CPM-2: Large-scale Cost-effective Pre-trained Language Models
Zhengyan Zhang, Yuxian Gu, Xu Han +16
In recent years, the size of pre-trained language models (PLMs) has grown by leaps and bounds. However, efficiency issues of these large-scale PLMs limit their utilization in real-…
cs.DC2021★ 1 cited
DFOGraph: An I/O- and Communication-Efficient System for Distributed Fully-out-of-Core Graph Processing
Jiping Yu, Wei Qin, Xiaowei Zhu +4
With the magnitude of graph-structured data continually increasing, graph processing systems that can scale-out and scale-up are needed to handle extreme-scale datasets. While exis…
cs.CL2020★ 22 cited
CPM: A Large-scale Generative Chinese Pre-trained Language Model
Zhengyan Zhang, Xu Han, Hao Zhou +22
Pre-trained Language Models (PLMs) have proven to be beneficial for various downstream NLP tasks. Recently, GPT-3, with 175 billion parameters and 570GB training data, drew a lot o…