57 citations · 57 across the 1 of their papers we have counts for
3 papers
cs.CL2020★ 57 cited
FastBERT: a Self-distilling BERT with Adaptive Inference Time
Weijie Liu, Peng Zhou, Zhe Zhao +3
Pre-trained language models like BERT have proven to be highly performant. However, they are often computationally expensive in many practical scenarios, for such heavy models can…
cs.CL2019
K-BERT: Enabling Language Representation with Knowledge Graph
Weijie Liu, Peng Zhou, Zhe Zhao +4
Pre-trained language representation models, such as BERT, capture a general language representation from large-scale corpora, but lack domain-specific knowledge. When reading a dom…
cs.CL2019
UER: An Open-Source Toolkit for Pre-training Models
Zhe Zhao, Hui Chen, Jinbin Zhang +7
Existing works, including ELMO and BERT, have revealed the importance of pre-training for NLP tasks. While there does not exist a single pre-training model that works best in all c…