134 citations · 264 across the 14 of their papers we have counts for
Showing 2020Show all
3 papers · 1 filter
cs.CL2020★ 2 cited
Improving Task-Agnostic BERT Distillation with Layer Mapping Search
Xiaoqi Jiao, Huating Chang, Yichun Yin +6
Knowledge distillation (KD) which transfers the knowledge from a large teacher model to a small student model, has been widely used to compress the BERT model recently. Besides the…
cs.CL2020★ 10 cited
TernaryBERT: Distillation-aware Ultra-low Bit BERT
Wei Zhang, Lu Hou, Yichun Yin +4
Transformer-based pre-training models like BERT have achieved remarkable performance in many natural language processing tasks.However, these models are both computation and memory…
cs.CL2020
HyperText: Endowing FastText with Hyperbolic Geometry
Yudong Zhu, Di Zhou, Jinghui Xiao +3
Natural language data exhibit tree-like hierarchical structures such as the hypernym-hyponym relations in WordNet. FastText, as the state-of-the-art text classifier based on shallo…