19 citations · 74 across the 10 of their papers we have counts for
Showing 2021Show all
3 papers · 1 filter
cs.CL2021★ 4 cited
A Short Study on Compressing Decoder-Based Language Models
Tianda Li, Yassir El Mesbahi, Ivan Kobyzev +6
Pre-trained Language Models (PLMs) have been successful for a wide range of natural language processing (NLP) tasks. The state-of-the-art of PLMs, however, are extremely large to b…
cs.CL2021★ 3 cited
How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding
Tianda Li, Ahmad Rashid, Aref Jafari +3
Knowledge Distillation (KD) is a model compression algorithm that helps transfer the knowledge of a large neural network into a smaller one. Even though KD has shown promise on a w…
cs.CL2021
Unsupervised Pre-training with Structured Knowledge for Improving Natural Language Inference
Xiaoyu Yang, Xiaodan Zhu, Zhan Shi +1
While recent research on natural language inference has considerably benefited from large annotated datasets, the amount of inference-related knowledge (including commonsense) prov…