147 citations · 396 across the 14 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2021
Distiller: A Systematic Study of Model Distillation Methods in Natural Language Processing
Haoyu He, Xingjian Shi, Jonas Mueller +3
We aim to identify how different components in the KD pipeline affect the resulting performance and how much the optimal KD pipeline varies across different datasets/tasks, such as…
cs.CL2019
Language Models with Transformers
Chenguang Wang, Mu Li, Alexander J. Smola
The Transformer architecture is superior to RNN-based models in computational efficiency. Recently, GPT and BERT demonstrate the efficacy of Transformer models on various NLP tasks…