Transformer to CNN: Label-scarce distillation for efficient text classification

Yew Ken Chia, Sam Witteveen, Martin Andrews

Significant advances have been made in Natural Language Processing (NLP) modelling since the beginning of 2018. The new approaches allow for accurate results, even when there is li…