Distillation of encoder-decoder transformers for sequence labelling

Marco Farina, Duccio Pappadopulo, Anant Gupta +3

Driven by encouraging results on a wide range of tasks, the field of NLP is experiencing an accelerated race to develop bigger language models. This race for bigger models has also…