Can We Use Probing to Better Understand Fine-tuning and Knowledge Distillation of the BERT NLU?

Jakub Hościłowicz, Marcin Sowański, Piotr Czubowski +1

In this article, we use probing to investigate phenomena that occur during fine-tuning and knowledge distillation of a BERT-based natural language understanding (NLU) model. Our ul…