What Happens To BERT Embeddings During Fine-tuning?

Amil Merchant, Elahe Rahimtoroghi, Ellie Pavlick +1

While there has been much recent work studying how linguistic information is encoded in pre-trained sentence representations, comparatively little is understood about how these mod…